Implement LLM Application Patterns

Provides production-ready patterns for building LLM applications, inspired by Dify and industry best practices - RAG pipeline architecture, agent architectures, prompt IDE tooling, LLMOps/observability, and production reliability patterns.

Use this skill when designing LLM-powered applications, implementing RAG, building AI agents with tools, setting up LLMOps monitoring, or choosing between agent architectures.

RAG pipeline flows ingest -> retrieve -> generate. Document ingestion covers chunking strategies (fixed-size, semantic, recursive-separator, document-aware) with recommended settings (512-token chunks, 50-token overlap). Vector database options are compared (Pinecone for managed production scale, Weaviate for self-hosted multi-modal, ChromaDB for prototyping, pgvector for existing Postgres infra), alongside embedding model tradeoffs (OpenAI's text-embedding-3-small/-large vs. local bge-large). Retrieval strategies include basic semantic search, hybrid search blending semantic and BM25 keyword results via Reciprocal Rank Fusion, multi-query retrieval for better recall, and contextual compression to retrieve then extract only relevant parts. Generation grounds responses in retrieved context with a strict prompt template and returns citations alongside the answer.

Agent architectures: the ReAct pattern (Thought/Action/Observation loop) with a working Python implementation capped at max_iterations; function calling with JSON-schema tool definitions and a message-passing loop handling tool_calls; Plan-and-Execute, which creates a step plan, executes each step, and replans if progress diverges from the task; and Multi-Agent Collaboration, where a coordinator decomposes a task across specialized agents (researcher, analyst, writer, critic) and iterates based on critic feedback.

Prompt IDE patterns: a PromptTemplate class validating required variables and supporting few-shot example injection; a PromptRegistry for versioning prompts, A/B testing variants by user-bucket hashing, and recording outcome metrics; and PromptChain for sequencing prompts where each step's output feeds the next (e.g. research -> analyze -> summarize).

LLMOps and observability: a metrics taxonomy spanning performance (latency p50/p99, tokens/sec), quality (user satisfaction, task completion, hallucination rate), cost (cost/request, cache hit rate), and reliability (error/timeout/retry rate); structured request/response logging with OpenTelemetry distributed tracing; and an evaluation framework scoring relevance, coherence, groundedness, accuracy (against ground truth), and safety, aggregated across a benchmark test set.

Production patterns: a Redis-backed LLMCache keyed on a SHA-256 hash of model/prompt/kwargs, only caching deterministic (temperature=0) outputs:

def _cache_key(self, prompt, model, **kwargs):
    content = f"{model}:{prompt}:{json.dumps(kwargs, sort_keys=True)}"
    return hashlib.sha256(content.encode()).hexdigest()

a sliding-window RateLimiter plus tenacity-based exponential-backoff retry that only retries rate-limit/server errors, never client errors; and an LLMWithFallback wrapper that tries a primary model then cascades through fallback models on failure.

An architecture decision matrix maps each pattern (Simple RAG, Hybrid RAG, ReAct Agent, Function Calling, Plan-Execute, Multi-Agent) to complexity and cost, from low-complexity FAQ search up to very-high-complexity multi-agent research tasks.

References Dify, LangChain, LlamaIndex, and the Anthropic Cookbook as inspiration and further reading, alongside vector databases (Pinecone, Weaviate, ChromaDB, pgvector), OpenAI/local embedding models, Redis for caching, OpenTelemetry for tracing, and tenacity for retry logic.

Engineers designing or hardening LLM applications who need concrete architecture choices - RAG pipeline design, agent pattern selection, prompt versioning, and production reliability (caching, rate limiting, fallback) - backed by a decision matrix rather than picking patterns ad hoc.