Implement RAG Systems

An 8-phase RAG (Retrieval-Augmented Generation) implementation workflow, each phase paired with named sub-skills. Phase 1 (Requirements Analysis) invokes ai-product and rag-engineer to define the use case, data sources, accuracy requirements, latency targets, and evaluation metrics. Phase 2 (Embedding Selection) invokes embedding-strategies to evaluate embedding models for domain relevance, quality, and cost or latency before selecting one. Phase 3 (Vector Database Setup) invokes vector-database-engineer and similarity-search-patterns to choose a vector database, design its schema, configure indexes, and test queries. Phase 4 (Chunking Strategy) invokes rag-engineer to choose a chunk size, implement chunking with overlap handling, attach metadata, and test retrieval quality. Phase 5 (Retrieval Implementation) invokes similarity-search-patterns and hybrid-search-implementation to implement vector search, add keyword search, configure hybrid search, set up reranking, and optimize latency. Phase 6 (LLM Integration) invokes llm-application-dev-ai-assistant and llm-application-dev-prompt-optimize to select an LLM provider, design a prompt template, inject retrieved context, add citation handling, and test generation quality. Phase 7 (Caching) invokes prompt-caching and rag-engineer to cache responses and embeddings, configure TTL, add cache invalidation, and monitor hit rates. Phase 8 (Evaluation) invokes llm-evaluation and evaluation to define metrics, build a test dataset, measure retrieval accuracy and generation quality, and iterate. The overall architecture is summarized as a single pipeline: a user query flows through an embedding model to vector search against a vector DB, retrieving chunks from a chunk store, which are combined with the prompt and context into the LLM to produce a response. A six-item quality gate checks that the embedding model is selected, the vector DB is configured, chunking is implemented, retrieval works, the LLM is integrated, and evaluation is passing.

Use when building a RAG-powered application, implementing semantic search, creating knowledge-grounded AI, setting up a document Q&A system, or optimizing retrieval quality end to end. Not for general AI/ML development without a retrieval component, which routes to the sibling ai-ml bundle, broader agent-building work beyond RAG, which is ai-agent-development, or standalone vector-database work with no LLM generation step, which is database.

Input is the target use case, data sources, and accuracy and latency requirements. Output across the eight phases is a selected embedding model, a configured vector database with schema and indexes, a chunking implementation with overlap and metadata, a hybrid retrieval pipeline with reranking, an LLM integration with prompt templates and citation handling, a caching layer, and an evaluation report on retrieval accuracy and generation quality.

User Query -> Embedding -> Vector Search -> Retrieved Docs -> LLM -> Response
                |              |              |              |
            Model         Vector DB     Chunk Store    Prompt + Context

Orchestrates named sub-skills ai-product, rag-engineer, embedding-strategies, vector-database-engineer, similarity-search-patterns, hybrid-search-implementation, llm-application-dev-ai-assistant, llm-application-dev-prompt-optimize, prompt-caching, and llm-evaluation.

AI and ML engineers building a RAG system from scratch who want a phased sequence - requirements, embeddings, vector DB, chunking, retrieval, LLM integration, caching, evaluation - rather than assembling a RAG pipeline ad hoc.