Implement LLM Operations for Production AI

LLM-OPS is a production-LLM-engineering skill covering RAG pipelines, embeddings, vector databases, fine-tuning, advanced prompt engineering, LLM cost reduction, quality evals, semantic caching, streaming, and agents - framed around the idea that the difference between an AI prototype and an AI product is operability.

It provides a full RAG architecture (documents to chunking to embeddings to vector DB, with query-time embedding, semantic search, and top-K context retrieval feeding the LLM) along with working indexing and query pipeline code using the Anthropic SDK and ChromaDB - a chunk_text function with configurable chunk size and overlap, an index_document function that upserts chunks, and a rag_query function that retrieves top-K results filtered by a distance threshold, assembles sourced context, and calls a Claude model. It compares five vector database options (Chroma, pgvector, Pinecone, Weaviate, Qdrant) by best-fit use case, hosting, and cost, and includes a working pgvector setup - extension creation, an embeddings table with a vector(1536) column, an ivfflat cosine-similarity index, and a similarity-ranked query.

CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE knowledge_embeddings (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    content TEXT NOT NULL,
    embedding vector(1536),
    metadata JSONB,
    created_at TIMESTAMPTZ DEFAULT NOW()
);
CREATE INDEX ON knowledge_embeddings
USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100);
SELECT content, 1 - (embedding <=> QUERY_VECTOR) AS similarity
FROM knowledge_embeddings ORDER BY similarity DESC LIMIT 5;

For prompt engineering, it documents an elite system-prompt structure (identity, behavioral rules, capabilities, limitations, and personalization variables, illustrated with a voice-assistant persona named Auri) and a chain-of-thought analysis function that walks a five-step reasoning template before producing a concise final answer. A SemanticCache class caches responses keyed by embedding similarity above a configurable threshold to avoid redundant LLM calls. A cost-estimation function holds per-model pricing (Opus, Sonnet, Haiku input/output rates) and projects monthly cost from average input/output tokens and daily request volume. An evaluation framework calls a cheaper model (Haiku) to score a response against expected output across named criteria (e.g. factual accuracy, relevance, clarity) on a 0-10 scale with JSON-formatted justification, and provides an example eval case list format.

Seven commands cover the operational workflow: /rag-setup (full RAG pipeline), /embed-docs (index documents into the vector DB), /prompt-optimize (optimize a prompt for quality and cost), /cost-estimate (project monthly LLM cost), /eval-run (run the quality eval suite), /cache-setup (configure semantic caching), and /model-select (choose the right model for a use case).

Use this skill when implementing RAG, building an embeddings pipeline, setting up Pinecone/Chroma/pgvector, fine-tuning, advanced prompt engineering, reducing LLM costs, running evals, setting up semantic caching, or building streaming or agent-based LLM systems.

Do not use it for tasks unrelated to LLM operations, when a simpler tool fits, or when the user needs general-purpose assistance without this domain expertise. All code should be reviewed before production use rather than applied as-is.

Inputs: documents to index, a query to run through RAG, a prompt to optimize, model/volume parameters for cost estimation, or question/expected/actual triples for evaluation.

Outputs: an indexed vector-DB collection, a RAG-grounded response with sourced context, an optimized system prompt, a monthly cost projection by model, an eval score with per-criterion justification, or a configured semantic cache.

Works with the Anthropic SDK (Claude Opus/Sonnet/Haiku), ChromaDB, and PostgreSQL's pgvector extension; documents alternative vector databases (Pinecone, Weaviate, Qdrant) for production-scale or multi-modal use cases.

Engineers building or operating production LLM systems who need RAG pipelines, vector database setup, prompt engineering structure, cost estimation, and quality evaluation in one place.