Ship Production-Ready AI Features

This skill covers building AI-powered product features on five core principles: LLMs are probabilistic, not deterministic (design for variance, validate output, never trust it blindly); prompt engineering is product engineering (version prompts, test them, A/B test them - a one-word change can flip behavior); RAG over fine-tuning for most use cases (fine-tuning is expensive and goes stale, RAG lets you add knowledge without retraining); design for latency (LLM calls take 1-30 seconds - stream responses, show progress, cache aggressively); and cost is a feature (measure cost per query, use smaller models where sufficient, cache everything cacheable).

It documents patterns for structured output with validation, streaming with progress indicators, prompt versioning and testing, caching expensive operations, a circuit breaker for LLM failures, and RAG with hybrid search. It also catalogs eight sharp edges: trusting LLM output without validation (fix: validate against a schema or use function calling, with a fallback); putting user input directly into prompts without sanitization (fix: separate user input, sanitize it, filter output, apply least privilege); stuffing too much into the context window (fix: calculate tokens before sending and apply reduction strategies); waiting for a complete response before showing anything (fix: stream responses with a frontend typing indicator and a fallback for structured output); not monitoring LLM API costs (fix: track cost per request, implement limits, optimize prompts); the app breaking whenever the LLM API fails (fix: defense-in-depth with a circuit breaker); not validating factual claims from LLM responses (fix: RAG with source verification, showing uncertainty, domain-specific validation); making LLM calls in synchronous request handlers (fix: streaming for chat, a job queue for processing, optimistic UI); and changing prompts in production without version control (fix: treat prompts as code, store in version control or a prompt-management system, A/B test); plus fine-tuning before exhausting RAG and prompting options first.

Use this when building or reviewing a production LLM-powered feature - anything from a chat assistant to a RAG pipeline - especially to catch the specific failure modes that come from treating LLM calls like a normal deterministic API call.

Not needed for non-AI feature work, and this skill explicitly recommends trying better prompts, then RAG, before reaching for fine-tuning as a last resort - so it's not the place to start if you've already decided to fine-tune without exhausting those cheaper options.

Inputs: the target LLM feature's requirements (latency tolerance, cost budget, factual-accuracy needs) and its current implementation if reviewing an existing feature.

Outputs: a production-ready LLM feature design with output validation, prompt versioning, cost tracking, streaming UX, circuit-breaker error handling, and RAG-based grounding where factual accuracy matters - checked against validation flags like unvalidated LLM output, unsanitized user input in prompts, missing rate limiting, sequential (non-batched) embedding generation, and single-provider-with-no-fallback risk.

Applies broadly across LLM providers and RAG/vector-store stacks; not tied to a specific vendor.

Product engineers building LLM-powered features who need the specific discipline - validation, versioning, cost/latency design, and failure handling - that production AI features require beyond a working demo.