Standardize AI engineering workflows with scoring baselines

AI Engineering Toolkit outlines six structured workflows for prompt evaluation, context allocation, retrieval architecture, agent safety, evaluation design, and product strategy. It emphasizes repeatable rubrics, decision trees, and checklists rather than unstructured requests for feedback. Scores can vary by evaluator, model, and context, so comparisons require calibrated rubrics and recorded versions.

This catalog bundle contains the overview only. The six workflow packages, 65 test cases, and CI templates are not included. A scored assessment or completed test run therefore requires separately supplied rubrics, cases, a harness, and actual execution evidence. The illustrative before-and-after prompt scores and sample security findings are report examples, not measured results.

Use the overview to choose a methodology before evaluating prompts, designing a RAG pipeline, allocating a context window, planning pre-launch safety work, building evaluations, or deciding whether to build a product. The suggested sequence connects RAG design, context optimization, prompt refinement, security audit, and evaluation setup. A single overall score should not replace the full dimension profile.

The security workflow is limited to education or authorized assessment. Target-affecting commands require an exact target, written owner authorization, stated scope, disclosure of the concrete commands and effects, and explicit confirmation. Each adversarial test phase also requires confirmation. A sandbox or disposable VM is preferred. This overview does not authorize external probes, data extraction, credential access, persistence, or changes to a target.

Prompt Evaluator describes eight dimensions: clarity, specificity, completeness, conciseness, structure, grounding, safety, and robustness. It aggregates weighted 1-to-10 judgments into a 0-to-100 score, identifies weak dimensions, and proposes rewrites for single, A/B, or batch comparisons.

Context Budget Planner divides tokens among system instructions, few-shot examples, user input, retrieval, and output, then chooses compression strategies. RAG Pipeline Architect connects document parsing and chunking decisions to embedding selection, vector, keyword, or hybrid retrieval, and measures such as faithfulness, relevancy, and context precision. It distinguishes Naive, Advanced, and Modular RAG patterns.

Agent Safety Guard describes a 65-point assessment covering direct and indirect prompt injection, information extraction, tool abuse, and goal hijacking. The documented test prompts remain inside an evaluation context rather than interacting with external systems. Actual execution needs the absent test assets and recorded evidence.

Eval Harness Builder designs metric systems and LLM-as-Judge scoring with attention to position, verbosity, and self-enhancement bias. Product Sense Coach uses five phases covering motivation, opportunity, paths, scenarios, and competition.

Installation contexts include Claude Code, WorkBuddy, and Cursor and identifies an MIT-licensed upstream repository. Its example prompt for an evaluation is:

You are a customer support agent. Help users with their questions. Be nice and helpful.

Separately installing upstream material requires review of a fixed revision and license, selected files, and the authorized workspace while preserving existing files. An external harness may read or write files, transmit prompts, or incur costs. Installation alone neither supplies proof of a scored run nor validates a production system.