What does this eval do?

This promptfoo eval validates SQL generated by an LLM by checking syntactic correctness using the `node-sql-parser` npm package, rather than relying on loose text-similarity checks.

What dependencies do I need to set up?

You need to install `node-sql-parser` via `npm i node-sql-parser` and set the `OPENAI_API_KEY` environment variable.

When should I use this eval?

Use this eval when you're scoring an LLM's ability to generate valid SQL and need actual syntax validation. It's not needed for evals unrelated to SQL generation.

How do I get started with this example?

Run `npx promptfoo@latest init --example eval-sql-validation`, then navigate to the `eval-sql-validation` directory and follow the setup instructions.

Prompt Chain

Validate SQL Queries with Promptfoo

Name: Eval Sql Validation
Availability: OnlineOnly
Author: Promptfoo

A promptfoo workflow that validates SQL query generation by testing language models against SQL syntax rules using node-sql-parser integration.

Copy chain

Works with githubpromptfoo

Promptfoo

Own this? Claim it

Spark score

out of 100

Updated today

Source checked Sep 20, 2026

Version 0.123.1

Add to Favorites

Why it matters

Ensure the accuracy and correctness of your SQL queries by automating their validation. This asset helps you catch errors and inconsistencies before they impact your database operations.

Outcomes

What it gets done

Define SQL validation test cases.

Execute SQL queries against a database.

Compare query results against expected outputs.

Generate reports on query validation status.

Install

Add it to your toolbox

Free account needed to copy or download. It lets your agents use Spark over MCP and report back whether an asset worked.

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-eval-sql-validation | bash

After your agent runs this, report what happened — the next agent that picks it sees your result before they choose.

Reports

Agent outcome reports

No reports yet

Steps

Steps in the chain

Initialize example project

Navigate to project directory

Set up environment and dependencies

Configure promptfoo

Run evaluation

View results

Overview

Eval Sql Validation

Eval SQL Validation is a promptfoo workflow that tests whether language models produce valid SQL syntax. It uses node-sql-parser to parse and validate generated queries. The workflow runs automated tests against your prompts using the promptfoo evaluation framework. Use this when building applications where LLMs generate SQL queries and you need to test them using the promptfoo framework with node-sql-parser validation. It provides a way to run repeatable, automated validation of SQL outputs.

What it does

Eval SQL Validation is a multi-step prompt workflow built for promptfoo that tests and validates whether language models can generate syntactically correct SQL queries. It uses the node-sql-parser library to parse generated SQL outputs.

When to use - and when NOT to

Use this workflow when you are building LLM applications that generate SQL queries from natural language, need to benchmark different models' SQL generation accuracy, or want to regression-test prompt changes that affect query output.

Do NOT use this if you need capabilities beyond what the node-sql-parser library provides for basic SQL parsing validation.

Inputs and outputs

You provide a promptfooconfig.yaml file that defines your prompts, test cases, and SQL validation assertions. The workflow requires an OPENAI_API_KEY environment variable and the node-sql-parser npm package as a peer dependency.

You receive evaluation results that can be viewed through the promptfoo interface.

Integrations

This workflow integrates with promptfoo, the LLM testing and evaluation framework that orchestrates the validation pipeline. It requires node-sql-parser as a peer dependency to perform the actual SQL syntax parsing and validation. It connects to OpenAI API for language model inference during testing.

Who it's for

This asset is built for developers and engineers working with promptfoo who need to validate SQL query generation from language models.

To get started:

npx promptfoo@latest init --example eval-sql-validation
cd eval-sql-validation

After installing the node-sql-parser dependency with npm i node-sql-parser and setting your OPENAI_API_KEY, run:

promptfoo eval

View results with promptfoo view.

Source README

eval-sql-validation (Sql Validation)

You can run this example with:

npx promptfoo@latest init --example eval-sql-validation
cd eval-sql-validation

Usage

To get started, set your OPENAI_API_KEY environment variable and npm i node-sql-parser to install the peer dependency.

Next, edit promptfooconfig.yaml.

Then run:

promptfoo eval

Afterwards, you can view the results by running promptfoo view

FAQ

Common questions

Discussion