What is the purpose of this example?

This example demonstrates how to define custom assertion logic in an external JavaScript file (`assert.js`) that is referenced from `promptfooconfig.yaml`, rather than writing assertions inline in the configuration.

When should I use an external JavaScript file for assertions?

Use an external file when your custom assertion logic is complex or reused frequently enough that keeping it in its own JavaScript file is cleaner than writing it inline in the config. For simple one-off assertions, inline config is easier.

What are the requirements to run this example?

You need `OPENAI_API_KEY` set, and the example uses `assert.js` and `promptfooconfig.yaml` files. The output is generated by running `promptfoo eval` or `promptfoo view`.

Does this approach work with providers other than OpenAI?

Yes, the external-assertion-file mechanism works with any provider, though this specific example runs against OpenAI via `OPENAI_API_KEY`.

Prompt Chain

Validate AI Code Generation with External JavaScript Tests

Name: Eval Javascript Assert External
Availability: OnlineOnly
Author: Promptfoo

Run external JavaScript assertions to validate AI-generated code quality and correctness with automated testing workflows.

Copy chain

Works with openaipromptfoo

Promptfoo

Own this? Claim it

Spark score

out of 100

Updated 19 days ago

Version 0.121.18

Add to Favorites

Why it matters

Evaluate and validate AI-generated code outputs by running custom JavaScript assertion functions against model responses, ensuring code quality and correctness before deployment.

Outcomes

What it gets done

Execute custom JavaScript assertion logic against AI model outputs

Configure test cases and validation rules in YAML configuration files

Run automated evaluation workflows to verify code generation quality

View detailed test results and assertion outcomes through built-in interface

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-eval-javascript-assert-external | bash

Steps

Steps in the chain

Initialize example project

Navigate to project directory

Set OpenAI API key

Edit configuration files

Run evaluation

View results

Overview

Eval Javascript Assert External

A promptfoo example showing how to use external JavaScript files for assertion logic. Use this example when you want to organize assertion logic in separate JavaScript files rather than embedding it directly in your configuration file.

What it does

This promptfoo example shows how to use external JavaScript files for assertion logic when evaluating prompts. You can reference external .js files in your configuration.

When to use - and when NOT to

Use this example when you want to organize assertion logic in separate JavaScript files rather than embedding it directly in your configuration file.

Inputs and outputs

You provide an assert.js file and a promptfooconfig.yaml configuration file. The example requires an OPENAI_API_KEY environment variable.

You receive evaluation results that can be viewed using the promptfoo command-line tools.

How to use

Initialize the example with:

npx promptfoo@latest init --example eval-javascript-assert-external
cd eval-javascript-assert-external

Set your OPENAI_API_KEY environment variable, then edit the assert.js and promptfooconfig.yaml files.

Run the evaluation:

promptfoo eval

View results with promptfoo view.

Integrations

This example requires an OpenAI API key.

Who it's for

This example is for users who want to organize their assertion logic in external JavaScript files when working with promptfoo.

Source README

eval-javascript-assert-external (Javascript Assert External)

You can run this example with:

npx promptfoo@latest init --example eval-javascript-assert-external
cd eval-javascript-assert-external

Usage

To get started, set your OPENAI_API_KEY environment variable.

Next, edit assert.js and promptfooconfig.yaml.

Then run:

promptfoo eval

Afterwards, you can view the results by running promptfoo view

FAQ

Common questions

Discussion