What's the difference between staticTests.ts and dynamicTests.ts in this example?

staticTests.ts provides a fixed set of test cases with TypeScript type checking, while dynamicTests.ts generates test cases dynamically from a simulated database at evaluation time.

When should I use function-generated test cases instead of YAML?

Use function-generated test cases when your test data has structure worth type-checking or when it needs to be pulled from a live source like a database or API, rather than hand-maintained in YAML.

What do I need to set up to run this example?

You need to install promptfoo with `npm install promptfoo` and set the `OPENAI_API_KEY` environment variable, then run `npx promptfoo eval` to execute the evaluation.

Which AI provider does this example use?

This example runs against OpenAI via the `OPENAI_API_KEY`, though the test-generation mechanism itself works with any provider.

Prompt Chain

Generate Javascript Test Cases

Name: Config Javascript Test Cases
Availability: OnlineOnly
Author: Promptfoo

A promptfoo example generating test cases from JavaScript/TypeScript functions, both statically and dynamically.

Copy chain

Works with github

Promptfoo

Own this? Claim it

Spark score

out of 100

Updated 27 days ago

Version 0.121.18

Models

gpt 4o

Add to Favorites

Why it matters

Automate the creation and validation of Javascript test cases for your codebase. This asset helps ensure code quality and reliability by generating comprehensive tests.

Outcomes

What it gets done

Generate unit tests for Javascript functions.

Validate generated test cases for correctness.

Integrate test generation into CI/CD pipelines.

Debug and refine test generation logic.

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-config-javascript-test-cases | bash

Steps

Steps in the chain

Install dependencies

Set up your API key

Run the evaluation

Overview

Config Javascript Test Cases

This promptfoo example generates test cases from TypeScript functions, demonstrating both statically type-checked test cases and dynamic test case generation from a simulated database. Use it as a template when promptfoo test cases need to be generated from code or a dynamic data source rather than written as static YAML.

What it does

This promptfoo example demonstrates different ways to generate test cases using JavaScript/TypeScript functions. It shows both static test case generation (with type-checking examples) and dynamic test case generation, simulated from a database.

npx promptfoo@latest init --example config-javascript-test-cases
cd config-javascript-test-cases

When to use - and when NOT to

Use this as a starting template when your promptfoo test cases need to be generated programmatically in TypeScript rather than written out as static YAML - for example pulling test inputs from a data source, or wanting compile-time type checking on your test case shape. It is not itself a specific eval subject; it's a pattern for test-case generation.

Inputs and outputs

The example ships three files: promptfooconfig.yaml (the configuration specifying prompts and providers), staticTests.ts (static test cases with type-checking examples), and dynamicTests.ts (dynamic test case generation from a simulated database). To run it: install dependencies with npm install promptfoo, set your API key via export OPENAI_API_KEY=your_api_key_here, and run the evaluation with npx promptfoo eval.

Who it's for

Developers who want their promptfoo test cases generated from code rather than hand-written YAML, particularly when test data comes from a database or other dynamic source, or when TypeScript's type checking should validate the test case shape before an eval runs.

Source README

config-javascript-test-cases (JavaScript/TypeScript Test Cases Example)

You can run this example with:

npx promptfoo@latest init --example config-javascript-test-cases
cd config-javascript-test-cases

This example demonstrates different ways to generate test cases using JavaScript/TypeScript functions. It shows both static and dynamic test case generation with optional type checking.

Files

promptfooconfig.yaml - Configuration file specifying prompts and providers
staticTests.ts - Static test cases with type checking examples
dynamicTests.ts - Dynamic test case generation from a simulated database

Usage

Install dependencies:
```
npm install promptfoo
```

Set up your API key:

export OPENAI_API_KEY=your_api_key_here

Run the evaluation:
```
npx promptfoo eval
```

FAQ

Common questions

Discussion