What does the Integration E2B prompt chain do?

It creates a complete pipeline that generates Python functions from prompts using an LLM, executes the generated code safely in e2b sandboxes, generates verification unit tests with OpenAI, and produces JSON metrics and markdown reports for evaluation.

What API keys and setup are required to use this?

You need E2B_API_KEY, OPENAI_API_KEY (or another LLM provider's key), and must set PROMPTFOO_PYTHON to your venv Python. Setup involves creating a virtual environment, installing e2b-code-interpreter and promptfoo globally via npm.

When should I use this instead of other code evaluation approaches?

Use this when you need to actually generate, execute, and verify code end-to-end in a safe sandbox. It's not needed for evals that only inspect generated code as text or when you prefer Docker-based sandboxes instead of hosted e2b sandboxes.

What output does this produce?

It produces promptfoo eval results viewable with `promptfoo view`, JSON metrics stored in `.promptfoo_results/`, and a human-friendly markdown report generated by `report.py`.

Prompt Chain

Automate Code Execution and Evaluation

Name: Integration E2B
Availability: OnlineOnly
Author: Promptfoo

A promptfoo pipeline that has an LLM write code, runs it in an e2b sandbox, and grades it with generated unit tests.

Copy chain

Works with githube2b

Promptfoo

Own this? Claim it

Spark score

out of 100

Updated 4 days ago

Version 0.121.19

Models

gpt 4o

Add to Favorites

Why it matters

Integrate prompts with a sandboxed execution environment to automate code generation, testing, and evaluation pipelines.

Outcomes

What it gets done

Define a prompt-based pipeline for code execution.

Execute generated code within a secure sandbox.

Evaluate code output against defined metrics.

Automate the review and debugging of generated code.

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-integration-e2b | bash

Steps

Steps in the chain

Set up environment variables

Create and activate Python virtual environment

Install required packages

Activate venv and verify environment variables

Run the evaluation

View interactive results

Overview

Integration E2B

This promptfoo example generates Python functions from a prompt, executes them safely in an e2b sandbox, generates and re-runs verification unit tests, and writes JSON metrics plus a markdown report. Use it as a template when LLM-generated code needs to be evaluated by actual sandboxed execution rather than static or text-only grading.

What it does

This example shows a complete prompt-to-LLM-to-sandboxed-execution-to-metric pipeline: promptfoo runs LLM prompts and manages evaluation cases, an LLM provider generates Python functions from a short problem prompt, e2b sandboxes (via e2b-code-interpreter) run the generated code safely, an OpenAI step generates small verification unit tests and re-runs them in the sandbox, and per-run JSON metrics are written to .promptfoo_results/ with a human-friendly markdown report produced by report.py.

npx promptfoo@latest init --example integration-e2b
cd integration-e2b

When to use - and when NOT to

Use this as a template when you need to evaluate LLM-generated code by actually executing it safely, rather than just grading the code text - useful for coding-assistant evals where correctness depends on runtime behavior, not just syntax. It requires both an e2b sandbox API key and an OpenAI (or other LLM provider) API key.

Inputs and outputs

Required environment variables: E2B_API_KEY for the e2b sandbox, OPENAI_API_KEY (or your chosen provider's key), and recommended PROMPTFOO_PYTHON pointing at the local virtual environment's Python so promptfoo knows which interpreter to use. Setup: create and activate a Python virtual environment, then pip install --upgrade pip and pip install e2b-code-interpreter, plus npm i -g promptfoo. To run: activate the venv, export the required environment variables, run promptfoo eval, then open the interactive viewer with promptfoo view.

Integrations

Combines promptfoo, an OpenAI (or compatible) LLM provider, and e2b's e2b-code-interpreter sandboxing package, with results written as JSON metrics and a markdown report.

Who it's for

Teams building or evaluating a coding-assistant or code-generation LLM feature who need to check generated code by actually running it - and its generated tests - in a safe sandbox rather than relying on static or LLM-graded review alone.

Source README

integration-e2b (E2B Code Evaluation)

What This Example Demonstrates

This example shows a complete prompt→LLM→sandboxed-execution→metric pipeline using:

promptfoo to run LLM prompts and manage evaluation cases.
An LLM provider to generate Python functions from a short problem prompt.
e2b sandboxes (via e2b-code-interpreter) to run generated code safely.
OpenAI step to generate small verification unit tests and re-run them in the sandbox.
Per-run JSON metrics written to .promptfoo_results/ and a human-friendly markdown report produced by report.py.

You can run this example with:

npx promptfoo@latest init --example integration-e2b
cd integration-e2b

Environment Variables

Set these in your shell before running the example.

### Required
export E2B_API_KEY="e2b_xxx_your_key_here"        # e2b sandbox API key
export OPENAI_API_KEY="sk_xxx_your_key_here"     # OpenAI key (or your chosen LLM provider)

### Recommended
export PROMPTFOO_PYTHON="$(pwd)/.venv/bin/python"  # tell promptfoo which Python/venv to use

If you use a different provider name in promptfooconfig.yaml, add that provider's key instead.

Prerequisites

Install and prepare a Python virtual environment, and install the required packages.

### create & activate venv
python -m venv .venv
source .venv/bin/activate

### install Python packages
pip install --upgrade pip
pip install e2b-code-interpreter
npm i -g promptfoo

Running the Example

Activate venv and ensure env vars are set:

source .venv/bin/activate
export E2B_API_KEY="e2b_xxx"
export OPENAI_API_KEY="sk_xxx"
export PROMPTFOO_PYTHON="$(pwd)/.venv/bin/python"

Run the evaluation:

promptfoo eval

Open the interactive viewer:

promptfoo view

FAQ

Common questions

Discussion