Prompt Chain

Generate and Test Code with Fireworks AI

Name: Generate and Test Code with Fireworks AI
Availability: OnlineOnly
Author: Promptfoo

A promptfoo example demonstrating how to configure and test prompts using the Fireworks AI provider for LLM evaluation and benchmarking workflows.

Copy chain

Promptfoo

Maintainer?

Spark score

out of 100

Updated 2 days ago

Version code-scan-action-0.1

Models

deepseek v3

Add to Favorites

Why it matters

This asset helps developers generate, test, and refine code using the Fireworks AI platform. It automates parts of the code development lifecycle, ensuring higher quality output.

Outcomes

What it gets done

Generate code snippets based on prompts.

Execute tests against generated code.

Debug and iterate on code based on test results.

Integrate with Fireworks AI for code generation.

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-provider-fireworks | bash

Capabilities

What this chain does

Generate code

Writes source code or scripts from a description.

Review code

Analyzes code for bugs, style issues, and improvements.

Debug

Traces errors to their root cause and suggests fixes.

Overview

Provider Fireworks

What it does

This is a working example configuration that demonstrates how to set up promptfoo to use Fireworks AI as a provider. It shows the necessary configuration structure for running prompt evaluations against Fireworks models within the promptfoo testing framework.

How it connects

Use this example when you want to add Fireworks AI to your promptfoo evaluation workflow, need a reference implementation for provider configuration, or are setting up automated prompt testing across multiple LLM providers including Fireworks.

Source README

provider-fireworks (Fireworks AI)

You can run this example with:

npx promptfoo@latest init --example provider-fireworks
cd provider-fireworks

Usage

Set your FIREWORKS_API_KEY environment variable. You can get a key from fireworks.ai, sign in, open Settings -> API Keys, and create one.

Then run:

promptfoo eval

View the results with promptfoo view.

What this shows

Three Fireworks serverless chat models (gpt-oss-120b, deepseek-v4-pro, kimi-k2p6) compared on a summarisation task. These are reasoning models, so max_tokens is set high enough to leave room for hidden reasoning tokens plus the visible answer.
A similar assertion graded by a Fireworks embedding model via the fireworks:embedding: prefix.

Models rotate in and out of the serverless tier - if a model 404s, pick a current one from the serverless catalogue.

Discussion