What is the CometAPI provider example for?

This example demonstrates how to use CometAPI to access language models through a unified OpenAI-compatible interface within a promptfoo setup, specifically showing how to evaluate the `gpt-5-mini` model on geography questions.

How do I set up and run the CometAPI provider example?

Get an API key from CometAPI's console, set it as the `COMETAPI_KEY` environment variable, then run `npx promptfoo@latest init --example provider-cometapi`, navigate to the directory, and execute `npx promptfoo@latest eval`.

What model does this example use?

The example uses the `gpt-5-mini` model accessed through CometAPI's OpenAI-compatible interface.

When should I use this example?

Use this example when you want to access models through CometAPI's unified OpenAI-compatible interface rather than individual provider APIs, and you want to evaluate them using promptfoo without writing custom integration code.

Prompt Chain

Access Multiple Language Models Through Unified API

Name: Provider Cometapi
Availability: OnlineOnly
Author: Promptfoo

Configures CometAPI's OpenAI-compatible interface in promptfoo to access multiple language models.

Copy chain

Works with cometapiopenai

Promptfoo

Own this? Claim it

Spark score

out of 100

Updated 19 days ago

Version 0.121.18

Add to Favorites

Why it matters

Enable developers to evaluate and access various language models through a single OpenAI-compatible interface without managing multiple API integrations or authentication schemes.

Outcomes

What it gets done

Configure CometAPI provider in promptfoo evaluation setup

Route requests to different language models using unified interface

Authenticate with CometAPI using single API key

Run evaluations against chat models for geography questions

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-provider-cometapi | bash

Steps

Steps in the chain

Get your API key from CometAPI

Set your API key

Run the evaluation

Overview

Provider Cometapi

Configures CometAPI's unified, OpenAI-compatible interface as a promptfoo provider for accessing multiple language models. Use when configuring the CometAPI provider in promptfoo.

What it does

This promptfoo example shows how to use CometAPI to access various language models through a unified OpenAI-compatible interface. It demonstrates basic usage of the CometAPI provider with chat models, using the gpt-5-mini model to answer geography questions about different countries' capitals, and how to configure CometAPI within a promptfoo setup.

When to use - and when NOT to

Use this example when configuring the CometAPI provider in promptfoo to access language models through its OpenAI-compatible interface.

Inputs and outputs

Requires a CometAPI key obtained from the CometAPI console, set via:

export COMETAPI_KEY=your_api_key_here

Run via npx promptfoo@latest init --example provider-cometapi then npx promptfoo@latest eval. The example configuration evaluates gpt-5-mini's answers to geography questions about country capitals.

Integrations

Built on CometAPI's OpenAI-compatible interface for unified access to multiple language models.

Who it's for

Engineers configuring promptfoo to evaluate models through CometAPI's unified, OpenAI-compatible provider interface rather than each underlying model provider directly.

Source README

provider-cometapi (CometAPI Provider)

This example shows how to use CometAPI to access various language models through a unified OpenAI-compatible interface.

You can run this example with:

npx promptfoo@latest init --example provider-cometapi
cd provider-cometapi

Setup

Get your API key from CometAPI
Set your API key:
```
export COMETAPI_KEY=your_api_key_here
```
Run the evaluation:
```
npx promptfoo@latest eval
```

What this example does

This example demonstrates:

Basic usage of CometAPI provider with chat models
Geography questions evaluation
How to configure CometAPI in your promptfoo setup

The example uses the gpt-5-mini model to answer geography questions about different countries' capitals.

FAQ

Common questions

Discussion