Summarize Web Content with Replicate

This promptfoo example compares openai:chat:gpt-4.1-mini against meta/llama-2-70b-chat accessed through Replicate's "Lifeboat" OpenAI-compatible proxy (apiBaseUrl: https://openai-proxy.replicate.com), letting a Replicate-hosted model be called with the standard openai:chat: provider syntax instead of Replicate's native provider format. Both run at temperature: 0.01 with a 512-token cap on the same conversational prompts.

Use this example when you want to query a Replicate-hosted model through an OpenAI-shaped client/API rather than promptfoo's native replicate: provider - useful if your tooling only supports the OpenAI chat completion interface. It is scoped to conversational, factual-honesty-style prompts (weather knowledge, hippo swimming, Henry VIII's grandchildren, a coding task); several prompts have no automated assertion and are left for manual review.

The Llama 2 provider config points the OpenAI-compatible client at Replicate's proxy endpoint instead of OpenAI's:

  - id: openai:chat:meta/llama-2-70b-chat
    config:
      apiKey: '...'
      apiBaseUrl: https://openai-proxy.replicate.com
      temperature: 0.01
      max_tokens: 512

Tests assert factual-honesty stances via llm-rubric on select prompts (e.g. "says that hippos are NOT good swimmers"), matching the same conversational test set used in other Claude/GPT/Llama comparison examples in this collection.

Runs openai:chat:gpt-4.1-mini against OpenAI directly and meta/llama-2-70b-chat through Replicate's OpenAI-compatible "Lifeboat" proxy endpoint, using promptfoo's llm-rubric assertion type.

Developers who want to access a Replicate-hosted model via an OpenAI-compatible API surface (rather than promptfoo's native Replicate provider) and compare its conversational responses against GPT-4.1-mini.