Integrate LLMs with LiteLLM
LiteLLM provider example demonstrating a unified interface to 400+ LLMs through a proxy server.
Why it matters
Connect to various LLM providers using the LiteLLM library. This asset helps you abstract away the complexities of different LLM APIs.
Outcomes
What it gets done
Integrate with multiple LLM providers (OpenAI, Cohere, Anthropic, etc.)
Abstract LLM API calls into a single interface
Facilitate easy switching between different LLM models
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/pfoo-litellm | bash Capabilities
What this chain does
Integrate with multiple LLM providers (OpenAI, Cohere, Anthropic, etc.)
Abstract LLM API calls into a single interface
Facilitate easy switching between different LLM models
Overview
Litellm
What it does
A configuration example showing how to use the LiteLLM provider to access multiple language models through a unified proxy interface.
How it connects
Use when you want to test the same prompts across different LLM providers using consistent syntax rather than managing separate integrations for each vendor.
Source README
yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
description: LiteLLM provider example - unified interface for multiple LLMs
This example demonstrates the LiteLLM provider, which provides a unified
interface to 400+ LLMs through a proxy server.
REQUIREMENTS:
1. LiteLLM proxy server must be running (use ./start-proxy.sh)
2. Set API keys: OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_AI_API_KEY
prompts:
- 'Translate this to {{language}}: {{text}}'
- 'Write a haiku about {{topic}}'
providers:
All providers use the litellm: prefix and connect through the proxy
- litellm:gpt-4.1
- litellm:claude-sonnet-4-6
- litellm:gemini-2.5-pro
Configure embedding provider for similarity assertions
This also goes through the LiteLLM proxy
defaultTest:
options:
provider:
embedding: litellm:embedding:text-embedding-3-large
tests:
vars:
language: French
text: 'Hello, world!'
assert:- type: contains
value: 'Bonjour' - type: similar
value: 'Bonjour, le monde!'
threshold: 0.8
- type: contains
vars:
topic: 'artificial intelligence'
assert:- type: javascript
value: |
// Check if output is a haiku (3 lines)
const lines = output.trim().split('\n').filter(line => line.trim());
return lines.length === 3;
- type: javascript
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.