How do I view the comparison results?

After running `promptfoo eval`, use `promptfoo view` to see the side-by-side results comparing Mistral and Llama model responses.

Can I use this to compare models on my own test cases?

Yes, this is designed as a starting point for your own comparisons. You edit the text files in `prompts/` and the `promptfooconfig.yaml` to customize your test cases and prompts before running the evaluation.

What models does this comparison tool support?

This example compares Mistral and Llama models accessed through OpenRouter, allowing you to evaluate both models side by side on the same test cases.

Prompt Chain

Compare LLM Performance

Name: Compare Mistral Vs Llama
Availability: OnlineOnly
Author: Promptfoo

Promptfoo starting point for comparing Mistral and Llama models via OpenRouter side by side on your own test cases.

Copy chain

Works with github

Promptfoo

Own this? Claim it

Maintainer of this project? Claim this page to edit the listing.

Spark score

out of 100

Updated last month

Version code-scan-action-0.1

Models

mistral large llama 3

Add to Favorites

Why it matters

Evaluate and compare the performance of different large language models (LLMs) like Mistral and Llama. This asset helps you understand which model is better suited for specific tasks by running comparative tests.

Outcomes

What it gets done

Run comparative tests between Mistral and Llama models.

Analyze and summarize LLM performance metrics.

Identify strengths and weaknesses of different LLMs.

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-compare-mistral-vs-llama | bash

Steps

Steps in the chain

Set up environment variable

Modify prompt files

Edit configuration

Run evaluation

View results

Overview

Compare Mistral Vs Llama

This is a minimal promptfoo starting point for comparing Mistral and Llama model responses via OpenRouter side by side on custom test cases. Use it as a bare-bones starting point for comparing Mistral and Llama models through a single OpenRouter key; not a ready-made comparison task.

What it does

This example runs a promptfoo evaluation comparing Mistral and Llama models, accessed via OpenRouter, side by side on the same test cases.

When to use - and when NOT to

Use it as a starting point for comparing Mistral and Llama model responses on your own prompts and test cases through a single OpenRouter key. Do not use it expecting a pre-built comparison task - the source provides only the generic setup and run steps, with prompt text files in prompts/ meant to be edited for your own comparison.

Inputs and outputs

Requires OPENROUTER_API_KEY. Inputs are the text files in prompts/ and the config in promptfooconfig.yaml; running promptfoo eval produces side-by-side results, viewable with promptfoo view.

Integrations

Set up via npx promptfoo@latest init --example compare-mistral-vs-llama, then edit the files in prompts/ and promptfooconfig.yaml before running the evaluation.

Who it's for

Developers who want a bare-bones promptfoo starting point for comparing Mistral and Llama model responses via OpenRouter on their own test cases.

Source README

compare-mistral-vs-llama (Mistral Llama Comparison)

You can run this example with:

npx promptfoo@latest init --example compare-mistral-vs-llama
cd compare-mistral-vs-llama

Usage

To get started, set your OPENROUTER_API_KEY environment variable.

Next, change a few of the text files in prompts/ and edit promptfooconfig.yaml.

Then run:

promptfoo eval

Afterwards, you can view the results by running promptfoo view

FAQ

Common questions

Discussion