Prompt Chain

Compare LLM Performance

Promptfoo starting point for comparing local Ollama-hosted Phi3 and Llama3 models side by side on your own test cases.


81
Spark score
out of 100
Updated 29 days ago
Version code-scan-action-0.1
Models

Add to Favorites

Why it matters

Evaluate and compare the performance of different large language models (LLMs) like Phi and Llama. This asset helps you understand which model is better suited for specific tasks by running comparative tests.

Outcomes

What it gets done

01

Set up comparative tests for LLMs.

02

Analyze and summarize LLM outputs.

03

Identify strengths and weaknesses of different models.

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-compare-phi-vs-llama | bash

Steps

Steps in the chain

01
Pull Llama3 model
02
Pull Phi3 model
03
Edit configuration
04
Run evaluation
05
View results

Overview

Compare Phi Vs Llama

This is a minimal promptfoo starting point for comparing local, Ollama-hosted Phi3 and Llama3 model responses side by side, with no API keys required. Use it as a bare-bones, local starting point for comparing Phi3 and Llama3; not a ready-made comparison task.

What it does

This example runs a promptfoo evaluation comparing Microsoft's Phi3 and Meta's Llama3 models, both run locally via Ollama, side by side on the same test cases.

When to use - and when NOT to

Use it as a starting point for comparing Phi3 and Llama3 responses locally, with no API keys or cloud costs required. Do not use it expecting a pre-built comparison task - the source provides only the generic setup and run steps, and you edit the config to define your own comparison.

Inputs and outputs

Requires Ollama installed with both models pulled locally (ollama pull llama3, ollama pull phi3). Inputs are whatever you define in promptfooconfig.yaml; running promptfoo eval produces side-by-side results, viewable with promptfoo view.

Integrations

Set up via npx promptfoo@latest init --example compare-phi-vs-llama, pull both models locally, then edit promptfooconfig.yaml before running the evaluation.

Who it's for

Developers who want a bare-bones, local, no-API-key promptfoo starting point for comparing Phi3 and Llama3 model responses on their own test cases.

Source README

compare-phi-vs-llama (Phi Vs Llama)

You can run this example with:

npx promptfoo@latest init --example compare-phi-vs-llama
cd compare-phi-vs-llama

Usage

To get started:

ollama pull llama3
ollama pull phi3

Next, edit promptfooconfig.yaml.

Then run:

promptfoo eval

Afterwards, you can view the results by running promptfoo view

FAQ

Common questions

Discussion

Questions & comments · 0

Sign In Sign in to leave a comment.