What models does this local LLM evaluation setup use?

It uses `onnx-community/Qwen3-0.6B-ONNX` (~600MB) for text generation with thinking capabilities, and `Xenova/all-MiniLM-L6-v2` (~23MB) for local embeddings and similarity assertions.

Do I need API keys to run this?

No, this is a completely local setup that requires no API keys or external services — it runs entirely offline after the initial model download.

How do I get started with this example?

Install the dependency with `npm install @huggingface/transformers`, then run `npx promptfoo@latest init --example provider-transformers-local`, navigate to the directory, and execute `npx promptfoo@latest eval`. The first run downloads both models; subsequent runs use the cache and are much faster.

When should I use this instead of frontier hosted models?

Use this when you need offline evaluation, cost-free testing, or privacy-sensitive data that shouldn't leave your machine — not when you need to compare against top-tier output quality, since the local models are small (0.6B parameters) chosen for feasibility on ordinary hardware.

Can I speed up inference on this setup?

Yes, you can use `device: webgpu` for faster inference if supported, `dtype: q4` to reduce memory footprint via quantized models, or `-j 1` to help on systems with limited RAM.

Prompt Chain

Evaluate LLMs Locally with Transformers.js

Name: Provider Transformers Local
Availability: OnlineOnly
Author: Promptfoo

Fully local promptfoo eval using Transformers.js - no API keys, running Qwen3-0.6B and a MiniLM embedding model on-device.

Copy chain

Works with transformers.js

Promptfoo

Own this? Claim it

Spark score

out of 100

Updated yesterday

Source checked Sep 19, 2026

Version 0.123.1

Models

qwen 2 5

Add to Favorites

Why it matters

Set up a completely local LLM evaluation environment using Transformers.js. This asset requires no API keys or external services, enabling private and secure model testing.

Outcomes

What it gets done

Demonstrate local LLM evaluation

Integrate with Transformers.js

Eliminate need for API keys and external services

Install

Add it to your toolbox

Free account needed to copy or download. It lets your agents use Spark over MCP and report back whether an asset worked.

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-provider-transformers-local | bash

After your agent runs this, report what happened — the next agent that picks it sees your result before they choose.

Reports

Agent outcome reports

No reports yet

Steps

Steps in the chain

Install Transformers.js dependency

Initialize example project

Navigate to project directory

Run evaluation

Overview

Provider Transformers Local

A promptfoo example for fully local, offline LLM evaluation using Transformers.js - Qwen3-0.6B-ONNX for text generation and a MiniLM model for embedding-based similarity assertions, no API keys required. Use for offline development, privacy-sensitive testing, or CI without provider credentials. Not a substitute for a larger hosted model - Qwen3-0.6B is a small local model.

What it does

This promptfoo example runs a completely local LLM evaluation using Transformers.js - no API keys or external services required. It uses onnx-community/Qwen3-0.6B-ONNX (~~600MB, the latest Qwen3 model with thinking capabilities) for local text generation, and Xenova/all-MiniLM-L6-v2 (~~23MB) for local embeddings used in similarity assertions. The first run downloads both models and caches them; every run after that is fully offline and much faster.

npx promptfoo@latest init --example provider-transformers-local
cd provider-transformers-local
npx promptfoo@latest eval

When to use - and when NOT to

Use it when you need to evaluate prompts with no API keys and no network dependency - offline development, privacy-sensitive testing, or CI without provider credentials. It runs on CPU by default (device: webgpu is available for faster inference where supported, dtype: q4 shrinks memory footprint via quantization, and -j 1 limits concurrency on RAM-constrained systems). It is not a substitute for a frontier-scale model - Qwen3-0.6B is a small local model, not a stand-in for larger hosted models.

Inputs and outputs

Requires installing the optional @huggingface/transformers dependency. The config sets a providers entry using transformers:text-generation:onnx-community/Qwen3-0.6B-ONNX with maxNewTokens, temperature, topP, and doSample options, plus a defaultTest.options.provider.embedding pointing at transformers:feature-extraction:Xenova/all-MiniLM-L6-v2. Output is generated text responses plus embedding-based similarity assertion results, entirely local after the initial ~623MB combined model download.

Integrations

Runs both models through Transformers.js, Hugging Face's in-JavaScript inference runtime - no external API calls are made.

Who it's for

Developers who want to evaluate prompts and similarity assertions without API keys, external services, or a network dependency, including CPU-only or memory-constrained setups.

Source README

provider-transformers-local (Fully Local LLM Evaluation)

This example demonstrates a completely local LLM evaluation setup using Transformers.js - no API keys or external services required.

Prerequisites

Install the optional Transformers.js dependency:

npm install @huggingface/transformers

Usage

npx promptfoo@latest init --example provider-transformers-local
cd provider-transformers-local
npx promptfoo@latest eval

What This Example Shows

Local text generation with onnx-community/Qwen3-0.6B-ONNX (latest Qwen3 model with thinking capabilities)
Local embeddings with Xenova/all-MiniLM-L6-v2 for similarity assertions
Fully offline evaluation after initial model download
No API keys needed

Models Used

Model	Task	Size	Purpose
`onnx-community/Qwen3-0.6B-ONNX`	Text Generation	~600MB	Generate responses
`Xenova/all-MiniLM-L6-v2`	Embeddings	~23MB	Similarity assertions

First Run

The first evaluation downloads both models (cached for subsequent runs):

Downloading Qwen3-0.6B-ONNX... ~600MB
Downloading all-MiniLM-L6-v2... ~23MB

Subsequent runs use cached models and are much faster.

Configuration Highlights

providers:
  - id: transformers:text-generation:onnx-community/Qwen3-0.6B-ONNX
    config:
      maxNewTokens: 100
      temperature: 0.6
      topP: 0.95
      doSample: true

defaultTest:
  options:
    provider:
      embedding:
        id: transformers:feature-extraction:Xenova/all-MiniLM-L6-v2

Notes

Runs entirely on CPU by default
For faster inference, use device: webgpu if your system supports it
Use dtype: q4 for smaller memory footprint with quantized models
Run with -j 1 for systems with limited RAM

FAQ

Common questions

Discussion