What models does Nscale support through this promptfoo integration?

Nscale supports a broad catalog including text models like OpenAI's gpt-oss-120b/20b, Qwen 3 variants, Meta's Llama 3.3-70b and Llama 4-scout, DeepSeek R1 distilled models, Mistral's devstral-small and mixtral-8x22b, an embedding model (Qwen3-Embedding-8B), and text-to-image models (FLUX.1-schnell, Stable Diffusion XL, and SDXL-Lightning).

How do I set up and run the Nscale provider example?

Sign up at Nscale, generate a service token from account settings, set it as the `NSCALE_SERVICE_TOKEN` environment variable or in a `.env` file, then scaffold with `npx promptfoo@latest init --example provider-nscale`. Run text evaluation with `promptfoo eval` and image evaluation with `promptfoo eval -c image-promptfooconfig.yaml`.

What are the pricing and performance benefits of Nscale?

Nscale offers highly competitive pricing starting from $0.01 input / $0.03 output per 1M tokens for text generation and $0.0008 per megapixel for image generation, with zero rate limits, no cold starts, and claimed savings of up to 80% versus other providers.

When should I use this Nscale example?

Use this example when you want to evaluate Nscale-hosted text or text-to-image models for quality and reasoning before adopting Nscale as your inference provider, particularly if you have bursty or unpredictable traffic where cost and latency consistency matter.

Prompt Chain

Evaluate Nscale Inference API Models

Name: Provider Nscale
Availability: OnlineOnly
Author: Promptfoo

Promptfoo example evaluating Nscale's cost-effective serverless inference across text and image generation models.

Copy chain

Works with nscale

Promptfoo

Own this? Claim it

Spark score

out of 100

Updated 4 months ago

Version 1.0.0

Models

qwen 2 5llama 3

Add to Favorites

Why it matters

Leverage promptfoo to benchmark and evaluate the performance of Nscale Serverless Inference API models. Ensure cost-effective and high-performance AI inference.

Outcomes

What it gets done

Integrate with Nscale Serverless Inference API

Evaluate model performance and cost-effectiveness

Automate AI inference testing

Identify optimal AI models for specific tasks

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-provider-nscale | bash

Steps

Steps in the chain

Navigate to account settings

Go to Service Tokens section

Generate and set service token

Overview

Provider Nscale

A promptfoo example evaluating Nscale's serverless inference across a broad open-weight text-generation catalog and text-to-image models like Flux.1 Schnell and SDXL Lightning. Use when evaluating Nscale as a cost-effective inference backend for text or image generation before committing to production use.

What it does

This example demonstrates how to use the Nscale provider with promptfoo to evaluate Nscale Serverless Inference API models, which offer cost-effective, high-performance AI inference with zero rate limits. It ships two configurations: a basic model evaluation comparing two popular Nscale models on reasoning problems and creative content generation (with metrics on accuracy, creativity, and response quality), and an image generation evaluation comparing Nscale's text-to-image models (Flux.1 Schnell, SDXL Lightning, Stable Diffusion XL) across landscapes, futuristic scenes, portraits, and abstract art prompts.

When to use - and when NOT to

Use this example when you want to evaluate Nscale-hosted open-weight text models or text-to-image models for quality, reasoning, or creative output before adopting Nscale as an inference provider. It is not a general Nscale account guide - it only documents the promptfoo-side eval configs for text and image generation.

Inputs and outputs

Sign up at Nscale, generate a service token from the "Service Tokens" section of account settings, and set it as NSCALE_SERVICE_TOKEN (via environment variable or .env file). Scaffold with:

npx promptfoo@latest init --example provider-nscale
cd provider-nscale

Run the basic text evaluation with promptfoo eval (default promptfooconfig.yaml) and the image evaluation with promptfoo eval -c image-promptfooconfig.yaml.

Integrations

Integrates the Nscale Serverless Inference API as a promptfoo provider, supporting a broad text-generation catalog including OpenAI's open-weight gpt-oss-120b/gpt-oss-20b, multiple Qwen 3 variants (qwen-3-235b-a22b-instruct, qwen-3-32b, qwen-3-8b, coder-tuned qwen-2.5-coder models), Meta's llama-3.3-70b-instruct and llama-4-scout-17b-16e-instruct (image-text-to-text), DeepSeek R1 distilled Llama/Qwen models from 1.5B to 70B, and Mistral's devstral-small-2505 and mixtral-8x22b-instruct-v0.1; an embedding model, Qwen3-Embedding-8B; and text-to-image models FLUX.1-schnell, stable-diffusion-xl-base-1.0, and SDXL-Lightning (4-step and 8-step). Pricing is highly competitive - text generation from $0.01 input / $0.03 output per 1M tokens, image generation from $0.0008 per megapixel, with zero rate limits and no cold starts, said to save up to 80% versus other providers.

Who it's for

Teams evaluating Nscale as a cost-effective inference backend for open-weight text models or text-to-image generation who need to validate quality and consistency across its model catalog before committing to it for production workloads. Because the catalog spans everything from small, fast coder-tuned models to 70B-parameter reasoning and distillation models, teams can use the same two example configs to size a model to their actual workload - a lightweight Qwen 2.5 Coder variant for quick code completions versus a larger DeepSeek R1 distillation for harder reasoning tasks - rather than assuming one model size fits every use case. The zero-rate-limit, no-cold-start pricing model also makes Nscale worth evaluating specifically for workloads with bursty or unpredictable traffic, where per-request cost and latency consistency matter as much as raw output quality.

Source README

provider-nscale (Nscale Example (Cost-Effective AI Inference))

You can run this example with:

npx promptfoo@latest init --example provider-nscale
cd provider-nscale

Prerequisites

Authentication Setup

Sign up for an account at Nscale
Navigate to your account settings
Go to "Service Tokens" section
Generate a service token and set it as an environment variable:

export NSCALE_SERVICE_TOKEN="your-service-token-here"

Alternatively, you can add it to your .env file:

NSCALE_SERVICE_TOKEN=your-service-token-here

Example Configuration

This repository contains an example configuration demonstrating Nscale's capabilities:

Basic Model Evaluation (`promptfooconfig.yaml`)

This configuration evaluates two popular Nscale models on their ability to solve reasoning problems and generate creative content.

promptfoo eval

Expected output: You'll see a comparison of how each model handles different types of tasks, with metrics on accuracy, creativity, and response quality.

Image Generation Evaluation (`image-promptfooconfig.yaml`)

This configuration compares Nscale's image generation models on various prompts to evaluate their quality and consistency.

promptfoo eval -c image-promptfooconfig.yaml

Expected output: You'll see generated images from different models (Flux.1 Schnell, SDXL Lightning, Stable Diffusion XL) for comparison across various image types including landscapes, futuristic scenes, portraits, and abstract art.

Model Capabilities

Nscale supports many popular models with competitive pricing:

Text Generation Models:

openai/gpt-oss-120b - OpenAI's 120B open-weight model
openai/gpt-oss-20b - OpenAI's 20B model
qwen/qwen-3-235b-a22b-instruct - Qwen 3 235B model
qwen/qwen-3-235b-a22b-instruct-2507 - Qwen 3 235B Instruct 2507
qwen/qwen-3-4b-thinking-2507 - Qwen 3 4B Thinking 2507
qwen/qwen-3-8b - Qwen 3 8B model
qwen/qwen-3-14b - Qwen 3 14B model
qwen/qwen-3-32b - Qwen 3 32B model
qwen/qwen-2.5-coder-3b-instruct - Qwen 2.5 Coder 3B Instruct
qwen/qwen-2.5-coder-7b-instruct - Qwen 2.5 Coder 7B Instruct
qwen/qwen-2.5-coder-32b-instruct - Qwen 2.5 Coder 32B Instruct
qwen/qwq-32b - Qwen QwQ 32B model
meta/llama-3.3-70b-instruct - Meta's Llama 3.3 70B model
meta/llama-3.1-8b-instruct - Meta's Llama 3.1 8B model
meta/llama-4-scout-17b-16e-instruct - Llama 4 Scout 17B model (Image-Text-to-Text)
deepseek/deepseek-r1-distill-llama-70b - DeepSeek R1 Distill Llama 70B
deepseek/deepseek-r1-distill-llama-8b - DeepSeek R1 Distill Llama 8B
deepseek/deepseek-r1-distill-qwen-1.5b - DeepSeek R1 Distill Qwen 1.5B
deepseek/deepseek-r1-distill-qwen-7b - DeepSeek R1 Distill Qwen 7B
deepseek/deepseek-r1-distill-qwen-14b - DeepSeek R1 Distill Qwen 14B
deepseek/deepseek-r1-distill-qwen-32b - DeepSeek R1 Distill Qwen 32B
mistral/devstral-small-2505 - Mistral's Devstral Small model
mistral/mixtral-8x22b-instruct-v0.1 - Mixtral 8x22B Instruct

Embedding Models:

Qwen/Qwen3-Embedding-8B - Qwen 3 8B Embedding model

Text-to-Image Models:

BlackForestLabs/FLUX.1-schnell - Flux.1 Schnell image generation model
stabilityai/stable-diffusion-xl-base-1.0 - Stable Diffusion XL 1.0
ByteDance/SDXL-Lightning-4step - SDXL Lightning 4-step
ByteDance/SDXL-Lightning-8step - SDXL Lightning 8-step

Pricing & Usage

Nscale offers highly competitive pricing with up to 80% cost savings compared to other providers:

Text Generation: Starting from $0.01 input / $0.03 output per 1M tokens
Image Generation: Starting from $0.0008 per mega-pixel
Zero rate limits and no cold starts

Check the official pricing page for the most current rates.

Learn More

FAQ

Common questions

Discussion