What AI models does this Vertex AI prompt chain support?

This prompt chain supports Google Vertex AI's model lineup including Gemini, Claude, and Llama, each with different capabilities like function calling, technical writing, and safety configuration.

What are the prerequisites to use this prompt chain?

You need Node.js ^20.20.0 or >=22.22.0, the `google-auth-library` package, a `VERTEX_PROJECT_ID` environment variable, and authentication set up via `gcloud auth application-default login` or a `GOOGLE_APPLICATION_CREDENTIALS` service-account path.

What capabilities can I test with this prompt chain?

You can test function calling, system instructions, safety settings, search grounding for real-time information, image understanding, and structured output via response schemas across the different models.

How do I run this prompt chain?

Run `npx promptfoo@latest init --example google-vertex`, then use `promptfoo eval -c` with the relevant config file (e.g., `promptfooconfig.gemini.yaml` for Gemini, `promptfooconfig.search.yaml` for search grounding), and view results with `promptfoo view`.

Prompt Chain

Test Google Vertex AI Models

Name: Google Vertex
Availability: OnlineOnly
Author: Promptfoo

A promptfoo example suite for testing Gemini, Claude, and Llama models on Google Vertex AI.

Copy chain

Works with google vertex

Promptfoo

Own this? Claim it

Spark score

out of 100

Updated 27 days ago

Version 0.121.18

Models

gemini 2 0

Add to Favorites

Why it matters

Automate the testing and evaluation of Google Vertex AI models. Ensure your AI models perform as expected by running them through a series of predefined tests and analyses.

Outcomes

What it gets done

Configure promptfoo for Google Vertex AI integration.

Define and execute test cases for Vertex AI models.

Analyze model outputs for accuracy and performance.

Generate reports on model quality and identify areas for improvement.

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-google-vertex | bash

Steps

Steps in the chain

Install dependencies

Configure authentication

Set your project ID

Overview

Google Vertex

A promptfoo example suite for testing Vertex AI's Gemini, Claude, and Llama models across function calling, safety, search grounding, image understanding, and structured output. Use it to evaluate Vertex AI-hosted models. Requires a Google Cloud account with Vertex AI enabled and Node.js 20.20.0+/22.22.0+ - Node 20 support ends July 30, 2026, so Node 24 LTS is recommended.

What it does

This promptfoo example provides configurations for testing Google Vertex AI models, covering Gemini, Claude, and Llama, with dedicated configs for function calling, system instructions, safety settings, search grounding, image understanding, and structured output via response schemas.

When to use - and when NOT to

Use this example when you want to evaluate or compare models hosted on Vertex AI - Gemini, Claude, or Llama - across capabilities like tool use, real-time search grounding, or structured JSON output. It requires a Google Cloud account with the Vertex AI API enabled, API credentials, and Node.js ^20.20.0 or >=22.22.0 - note that Node.js 20 support ends July 30, 2026, so Node.js 24 LTS is the recommended runtime going forward.

Inputs and outputs

Each configuration file targets a different capability or model family:

### Model-specific examples
promptfoo eval -c promptfooconfig.gemini.yaml
promptfoo eval -c promptfooconfig.claude.yaml
promptfoo eval -c promptfooconfig.llama.yaml

### Search grounding and image understanding
promptfoo eval -c promptfooconfig.search.yaml
promptfoo eval -c promptfooconfig.image.yaml

### Structured output with response schemas
promptfoo eval -c promptfooconfig.response-schema.yaml

Each config demonstrates a different capability, from function calling and tool use to safety settings and real-time information retrieval via search grounding.

Integrations

Install google-auth-library, then authenticate either as a user for development (gcloud auth application-default login) or as a service account (GOOGLE_APPLICATION_CREDENTIALS pointing at a credentials JSON file), and set VERTEX_PROJECT_ID to your Google Cloud project. Run promptfoo view after any eval to inspect results.

Who it's for

Teams evaluating or comparing Gemini, Claude, and Llama model behavior on Vertex AI - tool use, grounding, safety, or structured output - who want working example configs for each capability rather than building them from scratch.

Source README

google-vertex (Google Vertex AI Examples)

Example configurations for testing Google Vertex AI models with promptfoo.

You can run this example with:

npx promptfoo@latest init --example google-vertex
cd google-vertex

Purpose

Test Vertex AI's Gemini, Claude, and Llama models
Configure model-specific features and search grounding
Compare performance across different tasks

Prerequisites

Google Cloud account with Vertex AI API enabled
API credentials
Node.js ^20.20.0 or >=22.22.0 (Node.js 20 support ends July 30, 2026; Node.js 24 LTS recommended)

Environment Variables

VERTEX_PROJECT_ID - Your Google Cloud project ID
GOOGLE_APPLICATION_CREDENTIALS - Path to service account credentials (optional)

Setup

Install dependencies:
```
npm install google-auth-library
```

Configure authentication:

# User account (development)
gcloud auth application-default login

# Or service account
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/credentials.json

Set your project ID:

export VERTEX_PROJECT_ID=your-project-id

Configurations

This example includes:

promptfooconfig.gemini.yaml: Gemini models with function calling, system instructions, and safety settings
promptfooconfig.claude.yaml: Claude models for technical writing and code analysis
promptfooconfig.llama.yaml: Llama models with safety features and region configuration
promptfooconfig.search.yaml: Search grounding for real-time information
promptfooconfig.response-schema.yaml: Response schemas with structured output

Running Examples

### Basic example
promptfoo eval -c promptfooconfig.yaml

### Model-specific examples
promptfoo eval -c promptfooconfig.gemini.yaml
promptfoo eval -c promptfooconfig.claude.yaml
promptfoo eval -c promptfooconfig.llama.yaml

### Search grounding tool and image understanding
promptfoo eval -c promptfooconfig.search.yaml
promptfoo eval -c promptfooconfig.image.yaml

### Structured output with response schemas
promptfoo eval -c promptfooconfig.response-schema.yaml

### View results
promptfoo view

Expected Results

Each configuration demonstrates different model capabilities, from function calling and tool use to safety features and real-time information retrieval.