Automate Google AI Studio Tasks

google-aistudio-tools (Google AI Studio Tools)

This example demonstrates how to use Google AI Studio's function calling, search capabilities, code execution, and URL context features with promptfoo.

You can run this example with:

npx promptfoo@latest init --example google-aistudio-tools
cd google-aistudio-tools

Prerequisites

• Google AI Studio API key set as GOOGLE_API_KEY in your environment

Overview

This example shows how to:

1. Function Calling: Use Gemini to invoke predefined functions based on user queries
2. Google Search Integration: Get up-to-date information from the web using Gemini models with search grounding
3. Code Execution: Execute Python code to solve computational problems
4. URL Context: Extract and analyze content from web URLs

Function Calling Example

The function calling configuration (promptfooconfig.yaml) demonstrates:

• Defining a weather function in tools.json
• Validating that Gemini models correctly produce structured function calls
• Testing that the location parameter matches the user's query

Run with:

promptfoo eval -c promptfooconfig.yaml

Search Grounding Example

The search grounding configuration (promptfooconfig.search.yaml) demonstrates:

• Using Gemini 2.5 Flash with Google Search as a tool
• Using Gemini 2.5 Pro with thinking capabilities and Search grounding
• Using Gemini 2.5 Flash-Lite with Google Search grounding
• Testing queries that benefit from real-time web information
• Verifying responses include relevant information

Run with:

promptfoo eval -c promptfooconfig.search.yaml

Code Execution Example

The code execution configuration (promptfooconfig.codeexecution.yaml) demonstrates:

• Testing computational problems that require code to solve
• Verifying that the answer is correct from the code execution

Run with:

promptfoo eval -c promptfooconfig.codeexecution.yaml

URL Context Example

The URL context configuration (promptfooconfig.urlcontext.yaml) demonstrates:

• Using Gemini to extract and analyze content from web URLs
• Combining URL context with search capabilities

Run with:

promptfoo eval -c promptfooconfig.urlcontext.yaml

Example Files

• promptfooconfig.yaml: Function calling configuration
• promptfooconfig.search.yaml: Search grounding configuration
• promptfooconfig.codeexecution.yaml: Code execution configuration
• promptfooconfig.urlcontext.yaml: URL context configuration
• tools.json: Function definition for the weather example

Notes on Google Search Integration

When using Search grounding in your own applications:

• The API response includes search metadata and sources
• Google requires displaying "Google Search Suggestions" in user-facing apps
• Models can retrieve current information about events, prices, and technical updates

Search Methods

This example demonstrates three approaches to search:

1. Search as a tool (Gemini 2.5): Allows the model to decide when to use search

   tools:
     - googleSearch: {}
   

2. Search with thinking (Gemini 2.5): Adds thinking capabilities for better reasoning

   generationConfig:
     thinkingConfig:
       thinkingBudget: 1024
   tools:
     - googleSearch: {}
   

3. Search on Flash-Lite (Gemini 2.5): Uses the lower-cost Flash-Lite model with the same search tool

   tools:
     - googleSearch: {}
   

Further Resources

• Google AI Studio Function Calling documentation
• Google AI Studio Search Grounding documentation
• promptfoo Google Provider documentation