Route LLM calls through Neon's branch-scoped AI Gateway

Neon AI Gateway provides a model-inference layer associated with a Neon branch and powered by Databricks. A gateway endpoint and Neon credential can address catalog models across Anthropic, OpenAI, Google, Meta, Alibaba, DeepSeek, and Databricks. The skill guides inference requests, SDK integration, model selection, and branch-aware configuration.

Each branch has its own gateway host, following database branch lineage. This supports preview, feature, CI, and application environments with corresponding gateway configuration. Credentials can authorize descendants, however, so a branch endpoint is not an application-user authorization boundary. Application permissions still require their own enforcement.

Use the skill when an application needs consolidated model credentials, model portability, branch-scoped inference, or integration with Neon Functions. An existing single-provider application may be better served by its direct SDK when branch behavior and multi-model routing are unnecessary.

Before provisioning or sending data, confirm project and region eligibility, the model catalog, paid-plan and prepaid-credit requirements, pricing, and retention. An enabled configuration does not prove a completed inference request, deployment, or billing outcome. The sample applications also require adapters for their own tools, memory, and authentication.

Inputs include the linked Neon project and branch, neon.ts configuration, gateway credentials, selected model, SDK dialect, and required features. Outputs can be a configured branch gateway, an explicit SDK client, a working inference request, or a bounded tool workflow.

Enable preview.aiGateway in neon.ts and inspect the planned infrastructure change before applying it. The config status, plan, and apply commands distinguish live state, a dry-run difference, and provisioning; neon deploy is an apply alias. Creating a branch through checkout applies its policy when neon.ts is present. Checking out an existing branch does not reconcile it, so configuration changes need an explicit apply or deploy.

Provisioning, linking, and checkout also pull branch credentials into .env.local. Keep credential files private and ignored by version control. NEON_AI_GATEWAY_TOKEN is a server-side bearer with ai_gateway:invoke scope; NEON_AI_GATEWAY_BASE_URL is the bare HTTPS branch host. Do not assume OPENAI_* aliases exist or accidentally fall back to a direct provider key. Explicit revocation is needed where expiry is not enforced.

Match the endpoint to the API dialect: /v1 for all-provider chat completions, /openai/v1 for Responses, /anthropic for native Messages, and /gemini for google-genai. Native SDKs append their own API segments. Older longer routes remain aliases; model discovery uses GET /v1/models. Model switching requires checking the selected route and feature support, including streaming event handling.

The Vercel AI SDK supplies generation, streaming, tools, and structured-output primitives. A dedicated provider is imported with:

import { neon } from "@neon/ai-sdk-provider";

That provider routes models to the appropriate Messages, Responses, or MLflow endpoint. Use catalog model IDs directly rather than inventing provider prefixes. Compare external catalog mappings with the gateway model list and official product documentation before relying on identifiers or capabilities.

Tool loops use a stopWhen step budget alongside request deadlines, cancellation, concurrency, and cost limits. Tool adapters bind identity on the server and return allowlisted fields; AI SDK v5 and later use inputSchema. Mastra offers memory, tools, workflows, and tracing, with Postgres-backed threads through @mastra/pg and the chat-completions dialect. The parseEnv helper from @neon/env derives typed aiGateway settings from the same configuration.

The skill serves developers connecting inference or tool workflows to Neon branches. Plain OpenAI, Anthropic, and google-genai SDKs remain options when framework-level orchestration is unnecessary. The upstream skill uses Apache-2.0 licensing.