OpenAI Completions Usage API Extended Example

OpenAI Completions Usage API Extended Example

For most of our users, the default usage and cost dashboards are sufficient. However, if you need more detailed data or a custom dashboard, you can use the Completions Usage API.

This notebook demonstrates how to retrieve and visualize usage data from the OpenAI Completions Usage API and Costs API. We'll:

• Call the API to get completions usage data.
• Parse the JSON response into a pandas DataFrame.
• Visualize token usage over time using matplotlib.
• Use grouping by model to analyze token usage across different models.
• Display model distribution with a pie chart.

We also include placeholders for all possible API parameters for a comprehensive overview.

Setup API Credentials and Parameters

Set up an Admin Key - https://platform.openai.com/settings/organization/admin-keys

Replace 'PLACEHOLDER' with your actual ADMIN API key. It's best practice to load the key from an environment variable for security.

Inspect the JSON Response

Let's take a look at the raw JSON response from the API to understand its structure.

Parse the API Response and Create a DataFrame

Now we will parse the JSON data, extract relevant fields, and create a pandas DataFrame for easier manipulation and analysis.

Visualize Token Usage Over Time

We'll create a bar chart to visualize input and output token usage for each time bucket.

Visual Example: Grouping by Model

In this section, we retrieve and visualize usage data grouped by model and project_id. This can help you see the total tokens used by each model over the specified period.

Note on Grouping Parameter

• If you do not specify a group_by parameter, fields such as project_id, model, and others will return as null.
  Although the group_by parameter is optional, it is recommended to include it in most cases to retrieve meaningful data.

• You can specify multiple group fields by separating them with commas. For example: group_by=["model", "project_id"].

Parse the API Response into DataFrame and render a stacked bar chart

Now we will parse the JSON data, extract relevant fields, and create a pandas DataFrame for easier manipulation and analysis.

Visual Example: Model Distribution Pie Chart

This section visualizes the distribution of token usage across different models using a pie chart.

Costs API Example

In this section, we'll work with the OpenAI Costs API to retrieve and visualize cost data. Similar to the completions data, we'll:

• Call the Costs API to get aggregated cost data.
• Parse the JSON response into a pandas DataFrame.
• Visualize costs grouped by line item using a bar chart.

Parse the Costs API Response and Create a DataFrame

We will now parse the JSON data from the Costs API, extract relevant fields, and create a pandas DataFrame for further analysis.

Visualize Total Costs per Day

We'll create a bar chart to visualize the total costs aggregated by day. This helps give a high level perspective on organizational spend.

Visualize Costs by Line Item

We'll create a bar chart to visualize the total costs aggregated by line item. This helps identify which categories (e.g., models or other services) contribute most to the expenses.

Additional Visualizations (Optional)

You can extend this notebook with more visualizations for both the Completions and Costs APIs. For example:

Completions API:

• Group by user, project, or model to see which ones consume the most tokens.
• Create line plots for time series analysis of token usage over days or hours.
• Use pie charts to visualize distribution of tokens across models, users, or projects.
• Experiment with different group_by parameters (e.g., ["model", "user_id"]) to gain deeper insights.

Costs API:

• Group by project or line item to identify spending patterns.
• Create line or bar charts to visualize daily cost trends.
• Use pie charts to show how costs are distributed across projects, services, or line items.
• Try various group_by options (e.g., ["project_id"], ["line_item"]) for granular analysis.

Experiment with different parameters and visualization techniques using pandas and matplotlib (or libraries like Plotly/Bokeh) to gain deeper insights, and consider integrating these visualizations into interactive dashboards for real-time monitoring.

Integrating with Third-Party Dashboarding Platforms

To bring OpenAI usage and cost data into external dashboarding tools like Tableau, Power BI, or custom platforms (e.g., Plotly Dash, Bokeh), follow these steps:

1. Data Collection & Preparation:

   • Use Python scripts to regularly fetch data from the Completions and Costs APIs.
   • Process and aggregate the data with pandas, then store it in a database, data warehouse, or export it as CSV/JSON files.

2. Connecting to a Dashboard:

   • BI Tools (Tableau, Power BI):
     • Connect directly to the prepared data source (SQL database, CSV files, or web APIs).
     • Use built-in connectors to schedule data refreshes, ensuring dashboards always display current information.
   • Custom Dashboards (Plotly Dash, Bokeh):
     • Embed API calls and data processing into the dashboard code.
     • Build interactive visual components that automatically update as new data is fetched.

3. Real-Time & Automated Updates:

   • Schedule scripts using cron jobs, task schedulers, or workflow tools (e.g., Apache Airflow) to refresh data periodically.
   • Implement webhooks or streaming APIs (if available) for near real-time data updates.

By integrating API data into third-party platforms, you can create interactive, real-time dashboards that combine OpenAI metrics with other business data, offering comprehensive insights and automated monitoring.