Visualize AI Embeddings with Wandb
A notebook that logs embeddings to Weights & Biases and renders them as a 2D projection with PCA, UMAP, or t-SNE.
Why it matters
Visualize high-dimensional AI embeddings in a 2D space using Weights & Biases. Understand your data's structure and relationships through interactive projection techniques.
Outcomes
What it gets done
Log embeddings and original data to Weights & Biases tables.
Utilize the Wandb Embedding Projector for visualization.
Apply dimension reduction algorithms (PCA, UMAP, t-SNE).
Interpret embedding clusters and patterns.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/oai-visualizingembeddingsinwandb | bash Steps
Steps in the chain
Overview
Visualizing Embeddings In Wandb
A notebook that logs embeddings as a Weights & Biases Table and renders them as an interactive 2D projection using PCA, UMAP, or t-SNE. Use it when you're already tracking experiments in Weights & Biases and want to visualize embedding clusters there - it depends on a companion notebook to generate the embeddings first.
What it does
This notebook uploads embedding data to Weights & Biases and uses its Embedding Projector to visualize the embeddings with common dimension-reduction algorithms - PCA, UMAP, and t-SNE. Weights & Biases is a machine learning platform used by OpenAI and other ML teams to track experiments, evaluate model performance, reproduce models, visualize results, and share findings with colleagues. The underlying dataset comes from a companion embeddings-generation notebook.
The workflow logs the data as a W&B Table, with each review as a row and its 1536 embedding floats spread across individually named columns (emb_0, emb_1, and so on). From the resulting W&B run, opening the Table's settings and changing its render mode to a combined 2D projection turns those embedding columns into an interactive, dimension-reduced scatter view. A hosted example of the resulting projection is linked for reference.
When to use - and when NOT to
Use it when you're already tracking experiments in Weights & Biases and want to visualize and share embedding clusters there, with a choice of PCA, UMAP, or t-SNE for the projection, rather than building a standalone plotting script. It depends on a companion notebook to actually generate the embeddings first - this one only covers logging and visualizing data that already exists.
Inputs and outputs
Input is per-row text data with a 1536-dimension embedding, generated by a separate notebook. Output is a W&B Table logged to a run, rendered as an interactive 2D projection using a chosen dimension-reduction algorithm.
Integrations
It logs data to Weights & Biases via a W&B Table and uses its built-in Embedding Projector panel for PCA/UMAP/t-SNE visualization.
Who it's for
Teams already using Weights & Biases for experiment tracking who want to visualize and share embedding clusters alongside their other model evaluation artifacts.
Source README
Visualizing embeddings in W&B
We will upload the data to Weights & Biases and use an Embedding Projector to visualize the embeddings using common dimension reduction algorithms like PCA, UMAP, and t-SNE. The dataset is created in the Get_embeddings_from_dataset Notebook.
What is Weights & Biases?
Weights & Biases is a machine learning platform used by OpenAI and other ML teams to build better models faster. They use it to quickly track experiments, evaluate model performance, reproduce models, visualize results, and share findings with colleagues.
1. Log the data to W&B
We create a W&B Table with the original data and the embeddings. Each review is a new row and the 1536 embedding floats are given their own column named emb_{i}.
2. Render as 2D Projection
After navigating to the W&B run link, we click the ⚙️ icon in the top right of the Table and change "Render As:" to "Combined 2D Projection".
Example: http://wandb.me/openai_embeddings
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.