Cluster and Name Data Groups

This OpenAI Cookbook notebook demonstrates k-means clustering on embeddings (produced by the companion Get_embeddings_from_dataset notebook) to discover hidden groupings in a dataset. It runs the simplest form of k-means, visualizes the resulting clusters in a 2D projection, then uses GPT-4 to name each cluster based on a random sample of 5 reviews drawn from it.

Use this notebook when you already have embeddings for a dataset and want to discover natural groupings within it - for example, distinct themes in a set of reviews - rather than working from predefined categories. The number of clusters you choose changes what the clustering surfaces: more clusters focus on more specific patterns, while fewer clusters focus on the largest overall discrepancies in the data, so clusters won't necessarily line up neatly with whatever categories you originally had in mind.

The workflow has two stages: first find the clusters with k-means and visualize them in a 2D projection (the notebook's own example run shows one cluster standing out visually from the rest), then pull a handful of text samples per cluster and pass them to GPT-4 to generate a descriptive name for each cluster based on that sample.

This notebook is a direct continuation of Get_embeddings_from_dataset - it consumes the embeddings that notebook produces rather than generating its own. GPT-4 is used purely for the cluster-naming step, not for the clustering itself, which is done with a standard k-means algorithm.

Data analysts and developers who want to explore unlabeled text data for hidden themes or segments, using embeddings plus k-means to find the groups and an LLM to make sense of what each group represents.