Predict Review Scores Using Text Embeddings
A prompt workflow that predicts numerical review scores (1-5 stars) from text embeddings, achieving 0.53 average error on unseen data for regression tasks.
Why it matters
Leverage text embeddings to predict numerical review scores, enabling quantitative analysis of textual data and improving understanding of customer sentiment.
Outcomes
What it gets done
Generate embeddings from review text.
Train a regression model to predict review scores.
Evaluate model performance on unseen data.
Utilize embeddings as features in existing ML models.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/oai-regressionusingembeddings | bash Overview
Regression Using Embeddings
This workflow predicts numerical review scores between 1 and 5 stars from text embeddings using regression analysis. It splits data into training and testing sets to evaluate performance on unseen reviews, achieving an average prediction error of 0.53 per score. The embeddings can also be used to encode free text features within existing ML models. Use this when you need to predict continuous numerical ratings from review text rather than categorical labels, and when you have sufficient labeled data to split into training and testing sets. It is particularly suited for scenarios where you want to evaluate real-world performance on unseen data.
What it does
This prompt chain performs regression analysis using text embeddings to predict numerical review scores. It takes review text, converts it to embeddings, and trains a model to predict scores between 1 and 5 stars, where 1-star represents negative reviews and 5-star represents positive ones. The workflow achieves an average prediction error of 0.53 per score, roughly equivalent to predicting half of reviews perfectly and half off by one star.
When to use - and when NOT to
Use this workflow when you need to predict numerical ratings from text reviews rather than categorical classifications. It is ideal for scenarios where you have labeled training data with numerical scores and want to evaluate model performance on unseen test data. Do NOT use this when you need categorical classification (predicting discrete labels rather than continuous numbers) or when you lack sufficient training data with numerical scores to split into training and testing sets.
Inputs and outputs
The workflow requires a dataset split into training and testing sets, with review text and corresponding numerical scores between 1 and 5. The dataset is created in the Get_embeddings_from_dataset Notebook. Users provide review text that gets converted to embeddings. The output is a predicted numerical score for each review, along with performance metrics showing an average error of 0.53 per prediction.
Who it's for
This workflow is designed for data scientists and machine learning practitioners who need to predict numerical ratings from text data. It serves users who want to encode free text features within existing ML models or who need an alternative to training classifiers for label prediction. The approach is particularly valuable for teams working with review data who require continuous score predictions rather than binary or multi-class classifications.
Source README
Regression using the embeddings
Regression means predicting a number, rather than one of the categories. We will predict the score based on the embedding of the review's text. We split the dataset into a training and a testing set for all of the following tasks, so we can realistically evaluate performance on unseen data. The dataset is created in the Get_embeddings_from_dataset Notebook.
We're predicting the score of the review, which is a number between 1 and 5 (1-star being negative and 5-star positive).
We can see that the embeddings are able to predict the scores with an average error of 0.53 per score prediction. This is roughly equivalent to predicting half of reviews perfectly, and half off by one star.
You could also train a classifier to predict the label, or use the embeddings within an existing ML model to encode free text features.
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.