Detect Hate Speech in User-Generated Content
Automatically detect and classify hate speech in text using Hugging Face models to moderate content and protect online communities.
Why it matters
Automatically identify and flag hate speech in text content to maintain safe online communities and comply with content moderation policies. This asset helps teams screen user-generated content at scale using Hugging Face's classification models.
Outcomes
What it gets done
Classify text inputs as hate speech or acceptable content
Integrate Hugging Face API for real-time content analysis
Evaluate detection accuracy across test cases
Generate structured reports on flagged content
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/pfoo-hate-speech-detection | bash Steps
Steps in the chain
Overview
Hate Speech Detection
A promptfoo example configuration demonstrating hate speech detection evaluation with Hugging Face When you need a starting point for configuring hate speech detection evaluations with promptfoo and Hugging Face APIs
What it does
This example provides a promptfoo evaluation setup for hate speech detection using Hugging Face. It demonstrates how to configure and run evaluations using the promptfoo testing framework with Hugging Face models, then view the results.
When to use - and when NOT to
Use this example when you want to set up evaluation workflows for hate speech detection models accessible through Hugging Face APIs. It's useful as a starting point for building test configurations and running evaluations during development.
Do NOT use this if you lack access to Hugging Face API tokens, as the HF_API_TOKEN environment variable is required to run the example.
Inputs and outputs
You provide a Hugging Face API token (HF_API_TOKEN environment variable) and edit the promptfooconfig.yaml file to define your configuration. The example accepts the setup you define in the configuration file.
You can view results after running the evaluation by using the promptfoo view command.
Getting started
Initialize the example with:
npx promptfoo@latest init --example huggingface/hate-speech-detection
cd huggingface/hate-speech-detection
Set your Hugging Face API token as an environment variable, edit the promptfooconfig.yaml file, then run:
promptfoo eval
After evaluation completes, view results with promptfoo view.
Integrations
This example works with Hugging Face APIs and the promptfoo evaluation framework.
Who it's for
This example serves developers who want to evaluate hate speech detection models using promptfoo and Hugging Face.
Source README
huggingface/hate-speech-detection (Huggingface Hate Speech Detection)
You can run this example with:
npx promptfoo@latest init --example huggingface/hate-speech-detection
cd huggingface/hate-speech-detection
Usage
To get started, set your HF_API_TOKEN environment variable.
Next, edit promptfooconfig.yaml.
Then run:
promptfoo eval
Afterwards, you can view the results by running promptfoo view
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.