Generate and Test Image Prompts

This promptfoo example automatically discovers jailbreaks in OpenAI's DALL-E image generation model, using a set of pre-configured test cases that attempt to generate various different types of harmful content. When DALL-E's safety system rejects a prompt, promptfoo automatically retries with modified prompts in a loop until it succeeds or reaches a maximum number of iterations - by default 4 per test case, raised via PROMPTFOO_NUM_JAILBREAK_ITERATIONS.

Use this only for authorized red-teaming of your own DALL-E integration to understand where its safety boundaries actually are - running it may get your OpenAI account flagged for moderation or banned, per the example's own explicit warning. Error messages like "blocked by our content filters" during a run are expected behavior, not a failure, since the whole point is probing where the filters hold and where they don't.

export OPENAI_API_KEY=your_key_here
npx promptfoo@latest init --example redteam-dalle
npx promptfoo@latest eval
npx promptfoo@latest view

In the web UI, enable "Render model outputs as Markdown" to display generated images and set "Max text length" to unlimited, then use the magnifying-glass icon to inspect the final modified prompt, full conversation history, and scoring breakdown per iteration. DALL-E image URLs expire after 2 hours, so the example includes an extension hook that downloads each generated image locally to an images directory as soon as it's created, named by test description and timestamp.

Enable debug logging with LOG_LEVEL=debug npx promptfoo@latest eval -j 1 to see the internal jailbreak-retry mechanics; reduce concurrency with -j 1 if you hit rate limits, and if you hit a timeout error the evaluation is likely still running in the background rather than failed. Test cases mirror those in promptfoo's own DALL-E jailbreak blog post, and can be reviewed or modified directly in promptfooconfig.yaml before running your own evaluation against them.

Security and safety teams authorized to red-team their own DALL-E integration who need an automated, iterative jailbreak-discovery loop rather than manually crafting adversarial prompts one at a time. The example's own test cases are drawn from promptfoo's published "Automated jailbreaking techniques with Dall-E" blog post, so teams can compare their own results against a documented baseline rather than starting from an empty config, and promptfoo's broader red-team documentation covers the same iterative-retry technique applied to other model types beyond image generation.