Generate Images with OpenAI DALL-E
Generate images from text prompts in a LlamaIndex agent using OpenAI's DALL-E.
Why it matters
Leverage OpenAI's DALL-E model to create unique images from text descriptions. Integrate this capability into your workflows or use it directly for rapid visual content creation.
Outcomes
What it gets done
Generate images from textual prompts using DALL-E.
Integrate image generation into AI agents and workflows.
Retrieve image data in various formats (e.g., base64 JSON).
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/li-tool-tools-openai | bash Overview
OpenAI Image Generation Tool
A LlamaIndex tool that generates images from text prompts via OpenAI's DALL-E model, usable directly or through an agent. Use when an agent or app needs to turn a text description into a generated image.
What it does
The OpenAI Image Generation Tool lets a LlamaIndex agent create images from text prompts using OpenAI's DALL-E model. The OpenAIImageGenerationToolSpec exposes a single function, image_generation, that takes a text description and returns generated image data.
The tool can be used two ways: wired into an agent as part of its tool list, so the agent decides when to call it in response to a user's request, or called directly by a developer without an agent in the loop. When called directly, it accepts a response_format parameter (such as b64_json) controlling how the image data is returned, which can then be decoded and displayed or saved.
When to use - and when NOT to
Use it when a LlamaIndex agent or application needs to turn a text description into an actual image - illustrating a concept, generating a visual for a response, or letting a user request custom art through natural language. Do not use it for editing or modifying existing images, or for tasks needing precise pixel-level control - DALL-E generation is prompt-driven and does not guarantee exact compositional accuracy.
Capabilities
image_generation is the only tool: it takes a text prompt and returns generated image data, with an optional response_format for how that data is encoded (for example, base64-encoded JSON).
How to install
from llama_index.tools.openai import OpenAIImageGenerationToolSpec
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI
image_generation_tool = OpenAIImageGenerationToolSpec(
api_key=os.environ["OPENAI_API_KEY"]
)
agent = FunctionAgent(
tools=[*image_generation_tool.to_tool_list()],
llm=OpenAI(model="gpt-4.1"),
)
Requires an OpenAI API key set as an environment variable.
Who it's for
Developers building LlamaIndex agents or applications that need to generate custom images from natural-language descriptions, without hand-rolling a separate call to an image-generation API.
Source README
OpenAI Image Generation Tool
This tool allows Agents to generate images using OpenAI's DALL-E model. To see more and get started, visit https://openai.com/blog/dall-e/
Usage
This tool has a more extensive example usage documented in a Jupyter notebook here.
Usage with Agent
from llama_index.tools.openai import OpenAIImageGenerationToolSpec
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI
image_generation_tool = OpenAIImageGenerationToolSpec(
api_key=os.environ["OPENAI_API_KEY"]
)
agent = FunctionAgent(
tools=[*image_generation_tool.to_tool_list()],
llm=OpenAI(model="gpt-4.1"),
)
print(
await agent.run(
"A pink and blue llama in a black background with the output"
)
)
Usage directly
from llama_index.tools.openai import OpenAIImageGenerationToolSpec
image_generation_tool = OpenAIImageGenerationToolSpec(
api_key=os.environ["OPENAI_API_KEY"]
)
image_data = image_generation_tool.image_generation(
text="A pink and blue llama with a black background",
response_format="b64_json",
)
image_bytes = base64.b64decode(image_data)
img = Image.open(BytesIO(image_bytes))
display(img)
image_generation: Takes an text input and generates an image
This loader is designed to be used as a way to load data as a Tool in a Agent.
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.