Build Chatbots with Vision Capabilities
A single-node prompt flow that processes chat history and questions through OpenAI Vision to enable image-based conversational AI interactions.
Why it matters
Create interactive chatbots that can understand and respond to user queries, leveraging OpenAI's vision capabilities to interpret images within the conversation.
Outcomes
What it gets done
Process chat history for context.
Understand user questions.
Analyze images provided by the user.
Generate relevant responses based on text and images.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/pf-flows-chatflowwithopenaivisionimage | bash Capabilities
What this chain does
Handles multi-turn conversations within a defined domain.
Condenses long documents or threads into key takeaways.
Labels or categorizes text, files, or data points.
Overview
Chat Flow With Openai Vision Image
What it does
This prompt flow implements a 1-node workflow that integrates OpenAI Vision capabilities into a chat interface. It accepts two inputs-chat_history and question-and processes them through OpenAI's Vision model to enable image understanding within conversational exchanges.
How it connects
Use this flow when you need to add visual understanding to a chatbot or conversational interface, allowing users to ask questions about images while maintaining conversation context. Ideal for applications requiring multimodal AI where text and image inputs must be processed together.
Source README
message_format: openai-vision
inputs:
chat_history:
type: list
default:
- inputs:
question:
- the first question
- type: image_file
image_file:
path: logo.jpg
outputs:
answer:
- type: image_file
image_file:
path: logo.jpg
- inputs:
question:
- the second question
- type: image_file
image_file:
path: logo_2.png
outputs:
answer:
- type: image_file
image_file:
path: logo_2.png
is_chat_history: true
question:
type: list
default:
- the third question
- type: image_file
image_file:
path: logo.jpg
- type: image_file
image_file:
path: logo_2.png
is_chat_input: true
outputs:
answer:
type: string
reference: ${mock_chat_node.output}
is_chat_output: true
nodes:
- name: mock_chat_node
type: python
source:
type: code
path: mock_chat.py
inputs:
chat_history: ${inputs.chat_history}
question: ${inputs.question}
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.