Automate Customer Service with Tool Use
Multi-step prompt workflow demonstrating tool_choice='required' for deterministic customer service flows with multi-turn evaluation using a customer GPT.
Why it matters
Enhance customer service operations by creating a deterministic LLM agent that reliably uses defined tools for specific customer issues. This ensures consistent and controlled interactions, leading to improved customer satisfaction.
Outcomes
What it gets done
Configure LLM to use specific tools for customer service tasks.
Implement mandatory tool calls for deterministic agent behavior.
Simulate customer interactions using a GPT tester for evaluation.
Develop a contained customer service flow with defined exit points.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/oai-usingtoolrequiredforcustomerservice | bash Steps
Steps in the chain
Define `tools` and `instructions` which our LLM customer service agent will use. It will source the right instructions for the problem the customer is facing, and use those to answer the customer's query. As this is a demo example, ask the model to make up values where it doesn't have external systems to source info.
Test the setup by running an example for a customer who has experienced fraud, and see how the model handles it. Play the role of the user and provide plausible next steps to keep the conversation going.
Perform a simple evaluation where a GPT will pretend to be the customer. The two will go back and forth until a resolution is reached. Reuse the functions above, adding an `execute_conversation` function where the customer GPT will continue answering.
Overview
Using Tool Required for Customer Service
What it does
A demonstration notebook showing how to use tool_choice='required' to create deterministic customer service flows where a tool must be called with every request, plus an evaluation approach using a customer-imitating GPT.
How it connects
Use this when building customer service applications that require predictable tool usage patterns and defined conversation exit points, or when you want to test your LLM agent with a GPT acting as the customer.
Source README
Using Tool Required for Customer Service
The ChatCompletion endpoint now includes the ability to specify whether a tool must be called every time, by adding tool_choice='required' as a parameter.
This adds an element of determinism to how you build your wrapping application, as you can count on a tool being provided with every call. We'll demonstrate here how this can be useful for a contained flow like customer service, where having the ability to define specific exit points gives more control.
The notebook concludes with a multi-turn evaluation, where we spin up a customer GPT to imitate our customer and test the LLM customer service agent we've set up.
Config definition
We will define tools and instructions which our LLM customer service agent will use. It will source the right instructions for the problem the customer is facing, and use those to answer the customer's query.
As this is a demo example, we'll ask the model to make up values where it doesn't have external systems to source info.
Example
To test this we will run an example for a customer who has experienced fraud, and see how the model handles it.
Play the role of the user and provide plausible next steps to keep the conversation going.
Evaluation
Now we'll do a simple evaluation where a GPT will pretend to be our customer. The two will go back and forth until a resolution is reached.
We'll reuse the functions above, adding an execute_conversation function where the customer GPT will continue answering.
Conclusion
You can now control your LLM's behaviour explicitly by making tool use mandatory, as well as spin up GPT testers to challenge your LLM and to act as automated test cases.
We hope this has given you an appreciation for a great use case for tool use, and look forward to seeing what you build!
Step 1: Config definition
Define `tools` and `instructions` which our LLM customer service agent will use. It will source the right instructions for the problem the customer is facing, and use those to answer the customer's query. As this is a demo example, ask the model to make up values where it doesn't have external systems to source info.
Step 2: Example
Test the setup by running an example for a customer who has experienced fraud, and see how the model handles it. Play the role of the user and provide plausible next steps to keep the conversation going.
Step 3: Evaluation
Perform a simple evaluation where a GPT will pretend to be the customer. The two will go back and forth until a resolution is reached. Reuse the functions above, adding an `execute_conversation` function where the customer GPT will continue answering.
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.