Self-RAG

This directory is retained purely for archival purposes and is no longer updated. Please see the newly consolidated LangChain documentation for the most current information and resources.

Self-RAG

Self-RAG is a strategy for RAG that incorporates self-reflection / self-grading on retrieved documents and generations.

In the paper, a few decisions are made:

1. Should I retrieve from retriever, R -

• Input: x (question) OR x (question), y (generation)
• Decides when to retrieve D chunks with R
• Output: yes, no, continue

2. Are the retrieved passages D relevant to the question x -

• • Input: (x (question), d (chunk)) for d in D
• d provides useful information to solve x
• Output: relevant, irrelevant

3. Are the LLM generation from each chunk in D is relevant to the chunk (hallucinations, etc) -

• Input: x (question), d (chunk), y (generation) for d in D
• All of the verification-worthy statements in y (generation) are supported by d
• Output: {fully supported, partially supported, no support

4. The LLM generation from each chunk in D is a useful response to x (question) -

• Input: x (question), y (generation) for d in D
• y (generation) is a useful response to x (question).
• Output: {5, 4, 3, 2, 1}

We will implement some of these ideas from scratch using LangGraph.

Setup

First let's install our required packages and set our API keys

Retriever

Let's index 3 blog posts.

LLMs

Graph

Capture the flow in as a graph.

Graph state

Build Graph

The just follows the flow we outlined in the figure above.

LangSmith Traces -

• https://smith.langchain.com/public/55d6180f-aab8-42bc-8799-dadce6247d9b/r

• https://smith.langchain.com/public/1c6bf654-61b2-4fc5-9889-054b020c78aa/r