Agentic RAG

Build an agentic chatbot using Knowledgebase (Legal Scotus)

This lab focuses on building an agentic chatbot that leverages the SCOTUS legal knowledge base to provide precise and context-aware legal insights. You will integrate retrieval mechanisms, natural language processing (NLP), and autonomous reasoning to create an intelligent assistant for legal research and case analysis.

Use case:

• Build an Agentic chatbot with a Legal Supreme Court dataset.

Pre-requisites:

• S3 bucket path with Supreme Court reviews.

Step 1: Create Knowledgebase using QnA Recipe Template

1. Navigate to Recipe.

2. Click the Add new button in the top corner of Recipe.

3. Provide a suitable name and description of the recipe.

4. Select type QnA.

5. Drag the Source tile from the Elements panel on the left-hand side onto the Central Canvas. In the source tile, select Amazon S3 as the connector. Enter Supreme Court reviews bucket path.

6. Add Dataset tile (Click + to create a new dataset).

7. Connect the Source tile to the Dataset tile.

8. Use Amazon Textract for OCR.

9. Add Knowledgebase tile.

10. Connect the Dataset tile to the Knowledgebase tile.

11. In Data Processing Preferences, Use the “Recursive” chunking type with 512 token size and select Embedding model(Amazon Titan Text Embeddings V2).

12. Save and Publish the recipe.

13. Run the recipe.

Step 2: Create a Summarization prompt in the playground.

1. Navigate to Prompt Playground.

2. Click the Add new button in the top corner of Prompt Playground.

3. Select prompt template Legal Summary.

4. Ensure you rename your prompt as loaded from the prompt template before saving it.

5. In the Test & Compare tab, choose the preferred models, assess their outputs, and identify the most optimal model. Subsequently, click "Select as Best Answer" to designate it as the Primary Model. Additionally, if necessary, you may assign an alternative model as the Fallback Model. However, selecting a Primary Model is mandatory.

6. Click "Save prompt run" button provided at the right corner.

7. Save and publish the prompt.

Step 3: Create an Agent prompt in the playground

1. Navigate to Prompt Playground.

2. Click the Add new button in the top corner of Prompt Playground.

3. Select prompt template Scotus Opinion Assistant.

4. Increase the Max State Updates from 3 to 20.

5. Add your natural language question "What was the central legal issue in the case Allen v. Cooper?" to the Agent Input (mandatory) text box.

6. Ensure you rename your prompt as loaded from the prompt template before saving it.

7. After saving the prompt, the Tool tab becomes visible within the prompt.

8. Create the following tools in the Agent prompt.

   1. Set up Knowledgebase tool.

      1. Click Add new in the tool and enter an appropriate name and description.

      2. Select the type Dataset.

      3. Select the dataset created in step 1 (Point number 6).

      4. Select Use Embedding chunks in prompt context.

      5. Set

         1. top_k = 3

         2. Enable Reranker and top_n = 2

      6. Select Multi-quey rewrite in Advanced query reconstruction.

      7. Save the tool.

   2. Set up a tool with a Summarize prompt.

      1. Click Add new in the tool and enter an appropriate name and description.

      2. Select the type Prompt.

      3. Select the prompt created in step 2.

      4. Save tool.

   3. Set up a tool with a Messaging tool.

      1. Click Add new in the tool and enter an appropriate name and description.

      2. Select the type Messaging.

      3. Choose Email as the messaging type.

      4. Configure email tool.

      5. Enter Email tool input schema: <Update Default email>.

      6. Hit enter after adding your email ID to the recipient email addresses.

      7. Save the tool.

   4. In the Test & Compare tab, choose the preferred models, assess their outputs, and identify the most optimal model. Subsequently, click "Select as Best Answer" to designate it as the Primary Model. Additionally, if necessary, you may assign an alternative model as the Fallback Model. However, selecting a Primary Model is mandatory.

   5. Click "Save prompt run" button provided at the right corner.

   6. Save and publish the prompt.

Step 3: Create an Agent 2.0 Recipe

1. Navigate to Recipe.

2. Click the Add New button in the top corner of Recipe.

3. Provide a suitable name and description of the recipe.

4. Select type Agent 2.0.

5. Setup following tiles.

   1. Chat

      1. Select 4 messages from the Conversation history dropdown.

      2. Enable Generate follow-up questions.

   2. Start - Connect the Chat tile to the Start tile.

   3. Agent (Select the Agentic prompt created in Step 3).

      1. Select All Messages in "Messages" dropdown agent state flag in the Agent tile.

      2. Connect the Start tile to the Agent tile.

   4. End - Connect Agent tile to End tile.

6. Save and Publish the recipe.

7. Click Test button on Chat tile to test the recipe.

8. Ask questions in the recipe Test.

   1. What was the Supreme Court's decision regarding reconsidering a midtrial acquittal in Smith v. Massachusetts?

   2. How did the Court interpret the language of the ISDEAA in relation to government contracts?

NOTE: Click on the Prompt Lens to inspect the Chatbot’s Response. Refer below image.

Step 4: Create a copilot.

1. Click the "Export" button located at the top-right corner of the recipe interface.

2. Click Add new button.

3. Configure the export form by referring to the sample resource agent recipe provided below.

4. Save the export form.

5. The Copilot will be displayed in the Copilot section of the interface.

Step 5: Ask Questions in the chatbot

1. Go to the Copilot tab and open your chatbot.

2. Ask questions

   1. What was the Supreme Court's decision regarding reconsidering a midtrial acquittal in Smith v. Massachusetts?

   2. How did the Court interpret the language of the ISDEAA in relation to government contracts?

Sample resources

1. Prompts-SCOTUS Opinion Assistant

2. QnA Recipe - Scotus Opinion Recipe

3. Agent recipe - SCOTUS Opinion Agent

4. Copilot - SCOTUS OpinionGuide AGENT

Refer the following video to create recipe.