Intelligent Document Processing

This hands-on lab guides you through creating an Intelligent Document Processing (IDP) system that automates extraction and processing using AI technologies. You'll build a complete workflow integrating OCR capabilities, natural language processing and webhook triggers.

Use Case

Build an Intelligent Document Processing system powered by advanced AI services for automated document extraction and analysis.

Prerequisites

• S3 bucket with sample documents pre-loaded.

• Webhook API access credentials are configured.

Step 1: Build Document Extraction Prompt in Prompt Playground

1. Navigate to the Prompt Playground.

2. Click Add new in the top right corner to create a new prompt.

3. Open Prompt templates and select the Po Extractor template. This template will create a new summarization prompt.

4. Rename your prompt as "Document Extractor".

5. Save the prompt.

Test & Compare :

1. In Test & Compare tab, select different models from the dropdown to test the prompt [Hint: Compare Claude Sonnet 3.7, Claude Sonnet 3.5 v2, Amazon Nova Pro].

2. Test and compare the prompt responses with the selected models and the guardrail.

3. Select the best-performing model as the Primary Model.

4. Optionally assign a Fallback Model.

5. Click the Save prompt run in the right corner to save the prompt run.

6. Save and publish the prompt.

Step 2: Create Notification Agent Prompt

1. Return to Prompt Playground.

2. Click Add new in the top corner to add a new prompt.

3. Open Prompt templates and select the Notify Agent template.

4. Once the template is selected, increase the Max State Updates from 3 to 20 for complex reasoning.

5. In the agent input field, use the following sample input for testing.

{
    "po_number": "PO-2025-8975",
    "date": "20250312",
    "delivery_time": "10-14 Business Days",
    "delivery_mode": "Standard Freight",
    "payment_terms": "Net 30 Days",
    "supplier_ref": "SUPP-XYZ-7239",
    "project": "ABC Expansion 4",
    "pool": "General Procurement",
    "requestor": "Employee 64",
    "contact": "[phone]",
    "originator": "Procurement Team",
    "company": {
        "name": "XYZ ENGINEERING SOLUTIONS LTD.",
        "address": "789 Industrial Way, Houston, TX 77001, USA",
        "phone": "[phone]",
        "email": "[e-mail]"
    },
    "delivery_address": "ABC Manufacturing Inc., 321 Production Ave, Phoenix, AZ 85001, USA",
    "invoice_address": "XYZ Engineering Solutions Ltd., 789 Industr",
    "line_items": [
        {"line_no": 1, "item_no": "ITM0001", "description": "Industrial Equipment 861", "quantity": 8, "unit": "PCS", "unit_price": 47.00, "amount": 405.00},
       
    ],
    "grand_total": 405.00,
    "authorized_by": "",
    "received_by": ""
}

1. Name the prompt as Notification Agent and save the prompt.

2. Open your prompt and proceed to "Tools" tab to configure the following agent tools:

• Messaging Tool:

  • Click Add new to create a new tool.

  • Name: "Messaging tool".

  • Description: "Sends email notifications about document processing status".

  • Type: Messaging

  • Messaging type: Email

  • In the Input Schema section, use the following email schema and save the tool.

    {
      "$schema": "http://json-schema.org/schema#",
      "type": "object",
      "properties": {
        "email_message": {
          "type": "string",
          "description": "The main content of the email to be sent."
        },
        "subject": {
          "type": "string",
          "description": "The subject line of the email."
        },
        "recipient_email": {
          "type": "string",
          "description": "The email address of the recipient.",
          "default":"{{user_context.email}}"
        }
      },
      "required": [
        "email_message",
        "recipient_email",
        "subject"
      ]
    }

  • Enter the following credentials

    • Email

    • Password

    • SMTP Server

    • SMTP Port

    • Recipient email address

• Provide appropriate email subject and message body.

Test & Compare :

1. In Test & Compare tab, select different models from the dropdown to test the prompt

2. Click on Test button below and compare the agent responses with the selected models.

3. Click on Select as best answer to select the best-performing model as the Primary Model.

4. Optionally Select as best answer to assign a Fallback Model.

5. Click the Save prompt run in the right corner to save the prompt run.

6. Save and Publish the prompt.

Step 3: Create an Agent 2.0 Recipe

1. Navigate to the Recipes section.

2. Click Add New in the top right corner to create a new recipe.

3. Configure recipe details:

   1. Name: Intelligent Document Processing.

   2. Type: Agent 2.0.

4. Set up the the recipe workflow nodes by selecting each of the following elements and dragging them onto the recipe canvas:

   1. Webhook:

      1. From the right side panel, Copy the Webhook URL and Webhook Token to your notepad for later API use.

      2. Select the Query method as POST.

   2. Connector:

      1. Select Amazon S3 as the data source.

      2. Connect the Webhook node to Connector node.

   3. Processing:

      1. Enable the OCR option and select Amazon Textract.

      2. Enable Extract Layouts and Extract Table.

      3. Connect the Connector to the Processing node.

   4. Start:

      1. Connect the Processing node to the Start node.

   5. Prompt:

      1. Label: "Document Extractor".

      2. From the prompt dropdown, select the document extraction prompt created earlier.

      3. Scroll down and set the State settings:

         • Document Cache: Enable Document Cache and select the Type as Ephemeral.

         • Messages: Enable Messages, and select message context as Last Message.

      4. Connect the Start node to the Prompt node.

   6. Sink:

      1. Connect the Prompt Node to the Sink Node to store JSON output.

      2. Sink node Configuration:

         1. S3 bucket path:

            s3://karini-ai-workshop/idp-sink/

         2. File name pattern: (Ex. processed/{filename_prefix}{current_datetime}.json ) po_processing/{filename_prefix}{current_datetime}.json.

   7. Agent:

      1. Label: "Notification Agent".

      2. From the agents dropdown, select your notification agent created earlier.

      3. Scroll down and set the State settings:

         • Document Cache: Enable Document Cache and select the Type as Ephemeral.

         • Messages: Enable Messages, and select message context as Last Message.

         • Enable metadata.

      4. Connect the Document Extractor Prompt node to Notification Agent node.

   8. End:

      1. Connect the Notification Agent node to the End node.

5. On the right side, set the Number of state updates to a higher value such as 75.

6. Save and Publish the recipe by assigning an initial version.

Refer to the following video to create the recipe.

Step 4: Trigger Workflow via API

1. Open the linux terminal in your computer. Use the following curl command to initiate document processing [Hint: Ensure CURL command input payload is valid JSON]. Note the output of the command for next step.

curl -X 'POST' \
  '<Webhook URL from the Webhook Tile>' \
  -H 'accept: application/json' \
  -H 'x-api-token: <YOUR API TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
    "files": [
        {
            "content_type": "application/pdf",
            "file_content": "",
            "file_path": "<S3 path>"
        }
    ],
    "input_message": "Extract document metadata",
    "metadata": { "email": <YOUR EMAIL ID>}
}'

1. Retrieve webhook request status by ID:

curl -X 'GET' \
  '</api/webhook/request/{request_id}>' \
  -H 'accept: application/json' \
  -H 'x-api-token: <YOUR API TOKEN>'

3.View all webhooks for a specific recipe:

curl -X 'GET' \
  '</api/webhook/recipe/{recipe_id}>' \
  -H 'accept: application/json' \
  -H 'x-api-token: <YOUR API TOKEN>'

Refer the video to trigger workflow.

Step 5: Monitor and Verify Results

1. Navigate to the recipe icon on the LHS toolbar

2. Click on the Actions button on the Intelligent Document Processing recipe.

3. Select Webhook History. Here, you can review the history of all webhook requests along with their inputs, response, status, tokens and detailed traces.

4. Additionally, you can access the S3 location of your Sink node and review JSON output.

This IDP workflow automates document processing from ingestion through extraction to notification, creating a complete intelligent document handling system.

Refer the video for reviewing the webhook history.