> For the complete documentation index, see [llms.txt](https://docs.karini.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.karini.ai/agentic-ai-workshops/ai-foundation-labs/intelligent-document-processing.md). # Intelligent Document Processing **Lab time:** about 60 minutes ## The Problem Purchase orders arrive as PDFs, scans, and email attachments, each laid out a little differently. Before anything can happen downstream, someone has to open each one, find the PO number, dates, payment terms, supplier, addresses, and every line item, then key them into another system and tell the right people it's done. It's repetitive, it's error-prone, and the backlog grows every time volume spikes. ## What You Will Build An Intelligent Document Processing (IDP) workflow that runs on its own whenever a system calls it. Send it the S3 path of a purchase order and it will: 1. Read the document from S3 with OCR, including its layout and tables 2. Extract the purchase order into structured JSON: header fields, addresses, and line items 3. Save the JSON to an S3 sink folder 4. Email a notification about the processing status ## What You Will Learn * Trigger a workflow from an outside system with a webhook * Extract text, layouts, and tables from documents with Amazon Textract * Turn unstructured documents into structured JSON with a prompt * Write workflow output to S3 with a Sink node * Hand results to an agent that sends email notifications * Monitor every run with webhook history and traces ## Build Order Build the two prompts first, then the recipe. A recipe can only use prompts that are already published. | Step | You create | Contains | | ---- | ------------------------------------------------ | --------------------------------------------------------------- | | 1 | Prompt: Document Extractor | Prompt only | | 2 | Agent: Notification Agent | 1 tool: Messaging tool | | 3 | Workflow recipe: Intelligent Document Processing | Webhook, Connector, Processing, Start, Prompt, Sink, Agent, End | | 4 | API trigger | One run started with curl | | 5 | Verification | Webhook history and the S3 output | ## Before You Start * Access to the Karini AI platform * An S3 bucket with sample purchase order documents pre-loaded * Webhook API access credentials configured * The **Po Extractor** and **Notify Agent** prompt templates, available in Prompt Playground * A terminal with curl, to trigger the workflow ## Words You Will See * **IDP:** intelligent document processing, using AI to read documents and turn them into structured data * **OCR:** optical character recognition, turning the text in a scan or PDF into machine-readable text * **Prompt template:** a ready-made prompt stored in Karini. You load it instead of typing it. * **Tool:** something an agent can call, such as an email sender * **Webhook:** a URL that lets an outside system start your recipe * **Connector:** a node that reads files from a data source, here Amazon S3 * **Sink:** a node that writes workflow output to storage, here an S3 folder * **Recipe:** the workflow that connects everything on a canvas * **Publish:** saves a version that recipes can use. Saved but unpublished changes are not used. ## Step 1: Create the Document Extractor Prompt ### 1.1 Create the prompt 1. Navigate to Prompt Playground. 2. Click **Add new** in the top right corner. 3. Open Prompt templates and select the **Po Extractor** template. This creates a new summarization prompt. 4. Rename your prompt to **Document Extractor**. 5. **Save** the prompt. ### 1.2 Test & Compare 1. In the Test & Compare tab, select different models from the dropdown to test the prompt. Compare Claude Sonnet 3.7, Claude Sonnet 3.5 v2, and Amazon Nova Pro. 2. Test and compare the prompt responses with the selected models. 3. Select the best-performing model as the **Primary Model**. 4. Optional: assign a **Fallback Model**. 5. Click **Save prompt run** in the right corner. 6. **Save** and **publish** the prompt. ## Step 2: Create the Notification Agent ### 2.1 Create the prompt 1. Return to Prompt Playground and click **Add new**. 2. Open Prompt templates and select the **Notify Agent** template. 3. Once the template is selected, increase **Max State Updates** from **3** to **20** for complex reasoning. 4. In the agent input field, paste this sample input for testing: ```json { "po_number": "PO-2025-8975", "date": "20250312", "delivery_time": "10-14 Business Days", "delivery_mode": "Standard Freight", "payment_terms": "Net 30 Days", "supplier_ref": "SUPP-XYZ-7239", "project": "ABC Expansion 4", "pool": "General Procurement", "requestor": "Employee 64", "contact": "+1 555-987-654", "originator": "Procurement Team", "company": { "name": "XYZ ENGINEERING SOLUTIONS LTD.", "address": "789 Industrial Way, Houston, TX 77001, USA", "phone": "+1 234-567-890", "email": "info@xyzengineering.com" }, "delivery_address": "ABC Manufacturing Inc., 321 Production Ave, Phoenix, AZ 85001, USA", "invoice_address": "XYZ Engineering Solutions Ltd., 789 Industr", "line_items": [ {"line_no": 1, "item_no": "ITM0001", "description": "Industrial Equipment 861", "quantity": 8, "unit": "PCS", "unit_price": 47.00, "amount": 405.00} ], "grand_total": 405.00, "authorized_by": "", "received_by": "" } ``` 5. Name the prompt **Notification Agent** and save it. Save your changes before you navigate away to add tools, or they are lost. ### 2.2 Add the tool: Messaging tool 1. Open your prompt, go to the **Tools** tab, click **Add new**, and fill in: | Field | Value | | ---------------- | -------------------------------------------------------------------- | | Name | Messaging tool | | Description | Sends email notifications about document processing status | | Type | Messaging | | Messaging type | Email | | Input Schema | Use the schema below | | Credentials | Email, password, SMTP server, SMTP port, and recipient email address | | Subject and body | An appropriate email subject and message body | Input schema: ```json { "$schema": "http://json-schema.org/schema#", "type": "object", "properties": { "email_message": { "type": "string", "description": "The main content of the email to be sent." }, "subject": { "type": "string", "description": "The subject line of the email." }, "recipient_email": { "type": "string", "description": "The email address of the recipient.", "default":"{{user_context.email}}" } }, "required": [ "email_message", "recipient_email", "subject" ] } ``` 2. Save the tool. ### 2.3 Test & Compare 1. In the Test & Compare tab, select different models from the dropdown to test the prompt. 2. Click **Test** and compare the agent responses with the selected models. 3. Click **Select as best answer** to set the best-performing model as the Primary Model. 4. Optional: click **Select as best answer** on another output to assign a Fallback Model. 5. Click **Save prompt run** in the right corner. 6. **Save** and **Publish** the prompt. ## Step 3: Create a Workflow Recipe ### 3.1 Create the recipe 1. Navigate to **Recipes**. 2. Click **Add New** in the top right corner and configure: | Field | Value | | ------- | ------------------------------- | | Name | Intelligent Document Processing | | Type | Workflow | | Runtime | Karini | ### 3.2 Add and connect the nodes Drag each element onto the recipe canvas, then click it to configure it. | # | Element | Label | Settings | | - | ---------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------ | | 1 | Webhook | Webhook | Query method POST. Copy the Webhook URL and Webhook Token from the right panel for Step 4 | | 2 | Connector | Connector | Data source Amazon S3 | | 3 | Processing | Processing | OCR on, Amazon Textract. Extract Layouts on, Extract Table on | | 4 | Start | Start | None | | 5 | Prompt | Document Extractor | Prompt = Document Extractor. State settings: Document Cache on (Type Ephemeral), Messages on (Last Message) | | 6 | Sink | Sink | S3 bucket path `s3://karini-ai-workshop/idp-sink/`. File name pattern `po_processing/{filename_prefix}{current_datetime}.json` | | 7 | Agent | Notification Agent | Agent = Notification Agent. State settings: Document Cache on (Type Ephemeral), Messages on (Last Message), Metadata on | | 8 | End | End | None | ### 3.3 Connect the nodes | From | To | | ------------------ | ------------------ | | Webhook | Connector | | Connector | Processing | | Processing | Start | | Start | Document Extractor | | Document Extractor | Sink | | Document Extractor | Notification Agent | | Notification Agent | End | The Sink stores the extracted JSON in S3, while the Notification Agent uses the same output to send the status email. Metadata on the Notification Agent carries your email address from the webhook call. ### 3.4 Recipe settings and publish 1. On the right side, set **Number of state updates** to a higher value, such as 75. 2. Save and Publish the recipe. By default, the V1 version is deployed once it is published. 3. For later versions, click **Actions** and select **Deploy**. 4. Test the recipe once published. Refer to the following video to create the recipe. {% embed url="" %} ## Step 4: Trigger the Workflow via API Open a terminal and run the commands below. Replace each placeholder in angle brackets, and make sure the payload stays valid JSON. ### 4.1 Start document processing Note the request ID in the output; you'll use it in 4.2. ```bash curl -X 'POST' \ '' \ -H 'accept: application/json' \ -H 'x-api-token: ' \ -H 'Content-Type: application/json' \ -d '{ "files": [ { "content_type": "application/pdf", "file_content": "", "file_path": "" } ], "input_message": "Extract document metadata", "metadata": { "email": "" } }' ``` ### 4.2 Check the request status by ID ```bash curl -X 'GET' \ '' \ -H 'accept: application/json' \ -H 'x-api-token: ' ``` ### 4.3 View all webhook requests for the recipe ```bash curl -X 'GET' \ '' \ -H 'accept: application/json' \ -H 'x-api-token: ' ``` Refer to the video to trigger the workflow. {% embed url="" %} ## Step 5: Monitor and Verify Results 1. Open **Recipes** from the left toolbar. 2. Click **Actions** on the Intelligent Document Processing recipe and select **Webhook History**. You'll see every webhook request with its input, response, status, tokens, and detailed traces. 3. Open the S3 location of your Sink node to review the JSON output. 4. Check your inbox for the processing notification email. This is the moment that shows the value: a purchase order went from a file in S3 to structured data and a notification, with no one opening the document. Refer to the video for reviewing the webhook history. {% embed url="" %}