> For the complete documentation index, see [llms.txt](https://docs.karini.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.karini.ai/agentic-ai-workshops/automotive-and-manufacturing/assembly-to-bom-intelligence.md). # SpecToBOM: Build a Deep Agent That Turns a P\&ID Drawing into a Bill of Materials Build SpecToBOM, a deep agent that reads a P\&ID (Piping and Instrumentation Diagram) drawing, asks for missing specs, builds catalog-grounded part numbers, and returns an annotated bill of materials. **Lab time:** about 60 to 75 minutes ## The Problem A customer sends a P\&ID drawing for a custom fluid system assembly. Before anyone can quote it, an engineer has to read every symbol against the legend, trace the flow path, and find each component by digging through massive parts catalogs and product manuals, or by calling a colleague who knows the product line. Half the time the drawing leaves out what the part number depends on, such as the fluid, the maximum pressure, or the flow rate, so the engineer goes back to the customer with questions. Then the BOM is marked up on the drawing so the design team can turn it into a 3D model in SolidWorks, and the customer has to approve it before it goes to quoting and pricing. A single assembly can take days, and one wrong designator means the wrong part ships. Automating the first draft gives the engineer a head start on every assembly. ## The Art of the Possible (Future) The BOM is where manufacturing work starts, not where it ends. Once SpecToBOM produces a validated, part-numbered BOM, the same agentic pattern can carry it forward: * **Quote and order:** price each line and hand the BOM to order-to-cash * **Material availability:** check each part against inventory and open purchase orders * **Production planning:** generate work orders and routings, and check whether machine and labor capacity can absorb the assembly before you commit a ship date This lab builds the first link in that chain: drawing to BOM. ## What You Will Build SpecToBOM, a deep agent that turns a P\&ID drawing into a bill of materials with real part numbers. Upload a drawing image and it will: 1. Read the drawing, match every symbol against the legend, and list the components 2. Ask you, in one batch, for the specs the part numbers depend on, such as fluid, pressure, and flow rate 3. Search the product manuals for each component and build the ordering number from the catalog tables 4. Hand the BOM to a subagent that draws the BOM table and status markers onto the drawing 5. Return an HTML BOM table and a link to the annotated drawing ## What You Will Learn * Build a deep agent that reads an engineering drawing with a vision model * Pause for human input mid-task with a single batch of questions * Ground every part number in retrieved catalog tables, never in guesses * Delegate work to a subagent attached as an agent tool * Compare models side by side and pick a primary and fallback model ## Build Order Build the subagent first, because the main agent attaches it as a tool. Then build the main agent, the recipe, and the copilot last. | Step | You create | Contains | | ---- | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | | 1 | Subagent: Annotated P\&ID Generator | 1 tool (code interpreter), 1 skill (bom-table-overlay) | | 2 | Deep agent: SpecToBOM Agent | 3 tools (Code Interpreter, Knowledgebase tool, Annotated P\&ID Generator) and 2 skills (diagram-vlm-reader, bom-part-number-builder) | | 3 | Recipe: spectobom-workflow | Chat, Start, Deep Agent, End | | 4 | Copilot: SpecToBOM | Chat page for end users | | 5 | End-to-end test | One drawing, from image to annotated BOM | ## Before You Start Your instructor has prepared the following so you can focus on building the agents: * Login to Karini AI at app.karini.ai * The prompt templates **SpecToBOM Agent** and **Annotated P\&ID Generator**, available in Prompt Playground * The skills **diagram-vlm-reader**, **bom-part-number-builder**, and **bom-table-overlay**, available in the Skills Library * The knowledge base **Parts Catalog Library**, a vector database of the product manuals and parts catalogs, available in Karini datasets * A test P\&ID drawing (image file), provided by your instructor Use every tool name exactly as written in this guide. The prompts call the tools by these names, so a renamed tool breaks the agent. ## Words You Will See * **P\&ID:** piping and instrumentation diagram, the engineering drawing of a fluid system: its components, the lines between them, and a legend of symbols * **BOM:** bill of materials, the list of every part needed to build the assembly, with its ordering number * **Deep Agent:** an agent type for long, multi-step tasks. It can pause to ask the user questions, then continue. * **Subagent:** an agent that another agent calls as a tool and hands work to. Here, the main agent hands the finished BOM to a subagent that draws it on the diagram. * **Skill:** a folder of instructions and scripts the agent reads when it needs them * **Tool:** something the agent can call, such as a code runner, a knowledge base, or a subagent * **Recipe:** the workflow that connects everything on a canvas * **Copilot:** the chat assistant end users open to use the recipe * **Publish:** saves a version that recipes and other agents can use. Saved but unpublished changes are not used. * **Deploy:** makes a published recipe version live, so the copilot or webhook uses it. click Actions → Deploy after you publish. ## Step 1: Create the Annotated P\&ID Generator Subagent This agent draws the finished BOM onto the drawing. You build it first because the main agent attaches it as a tool in Step 2. ### 1.1 Create the prompt 1. From the left menu, open Prompt Playground. 2. Click Add new in the top right corner. 3. Click Prompt templates, Select **Name** from the dropdown next to the search field, search for **Annotated P\&ID Generator**, select it, and click Use template. 4. Configure the prompt: | Field | Value | | ----------------- | ------------------------- | | Name | Annotated P\&ID Generator | | Task | Agent | | Max State Updates | 30 | 5. Click Save. Save before you move to the Tools or Skills tab, or your changes are lost. ### 1.2 Add the tool: code interpreter 1. Open the Tools tab, click Add new, and select Agent tools. 2. Fill in: | Field | Value | | ----------------------------------- | ------------------------------ | | Name | code\_interpreter | | Description | \ | | Type | Code Interpreter | | Input Schema and Input Test Payload | \ | | Internet Access | Enable Public Internet Access | Description text: {% code overflow="wrap" %} ``` Runs Python code with AWS access. Use it to read the uploaded supplier catalog, run the catalog-extract and catalog-report skill scripts, and create the download link for the normalized Excel file. ``` {% endcode %} Input Schema : {% code overflow="wrap" %} ``` { "$schema": "http://json-schema.org/schema#", "type": "object", "properties": { "code": { "type": "string", "description": "The source code or code snippet to be processed" }, "method": { "type": "string", "description": "The method or function name to be executed" }, "Programming_language": { "type": "string", "description": "The programming language of the provided code" } }, "required": ["Programming_language","code","method"] } ``` {% endcode %} Input Test Payload: {% code overflow="wrap" %} ``` {"code": "import json\nresult = {'status': 'ok', 'value': round(55/8.0, 3)}\nprint(json.dumps(result))"} ``` {% endcode %} 3. Click Test. The result should print `{"status": "ok", "value": 6.875}`. Then click Save. ### 1.3 Add the skill 1. Open the Skills tab, click Add new, and select Custom. 2. Search for **bom-table-overlay** and click Add. ### 1.4 Test & Compare 1. Open the Test & Compare tab. 2. Select model: Claude Sonnet 5.5. 3. In Agent input, enter: 
Create the annotated P&ID with the BOM overlay. S3 URL: s3://karini-prod-6564fb2f4ab9f6bb6a195327/6adb1047a039ff854ca11cf5/Assembly_2.png. BOM JSON: {"bom": [{"component_id": "C1", "description": "CGA", "part_number": null, "status": "not_applicable", "note": "Labeled block, not a component requiring part number"}, {"component_id": "C3", "description": "Reducing union", "part_number": "SS-4-TSW-6", "status": "matched", "source": "ms-01-149.pdf"}, {"component_id": "C5", "description": "Filter", "part_number": null, "status": "not_applicable", "note": "Filters are out of scope"}, {"component_id": "C7", "description": "Union", "part_number": "SS-4-TSW-6", "status": "matched", "source": "ms-01-149.pdf"}, {"component_id": "C9", "description": "Regulator", "part_number": null, "status": "not_applicable", "note": "Regulators are out of scope"}, {"component_id": "C11", "description": "Check valve", "part_number": "SS-4C-10", "status": "matched", "source": "ms-01-176.pdf"}, {"component_id": "C13", "description": "45° union elbow", "part_number": "SS-4-TSW-5", "status": "matched", "source": "ms-01-149.pdf"}, {"component_id": "C15", "description": "Solenoid valve", "part_number": null, "status": "not_found", "note": "Requires additional specifications for ASCO 8262 series lookup"}, {"component_id": "C17", "description": "Tube support clamp", "part_number": "304-S1-PP-4T", "status": "matched", "source": "ms-01-181.pdf"}, {"component_id": "C19", "description": "Rotometer (gas)", "part_number": "VAF-G2-02S-1-1-Z", "status": "matched", "source": "ms-02-346.pdf"}, {"component_id": "C21", "description": "90° union elbow", "part_number": "SS-4-TSW-9", "status": "matched", "source": "ms-01-149.pdf"}, {"component_id": "C23", "description": "Ball valve", "part_number": null, "status": "not_applicable", "note": "Ball valves are out of scope"}, {"component_id": "C25", "description": "Union", "part_number": "SS-4-TSW-6", "status": "matched", "source": "ms-01-149.pdf"}]}
4. Click Test. Model generates Annotated P$ID img in few minutes. 5. Click Select as best answer . 6. Click Save prompt run in the Upper right corner. 7. Click Save, reopen the Annotated P\&ID Generator prompt then Publish. ### 1.5 Publish Click Save, then Publish. Only published agents can be attached as an agent tool, so do not skip Publish. You'll test this subagent end to end in Step 2, when the main agent hands it a real BOM. ## Step 2: Create the SpecToBOM Agent ### 2.1 Create the prompt 1. In Prompt Playground, click Add new. 2. Click Prompt templates, Select **Name** from the dropdown next to the search field, search for **SpecToBOM Agent**, select it, and click Use template. 3. Configure the prompt: | Field | Value | | ----------------- | --------------- | | Name | SpecToBOM Agent | | Task | Deep Agent | | Max State Updates | 50 | 4. Save the prompt. 5. Reopen the SpecToBOM agent prompt. 6. Scroll down,In Agent input paste the text below. {% code overflow="wrap" %} ``` No upload: the image is already in S3 (instructor copies it there once) - Paste this whole block into Agent input: - for prompt playground test only. Generate the BOM for this P&ID drawing. Available files: {'files': [{'content_type': 'image/png', 'file_path': 's3://karini-prod-6564fb2f4ab9f6bb6a195327/6adb1047a039ff854ca11cf5/Assembly_2.png'}] ``` {% endcode %} ### 2.2 Add tool 1: Code Interpreter 1. Reopen the SpecToBOM Agent prompt. 2. Open the Tools tab, click Add new, and select Agent tools. 3. Fill in: | Field | Value | | ----------------------------------- | ------------------------------ | | Name | code\_interpreter | | Description | \ | | Type | Code Interpreter | | Input Schema and Input Test Payload | \ | | Internet Access | Enable Public Internet Access | Description text: {% code overflow="wrap" %} ``` Runs Python code with AWS access. Use it to read the uploaded supplier catalog, run the catalog-extract and catalog-report skill scripts, and create the download link for the normalized Excel file. ``` {% endcode %} Input Schema : {% code overflow="wrap" %} ``` { "$schema": "http://json-schema.org/schema#", "type": "object", "properties": { "code": { "type": "string", "description": "The source code or code snippet to be processed" }, "method": { "type": "string", "description": "The method or function name to be executed" }, "Programming_language": { "type": "string", "description": "The programming language of the provided code" } }, "required": ["Programming_language","code","method"] } ``` {% endcode %} Input Test Payload: {% code overflow="wrap" %} ``` {"code": "import json\nresult = {'status': 'ok', 'value': round(55/8.0, 3)}\nprint(json.dumps(result))"} ``` {% endcode %} 3. Click Test. The result should print `{"status": "ok", "value": 6.875}`. Then click Save. ### 2.3 Add tool 2: Knowledgebase tool 1. In the Tools tab, click Add new and select Agent tools. 2. Fill in: | Field | Value | | -------------- | --------------------------------------------------------------------------------------------------------- | | Name | Knowledgebase tool | | Description | Searches the product manuals and parts catalogs for complete part numbers and ordering designator tables. | | Type | Dataset | | Dataset | Parts Catalog Library | | Prompt context | Use embedding chunks - Enable Check | | Top-K | 5 | 3. Click Save. ### 2.4 Add tool 3: Annotated P\&ID Generator This attaches the subagent you built in Step 1. 1. In the Tools tab, click Add new and select Agent tools. 2. Fill in: | Field | Value | | --------------- | ---------------------------------------------------- | | Name | Annotated P\&ID Generator | | Description | \ | | Type | Agent | | Agent | Annotated P\&ID Generator (latest published version) | | Use as subagent | Enable Checkbox | Description text: {% code overflow="wrap" %} ``` Draws the BOM table and numbered markers on the P&ID image, saves it to S3 and replies with an HTML BOM table and a link. Input: the S3 URL and the BOM JSON. ``` {% endcode %} 3. Click Save. The Tools tab now shows 3 tools: Code Interpreter, Knowledgebase tool, and Annotated P\&ID Generator. ### 2.5 Add the skills 1. Open the Skills tab, click Add new, and select Custom. 2. Search for **diagram-vlm-reader** and click Add. This skill runs a vision model over the drawing, reads the legend, traces the flow path, and returns the components and a BOM. 3. follow the same step as before and Search for **bom-part-number-builder** and click Add. This skill holds the ordering formats, catalog query patterns, and validation rules for building part numbers. ### 2.6 Test & Compare 1. Open the Test & Compare tab. 2. Select the models to compare: Claude Sonnet 5.5. 3. Make sure prompt caching is enabled for each model. 4. Click Test. 5. When the deepagent stops for Human input we need to hit the edit button. 6. The agent reads the drawing, then pauses with one batch of questions about the specs it could not find on the drawing, such as fluid, maximum pressure, line size, body material, and flow rate. Answer every question in plain words, for example: | Question | Answer | | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | | Line size | 1/4 in. tube fitting | | Body material | 316 stainless steel | | Special cleaning | No | | Tube size and wall thickness |

3/8 in. × 0.035 in. from INLET to filter F-401 (Larger tube size);
1/4 in. × 0.035 in. everywhere else (Larger tube size)

| | Joining method | Manual tube socket weld | | Rotometer – flow rate | 1 std ft³/h | | Rotometer – accuracy | 2.5 | | Rotometer – alarm switches | One | | Rotometer – mounting | Panel mount | | Rotometer – needle valve | Top | | Check valve – cracking pressure | 10 psi | | Solenoid valve – action | Normally closed | | Solenoid valve – coil | 24 VDC | | Solenoid valve – enclosure | General purpose | | Tube support clamp | Single clamp | | Fluid | Nitrogen | | Max pressure | 100 psig | | Special cleaning | No | | Tube support clamp – mounting | None | | CGA connection – components | All | | CGA connection – nipple outlet | 3/8 in. tube fitting | | CGA connection – nipple style | Plain | | Weld fittings – joint type | Manual tube socket weld | 7. The agent searches the catalogs, builds the part numbers, and hands the BOM to the subagent. 8. The final answer is an HTML BOM table and a line starting "Annotated P\&ID with BOM overlay:". Click **Format** to view the annotated drawing and download it. 9. Compare the output, then click Select as best answer on the output to set the Primary Model. 10. Click Save prompt run in the right corner. 11. Click Save, the Reopen the SpecToBOM Agent then Publish. Each line of the BOM carries a status that tells you how far to trust it: | Status | Meaning | | --------------- | --------------------------------------------------------------------- | | matched | A complete part number was printed in the catalog | | selected | Part number assembled from catalog designator tables and your answers | | configurable | Ordering format found, but at least one position could not be filled | | Pending | No useful catalog result after the allowed searches | | not\_applicable | Connector lines and flow direction symbols | | not\_in\_legend | Symbol not in the drawing's legend, so no catalog lookup | SpecToBOM never invents a part number. When it can't trace a position to a catalog table, it says so with a status instead of guessing. ## Step 3: Create the Karini Recipe with Bedrock AgentCore Runtime ### 3.1 Create the recipe 1. From the left menu, open Recipes and click Add new in the top corner. 2. Configure the recipe: | Field | Value | | ----------- | ------------------------------------------------------------ | | Name | spectobom-workflow | | Description | Turns a P\&ID drawing into a part-numbered bill of materials | | Type | Workflow | | Runtime | AgentCore | ### 3.2 Add and connect the nodes Drag each element from Elements onto the canvas, then click it to fill in the right panel. | # | Element | Label | Settings in the right panel | | - | ---------- | --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | Chat | Chat | **Conversation history** - No history | | 2 | Start | Start | **None** | | 3 | Deep Agent | SpecToBOM Agent |

Deep Agent = SpecToBOM Agent.
1) State settings: Messages on (Last message)
2) Metadata -
Enable Checkbox

| | 4 | End | End | **None** | When you select SpecToBOM Agent, its 3 tools and 2 skills appear on the canvas automatically, with the subagent's own tool and skill nested under it. Connect them in a straight line: Chat → Start → SpecToBOM Agent → End. ### 3.3 Recipe settings, test, and deploy 1. Click an empty area of the canvas and set Number of state updates to 50. 2. Click Save, reopen the spectobom-workflow recipe then Publish. 3. Click Action then Deploy. {% embed url="" %} ## Step 4: Create the SpecToBOM Copilot Make sure spectobom-workflow is published before you start. 1. On the recipe page, click Actions in the top right corner and select Create Copilot. 2. On the Copilots page, click Add new and fill in: | Field | Value | | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | | Copilot name | SpecToBOM | | Welcome message | I'm SpecToBOM. Upload a P\&ID drawing and I'll read it, ask you for any missing specs, find the part numbers, and give you an annotated BOM. | | Description | Turns P\&ID drawings into part-numbered bills of materials | | Seed questions |

Click on Add button and Add -
Create a BOM for this drawing

| | Show traces | Enable Checkbox | | Other fields | Keep the defaults | 3. Click Save. ## Step 5: Test SpecToBOM End to End 1. From the left menu, open Copilots and select SpecToBOM. 2. Click the attachment icon and upload the test drawing image, then click the seed question or type Create a BOM for this drawing and send it. (Instrtuctor will provide Box folder link for those test Drawing images). 3. When SpecToBOM asks for the missing specs, answer every question in one reply (The Ideal reply for asked question will be provided by Instructor ). 4. Wait for the HTML BOM table and the annotated drawing link. 5. Check the status of each line, then open the annotated drawing to see each component marked with its status color and number. While the agent runs, open Traces to watch it work: the diagram reader call, the batch of questions, each catalog search, and the handoff to the subagent. This is the moment that shows the value: a drawing that would take an engineer days to read, look up, and mark up comes back as a validated, part-numbered BOM, with every number traceable to a catalog table.