> For the complete documentation index, see [llms.txt](https://docs.karini.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.karini.ai/agentic-ai-workshops/automotive-and-manufacturing/specmatch-build-a-knowledge-graph-agent-for-industrial-parts.md). # SpecMatch: Build a Knowledge Graph Agent for Industrial Parts Build SpecMatch, an agent that matches a plain-language fitting spec to the exact ordering number, using an Amazon Neptune knowledge graph, a catalog knowledge base, a guardrail, and memory. **Lab time:** about 45 minutes, including the evaluation in Step 6 ## The Problem A customer calls asking for a 316 stainless union for 1/4 inch tube. Somewhere in a catalog of thousands of fittings is the exact ordering number, but finding it means flipping through PDF pages, cross-checking tube sizes against end connections, and confirming the part comes in the right material. Keyword search fails here because the question is about relationships: which part, in which family, available in which material, for which tube size. That is exactly what a knowledge graph is built for. ## What You Will Build SpecMatch, a copilot that matches a plain-language spec to the exact ordering number and returns a specifications table. Here is how a question flows through what you build: 1. **Guardrail** screens the request against usage guidelines 2. **SpecMatch Agent** receives the question and decides how to answer it 3. **Spec-to-Gremlin Translator** turns the question into a Gremlin query 4. **Knowledge graph (Amazon Neptune)** returns the part, its family, and available materials 5. **Catalog documents** serve as fallback when the graph has no answer, and supply details like pressure ratings 6. **Memory** keeps context so follow-up questions work naturally ## What You Will Learn * Turn natural language into structured graph queries with a prompt tool * Compare models side by side and pick a primary and fallback model * Combine a knowledge graph and a knowledge base so the agent uses each for what it does best * Protect an agent with a guardrail at the workflow level * Measure agent quality with built-in evaluation metrics ## Build Order Build the prompt tool first, then the agent that uses it, then the recipe, and the copilot last. A recipe can only use prompts and tools that already exist. | Step | You create | Contains | | ---- | ---------------------------------- | ----------------------------------------------------------------------------------------- | | 1 | Prompt: Spec-to-Gremlin Translator | Prompt only, used as a tool | | 2 | Agent: SpecMatch Agent | 4 tools: spec-to-gremlin, fittings-catalog-graph, fittings-catalog-docs, specmatch-memory | | 3 | Recipe: specmatch-workflow | Chat, Start, Usage Guardrail, SpecMatch Agent, End | | 4 | Copilot: SpecMatch | Chat page for end users | | 5 | End-to-end test | Five questions plus a guardrail test | | 6 | Evaluation | Score the agent against the SpecMatch Eval Set | ## Before You Start Your instructor has prepared the following so you can focus on building the agent: * Access to Karini AI at app.karini.ai * The KnowledgeGraph **fittings-catalog-knowledge-graph** , knowledge base **Fittings Catalog Documents** and the memory **SpecMatch User Memory**, already available in Karini * The prompt templates **Spec-to-Gremlin Translator** and **SpecMatch Agent**, available in Prompt Playground * A Neptune knowledge graph already built with the fittings catalog. Your instructor will provide the endpoint, role ARN, and region. Every template and resource name in this guide matches what is set up in Karini. Search for them exactly as written. Here is the schema for the Parts catalog KnowlegeGraph (**fittings-catalog-knowledge-graph)**
## Words You Will See * **Prompt Playground:** where you create agents and prompts, attach tools, and test them * **Prompt template:** a ready-made prompt stored in Karini. You load it with Use template instead of typing it. * **Tool:** something an agent can call, such as a prompt, a knowledge graph, a knowledge base, or memory * **Knowledge graph:** a database of things and how they connect, such as a part, its family, and its materials * **Gremlin:** the query language Amazon Neptune uses to walk a knowledge graph * **Primary and fallback model:** the model the prompt uses by default, and the one it switches to if the primary is unavailable * **Recipe:** the workflow that connects everything on a canvas * **Guardrail:** a filter that blocks requests outside usage guidelines before the agent sees them * **Copilot:** the chat page end users open to use the recipe * **Publish:** saves a version that recipes can use. Saved but unpublished changes are not used. * **Deploy:** makes a published recipe version live, so the copilot or webhook uses it. click Actions → Deploy after you publish. ## Step 1: Create the Spec-to-Gremlin Translator Prompt This prompt turns a fittings question into a Gremlin query for the knowledge graph. The SpecMatch Agent uses it as a tool. 1. From the left menu, open Prompt Playground. 2. Click Add new in the top right corner and configure: | Field | Value | | ----- | -------------------------- | | Name | Spec-to-Gremlin Translator | | Task | Summarization | 3. Click Prompt templates, Select **Name** from the dropdown next to the search field, search for Spec-to-Gremlin Translator, select it, and click Use template. 4. Check that the question variable appears under Variables, then click Save. ### Test & Compare 1. Open Spec-to-Gremlin Translator Agent prompt in Prompt playground. 2. Open the Test & Compare tab. 3. Select two models to compare: Claude Sonnet 5.5 and Claude Haiku 5.5. 4. Click Test. 5. Compare the outputs. Each should be a JSON query plan with a steps array that contains a Gremlin query. 6. Click Select as best answer on the best output to set that model as the Primary Model. 7. Optional: click Select as best answer on the other output to set it as the Fallback Model. 8. Click Save prompt run in the right corner. 9. Click Save, reopen prompt then Publish. Only published prompts can be selected as a prompt tool, so do not skip Publish. This prompt only writes a query, so a smaller, faster model like Haiku often performs as well as a larger one. Comparing them side by side is how you find out. ## Step 2: Create the SpecMatch Agent ### 2.1 Create the prompt 1. In Prompt Playground, click Add new and configure: | Field | Value | | ----------------- | --------------- | | Name | SpecMatch Agent | | Task | Agent | | Max State Updates | 30 | 2. Click Prompt templates, Select **Name** from the dropdown next to the search field, search for **SpecMatch Agent**, select it, and click Use template. 3. Check that Name, Task, and Max State Updates still have the values above. Set them again if they changed. 4. Click Save. 5. Click and Open SpecMatch Agent prompt. ### 2.2 Add tool 1: spec-to-gremlin 1. Open the Tools tab, click Add new, and select Agent tools. 2. Fill in: | Field | Value | | ------------ | ------------------------------ | | Name | spec-to-gremlin | | Description | \ | | Type | Prompt | | Prompt | Spec-to-Gremlin Translator | | Input Schema | \ | Description text: {% code overflow="wrap" %} ``` Turns a fittings question in plain language (ordering number, fitting type, tube size, end connection or material) into a query plan for the fittings-catalog-graph tool. Returns only a JSON query plan with a steps array. ``` {% endcode %} Input schema: {% code overflow="wrap" %} ```json [ { "name": "question", "type": "string", "description": "question" } ] ``` {% endcode %} 3. Click Save. ### 2.3 Add tool 2: fittings-catalog-graph 1. In the Tools tab, click Add new and select Agent tools. 2. Fill in: | Field | Value | | ------------------- | ------------------------------------------------------- | | Name | fittings-catalog-graph | | Description | \ | | Type | KnowledgeGraph | | Provider | Neptune | | Input Schema | \ | | Embeddings model | Amazon Titan Text Embeddings V2 | | Top K | 5 | | Neptune Credentials | Endpoint, Role ARN, and AWS region from your instructor | Description text: {% code overflow="wrap" %} ``` Structured fitting catalog graph. PART nodes (ordering number, fitting type, tube OD, end connection, dimensions, page) linked to PRODUCT_FAMILY, which links to available MATERIALs and the CATALOG. Use first for every parts question. ``` {% endcode %} Input schema (replace the whole default schema): {% code overflow="wrap" %} ```json { "$schema": "http://json-schema.org/schema#", "type": "object", "properties": { "steps": { "type": "array", "description": "Array of steps to execute in sequence to answer the user question using the Graph Database.", "items": { "oneOf": [ { "type": "object", "description": "Gremlin query step for graph traversal and data retrieval.", "properties": { "type": {"type": "string", "enum": ["gremlin"]}, "query": {"type": "string", "description": "The Gremlin traversal string (e.g., g.V().hasLabel('SPARE_PART')...). Do NOT use Neptune#fts side-effects."}, "parameters": {"type": "object", "description": "Key-value pairs for parameters used in the Gremlin query string.", "additionalProperties": true} }, "required": ["type", "query"] }, { "type": "object", "description": "Transformation step to process results between query steps (e.g., extracting IDs to use in the next query).", "properties": { "type": {"type": "string", "enum": ["transform"]}, "operation": {"type": "string", "enum": ["filter", "map", "group", "aggregate", "to_gremlin"], "description": "The type of data transformation to apply. 'to_gremlin' formats data for the next query."}, "mapping": {"type": "object", "description": "Field mapping configuration for the transformation.", "additionalProperties": true}, "config": {"type": "object", "description": "Additional configuration options.", "additionalProperties": true} }, "required": ["type", "operation"] } ] } } }, "required": ["steps"] } ``` {% endcode %} Replace the whole default input schema. With the default schema, every graph call fails with "Query plan is empty". 3. Click Test Neptune Connection, then Save. ### 2.4 Add tool 3: fittings-catalog-docs 1. In the Tools tab, click Add new and select Agent tools. 2. Fill in: | Field | Value | | -------------------- | ------------------------------ | | Name | fittings-catalog-docs | | Description | \ | | Type | Dataset | | Dataset | Fittings Catalog Documents | | Prompt context | Use embedding chunks | | Top-K | 5 | | Enable hybrid search | Enable Checkbox | Description text: {% code overflow="wrap" %} ``` Full text of the fittings parts catalog PDFs. Use it when the knowledge graph returns nothing, to confirm a part found in the graph, or for notes and details the graph lacks. ``` {% endcode %} 3. Click Save. ### 2.5 Add tool 4: specmatch-memory 1. In the Tools tab, click Add new and select Agent tools. 2. Fill in: | Field | Value | | -------------------------- | ------------------------------------------------------------------------------------------ | | Name | specmatch-memory | | Description | \ | | Type | Agent Memory | | Memory | Spec Match User Memory | | Read from memory :- Enable | Memory type :- Short-term, Memory scope Current user, Enable:-Limit results, Max results 5 | | Write to memory | Enable Checkbox | Description text: {% code overflow="wrap" lineNumbers="true" %} ``` Short-term memory for this user. Read it first to recall the parts, sizes and materials the user already asked about, so follow-up questions such as 'is it available in brass?' are answered in context. Write the confirmed ordering numbers and the user's stated requirements after each answer. ``` {% endcode %} 3. Click Save. The Tools tab now shows 4 tools: spec-to-gremlin, fittings-catalog-graph, fittings-catalog-docs, specmatch-memory. ### 2.6 Test & Compare 1. Open the Test & Compare tab. 2. Select two models to compare: Claude Sonnet 5.5 and Claude Haiku 5.5. 3. In Agent input, enter: `Show part SS-400-6 with its family and materials` 4. Click Test. Each model generates a query plan, runs it on the knowledge graph, and answers with the ordering number and a specifications table. 5. Compare the outputs and click Select as best answer to set the Primary Model. 6. Optional: click Select as best answer on the other output to set the Fallback Model. 7. Click Save prompt run in the right corner. 8. Click Save, Open SpecMatch Agent prompt then Publish. ## Step 3: Create the Karini Recipe with Bedrock AgentCore Runtime ### 3.1 Create the recipe 1. From the left menu, open Recipes and click Add new. 2. Set Recipe name = specmatch-workflow, Type = Workflow, Runtime = AgentCore. ### 3.2 Add the nodes Drag each element from Elements onto the canvas, then click it to fill in the right panel. | # | Element | Label | Settings in the right panel | | - | --------- | --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | Chat | Chat | **Conversation history** - No History | | 2 | Start | Start | **None** | | 3 | Guardrail | Usage Guardrail |

Guardrail = Bedrock-Guardrail.
1) Blocked message: \
2) Messages - on,
3) Message context - Last message

| | 4 | Agent | SpecMatch Agent |

Agent = SpecMatch Agent.
Document cache on, Metadata on

| | 5 | End | End | None | Guardrail blocked message: {% code overflow="wrap" %} ``` Your request violates usage guidelines. Please contact administrator if you think this is in error. ``` {% endcode %} When you select SpecMatch Agent, its 4 tools appear on the canvas automatically, linked to the agent node. {% embed url="" %} ### 3.3 Connect the nodes | From | To | | --------------- | --------------- | | Chat | Start | | Start | Usage Guardrail | | Usage Guardrail | SpecMatch Agent | | Usage Guardrail | End | | SpecMatch Agent | End | Usage Guardrail connects to End as well, so a blocked request ends the run without reaching the agent. Click Test guardrail on the Usage Guardrail node to check it. ### 3.4 Recipe settings, test, and deploy 1. Click an empty area of the canvas to open the recipe settings and set Number of state updates to 50. 2. Click Save and Test. test with question - `Show part SS-400-6 with its family and materials ?`\ Fix anything it reports, then click **Test** again until it passes. 3. Click Save, reopen **specmatch-workflow** recipe click **Publish**, then **Actions → Deploy**. ## Step 4: Create the SpecMatch Copilot Make sure specmatch-workflow is published and deployed before you start. 1. On the recipe page, click Actions in the top right corner and select Create Copilot. 2. On the Copilots page, click Add new and fill in: | Field | Value | | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Copilot name | SpecMatch | | Description | AI assistant that matches tube fitting specs to exact ordering numbers using the parts knowledge graph and catalog | | Seed questions |

Click on Add button and each question sepetately.

Question 1 :- 316 SS union for 1/4 in. tube
Question 2 :- Male connector, 1/8 in. tube to 1/8 NPT
Question 3 :- Is -400-6 available in brass?

| | Show traces | Enable Checkbox | | Other fields | Keep the defaults | 3. Click Save. ## Step 5: Test SpecMatch End to End Open the copilot as: * Open Copilots from the left menu and select SpecMatch Try each question: | Question | What to expect | | ------------------------------------------------ | --------------------------------------------------------------- | | Show part SS-400-6 with its family and materials | Answer from fittings-catalog-graph, with a specifications table | | 316 SS union for 1/4 in. tube | Exact ordering number from the graph | | Is -400-6 available in brass? | Yes or no, based on the materials linked to the part family | | Union tees for 3/8 inch tube | A list of matching parts | | What is the pressure rating of a 1/4 in. union? | Answer from fittings-catalog-docs | | A question that breaks usage guidelines | Usage Guardrail blocks it and shows the blocked message | With Show traces on, open the trace on any answer to see which tool the agent called and the Gremlin query it ran. ## Step 6: Evaluate SpecMatch 1. Download the **SpecMatch Eval Set** file from the shared Box folder provided by Instructor. 2. In Prompt Playground, open SpecMatch Agent and open the Evaluation tab. 3. Click Select prompt version and select the latest published version. 4. Click Upload dataset and upload the SpecMatch Eval Set file. The preview shows the ground truth, input, and metadata columns. 5. Under Evaluation Strategy, enable Default evaluation. This selects all eight metrics: answer relevancy, answer faithfulness, answer similarity, context sufficiency, guideline adherence, tool accuracy, goal achievement, and execution efficiency. 6. Click Run. 7. Wait until Last evaluation status shows Completed. 8. Click View last evaluation result to see the score for each metric and each question. ## Troubleshooting | Problem | Likely cause | Fix | | ------------------------------------------------- | -------------------------------------------------------- | ----------------------------------------------------------------------- | | Prompt box empty after Add new | Template not loaded | Prompt templates → search the template name → Use template → Save | | Template not found | Name typed differently from the guide | Search Spec-to-Gremlin Translator or SpecMatch Agent exactly as written | | spec-to-gremlin can't select its prompt | Spec-to-Gremlin Translator saved but not published | Open it in Prompt Playground and click Publish | | Every graph call fails with "Query plan is empty" | Default input schema kept on fittings-catalog-graph | Replace the whole input schema with the one in 2.3 | | Neptune connection test fails | Wrong endpoint, role ARN, or region | Re-enter the values from your instructor | | Agent says it cannot find a tool | Tool name changed from the guide | Use the tool names exactly as written in Step 2 | | Pressure rating question returns nothing | fittings-catalog-docs missing, or hybrid search off | Check fittings-catalog-docs: Top-K 5, hybrid search on | | Follow-up questions lose context | Memory read or write disabled | Turn on both Read from memory and Write to memory in specmatch-memory | | Guardrail never blocks | Usage Guardrail not connected, or no guardrail selected | Connect Start → Usage Guardrail and select Bedrock Guardrail | | Recipe changes not visible in the copilot | specmatch-workflow not published and deployed | Publish, then Actions → Deploy | | Recipe Test fails | A node has no agent selected, or a connection is missing | Fix the node it names, then click Test again |