Test Agentic Recipe
Last updated
Last updated
A recipe can be tested by bringing and configuring the element into the recipe canvas (see ).
Click the Test button to open a chat window, allowing interaction through queries. Submit your question review the response generated by the recipe. The response includes the following:
You can see the real-time response from your recipe pipeline that includes the answer to the question, prompt lens icon, a trace icon, and statistics. If the model selected in the prompt for the recipe supports streaming, you will see a streaming response.
Prompt lens let you peek behind the scenes as the request is being executed. Here, you can inspect the input we are sending to the language models (LLMs) - including system instructions, questions, and prompt. This empowers you to analyze the quality of your retrieved context from the vector store and make necessary adjustments to the strategy if needed.
You can view the streaming in the prompt lens. Once the response in the prompt lens is completed, it auto-refreshes, and then the answer is displayed in the chat widget.
You can view the information in the prompt lens after the request is processed.
Agent scratch pad: The scratch pad aids in refining prompts, documenting interactions, or brainstorming ideas based on the outputs received from the selected models.
Agent response: The agent response refers to the output or action taken by the selected model in response to a user's prompt or query.
Tool response: It gives the insights or summaries related to the tools used, performance metrics, or operational status.
Trace has two sections as Prompt and Attributes.
Prompt: You can view the traces of each operation executed during the processing . It includes the following:
Input
Output
Attributes: These include various parameters and metrics associated with each request. Some of the attributes include:
Input Tokens
Completion tokens
Model parameters such as temperature, max tokens etc.
You can view the following statistics when the response is generated after a test.
LLM Response Time: The amount of time in milliseconds taken by the LLM to generate complete response for the given prompt request.
LLM Request Timestamp: Represents the specific time a request was made to the Language Learning Model (LLM).
Time to First Token: The time that it takes for the model to produce the first token of the response after receiving the prompt. TTFT is particularly relevant for applications utilizing streaming, where providing immediate feedback is crucial.
Input Tokens: Total number of input tokens in the LLM request. This includes the prompt instructions, system prompt, context and user query.
Output Tokens: Total number of output tokens generated by the LLM in response to the prompt request. This number does not exceed the Max Tokens value setup during the prompt testing.
Input Unsafety Score: It measures the potential risk or danger associated with a given input. A higher score indicates a greater level of unsafety.
Input Toxicity Score: This score represents the likelihood that the input text could be perceived as toxic or harmful.
To export the recipe and deploy copilot, please refer to the detailed instructions provided in section. This section includes step-by-step guidelines that will guide you through the entire process, ensuring accuracy and efficiency.
To explore the various features and functionalities offered by Copilot, including its capabilities, settings, and customization options, please refer to the detailed section titled .