Spend by Batch Recipes
Cost and usage details for the Batch recipes can be visualized using the following dashboards: Total Cost, Total API Requests, and Total API Tokens. These dashboards provide valuable insights into the operational metrics and usage patterns of your copilots, helping you to understand the usage patterns, identify the cost drivers and manage and optimize their cost effectively.
Total cost
You can visualize total cost of the Batch recipes within your Organization. You have an option to chose all, or a set of specific Batch recipes from the dropdown of available Batch recipes. You can also select appropriate date range and granularity of daily or monthly.
For each Batch recipe, the cost is determined by summing up the costs of all the model endpoints involved in processing the batch request. This calculation factors in the respective model prices set in the model hub and the number of Input/Output tokens for each endpoint.
By default, when no batch recipes selected from the dropdown, the graph displays the costs associated with the Top 5 batch recipes, with expenses for all other batch recipes consolidated under Others.
The right-hand side features a color-coded legend, identifying each of the Top 5 batch recipes and Others for easy reference.
When you select Batch recipe(s) from the dropdown, the cost of the selected batch recipe(s) is displayed, accompanied by a color-coded legend on the right-hand side of the graph.
You can adjust the date filter or granularity, the chart dynamically refreshes to display the corresponding data.
Total API Requests
This graph displays the total number of API requests processed by the Batch recipe within the selected date range.
By default, when no batch recipe is selected from the dropdown, the graph displays the Total API Requests associated with All batch recipes.
When you select Batch recipe(s) from the dropdown, the Total API Requests of the selected batch recipe(s) is displayed.
This visualization helps in tracking the volume of requests and observing trends or patterns over the selected time frame.
Total tokens
This graph shows the total number of tokens consumed (Input tokens) and generated (Output tokens) by all the model endpoint(s) when processing batch recipe requests during the selected timeframe.
By default, when no batch recipe is selected from the dropdown, the graph displays the Total tokens associated with All batch recipes.
When you select Batch recipe(s) from the dropdown, the Total tokens of the selected batch recipe(s) is displayed.
Last updated
