Model Endpoints Dashboard

You can view granular cost and usage dashboards for each registered model endpoint on the Model Endpoint page.

Click on View model link of the desired model endpoint to explore the associated dashboards.

Below are the particulars for comprehending the model endpoints dashboard for LLM endpoints as well as embedding model endpoints.

1. Data can be viewed and analyzed at both daily and hourly levels of detail. The default level of granularity is set to hourly.

2. The dashboard is set to showcase data ranging from the previous day's date to the current date, with hourly granularity.

3. When selecting Hourly Granularity, select a date range that does not exceed 48 hours. Your start and end dates must fall within this 2-day period.

4. When selecting Daily Granularity, the date range cannot exceed 30 days. Ensure that your selected dates are no more than a month apart.

5. When a date filter or granularity is changed, all the charts update with the appropriate values dynamically.

6. When you click on the Clear filter option, the selected date range, granularity reset to their default values.

Total cost

This dashboard provides a daily or hourly breakdown of the costs associated with the selected model endpoint for the specified time period.

The model price set in the model hub, and the number of Input/Output tokens are used to calculate the endpoint cost for the selected date range.

Total API Requests

This dashboard illustrates the total number of API requests served by the model endpoint within the selected timeframe.

Total Tokens

This dashboard illustrates the count of input and output tokens processed by the model endpoint over the selected timeframe.