# Model Monitoring

Model Monitoring provides a multi-dimensional metrics dashboard and per-model in-depth analysis for your model API calls, helping you monitor call health, performance, and Token usage in real time.

Model Monitoring includes two main features:

- **Monitoring Overview**: Displays summary metrics and a model list for all models, giving a quick overview of overall call status.
- **Monitoring Detail**: Provides in-depth multi-chart analysis for a single model, organized into Statistics and Performance tabs.

> Monitoring data is stored in audit logs. Data ingestion and display may have a slight delay. Please refresh the page if data appears outdated.

## Monitoring Overview

Navigate to **Model Service Platform > Model Monitoring**. The page loads data for the **last 3 hours** by default.

### Filters

- **Time Range** (required): Supports quick presets and custom time selection.
- **Region** (required): Select an inference region. The first region in the list is selected by default.

> The overview page automatically uses all API Keys under your account. API Key filtering is not available on this page.

### Summary Metrics

The page displays 5 summary metric cards. Hover over any card to see a detailed description. A manual refresh button is available on the right.

| Metric | Description |
|--------|-------------|
| Model Count | Total number of models with calls in the selected time range |
| Total Calls | Total number of calls across all models |
| Total Failures | Total number of failed calls across all models |
| Avg Latency | The average call duration across all model API calls in the selected time range |
| Avg First Token Latency | The average first Token latency across all streaming model API calls in the selected time range |

### Model List

The model list displays per-model call statistics in a table. Use the search box in the top-right corner to filter by model name.

| Column | Description |
|--------|-------------|
| Model Name | Model identifier |
| Call Count | Total calls for this model in the selected time range |
| Failure Count | Number of failed calls for this model |
| Failure Rate | Percentage display = Failure Count / Call Count |
| Avg Latency | Average call duration for this model. Automatically switches to seconds when ≥ 1000ms |
| Avg First Token Latency | Average first Token latency for this model. Automatically switches to seconds when ≥ 1000ms |
| Action | Click "Monitor" to view detailed monitoring for this model |

## Model Monitoring Detail

In the monitoring overview, click the "Monitor" link in the target model's row to access its detail page.

> When clicking "Monitor", the system passes the currently selected time range from the overview page to the detail page as the initial filter.

### Filters

The detail page provides the following filters. Data refreshes automatically when filters are changed:

- **Time Range**: Supports quick presets and custom time selection. The display format adapts to the aggregation granularity — minute granularity shows HH:mm, hour granularity shows HH, and day granularity shows only the date. The maximum supported time range is 60 days.
> Aggregation granularity options are automatically disabled based on the selected time range to avoid overly dense or sparse data points. See the table below for the full rules.

  | Time Range | Available Granularities |
  |------------|------------------------|
  | ≤ 12 hours | Minute, 5-Minute, Hour (Day disabled) |
  | 12 hours ~ 24 hours | 5-Minute, Hour (Minute, Day disabled) |
  | 24 hours ~ 7 days | Hour, Day (Minute, 5-Minute disabled) |
  | > 7 days | Day only |

- **API Key**: Select "All API Keys" or a specific API Key. Defaults to all.
- **Region**: Select an inference region.
- **Request Mode**: Filter by request type — All, Streaming, or Non-Streaming.

### Statistics

The Statistics tab contains the following metrics, with the following layout:

1. Call Count (full width)
2. Status Code Distribution (full width)
3. Failure Count + Failure Rate (side by side)
4. Rate Limit Errors (full width)
5. Token Usage (full width)
6. Avg Token Usage Per Request (full width)
7. Cache Hit Rate (full width)

| Metric | Description |
|---|---|
| Call Count | Total number of requests received by the model API within the selected time range. |
| Status Code Distribution | Request count distribution for each HTTP status code within the selected time range. |
| Failure Count | Number of requests that did not complete successfully within the selected time range. |
| Failure Rate | Proportion of failed requests out of total requests within the selected time range. |
| Rate Limit Error Count | Number of requests that failed due to rate limiting (status code 429) within the selected time range. |
| Token Usage | Total tokens consumed by model API requests within the selected time range, including total tokens, input tokens, output tokens, and cache hit tokens. (Input tokens include cache hit tokens.) |
| Avg Token Usage Per Request | Average tokens consumed per model API request within the selected time range, including total tokens, input tokens, output tokens, and cache hit tokens. (Input tokens include cache hit tokens.) |
| Cache Hit Rate | Ratio of cache-hit input tokens to total input tokens within the selected time range. |
| Performance Metrics | RPM | Number of requests received by the model API per minute. |
| Performance Metrics | TPM | Number of tokens processed by the model API per minute, including total tokens, input tokens, output tokens, and cache hit tokens. (Input tokens include cache hit tokens.) |
| Performance Metrics | Total Latency | Total elapsed time from when the gateway receives a request to when the complete response is returned. |
| Performance Metrics | First Token Latency (TTFT) | Time from when the gateway receives a request to when the first output token is returned. Only collected for successful streaming requests that support first-token measurement. |
| Performance Metrics | Token Output Speed (TPS) | Average token generation speed during the output phase of a single request, calculated as output token count divided by output duration, in tokens/s. Only collected for successful requests that support token output speed measurement. |

### Performance

The Performance tab contains the following metrics, with the following layout:

1. RPM + TPM (side by side)
2. Total Latency (full width)
3. First Token Latency + Token Output Speed (side by side)

| Metric | Description |
|---|---|
| RPM | Number of requests received by the model API per minute. |
| TPM | Number of tokens processed by the model API per minute, including total tokens, input tokens, output tokens, and cache hit tokens. (Input tokens include cache hit tokens.) |
| Total Latency | Total elapsed time from when the gateway receives a request to when the complete response is returned. |
| First Token Latency (TTFT) | Time from when the gateway receives a request to when the first output token is returned. Only collected for successful streaming requests that support first-token measurement. |
| Token Output Speed (TPS) | Average token generation speed during the output phase of a single request, calculated as output token count divided by output duration, in tokens/s. Only collected for successful requests that support token output speed measurement. |

> **Percentile Explanation**: P50 is the median (50% of requests are below this value). P95 means 95% of requests are below this value, and P99 means 99% are below this value. Higher percentile values indicate greater tail latency.

### Chart Actions

Each chart card provides the following action buttons in the top-right corner:

- **Fullscreen**: Click to enlarge the chart in an overlay. Press ESC to exit.
- **Download PNG**: Export the chart as a 2x resolution PNG image.
- **Copy to Clipboard**: Copy the chart as an image to the system clipboard.
- **Retry**: Appears when a chart fails to load. Click to re-fetch data for that specific chart.

While loading, charts display a skeleton animation (pulsing line placeholders) instead of a blank state.

## Prerequisites

- Your account has created API Keys and generated model call data.
- Monitoring data is stored in audit logs. Select the correct region.

## FAQ

### Why are some charts not showing data?

Possible reasons include:

- No call records exist in the selected time range;
- The selected API Key does not have permission for the target model;
- The selected region is incorrect.

Check your filters and try again.

### Why are some aggregation granularity options grayed out?

Aggregation granularity options are automatically disabled based on the selected time range to avoid overly dense or sparse data points. For example, when the time range exceeds 7 days, only daily aggregation is available. See the Aggregation Granularity table in the Filters section above for the full rules.

### Why can't I select a time range longer than 60 days?

The system limits each query to a maximum of 60 days. To view data over a longer period, please query in separate segments.