# OpenAI Batch API

This document describes how to use the **OpenAI-compatible Batch API** through ModelVerse. It is designed for processing large numbers of requests asynchronously and at lower cost, such as batch chat completions. The overall workflow is consistent with the official OpenAI and Azure OpenAI Batch APIs.

> **You can view the models that support batch processing in the Model Marketplace.**

The complete batch workflow consists of five steps:

```
1. Upload a file → 2. Check the file status → 3. Create a batch → 4. Poll the batch status → 5. Download the results
```

---

## Authentication

Pass your API key in the HTTP header for every endpoint. Do not place a real key in documentation or source code; use an environment variable or placeholder instead.

- `Authorization: Bearer <your_api_key>`

Base URL example: `https://api.modelverse.cn` (use `https://api.umodelverse.ai` outside China).

> When a file is uploaded, the service **automatically reads `body.model` from the first line of the JSONL input file**. It then routes the file and subsequent batch to a provider that supports that model. You do not need to specify a provider when uploading a file or creating a batch.

---

## Input File Format

The input must be a **JSONL** file containing one JSON object per line. Each line represents an independent request.

| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `custom_id` | string | Yes | A unique custom request identifier used to match input requests with result records |
| `method` | string | Yes | Request method; must be `POST` |
| `url` | string | Yes | Target endpoint, such as `/v1/chat/completions` |
| `body` | object | Yes | Request body; `model` determines the model and provider routing |

**Example (`batch_input.jsonl`)**

```jsonl
{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-5-batch", "messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Describe Hangzhou in one sentence."}]}}
{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-5-batch", "messages": [{"role": "user", "content": "What is 1 plus 1?"}]}}
```

> All requests in the same input file should use the same `model`. The service uses the model in the first line to route the entire batch.

---

## 1. Upload the Input File

Upload the JSONL input file and obtain a `file_id` for creating the batch.

**Request**

- **Method / Path**: `POST /v1/files`
- **Content-Type**: `multipart/form-data`
- **Form fields**:
  - `purpose`: Must be `batch`.
  - `file`: The JSONL input file.

**Response fields**

| Field | Type | Description |
| --- | --- | --- |
| `id` | string | File ID used in steps 2 and 3 |
| `object` | string | Always `file` |
| `bytes` | int64 | File size in bytes |
| `created_at` | int64 | Creation timestamp |
| `filename` | string | File name |
| `purpose` | string | The submitted value, `batch` |
| `status` | string | File status, such as `processed` |

**Example**

```bash
curl -X POST "https://api.umodelverse.ai/v1/files" \
  -H "Authorization: Bearer $MODELVERSE_API_KEY" \
  -F "purpose=batch" \
  -F "file=@batch_input.jsonl"
```

**Response**

```json
{
  "id": "file-abc123",
  "object": "file",
  "bytes": 512,
  "created_at": 1781600000,
  "expires_at": 0,
  "filename": "batch_input.jsonl",
  "purpose": "batch",
  "status": "processed"
}
```

---

## 2. Check the File Status

> **Note:** The file-status endpoint currently supports only GPT-series models, such as `gpt-5-batch`. Batch files for other models cannot be queried through this endpoint.

After uploading the file, check its status and wait until it is `processed` before creating a batch.

**Request**

- **Method / Path**: `GET /v1/files/{file_id}`
- **Path parameter**: `file_id` is the `id` returned in step 1.

**Main response fields**

| Field | Type | Description |
| --- | --- | --- |
| `id` | string | File ID |
| `status` | string | File status; `processed` means the file is ready |
| `bytes` | int64 | File size |
| `purpose` | string | `batch` |

**Example**

```bash
curl -X GET "https://api.umodelverse.ai/v1/files/file-abc123" \
  -H "Authorization: Bearer $MODELVERSE_API_KEY"
```

**Response**

```json
{
  "id": "file-abc123",
  "object": "file",
  "bytes": 512,
  "created_at": 1781600000,
  "filename": "batch_input.jsonl",
  "purpose": "batch",
  "status": "processed"
}
```

---

## 3. Create a Batch

Create a batch using the uploaded input file.

**Request**

- **Method / Path**: `POST /v1/batches`
- **Content-Type**: `application/json`

**Request body**

| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `input_file_id` | string | Yes | File ID returned in step 1 |
| `endpoint` | string | Yes | Target endpoint, such as `/v1/chat/completions` |
| `completion_window` | string | Yes | Completion window; must be `24h` |
| `metadata` | object | No | Custom metadata with string keys and values |

**Main response fields**

| Field | Type | Description |
| --- | --- | --- |
| `id` | string | Batch ID used for polling in step 4 |
| `object` | string | Always `batch` |
| `status` | string | Batch status; see the status table below |
| `input_file_id` | string | Input file ID |
| `output_file_id` | string | Result file ID, available after completion and used in step 5 |
| `error_file_id` | string | Error details file ID, present when some requests fail |
| `request_counts` | object | Request counts: `total`, `completed`, and `failed` |

**Example**

```bash
curl -X POST "https://api.umodelverse.ai/v1/batches" \
  -H "Authorization: Bearer $MODELVERSE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file-abc123",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h",
    "metadata": {"job": "demo-batch"}
  }'
```

**Response**

```json
{
  "id": "batch_abc123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "errors": null,
  "input_file_id": "file-abc123",
  "completion_window": "24h",
  "status": "validating",
  "created_at": 1781600010,
  "request_counts": {"total": 0, "completed": 0, "failed": 0},
  "metadata": {"job": "demo-batch"}
}
```

---

## 4. Poll the Batch Status

Poll the batch until it reaches a terminal state: `completed`, `failed`, `expired`, or `cancelled`.

**Request**

- **Method / Path**: `GET /v1/batches/{batch_id}`
- **Path parameter**: `batch_id` is the `id` returned in step 3.

**Batch statuses**

| Status | Meaning |
| --- | --- |
| `validating` | The input file is being validated before processing begins |
| `in_progress` | Validation passed and the batch is being processed |
| `finalizing` | Processing is complete and the result file is being prepared |
| `completed` | Processing is complete and the result file is ready |
| `failed` | Input validation or batch processing failed |
| `expired` | The batch did not finish within the 24-hour window |
| `cancelling` | Cancellation is in progress and may take up to approximately 10 minutes |
| `cancelled` | The batch was cancelled |

When `status` becomes `completed`, use `output_file_id` for successful results and `error_file_id`, if present, for failure details in step 5.

**Example**

```bash
curl -X GET "https://api.umodelverse.ai/v1/batches/batch_abc123" \
  -H "Authorization: Bearer $MODELVERSE_API_KEY"
```

**Response**

```json
{
  "id": "batch_abc123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "errors": null,
  "input_file_id": "file-abc123",
  "completion_window": "24h",
  "status": "completed",
  "output_file_id": "file-out456",
  "error_file_id": "",
  "created_at": 1781600010,
  "in_progress_at": 1781600030,
  "finalizing_at": 1781600300,
  "completed_at": 1781600360,
  "request_counts": {"total": 2, "completed": 2, "failed": 0}
}
```

> Use a polling interval of at least 30 seconds to avoid sending requests too frequently.

---

## 5. Download the Results

After completion, use `output_file_id` for results or `error_file_id` for failure details to download the corresponding JSONL file.

**Request**

- **Method / Path**: `GET /v1/files/{file_id}/content`
- **Path parameter**: `file_id` is the `output_file_id` or `error_file_id` returned in step 4.

**Response**

- On success, the endpoint returns a JSONL file stream. `Content-Type` is typically `application/octet-stream` or `application/jsonl`.
- Each line corresponds to one input request and is associated through `custom_id`.

**Example**

```bash
curl -X GET "https://api.umodelverse.ai/v1/files/file-out456/content" \
  -H "Authorization: Bearer $MODELVERSE_API_KEY" \
  -o batch_output.jsonl
```

**Result file example (one record per line)**

```jsonl
{"id": "batch_req_1", "custom_id": "request-1", "response": {"status_code": 200, "request_id": "req_aaa", "body": {"id": "chatcmpl-1", "object": "chat.completion", "model": "gpt-5-batch", "choices": [{"index": 0, "finish_reason": "stop", "message": {"role": "assistant", "content": "Hangzhou, the capital of Zhejiang Province, is known for West Lake and its thriving digital economy."}}], "usage": {"prompt_tokens": 30, "completion_tokens": 18, "total_tokens": 48}}}, "error": null}
{"id": "batch_req_2", "custom_id": "request-2", "response": {"status_code": 200, "request_id": "req_bbb", "body": {"id": "chatcmpl-2", "object": "chat.completion", "model": "gpt-5-batch", "choices": [{"index": 0, "finish_reason": "stop", "message": {"role": "assistant", "content": "1 plus 1 equals 2."}}], "usage": {"prompt_tokens": 12, "completion_tokens": 6, "total_tokens": 18}}}, "error": null}
```

---

## Summary

| Step | Endpoint | Description |
| --- | --- | --- |
| 1. Upload the file | `POST /v1/files` | Upload JSONL with `purpose=batch`; returns `file_id`; the model is detected from the first line |
| 2. Check file status | `GET /v1/files/{file_id}` | Confirm that `status=processed` |
| 3. Create the batch | `POST /v1/batches` | Provide `input_file_id`, `endpoint`, and `completion_window` |
| 4. Poll batch status | `GET /v1/batches/{batch_id}` | Poll until `completed`, then obtain `output_file_id` |
| 5. Download results | `GET /v1/files/{file_id}/content` | Download the result JSONL using `output_file_id` |