# Gemini Media Analysis and Upload Files to GCS This document describes how to use the ModelVerse gateway to call Gemini’s media analysis capabilities and upload files to GCS. Supported inputs include images, videos, and audio. [Gemini Official Documentation](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/multimodal/image-understanding) --- ## Authentication All APIs support either of the following methods to pass the API Key (placeholders are used in examples): - `Authorization: Bearer ` - `x-goog-api-key: ` Base URL example: `https://api-us-ca.umodelverse.ai` --- ## Workflow Overview 1. Upload media files to the GCS platform: `POST /v1/files` 2. Retrieve the file URL (`s3_url`) from the upload response 3. Call the Gemini media analysis API: `POST /v1beta/models/{model}:generateContent` 4. Extract the text analysis result from the response --- ## 1. Upload Media File Upload an image, video, or audio file to the platform and obtain an `s3_url` for subsequent requests. **Request** - **Method / Path**: `POST /v1/files` - **Content-Type**: `multipart/form-data` - **Form Fields**: - `file`: media file (binary) - `purpose`: required; (`batch:gcs`) **Example** ```bash curl https://api-us-ca.umodelverse.ai/v1/files \ -H "Authorization: Bearer <$MODELVERSE_API_KEY>" \ -F purpose="batch:gcs" \ -F "file=@gemini-3.1-pro-preview.jpg" ``` **Successful Response Example** ```json { "id": "1773236148797418904_demo-image.jpg", "object": "file", "bytes": 224123, "created_at": 1773236150, "expires_at": 0, "filename": "1773236148797418904_demo-image.jpg", "purpose": "", "s3_url": "https://example-file-host/modelverse/1773236148797418904_demo-image.jpg" } ``` Subsequent media analysis requests can use `s3_url` as `file_data.file_uri`. **Error Response Example** ```json { "error": { "message": "this is error meesage", "type": "internal_error", "param": "e4b4cf3f-24a0-4620-b53e-fee919c9d914", "code": "model_server_error" } } ``` --- ## 2. Call Gemini Media Analysis API **Request** - **Method / Path**: `POST /v1beta/models/{model}:generateContent` - **Content-Type**: `application/json` Common fields in `contents[].parts[]`: | Field | Type | Description | | --- | --- | --- | | `text` | string | Question or instruction text | | `file_data.mime_type` | string | Media MIME type, e.g., `image/jpeg`, `video/mp4`, `audio/mpeg` | | `file_data.file_uri` | string | Media URL, recommended to use the `s3_url` returned by the upload API | --- ## 3. Example: Text Analysis ```bash IMAGE_URL="https://example-file-host/modelverse/1773236148797418904_demo-image.jpg" curl -X POST "https://api-us-ca.umodelverse.ai/v1beta/models/gemini-3-flash-preview:generateContent" \ -H "x-goog-api-key: $MODELVERSE_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "contents": [ { "parts": [ { "fileData": { "fileUri": "https://storage.googleapis.com/gemini-batch-001/1774000031757082682_gemini-3-flash-preview.jsonl?Expires=1774604833&GoogleAccessId=bucket%40stacyzhang0708-1218-01.iam.gserviceaccount.com&Signature=c4QsHdf89aWeTNX0JzVqT%2BL4VtM2%2Bx7PEpjjNSVU%2FBNjrvJB2kKbpbIWmKvexQvOA4BO9no0KbZxazCvAQbNttK4Ecng7Za4K4FslNLBJ7fucapF7xdE0b%2BSO690ph%2BXWX5erZJixmzV%2BhiGZay0mBdYAxOfVwZb%2Bc5RA%2FQzPqBsdbdKb1%2FzzD4AcwCHu0w0So54JMQt3qcDQVi56NZaOZA4aC9SLZGcVdYakMF75XKn9Z99e2rXgFtEo2P7%2BuRBY8hD%2Bv%2B6dHC9k6fa9nABZEw38wi3Ozfsf7eQ0PmTmrQ2ue6n97zMjbg6rSn2RmKFWc4iWBl48ikkyFmA1GhKsA%3D%3D", "mimeType": "text/plain" } }, { "text": "What is in the text?" } ], "role": "user" } ] } ' ``` --- ## 4. Example: Video Analysis ```bash VIDEO_URL="https://example-file-host/modelverse/1773236148797418904_demo-video.mp4" curl -X POST "https://api-us-ca.umodelverse.ai/v1beta/models/gemini-3-flash-preview:generateContent" \ -H "Authorization: Bearer $MODELVERSE_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "contents": [ { "role": "user", "parts": [ { "file_data": { "mime_type": "video/mp4", "file_uri": "'"$VIDEO_URL"'" } }, { "text": "Please summarize the main events of this video and list key segments in chronological order." } ] } ] }' ``` --- ## 5. Example: Audio Analysis ```bash AUDIO_URL="https://example-file-host/modelverse/1773236148797418904_demo-audio.mp3" curl -X POST "https://api-us-ca.umodelverse.ai/v1beta/models/gemini-3-flash-preview:generateContent" \ -H "x-goog-api-key: $MODELVERSE_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "contents": [ { "role": "user", "parts": [ { "file_data": { "mime_type": "audio/mpeg", "file_uri": "'"$AUDIO_URL"'" } }, { "text": "Please transcribe this audio and summarize its key points." } ] } ] }' ``` --- ## 6. SDK Python The following example uses Python to: 1. Upload a media file to `/v1/files` 2. Use the returned `s3_url` to call `/v1beta/models/{model}:generateContent` ```python from google import genai from google.genai.types import HttpOptions,Part client = genai.Client( http_options=HttpOptions( api_version="v1beta", base_url="https://api-us-ca.umodelverse.ai", ), api_key='<$MODELVERSE_API_KEY>', ) prompt = "Describe the key events in this video, providing both audio and visual details. Include timestamps for salient moments." response = client.models.generate_content( model="gemini-3-flash-preview", contents=[ Part.from_uri( file_uri="https://storage.googleapis.com/gemini-batch-001/1774000031757082682_gemini-3-flash-preview.jsonl?Expires=1774604833&GoogleAccessId=bucket%40stacyzhang0708-1218-01.iam.gserviceaccount.com&Signature=c4QsHdf89aWeTNX0JzVqT%2BL4VtM2%2Bx7PEpjjNSVU%2FBNjrvJB2kKbpbIWmKvexQvOA4BO9no0KbZxazCvAQbNttK4Ecng7Za4K4FslNLBJ7fucapF7xdE0b%2BSO690ph%2BXWX5erZJixmzV%2BhiGZay0mBdYAxOfVwZb%2Bc5RA%2FQzPqBsdbdKb1%2FzzD4AcwCHu0w0So54JMQt3qcDQVi56NZaOZA4aC9SLZGcVdYakMF75XKn9Z99e2rXgFtEo2P7%2BuRBY8hD%2Bv%2B6dHC9k6fa9nABZEw38wi3Ozfsf7eQ0PmTmrQ2ue6n97zMjbg6rSn2RmKFWc4iWBl48ikkyFmA1GhKsA%3D%3D", mime_type="text/plain", ), "What is in the text?", ], ) print(response.text) ``` --- --- ## 7. Common Errors and Troubleshooting - `400 INVALID_ARGUMENT`: `mime_type` does not match the actual file, or request fields are misspelled - `400 FAILED_PRECONDITION`: Media URL is not accessible, or the file is not yet readable - `401 UNAUTHENTICATED`: API Key is invalid, missing, or incorrectly formatted - `403 PERMISSION_DENIED`: The current key does not have permission to call the model - `429 RESOURCE_EXHAUSTED`: Rate limit exceeded; please retry with backoff Recommended checks: 1. Ensure the API Key placeholder is replaced with a real value at runtime 2. Verify that `file_data.file_uri` is accessible by the server 3. Confirm that `mime_type` matches the media format 4. Ensure the `model` is a valid and available Gemini model ID