# Gemini-Embedding-2

Gemini-Embedding-2 is a multimodal embedding model from Google that supports vector embeddings for text, images, video, audio, and documents.

## What is Gemini-Embedding-2

Gemini-Embedding-2 converts text, images, video, audio, and PDF documents into high-dimensional vector representations. These embeddings can be used for:

- **Multimodal search** — Semantic search across text, images, video, and audio
- **Content recommendation** — Similarity-based recommendations across modalities
- **Clustering** — Grouping mixed-type content
- **Anomaly detection** — Identifying unrelated or anomalous content

## Quick Start

### Request Example

```bash
curl --location --globoff 'https://api-us-ca.umodelverse.ai/v1beta/models/gemini-embedding-2:embedContent' \
--header 'Authorization: Bearer YOUR_MODELVERSE_API_KEY' \
--header 'Content-Type: application/json' \
-d '{
  "content": {
    "parts": [
      {
        "text": "Whats this"
      },
      {
        "text": "Whats this like"
      },
      {
        "file_data": {
          "mime_type": "video/mp4",
          "file_uri": "https://example.com/video.mp4"
        }
      },
      {
        "file_data": {
          "mime_type": "image/png",
          "file_uri": "https://example.com/image.png"
        }
      },
      {
        "file_data": {
          "mime_type": "audio/wav",
          "file_uri": "https://example.com/audio.wav"
        }
      },
      {
        "file_data": {
          "mime_type": "application/pdf",
          "file_uri": "https://example.com/document.pdf"
        }
      }
    ]
  }
}'
```

### Response Example

```json
{
  "embedding": {
    "values": [
      0.0015015427,
      0.014678672,
      -0.018105509,
      -0.013457718,
      0.015227248,
      0.003112343
    ]
  },
  "usageMetadata": {
    "promptTokenCount": 2517,
    "totalTokenCount": 2517,
    "promptTokensDetails": [
      {
        "modality": "VIDEO",
        "tokenCount": 330
      },
      {
        "modality": "TEXT",
        "tokenCount": 5
      },
      {
        "modality": "DOCUMENT",
        "tokenCount": 1548
      },
      {
        "modality": "IMAGE",
        "tokenCount": 258
      },
      {
        "modality": "AUDIO",
        "tokenCount": 376
      }
    ]
  }
}
```

## API Reference

**POST** `https://api-us-ca.umodelverse.ai/v1beta/models/gemini-embedding-2:embedContent`

### Request Headers

| Parameter | Description |
|-----------|-------------|
| Authorization | Bearer YOUR_MODELVERSE_API_KEY |
| Content-Type | application/json |

### Request Body Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| content | object | Yes | Contains the content to embed |
| content.parts | array | Yes | Array of content parts, supports multiple modalities |
| outputDimensionality | integer | No | Output embedding dimension. Range: 128 to 3072. Default: 3072 |

### Output Dimensionality

Gemini-Embedding-2 supports configurable output dimensions:

| Model | Default (Max) | Supported Range | Recommended Lower Values |
|-------|---------------|-----------------|--------------------------|
| gemini-embedding-2 | 3072 | 128 to 3072 | 128, 768, or 1536 |

**Guidance**:
- Lower dimensions (e.g. 768 or 1536) reduce storage and computation costs
- Slightly lower semantic search accuracy, but negligible difference in most use cases
- Use the default 3072 for highest accuracy

### content.parts Types

#### Text Part

```json
{
  "text": "text content to embed"
}
```

#### File Part

```json
{
  "file_data": {
    "mime_type": "video/mp4",
    "file_uri": "https://example.com/video.mp4"
  }
}
```

Supported MIME types:

| Type | MIME Types |
|------|-----------|
| Video | video/mp4, video/mpeg, video/mov, video/avi, etc. |
| Image | image/png, image/jpeg, image/webp, image/gif, etc. |
| Audio | audio/wav, audio/mp3, audio/mpeg, audio/ogg, etc. |
| Document | application/pdf |

### Response Fields

| Field | Type | Description |
|-------|------|-------------|
| embedding.values | array | Embedding vector (list of floats) |
| usageMetadata.promptTokenCount | integer | Total token count for input |
| usageMetadata.totalTokenCount | integer | Total tokens |
| usageMetadata.promptTokensDetails | array | Per-modality token usage details |
| promptTokensDetails.modality | string | Modality type: TEXT, IMAGE, VIDEO, AUDIO, DOCUMENT |
| promptTokensDetails.tokenCount | integer | Token count for this modality |

## Use Cases

### 1. Multimodal Semantic Search

Combine text and images for semantic search:

```bash
curl --location --globoff 'https://api-us-ca.umodelverse.ai/v1beta/models/gemini-embedding-2:embedContent' \
--header 'Authorization: Bearer YOUR_MODELVERSE_API_KEY' \
--header 'Content-Type: application/json' \
-d '{
  "content": {
    "parts": [
      {
        "text": "Find video tutorials about machine learning"
      },
      {
        "file_data": {
          "mime_type": "image/png",
          "file_uri": "https://example.com/ml-diagram.png"
        }
      }
    ]
  }
}'
```

### 2. Video Content Analysis

Extract embeddings from video for classification or recommendation:

```bash
curl --location --globoff 'https://api-us-ca.umodelverse.ai/v1beta/models/gemini-embedding-2:embedContent' \
--header 'Authorization: Bearer YOUR_MODELVERSE_API_KEY' \
--header 'Content-Type: application/json' \
-d '{
  "content": {
    "parts": [
      {
        "file_data": {
          "mime_type": "video/mp4",
          "file_uri": "https://example.com/tutorial.mp4"
        }
      }
    ]
  }
}'
```

### 3. Document Understanding

Convert a PDF document to an embedding vector:

```bash
curl --location --globoff 'https://api-us-ca.umodelverse.ai/v1beta/models/gemini-embedding-2:embedContent' \
--header 'Authorization: Bearer YOUR_MODELVERSE_API_KEY' \
--header 'Content-Type: application/json' \
-d '{
  "content": {
    "parts": [
      {
        "text": "Summarize the key points of this report"
      },
      {
        "file_data": {
          "mime_type": "application/pdf",
          "file_uri": "https://example.com/report.pdf"
        }
      }
    ]
  }
}'
```

### 4. Audio Embedding

Process audio files to obtain semantic representations:

```bash
curl --location --globoff 'https://api-us-ca.umodelverse.ai/v1beta/models/gemini-embedding-2:embedContent' \
--header 'Authorization: Bearer YOUR_MODELVERSE_API_KEY' \
--header 'Content-Type: application/json' \
-d '{
  "content": {
    "parts": [
      {
        "file_data": {
          "mime_type": "audio/wav",
          "file_uri": "https://example.com/speech.wav"
        }
      }
    ]
  }
}'
```

### 5. Custom Output Dimensionality

Use `outputDimensionality` to specify the output vector dimension (e.g. 768):

```bash
curl --location --globoff 'https://api-us-ca.umodelverse.ai/v1beta/models/gemini-embedding-2:embedContent' \
--header 'Authorization: Bearer YOUR_MODELVERSE_API_KEY' \
--header 'Content-Type: application/json' \
-d '{
  "outputDimensionality": 768,
  "content": {
    "parts": [
      {
        "text": "Whats this"
      },
      {
        "file_data": {
          "mime_type": "video/mp4",
          "file_uri": "gs://cloud-samples-data/generative-ai/video/pixel8.mp4"
        }
      }
    ]
  }
}'
```

## Technical Specifications

### Images

| Limit | Specification |
| --- | --- |
| Max images per prompt | 6 |
| Max file size (inline or direct upload) | Unlimited |
| Max file size (Google Cloud Storage) | Unlimited |
| Supported MIME types | `image/png`, `image/jpeg`, `image/webp`, `image/bmp`, `image/heic`, `image/heif`, `image/avif` |

### Documents

| Limit | Specification |
| --- | --- |
| Max files per prompt | 1 |
| Max pages per file | 6 |
| Supported MIME types | `application/pdf` |

### Video

| Limit | Specification |
| --- | --- |
| Max duration (with audio) | 80 seconds |
| Max duration (without audio) | 120 seconds |
| Max videos per prompt | 1 |
| Supported MIME types | `video/mpeg`, `video/mp4` |

### Audio

| Limit | Specification |
| --- | --- |
| Max audio length per prompt | 180 seconds |
| Max audio files per prompt | 1 |
| Supported MIME types | `audio/mp3`, `audio/wav` |

## Notes

1. **File URIs**: Files must be provided via publicly accessible URLs, or use the Gemini File Upload API to obtain a URI
2. **File size limits**: Different modalities have different size limits; refer to the technical specifications above
3. **Token calculation**: Multimodal content token counts differ by modality; check `usageMetadata` in the response
4. **Embedding dimensions**: Use `outputDimensionality` to customize output dimensions (128 to 3072, default 3072)

## Comparison with Standard Embeddings API

| Feature | Gemini-Embedding-2 | Standard Embeddings API |
|---------|-------------------|-------------------------|
| Supported modalities | Text, image, video, audio, PDF | Text only |
| API endpoint | `/v1beta/models/gemini-embedding-2:embedContent` | `/v1/embeddings` |
| Request format | Gemini native format | OpenAI-compatible format |
| Mixed multimodal | Supported in single request | Not supported |

## Related Documentation

- [Embeddings](/docs/modelverse/modelverse/text_api/embeddings) — Standard text embedding API
- [Gemini Quick Start](/docs/modelverse/modelverse/text_api/gemini_compatible) — Gemini model usage guide
- [Gemini Media Analysis](/docs/modelverse/modelverse/text_api/gemini-media-analysis) — Media file processing