# Multi-Vector Embeddings

> Embedding approach where documents/images are represented by multiple vectors (one per token/patch) rather than a single vector, enabling fine-grained semantic matching.

- **URL:** https://jina.ai/news/jina-embeddings-v4-universal-embeddings-for-multimodal-multilingual-retrieval/
- **Category:** Concepts & Definitions
- **Tags:** Embeddings, colbert, retrieval
- **Updated:** 2026-03-24 13:36
- **Canonical page:** https://vectordb.ever.works/th/items/multi-vector-embeddings

## Details

## Overview

Multi-Vector Embeddings represent documents or images as sequences of vectors rather than single vectors, enabling more nuanced semantic matching and better retrieval quality.

## Architecture

- Each token/patch gets its own vector
- Typical output: 128 dimensions per token
- Variable length based on content
- Stored and indexed together

## Comparison to Single-Vector

### Single-Vector (Dense)
- One vector per document
- Fixed dimensionality (e.g., 768 or 1536)
- Fast similarity computation
- Less storage

### Multi-Vector
- Multiple vectors per document
- Captures fine-grained semantics
- Better retrieval quality
- More storage needed

## Implementation Examples

### ColBERT
Pioneered multi-vector retrieval for text with MaxSim aggregation.

### Jina Embeddings v4
Supports both single-vector (2048-dim) and multi-vector (128-dim per token) embeddings.

## Retrieval Process

1. Encode query into multiple vectors
2. Encode documents into multiple vectors
3. Compute token-level similarities
4. Aggregate (typically MaxSim)
5. Rank documents

## Trade-offs

- **Quality**: Significantly better than single-vector
- **Storage**: 5-20x more space
- **Speed**: Slower similarity computation
- **Scalability**: Requires specialized indexing

## Pricing

Implementation approach, supported by various tools.

---

_This Markdown mirror is generated by Ever Works for AI agents. The canonical HTML page is at https://vectordb.ever.works/th/items/multi-vector-embeddings._