# Cross-Encoder Reranking

> Two-stage retrieval where initial results from bi-encoder vector search are reranked using more expensive cross-encoder models for higher accuracy. Used in Hindsight and other systems.

- **URL:** https://hindsight.vectorize.io/
- **Category:** Concepts & Definitions
- **Tags:** Reranking, retrieval, accuracy
- **Updated:** 2026-03-24 13:36
- **Canonical page:** https://vectordb.ever.works/it/items/cross-encoder-reranking

## Details

## Overview

Cross-Encoder Reranking is a two-stage retrieval approach where fast approximate search provides candidates, which are then reranked using slower but more accurate cross-encoder models.

## Two-Stage Process

### Stage 1: Candidate Retrieval
- Use bi-encoder for fast similarity search
- Retrieve top-k candidates (e.g., k=100)
- Low cost, high recall
- Millisecond latency

### Stage 2: Reranking
- Pass query+candidate pairs to cross-encoder
- Model scores each pair jointly
- High accuracy
- Return top-n (e.g., n=10)

## Why It Works

Bi-encoders encode query and document separately, limiting interaction. Cross-encoders encode them together, capturing fine-grained relationships but at higher computational cost.

## Implementation in Hindsight

Hindsight uses four parallel retrieval strategies, then applies cross-encoder reranking to combine and refine results, achieving 91.4% on LongMemEval.

## Popular Cross-Encoders

- BGE Reranker models
- Cohere Rerank
- Cross-encoder/ms-marco models
- Custom fine-tuned models

## Performance Trade-offs

- **Accuracy**: 10-30% improvement over bi-encoder alone
- **Latency**: Adds 50-200ms
- **Cost**: Higher compute requirements
- **Scalability**: Limited by reranking budget

## Best Practices

- Limit reranking to top-100 candidates
- Use GPUs for cross-encoder inference
- Batch reranking requests
- Cache results when possible

## Pricing

Technique/pattern, various implementations available.

---

_This Markdown mirror is generated by Ever Works for AI agents. The canonical HTML page is at https://vectordb.ever.works/it/items/cross-encoder-reranking._