Overview
Binary Quantization is an extreme compression technique that converts each vector dimension to a single bit (0 or 1), achieving up to 32x memory reduction compared to 32-bit floats and enabling ultra-fast similarity search using Hamming distance.
How Binary Quantization Works
Quantization Process
- Threshold: Determine a threshold (often 0 or mean value)
- Binarize: Convert each dimension to 1 if above threshold, 0 otherwise
- Pack: Pack bits efficiently for storage
- Store: Store binary vectors (1 bit per dimension)
Search Process
- Binarize query vector using same threshold
- Compute Hamming distance (count differing bits)
- Use XOR and popcount for ultra-fast computation
- Optionally rescore top candidates with original vectors
Memory Reduction
- 32-bit float → 1 bit: 32x compression
- Example: 768-dim vector: 3,072 bytes → 96 bytes
- Enables keeping billions of vectors in memory
Speed
- Hamming distance via XOR + popcount: extremely fast
- Modern CPUs have hardware popcount instructions
- Can process vectors 10-100x faster than float comparisons
Accuracy Considerations
- Accuracy loss more significant than PQ or SQ
- Works well with models trained for binary embeddings
- Typically 5-15% recall reduction vs full precision
When Binary Quantization Works Best
Model Characteristics
- High-dimensional embeddings (768+)
- Models with well-distributed values
- Embeddings specifically trained for binary quantization
Use Cases
- First-stage retrieval (rerank with full precision)
- Massive-scale search (billions of vectors)
- Memory-constrained environments
- Real-time search requirements
Limitations
- Significant accuracy loss for some datasets
- Not suitable for all embedding models
- Requires careful threshold selection
- May need rescoring step
Modern Implementations
Many embedding models now support Matryoshka Representation Learning combined with binary quantization for flexible compression-accuracy trade-offs.
Pricing
Implemented in vector databases (Qdrant, Milvus, Weaviate, etc.)
