# Embedding Cache

> Caching mechanism for storing and reusing previously computed embeddings to reduce API costs and latency. Essential optimization for production RAG systems processing repeated or similar content.

- **URL:** https://github.com/zilliztech/GPTCache
- **Category:** Concepts & Definitions
- **Tags:** Caching, optimization, cost-reduction
- **Updated:** 2026-03-15 12:39
- **Canonical page:** https://vectordb.ever.works/vi/items/embedding-cache

## Details

## Overview

Embedding caches store computed embeddings to avoid redundant API calls. Critical for cost reduction and performance in production systems.

## Why Cache Embeddings?

### Cost Savings
- OpenAI embeddings: $0.13 per 1M tokens
- Repeated content → wasted money
- Cache hit rate of 30-50% typical
- 30-50% cost reduction

### Performance
- API latency: 100-500ms
- Cache lookup: <10ms
- 10-50x faster for cache hits

## Implementation

```python
import hashlib
import redis

class EmbeddingCache:
    def __init__(self, redis_client):
        self.redis = redis_client
        
    def get_embedding(self, text):
        # Create cache key
        key = hashlib.sha256(text.encode()).hexdigest()
        
        # Check cache
        cached = self.redis.get(key)
        if cached:
            return json.loads(cached)
            
        # Compute and cache
        embedding = embedding_model.encode(text)
        self.redis.set(key, json.dumps(embedding), ex=86400)  # 24h TTL
        return embedding
```

## Cache Strategies

### Exact Match
- Hash text content
- 100% accuracy
- Lower hit rate

### Semantic Cache
- Find similar cached texts
- Higher hit rate
- Approximate matches
- Uses vector DB for cache lookup

## GPTCache

Dedicated library for LLM caching:
- Semantic caching
- Multiple backends (Redis, Milvus)
- Similarity thresholds
- TTL management

## Best Practices

1. **Normalize Text**: Lowercase, trim whitespace
2. **Set TTL**: Expire old embeddings
3. **Monitor Hit Rate**: Track cache effectiveness
4. **Size Limits**: Prevent unbounded growth
5. **Invalidation**: Clear when model updates

## Pricing

Cache storage costs (Redis/Memcached) typically < embedding API costs.

---

_This Markdown mirror is generated by Ever Works for AI agents. The canonical HTML page is at https://vectordb.ever.works/vi/items/embedding-cache._