# preprocessing

> Browse 5 items tagged with preprocessing.

## Items

- [Vector Deduplication](https://vectordb.ever.works/ja/items/vector-deduplication) — Techniques for identifying and removing duplicate or near-duplicate vectors in databases using similarity thresholds. Deduplication reduces storage costs, improves search quality, and prevents redundant results in RAG systems by detecting semantically identical content even when textual representations differ.
- [Vector Normalization](https://vectordb.ever.works/ja/items/vector-normalization) — The process of scaling vectors to unit length (L2 normalization) or other standard forms. Normalized vectors enable cosine similarity computation via simple dot product and are essential for many embedding models and distance metrics used in vector databases.
- [Document Parsing for RAG](https://vectordb.ever.works/ja/items/document-parsing-for-rag) — Critical preprocessing step for RAG systems involving extraction of text, tables, and images from various document formats (PDF, DOCX, HTML) using tools like Unstructured, LlamaParse, and PyPDF.
- [L2 Normalization (Vector Normalization)](https://vectordb.ever.works/ja/items/l2-normalization-vector-normalization) — A preprocessing technique that scales vectors to unit length, ensuring all vectors lie on a hypersphere. Essential for making cosine similarity equivalent to inner product and improving embedding quality in many applications.
- [Vector Normalization (L2 Normalization)](https://vectordb.ever.works/ja/items/vector-normalization-l2-normalization) — Essential preprocessing technique that scales embedding vectors to unit length using L2 norm, ensuring consistent magnitude and making cosine similarity equivalent to dot product for faster computation.

---

_Canonical page: https://vectordb.ever.works/ja/tags/preprocessing_