卓越
目录平台模板

Massive Text Embedding Benchmark leaderboard covering 58 datasets across 112 languages and 8 embedding tasks. Industry-standard benchmark for comparing text embedding models.

Comprehensive benchmark dataset comparing 10 vector databases across 19 fields including query latency (p50/p99), throughput, scalability limits, features like hybrid search and ACID compliance, SDK support, and managed pricing. Tested with 1M vectors at 1536 dimensions for RAG and AI search applications. Key highlights include Qdrant for lowest latency, Pinecone for managed scalability, and pgvector for ACID transactions.

Zero-shot benchmark for embedding model evaluation on 18 diverse datasets with NDCG@10 and Recall@100 metrics correlating to vector DB QPS/latency in production. Features heterogeneous tasks like QA, fact-checking, biomedical retrieval for robust comparisons. Use cases include selecting embeddings for RAG pipelines in vector DBs; complements ANN-Benchmarks indexing focus with retrieval task evaluation, differs from VectorDBBench full-DB tests.

Tests vector DBs on multimodal QPS/latency for heterogeneous embeddings and compound queries including GPU setups. Features Docker-based eval for Milvus etc. on cross-modal retrieval. For selecting multimodal vector DBs; differs from ANN-Benchmarks text-only by adding hybrid workloads vs custom single-DB tests.

Provides QPS/latency/recall benchmarks for ANNS algorithms on billion-point datasets via NeurIPS tools for dataset prep and evaluation. Features scalable testing for extreme throughput and visualization. Key for production vector DBs at scale; extends ANN-Benchmarks with billion-scale tools unlike full-system DB benchmarks.

Evaluates embeddings on 58 datasets/112 languages with retrieval/clustering metrics for vector DB model selection via nDCG/Recall throughput proxies. Features 8 task types for comprehensive perf eval. Standard for RAG embedding choice; text-focused unlike BigVectorBench multimodal, complements ANN-Benchmarks index benchmarks.

BEIR (Benchmarking IR) is a benchmark suite for evaluating information retrieval and vector search systems across multiple tasks and datasets. Useful for comparing vector database performance.

Main competition for large-scale vector database algorithms held at NeurIPS conferences. Framework for evaluating approximate nearest neighbor search algorithms on billion-scale datasets with standardized metrics and datasets.

Massive Text Embedding Benchmark (MTEB) - a comprehensive benchmark for evaluating text embedding models across 8 embedding tasks and 58 datasets in 112 languages. Provides a standardized leaderboard for comparing embedding quality across classification, clustering, retrieval, reranking, semantic textual similarity, and summarization tasks.

Vector Bible is a GitHub repository comparing popular vector databases across features, performance, and use cases in a structured table. It serves as a quick reference for selection by aggregating benchmarks and pricing information. It complements ANN-Benchmarks as an essential resource for DB evaluation and decision-making, not a tool itself.

Open-source vector database benchmarking tool testing databases across production-critical scenarios including static collection, filtering, and streaming cases with modern embedding model datasets.

Confident AI evaluates vector DB-integrated LLM apps with 50+ metrics on faithfulness, relevance, tracking QPS/latency in production traces for RAG performance. Key features include DeepEval-powered scoring, observability dashboards, and quality-aware alerting across datasets. Supports prod vector DB RAG selection via real-world eval; broader than ANN-Benchmarks (indexing) or VectorDBBench (DB perf).