BenchmarkQED standardizes QPS/latency/accuracy evaluations for RAG pipelines including vector DB retrieval on diverse datasets. Features comparable methodologies for fair benchmarking of full RAG stacks. Essential for selecting production vector DBs in RAG; emphasizes retrieval fairness unlike ANN-Benchmarks indexing focus or VectorDBBench system-level throughput tests.