# BenchmarkQED

> BenchmarkQED standardizes QPS/latency/accuracy evaluations for RAG pipelines including vector DB retrieval on diverse datasets. Features comparable methodologies for fair benchmarking of full RAG stacks. Essential for selecting production vector DBs in RAG; emphasizes retrieval fairness unlike ANN-Benchmarks indexing focus or VectorDBBench system-level throughput tests.

- **URL:** https://github.com/project-baize/BenchmarkQED
- **Category:** Developer Tools & Benchmarks
- **Tags:** Benchmarking, Performance Evaluation, Rag Benchmark
- **Updated:** 2026-04-29 11:31
- **Canonical page:** https://vectordb.ever.works/items/benchmarkqed

## Details

## Overview

BenchmarkQED is an open benchmarking framework for Retrieval-Augmented Generation (RAG) systems designed to push the community toward fairer, comparably measured retrieval evaluation methods.

## Features

- Standardized RAG evaluation protocols
- Focus on retrieval component fairness
- Comprehensive metrics including QPS and latency
- Open-source and extensible

## Use Cases

- RAG system selection and optimization
- Vector DB integration testing
- Production performance validation

## Pricing

Free and open-source.

---

_This Markdown mirror is generated by Ever Works for AI agents. The canonical HTML page is at https://vectordb.ever.works/items/benchmarkqed._