Arize Phoenix

Overview

Arize Phoenix is an open-source observability and evaluation platform for LLM applications built on the OpenTelemetry standard. It provides comprehensive tracing and evaluation capabilities for RAG systems and AI agents.

Features

• OpenTelemetry-Based: Built on industry-standard observability framework
• Automated Instrumentation: Automatic tracing without code changes
• RAG Evaluation: Specialized metrics for RAG pipelines
• Trace Visualization: Detailed execution flow diagrams
• LLM Scorers: Multiple evaluation methods including LLM-as-judge
• Real-Time Monitoring: Production-ready observability
• Dataset Management: Track and version evaluation datasets
• Integration: Works with LangChain, LlamaIndex, and other frameworks

Evaluation Metrics

• Context relevance
• Answer correctness
• Hallucination detection
• Response quality
• Custom metrics support

Use Cases

• Debugging LLM applications
• RAG pipeline optimization
• Production monitoring
• Evaluation dataset curation
• Performance benchmarking

Integration

MLflow's third-party scorer framework supports Phoenix alongside RAGAS and TruLens. The platform is part of an ecosystem with 32M+ monthly PyPI downloads.

Pricing

Open-source under Apache 2.0 license. Arize also offers a commercial observability platform.