Framework
DeepEval
DeepEval is an open-source AI framework, launched in 2023, made for developers and businesses.
A Python framework for unit-testing and evaluating LLM apps, RAG pipelines and agents.
Main jobs: Evaluate and trace LLM apps. It is open source and free to use. You can use it on your own server, in the terminal and through an API.
Markdown version of https://godooo.ai/en/framework/deepeval. Every page on this site has one: add .md to its address.
At a glance
- Type
- Framework
- Launched
- 2023
- Galaxy
- Code
- Open source
- Yes · Apache-2.0
- Source code
- github.com/confident-ai/deepeval
- GitHub stars
- 18,426
- How you run it
- Self-host · Terminal · API
- Made for
- Developers · Businesses
- Website
- deepeval.com
What it can do
Evaluate and trace LLM appsmain jobunconfirmed
“DeepEval is the open-source LLM evaluation framework for testing and benchmarking LLM applications.”— deepeval.com, 2026-09-24
Test writing and QAsupportedunconfirmed
“Pytest-native evals that run in CI/CD or as Python scripts.”— deepeval.com, 2026-09-24
unconfirmedproposed by a machine, awaiting calibration
Alternatives
- RagasOpen source, freeAlso does: Evaluate and trace LLM apps; runs offline
- LangfuseOpen source, freeAlso does: Evaluate and trace LLM apps
- PhoenixOpen source, freeAlso does: Evaluate and trace LLM apps; runs offline
- promptfooOpen source, freeAlso does: Evaluate and trace LLM apps; runs offline
- Pydantic AIOpen source, freeAlso does: Evaluate and trace LLM apps; runs offline
Questions
What is DeepEval?
DeepEval is an open-source AI framework, launched in 2023, made for developers and businesses. A Python framework for unit-testing and evaluating LLM apps, RAG pipelines and agents.
Is DeepEval free?
Yes. DeepEval is open source (Apache-2.0) and free to use.
Is DeepEval open source?
Yes, under the Apache-2.0 license. The source code is at https://github.com/confident-ai/deepeval.
Can DeepEval run locally?
DeepEval can be self-hosted on your own server.
What can DeepEval do?
Main jobs: Evaluate and trace LLM apps. It can also do: Test writing and QA.
What are open-source alternatives to DeepEval?
Open-source ones: Ragas, Langfuse, Phoenix, promptfoo and Pydantic AI.
Sources
- deepeval.com/checked 2026-09-24
- github.com/confident-ai/deepeval