CLI
promptfoo
promptfoo is an open-source AI command-line tool from OpenAI, launched in 2023, made for developers and businesses.
A command-line tool for testing prompts and models and red-teaming LLM apps.
Main jobs: Evaluate and trace LLM apps. It is open source and free to use. You can use it in the terminal, offline on your own machine and on your own server.
Markdown version of https://godooo.ai/en/cli/promptfoo. Every page on this site has one: add .md to its address.
At a glance
- Type
- CLI
- Made by
- OpenAI
- Launched
- 2023
- Galaxy
- Code
- Pricing
- Open source, free
- Open source
- Yes · MIT
- Source code
- github.com/promptfoo/promptfoo
- GitHub stars
- 25,416
- How you run it
- Terminal · Runs offline · Self-host
- Made for
- Developers · Businesses
- Website
- promptfoo.dev
What it can do
Evaluate and trace LLM appsmain jobunconfirmed
“Evaluations Test and evaluate your prompts, models, and RAG pipelines”— promptfoo.dev, 2026-09-24
unconfirmedproposed by a machine, awaiting calibration
Alternatives
- DeepEvalOpen source, freeAlso does: Evaluate and trace LLM apps
- LangfuseOpen source, freeAlso does: Evaluate and trace LLM apps
- PhoenixOpen source, freeAlso does: Evaluate and trace LLM apps
- RagasOpen source, freeAlso does: Evaluate and trace LLM apps
- Pydantic AIOpen source, freeAlso does: Evaluate and trace LLM apps
Questions
What is promptfoo?
promptfoo is an open-source AI command-line tool from OpenAI, launched in 2023, made for developers and businesses. A command-line tool for testing prompts and models and red-teaming LLM apps.
Is promptfoo free?
Yes. promptfoo is open source (MIT) and free to use.
Is promptfoo open source?
Yes, under the MIT license. The source code is at https://github.com/promptfoo/promptfoo.
Can promptfoo run locally?
Yes. promptfoo can run offline on your own machine. It can also be self-hosted on your own server.
What can promptfoo do?
Main jobs: Evaluate and trace LLM apps.
Who makes promptfoo?
promptfoo is made by OpenAI.
What are open-source alternatives to promptfoo?
Open-source ones: DeepEval, Langfuse, Phoenix, Ragas and Pydantic AI.
Sources
- www.promptfoo.dev/checked 2026-09-24
- promptfoo.dev
- github.com/promptfoo/promptfoo