DEV Community
•
2026-07-28 21:10
Run and Compare AI Evaluations with a CLI for Developers and Coding Agents
TL;DR: This walkthrough shows how developers and coding agents can use Quantiles, an open-source AI evaluation platform licensed under Apache 2.0, to quickly run, analyze, and compare AI evaluations locally. We'll use the SimpleQA Verified benchmark as an example throughout this post, letting you follow the commands, inspect the evaluation results, and configure your own model for the same workflo...