What Is LLM Evaluation? Evals Explained for People Who Ship, Not Publish

LLM evaluation, evals for short, is the practice of running a model or an AI feature against a fixed set of cases and scoring the results against a written standard, so that a change to the prompt, the model or the data can be called better or worse with evidence instead of a feeling.

The feeling is the problem. You change a prompt, you try three questions, the answers look nicer, you ship. A week later a user tells you the bot stopped mentioning the refund window, and you have no idea which of the four prompt edits since Tuesday did it. Every guide on this term is written for ML teams with a benchmark budget. This one is written for the person who built the feature in an afternoon with an AI app builder and now has to know whether it is getting better or worse.

Model evals and product evals are different jobs

The first thing to separate, because the search results mix them.

A model eval is what a benchmark score is. EleutherAI's lm-evaluation-harness, the backend of the Hugging Face Open LLM Leaderboard, is the standard tool, and it is MIT licensed and free. It is also almost useless to you. It scores the model on somebody else's questions, and your feature is not the model. Your feature is the model plus your prompt plus whatever you retrieve plus the code that parses the answer, and any one of those can break while the leaderboard number stays exactly where it was.

A product eval scores that whole stack on your questions. Everything below is about product evals, because that is the kind a builder can run and the kind that changes what ships.

The four approaches, and which one you will use

Sebastian Raschka's four approaches are the cleanest map of the field, and the labels are his: evaluating answer-choice accuracy, using verifiers to check answers, comparing models using preferences and leaderboards, and judging responses with other LLMs.

The first is the multiple-choice benchmark. The second, the verifier, is any check a program can run without judgement: the code compiles, the JSON parses, the number matches, the test passes. The third is preference voting at scale, which is how the chatbot arenas rank models. The fourth is LLM-as-a-judge: one model reads another's answer and scores it against a rubric.

For a shipped feature you will use two of the four. Verifiers wherever the output has a checkable shape, because they are cheap, deterministic and never wrong about what they check. A judge wherever the output is prose, because nothing else scales past the number of answers you can read yourself. The other two are model evals, and you will read about them rather than run them.

Why vibes are not an eval

A vibe check is a product eval with a sample size of three and no record. It catches the failures you happened to try and none of the others, and it cannot tell you whether the version from last week was better, because last week's answers are gone.

An eval fixes three things a vibe check lacks. The cases are written down, so the same questions get asked every time. The standard is written down, so "good" means the same thing on Friday as on Monday. The results are kept, so a change can be compared with the version before it. None of that requires a platform. It requires a spreadsheet and the discipline to run it before shipping.

The smallest eval loop that works

Suppose the feature is a support bot for a small online shop, built in an app builder and answering from the shop's policy pages. Here is the loop, with nothing to install.

1. Write the golden set. Twenty to fifty questions real users would ask. Ten from the normal path ("how long does delivery take"), ten edge cases ("I ordered two, can I return one"), and every question that produced a wrong answer in the past. Each row gets the question and the facts a correct answer must contain: for the returns question, the window, the condition, the address.

2. Write the standard. For each row, a pass condition a stranger could apply. "Mentions the 14-day window and that the item must be unused" is a standard. "Answers well" is not. Where the output has a shape, the standard is a verifier: the response is valid JSON, the order number it quotes is one that exists.

3. Run it and keep the run. Every case through the current feature, results in a column dated today. The first run is your baseline, and the baseline is allowed to be bad. Its job is to exist.

4. Change one thing, run it again. A prompt edit, a model swap, a new document in the retrieval set. Compare the column with the baseline. A change that improves fifteen cases and breaks four has told you something the three-question vibe check never would, and the four broken rows tell you exactly what to fix.

5. Add every failure from production to the set. The set grows from what users actually broke, which is the only source of cases worth having.

That is the whole loop. At thirty cases it takes minutes by hand. At a hundred, reading every answer stops being fun, and that is the moment a judge earns its place: give a second model the question, the answer and the standard from step two, and have it return pass or fail with a reason. Then, before trusting it, check its verdicts against twenty of yours. A judge that was never checked against a human is the failure practitioners keep describing in the community threads on this topic, and the fix is a one-time afternoon.

What a production eval looks like

The loop scales without changing shape. GitHub's secret-scanning team described running exactly this pattern before shipping an LLM-backed change, evaluating offline against a fixed dataset and gating on the result. They report a 95% reduction in false positives on that offline dataset, with recall held inside a guardrail they set in advance, in a write-up published on 25 August 2026. Same ingredients as the support bot: a fixed set, a written standard, a baseline, a gate. The dataset was bigger and the standard was stricter, and nothing else was different.

The detail worth copying is the guardrail on recall. An eval that measures one thing will be gamed by a change that improves that thing at the cost of another. Two metrics, one you want to raise and one you refuse to let fall, is the minimum for a gate that means something.

The three kinds of check

Whatever you are evaluating, the cases sort into three groups, and each group wants a different kind of check.

Correctness. Did the answer contain the facts, do the task, match the reference? Verifiers where possible, a judge with the facts in the rubric where not. If the feature retrieves documents, correctness includes groundedness: the answer used what was retrieved rather than inventing around it, which is the eval-time view of hallucination.

Safety. Did it refuse what it should, avoid what it must, keep private data private? These cases are the ones you write from the failures your guardrails exist to catch, and an eval is how you find out whether the guardrails are catching them.

Format. Did it return the shape the code expects, in the length the interface allows, in the tone the brand set? Almost entirely verifiers. A format regression is the cheapest kind to catch and the most embarrassing kind to ship.

Most golden sets start heavy on correctness and light on the other two. Rebalance after the first production incident, which is usually a format or safety case nobody wrote down.

When you need tooling, and what it costs

The spreadsheet loop holds up longer than people expect. The signals that it is time to graduate are concrete: more than one person runs the evals, the set has passed a few hundred cases, or you need to score live traffic rather than a fixed set.

The open-source frameworks are the next step and they are free. DeepEval, an Apache 2.0 framework that describes itself as Pytest for LLM apps, turns the loop into test cases a CI run can execute. Evidently's library is open source under Apache 2.0 and covers evaluation and monitoring; the company does not publish cloud pricing. Both run against your existing golden set.

The hosted platforms add tracing and a shared dashboard, and they charge for it. Read on 6 September 2026, LangSmith is free for one developer up to 5,000 traces a month, and Plus is $39 per seat per month. Confident AI, the platform behind DeepEval, has a free tier, and paid plans start at $200 a month per organisation with unlimited seats. Both prices are the vendor's own pages on that date and both change; check before you budget.

The order matters. Spreadsheet, then framework, then platform. A team that buys the platform first tends to end up with beautiful traces of a feature nobody wrote a standard for.

How evals fit a vibe-coding loop

The build loop most readers of this site run is prompt, generate, look, ship. Evals slot into it as the "look" step, made repeatable. Before a change goes out, the golden set runs. If the set is green, ship. If it is not, the red rows are the to-do list.

That turns the AI development workflow from a sequence of hopeful edits into something closer to engineering, and it does it without slowing the fast part. Generating the change is still minutes. The eval is the minute after, and it is the minute that decides whether the next week is spent building or apologising.

FAQ

What is LLM evaluation in simple terms? Fixed cases, a written standard, kept results. Run them before a change ships so "better" has evidence.

Model evals or product evals? Model evals rank models on public benchmarks. Product evals score your feature on your users' tasks. You run the second one.

What are the four approaches? Raschka's labels: answer-choice accuracy, verifiers, preference comparisons and leaderboards, and LLM-as-a-judge. A shipped feature uses verifiers and a judge.

How many cases to start? Twenty to fifty, weighted toward edge cases and past failures.

Can I trust an LLM judge? After you have checked its verdicts against your own on a sample. Not before.

What do the tools cost? Frameworks are free and open source. LangSmith is free to 5,000 traces then $39 per seat; Confident AI is free then $200 a month per organisation, at the time of writing.

The short version: an eval is a vibe check with a memory, thirty cases in a spreadsheet is a real one, and the tools are for when the spreadsheet stops fitting the team. Start with the golden set this week, before the next prompt change, and pick the tooling when the set outgrows you rather than before.