Metrics & Scorers
A scorer turns an output into a number or a label. Picking one means matching it to the task and being clear about what it can't see. No scorer works for everything.
SafeGuard tunes the wrong number
SafeGuard built a content-moderation classifier and pushed its precision to 0.95. Of everything it flagged, 95% really was harmful, and leadership was happy. Harmful posts kept reaching users anyway. Recall was sitting at 0.60, so the filter was missing four in ten harmful items.
For a safety filter, the miss is the expensive mistake. A post flagged by accident costs a few seconds of a reviewer's time. The team retuned for recall, accepted more false flags, sent those to a human queue, and the amount of harmful content getting through dropped sharply.
The main kinds of scorer
Deterministic checks cover exact match, regular expressions, JSON-schema validity, "does it cite a source," code that compiles and math that comes out right. For code and math, running the output and checking the result is the best scorer there is. These checks are cheap and completely reliable, so use them whenever the task allows it.
Overlap metrics such as BLEU and ROUGE count shared word sequences between the output and a reference. They're cheap but weak: a correct paraphrase scores badly and a wrong answer that reuses the right words scores well. Treat them as rough signals.
Semantic similarity embeds the output and the reference and compares them. It picks up meaning better than word overlap, but a high score only says the two are about the same thing. They can still disagree on the facts.
Model-graded scoring, where a language model acts as the judge, is the flexible option for open-ended quality. It gets the whole next chapter.
A lot of LLM tasks are classification
Routing a message, detecting intent, filtering unsafe content and checking extracted fields are all classification problems, and interviewers expect the standard vocabulary. Precision is the share of flagged items that were right. Recall is the share of items that should have been flagged that you caught. F1 combines the two. Which one to push depends on which error hurts more. A safety filter cares about recall, since a missed unsafe message is the costly mistake. An action that deletes someone's email cares about precision, since a wrong deletion can't be undone.
Choose the scorer separately for each task, and say out loud what it can't detect. If its blind spot doesn't matter for the decision you're making, it's a good choice.
Score a safety filter by hand
Compute precision, recall and F1 for a safety filter from its predictions and the true labels, using only the standard library. Then decide whether this filter's weakness is acceptable for a safety use case, and which number tells you.
Lab code · runs free
import json
def exact_match(out, ref):
return float(out.strip() == ref.strip())
def valid_json(out, _ref):
try:
json.loads(out); return 1.0
except ValueError:
return 0.0
# Safety filter: 1 = flagged unsafe. Compare predictions to ground truth.
y_true = [1, 1, 1, 0, 0, 0, 0, 0, 1, 1] # what was actually unsafe
y_pred = [1, 1, 0, 0, 0, 1, 0, 0, 1, 0] # what the filter flagged
tp = sum(t == 1 and p == 1 for t, p in zip(y_true, y_pred))
fp = sum(t == 0 and p == 1 for t, p in zip(y_true, y_pred))
fn = sum(t == 1 and p == 0 for t, p in zip(y_true, y_pred))
precision = tp / (tp + fp)
recall = tp / (tp + fn)
f1 = 2 * precision * recall / (precision + recall)
print(f"precision={precision:.2f} recall={recall:.2f} f1={f1:.2f}")precision=0.75 recall=0.60 f1=0.67A precision of 0.75 means three of every four flags were correct. The number to look at for safety is recall: at 0.60, the filter let 40% of the unsafe content through. You'd lower the threshold to catch more, accepting some extra false flags, because each miss costs far more than a cautious flag. The math is the same for every classifier. Which number you push depends on what each kind of mistake costs.
Meridian chooses a scorer for each of Remi's jobs. A deterministic check confirms that every processed refund matches policy. Routing a ticket to the right queue is classification, and for the "possible fraud" queue the team optimizes recall. Tone and helpfulness have no answer key, so they'll go to a model judge. That judge has to earn their trust first.
Precision and recall are easy to mix up under interview pressure. One way to keep them straight: precision is about the things you flagged, recall is about the things you should have flagged. Say which one a use case needs and why, in terms of cost. "For a fraud alert I'd favor recall, because a missed fraud case costs more than an extra review" is a complete answer.
You'll also hear "accuracy" used loosely. When one class is rare, accuracy can look great while the system misses everything important. Pointing that out, with a quick example, is an easy way to show you understand the metrics.
You don't need the formulas. You need to know which kind of mistake is more expensive for your business, because that should decide which number the team pushes. When a team picks a metric, ask them: "Which costs us more here, a miss or a false alarm?" If they can't link the metric to a real cost, the choice was arbitrary.
Watch for "99% accurate" on a rare-event problem. If 2% of content is harmful, a filter that flags nothing is 98% accurate. Ask instead what share of the harmful items it catches, and accept that raising that number means more false flags for a person to review.
Quiz · Chapter 4
- What's the best scorer for generated code?
- What's the main weakness of BLEU and ROUGE?
- For a safety filter, which metric do you usually push hardest?
- What does precision measure?
- A high semantic-similarity score tells you the output: