Day 6 · Scoring an AI

🏷️ The pro wordThe 4-box table is a confusion matrix. The scores are accuracy, precision, recall (sensitivity) and F1. The full tradeoff curve is a precision-recall / ROC curve.

🐱 Lab · The Cat Detector

One slider changes everything

Each 🐱/🐶 sits at the score the AI gave it (how "catty" it looks). Everything to the right of the yellow line, the AI calls CAT; to the left, DOG. Drag the line and watch who gets caught — and who gets it wrong (red ring).

← calls almost everything a CAT (catch them all)line at 0.50calls almost nothing a CAT (be super sure) →

🔢 The 4 Outcomes

Every guess lands in one of 4 boxes

This is a confusion matrix. Whenever the AI guesses, it's either right or wrong, about a cat or a dog. That makes exactly 4 possibilities:

AI says 🐱 CAT

AI says 🐶 DOG

Really a CAT

TRUE POSITIVE

caught a real cat ✅

FALSE NEGATIVE

missed a cat 😿

Really a DOG

FALSE POSITIVE

false alarm! 🚨

TRUE NEGATIVE

correctly skipped a dog ✅

Move the slider up top and watch the numbers flow between boxes. Green = correct, red = mistakes.

📊 The Scores

Precision, Recall & F1 — decoded

Accuracyhow many right overall

Precisionwhen it says CAT, how often it's right

Recall (Sensitivity)of all real cats, how many it caught

F1 Scorethe balance of precision & recall

🎚️ The big secret: slide left and you catch every cat (high recall) but get false alarms (low precision). Slide right and you're rarely wrong (high precision) but miss cats (low recall). You can't max both — you pick what matters!

🔍 Curious? The exact recipes

Precision = TP ÷ (TP + FP). Recall = TP ÷ (TP + FN). Accuracy = (TP + TN) ÷ everything. F1 = 2 × (Precision × Recall) ÷ (Precision + Recall) — it's only high when BOTH are high. Recall is also called sensitivity; TN ÷ (TN+FP) is called specificity.

🏆 Boss Build · Three Missions

Beat the tradeoff

Use the slider up top to hit each target. Watch how winning one can cost you another!

🐱

Catch every cat

Get Recall = 100% — don't miss a single cat.

not yet

🎯

Zero false alarms

Get Precision = 100% — never call a dog a cat.

not yet

⚖️

Best balance

Get F1 ≥ 90% — the sweet spot between both.

not yet

🚀 Beyond the Basics

How the pros really do it

Accuracy is the beginner metric. Real benchmarks use sharper numbers:

🟢

Accuracy (the simple one)

✅ Easy. ⚠️ Lies when classes are imbalanced (99% by always saying 'no').

📈

Precision / Recall / F1 + ROC-AUC

The real report card; ROC-AUC summarizes every threshold at once. ✅ Honest.

🎯

mAP (for detection)

mean Average Precision - the single number that ranks detectors on leaderboards.

🏷️ Pro names to look upconfusion matrix · precision/recall/F1 · ROC-AUC · mAP.

🎉 Day 6 Complete!

You can now read an AI's report card like a pro: TP, FP, FN, TN, and the scores built from them — accuracy, precision, recall, and F1.

Day 7: What AND Where →

Tomorrow: so far the AI says what something is. Next it learns where — drawing boxes around objects, the way self-driving cars see the road. 🚗