Yesterday your AI aced its practice but scored lower on the exam. Why? Because there are two ways to "pass": memorize every answer (works only on stuff you've seen) or actually learn the pattern (works on anything). Let's meet both.
The dots are two teams (🔵 and 🟣) that make an X-shape — you can't split them with one straight line! Try three "brains" and watch two scores: Practice (data it learned) and Exam (fresh data). The gap between them tells the whole story.
Brain too simple — it can't even learn the practice data. Both scores low. Like studying too little.
Learns the real pattern. Practice and exam both good, small gap. This is what you want!
Memorizes every dot, even the mistakes. Practice ~100%, exam way lower. A big gap is the warning sign.
Picking 'just right' complexity is the idea. Pros have real tools to prevent overfitting:
✅ Shows the tradeoff. ⚠️ By hand only.
Dropout, weight decay, early stopping - nudge the model to stay simple and general.
More varied data and a validation set catch overfitting before it ships.
You can now spot a memorizer: watch the gap between practice and exam scores. A real learner generalizes to new stuff.