There are three big ways an AI can "see" a picture — from a quick label, to boxes, to coloring in every single pixel. Let's line them up side by side and feel the difference.
Detection draws a box (fast, but grabs background too). Segmentation is way pickier: it labels every single pixel as "object" or "not". Click and drag to paint the cat's pixels. Cover the cat, avoid the background — your overlap score climbs!
Real engineers pick the right kind of vision for each task. Read each mission and choose!
Here's the mind-blowing one. CLIP is a real AI that matches pictures to words — and you pick the words! Type any labels you invent ("a happy dog", "my homework", "a toy robot") and it scores your photo against them. No retraining, ever. Running on your device. (~150 MB one-time — the biggest model here, so it's best on a laptop with a bit of room.)
You painted a mask by hand. Real segmentation uses trained networks:
✅ Shows what a mask is. ⚠️ Way too slow for real use.
U-Net, DeepLab, Mask R-CNN - trained to mask objects automatically.
SAM (Segment Anything) cuts out almost any object with a click. ✅ Magical. ⚠️ Big.
You now know the three flavors of computer vision — classification (what), detection (what + where, boxes), and segmentation (every pixel) — and when to use each.