Module 2
Machines That Hear
Give your AI ears and a voice! From sound waves and spectrograms to talking to your computer, hearing it talk back, and the real engineering that makes it all work — running privately on your own device. An 18-day journey.
How to get the most out of this module
This module isn't just for fun — it's to make you a real builder. Do these and you'll be able to build voice apps yourself (or tell an AI assistant exactly what to build):
🌊 Part 1 · How Machines Hear
Sound becomes numbers — waves, pictures, and pitch.
Sound Is a Wave
See and hear sound as wiggles: frequency = pitch, size = loudness.
Sound as a Picture
The spectrogram — how machines turn sound into an image, then use vision tricks.
Pitch Detective
A live note detector — sing and it names the note.
🧠 Part 2 · Understanding Sound
Train ears, talk to text, and text to talk — on your device.
Teach It a Sound
Train your own audio AI to tell claps, whistles, or words apart.
Talk to Text
Real speech-to-text (Whisper), 100% on your device.
Text to Talk
Type anything and hear the AI say it — design its voice.
How It Understands
Phonemes, context, and the language model ("I scream" vs "ice cream").
🛠️ Part 3 · Build & Decide
Voice commands, scoring, ethics, and your own assistant.
Wake Words & Commands
Real on-device voice control — drive a robot maze with your voice.
Scoring Speech AIs
Word Error Rate — inserts, deletes, swaps, and how noise wrecks it.
Real AI Voices & Ethics
Hear how human AI voices are + deepfake-detective + responsible building.
Your Voice Assistant
Talk → it understands → acts → talks back. Build your own assistant!
⚙️ Part 4 · Real Voice Engineering
The pro topics that make real voice systems actually work.
Is Anyone Talking? (VAD)
Voice Activity Detection — know when speech starts, stops, and ends.
Stop the Echo (AEC)
Acoustic Echo Cancellation — subtract the speaker's voice out of the mic.
Clean the Voice
Noise suppression — pull a clean voice out of a noisy signal.
Fast or Accurate?
Streaming vs batch ASR and why latency matters.
Tiny Audio, Big Tradeoffs
Sample rate, bit depth & codecs — quality vs file size.
Listen This Way
Beamforming & mic arrays — steer a listening beam at a voice.
Cancel the Noise Wave
Active Noise Cancellation — erase noise with an opposite wave.
Real tools & open-source models
You know the ideas AND the pro words now. Here are the real, free tools engineers use — for hobby projects and professional apps. Most run in a browser, on a laptop, or on a Raspberry Pi.
🧰 Platforms & starting points
- Hugging Face — find open models & demos beginner
- Teachable Machine (Audio) — train sound models, no code beginner
- transformers.js — run models in the browser beginner
🎙️ Speech-to-text (ASR)
- Whisper — top speech-to-text beginner
- whisper.cpp — Whisper on small devices pro
- Vosk — offline speech recognition beginner
🔊 Text-to-speech (TTS)
- Kokoro TTS — small open neural voice beginner
- Piper — fast local text-to-speech beginner
- Coqui TTS — train & clone voices pro
⚙️ Engineering building blocks
- Silero VAD — detect speech vs silence beginner
- openWakeWord — custom wake words pro
- RNNoise — neural noise suppression pro
- librosa — spectrograms & audio analysis pro