The Signal Processing Behind Voice Assistants: Alexa, Siri, and Google Home
Voice assistants capture sound through microphone arrays then process it in stages from local wake-word detection to cloud automatic speech recognition. The entire pipeline balances always-on listening with power limits and real-time response requirements.
The first stage runs on the device itself. A tiny model listens for the specific wake word. Everything else stays silent until that model fires.
How the Signal Chain Works
The signal chain begins with microphone arrays converting pressure waves into voltage. It continues through analog amplification, digital conversion, feature extraction, wake-word detection, and cloud offload for full automatic speech recognition.
- Microphone arrays and analog front-end capture raw audio.
- ADCs sample at 12-bit or 16-bit resolution.
- Pre-processing applies noise reduction and gain control.
- Feature extraction computes MFCC or spectrogram data.
- The wake-word model decides whether to trigger the full pipeline.
Microphone Arrays to Analog Front-End
Your Echo Dot or Nest Mini holds multiple microphones spaced a few centimeters apart. These MEMS microphones convert pressure waves into tiny voltage changes. The analog front-end amplifies those signals and applies initial filtering to remove frequencies outside human speech.
The part nobody mentions is the gain staging. Each microphone channel needs its own programmable gain amplifier. Too much gain and you clip on loud sounds. Too little and distant voices disappear in noise. Devices typically run 12-bit to 16-bit ADCs here.
On a recent install I placed four different voice devices in the same room for a client. The one with the better analog front-end consistently picked up commands from the kitchen while others struggled. The difference lived in the pre-amplification stage before digital conversion.
The signal then moves to the digital section. (Espressif ESP32-S3 Technical Reference Manual, 2025)
ADC Sampling and Pre-Processing at 12-Bit Resolution
Consumer voice devices typically use 12-bit ADCs for the audio path. This gives 4096 discrete levels. At 16 kHz sampling rate the system captures enough data for speech while keeping computational load manageable.
The ADC runs continuously. Its output feeds a buffer managed by the RTOS. FreeRTOS handles these buffers with worst-case interrupt latency around 3 μs on ESP32 hardware. That timing matters when you need to keep audio frames aligned for later processing. (FreeRTOS Developer Documentation, 2025)
Pre-processing applies noise reduction and automatic gain control before the wake-word model sees any data. These filters run in fixed-point math on the same microcontroller that handles the network stack. (ARM Cortex-M4 Technical Reference Manual, 2024)
Feature Extraction Before Wake-Word Trigger
The system extracts MFCC features or similar spectrogram representations from the audio stream. A 512-point FFT forms the core of this step. On an ESP32-S3 with vector instructions this takes roughly 50 μs. The same operation on an STM32F4 using CMSIS-DSP takes about 120 μs. (TinyML Foundation Benchmarks, 2024)
These features feed the tinyML wake-word model. The model itself occupies 200-500 KB and consumes 1-2 mW when running continuously.
Wake Word Detection: The Always-On DSP Pipeline
Wake word detection refers to the continuous low-power analysis of audio streams by a compact on-device model that triggers the full voice assistant pipeline only when a specific phrase is recognized, keeping power draw to milliwatts until activation.
200-500KB TinyML Models and Power Budget
Wake-word detection models stay small by design. They only need to recognize one or two phrases. The entire neural network fits in a few hundred kilobytes and runs inference at milliwatt power levels.
Espressif designed the vector instruction unit in the ESP32-S3 specifically for this type of workload. "We designed the ESP32-S3 vector instruction unit specifically to enable on-device wake-word detection and simple ML inference. The goal was a $3 chip that can listen, not just connect," says Teo Swee Ann, CEO and founder of Espressif Systems (Espressif Developer Conference 2024). The ESP32-S3 carries a BOM cost of $2.50-$3.50 depending on flash and PSRAM configuration. (Espressif ESP32-S3 Technical Reference Manual, 2025)
RTOS Interrupt Latency for Real-Time Triggers
FreeRTOS runs on an estimated 40 percent of all embedded MCUs with an RTOS. Its context switch time on STM32F4 hardware sits between 2 and 5 μs. "FreeRTOS dominance isn't because it's the best RTOS. It's because it's free, well-documented, and runs on everything. Good enough wins in embedded," says Richard Barry, creator of FreeRTOS, Principal Engineer at AWS.
When the wake-word model triggers an interrupt the RTOS must move the audio buffer to the streaming pipeline without dropping frames. ST Microelectronics has shipped over 5 billion STM32 MCUs cumulatively. The Cortex-M4 remains the sweet spot for most IoT signal processing tasks. (ARM Cortex-M4 Technical Reference Manual, 2024)
ARM Cortex-M4 vs RISC-V Implementations
The ARM Cortex-M4 remains popular for these tasks. Its DSP instructions and hardware FPU handle the filtering and feature extraction efficiently. RISC-V implementations like the ESP32-C3 achieve similar results at lower BOM cost of $1.50 to $2.00 in volume. RISC-V cumulative chip shipments exceeded 10 billion in 2023.
How Does Beamforming Improve Voice Recognition Accuracy?
Beamforming combines signals from multiple microphones to focus on the speaker while suppressing noise from other directions. It can improve recognition accuracy by 20 to 40 percent in typical living room conditions compared to single microphone setups.
Delay-and-Sum vs Adaptive Beamforming
Delay-and-sum beamforming uses fixed coefficients. Adaptive beamforming tracks moving speakers by updating the filter coefficients in real time. Most consumer devices use a hybrid approach.
The difference becomes obvious in noisy kitchens. Fixed beamforming struggles with moving speakers. Adaptive versions maintain accuracy but consume more power and MIPS.
Multi-Mic Array Configurations in Devices
Google Nest devices typically use four microphones. Apple HomePod Mini uses a circular array of six. Amazon Echo units vary from three to seven depending on model.
What the Spec Sheet Doesn't Tell You About Latency?
Local Wake-Word to Cloud ASR Round Trip
The local wake-word detector fires within 200 to 400 milliseconds of the user finishing the phrase. Audio then travels to cloud servers for automatic speech recognition. Total round trip time usually lands between 500 ms and 2 seconds.
The device must continue recording while waiting for the cloud response without creating audible gaps or duplicated audio. (FreeRTOS Developer Documentation, 2025)
How Much Does Advanced Voice Assistant Hardware Cost in 2026?
The average cost of core electronics in a voice assistant device is $3 to $8 in 2026. The microcontroller or SoC might cost $2.50 to $3.50 in the ESP32 family. Microphones add another $1 to $2.
Premium units spend more on better microphone arrays and additional memory for larger on-device models. The $25 to $40 total BOM range appears consistently across teardowns.
Failure Mode Analysis in Real-World Deployments
Television audio creates the most common false wake triggers. Placement matters enormously. Devices near windows or kitchen appliances see higher false positive rates.
The noise suppression algorithms have limits. Two people talking at once usually confuses the system. Loud background noise like vacuums or power tools can mask commands completely.
Always-on listening creates steady power draw even in low power modes. Devices in hot environments throttle their processing to manage heat.
How Do Voice Assistants Integrate with Modern Smart Home Protocols?
Matter-certified devices reached 2,800+ as of March 2025. Matter 1.4 and later versions added improved support for energy management and better voice command interoperability across ecosystems. (Connectivity Standards Alliance - Matter, 2025)
Zigbee mesh hop latency sits at 10-30 ms per hop. A command crossing four hops experiences 40-120 ms total latency. Thread offers IPv6 native operation and self-healing mesh as the transport layer for Matter.
Home Assistant now exceeds one million active installations and provides a local-first alternative that reduces cloud dependency for users who prioritize privacy. (Home Assistant Statistics, 2024)
Comparing DSP Architectures: Alexa, Siri, and Google Home
| Architecture | 512-point FFT Time | Power Profile | Typical BOM Cost | Best For |
|---|---|---|---|---|
| ESP32-S3 | ~50 μs | Vector-accelerated | $2.50-$3.50 | On-device wake-word |
| STM32F4 (Cortex-M4) | ~120 μs | 100 μW/MHz | $1-$3 | General DSP filtering |
| Custom SoC | Variable | Highly optimized | Higher | Volume consumer devices |
| RISC-V (ESP32-C3) | Competitive | Low cost | $1.50-$2.00 | Cost-sensitive designs |
Custom SoCs from Amazon, Google, and Apple integrate dedicated neural accelerators. The ESP32-S3 completes operations faster thanks to its vector unit. This performance difference lets manufacturers either reduce power or add more sophisticated noise suppression. (Ambarella CV2x/CV5x Series, 2024)
On-Device vs Cloud Tradeoffs
Full on-device speech recognition remains limited to simple commands. Complex queries still require cloud resources. The privacy benefit of on-device processing comes with accuracy costs.
The signal processing that makes voice assistants work sits mostly out of sight. Tiny models running at milliwatt levels hand off to cloud servers that do the heavy lifting. Understanding where each piece runs helps learn more why some commands feel instant while others stumble.
The next improvements will come from better on-device models and tighter integration between the local DSP pipeline and the cloud backend. Your choice of hardware determines how much of that pipeline stays under your control.
Related resources: Persistent Agents and Always-On AI Systems. Hardware Risks AI Coding Cheatsheet 2026: Local Edge AI Costs
