One of the most fascinating applications of AI is humanoid robots. As audio specialists, we're particularly interested in the audio systems onboard, and these systems are evolving rapidly. Figure AI’s Helix model represents a shift in both humanoid robot capabilities and how audio is handled. It combines visual understanding, natural language comprehension, and physical control into a unified system. The result? Robots that follow verbal instructions and perform physical tasks, all while adapting to new environments. Here’s a quick overview, as well as how audio factors into such systems.

A Two-Brain System: Language + Motion

Helix is structured as a dual-model system:

• System 2: A 7-billion-parameter multimodal model that handles high-level reasoning. It processes RGB-D (includes depth) camera input and speech commands to understand intent.

• System 1: An 80-million-parameter motion model that handles joint-level execution across 35 degrees of freedom—fingers, wrists, torso, and head—at 200Hz. This system is optimized to be super fast.

The models exchange information through shared latent representations, allowing abstract instructions like “put the milk in the fridge” to flow smoothly into real-world movements. System 2 allows the robot to “think slow” while System 1 can “think fast” and adjust in real time.

Vision-Language-Action Integration

Most robots follow a rigid pipeline—first they perceive, then plan, then act. Helix trains all these steps together in one neural network using human demonstration data. This structure allows it to handle previously unseen objects by grounding language (e.g., “soft,” “slippery”) to visual features learned at scale.

For example, given the command “Pass the cereal box to the other robot,” Helix maps that instruction to both a visual search pattern and a series of handoff actions—without hardcoding.

Embedded and Efficient

Helix doesn’t rely on the cloud. It runs entirely on embedded GPUs (Jetson Orin), using 4-bit quantization and model parallelism to stay under 60W. This design delivers sub-100ms control loop latency—critical for responsive, safe operation around humans—and makes Helix viable in environments with poor connectivity.

Real-Time Audio Processing in Helix

Audio is central to Helix’s ability to understand and respond to human intent. The system continuously processes spoken commands through onboard microphones using a speech recognition pipeline integrated into the multimodal model.

Key Characteristics:

• Embedded Speech-to-Text (STT): Likely built on quantized transformer-based models for latency and efficiency.

• Multimodal Fusion: Audio is fused with visual data to disambiguate intent. For example, “Give me that” is grounded visually via attention over camera input.

• Low-Latency Feedback: Sub-100ms audio command-to-action pipeline enables natural interaction pacing for tasks like collaboration, correction, or clarifying questions.

Potential Areas for Future Improvement:

We're speculating here, but there's likely interesting work to be done including:

• On-device speaker diarization and emotional tone detection to improve multi-human interaction.

• Noise robustness in environments like kitchens or workshops.

• Bidirectional interaction with real-time voice synthesis for robots that can ask clarifying questions or explain actions.

As Helix evolves, more advanced real-time audio features—such as interruptibility, conversational memory, and continuous background listening with energy efficiency—will be key to scaling up interaction complexity.

We will be watching the space closely.