Apple researchers have created an AI model that identifies hand gestures absent from its training data. Dubbed EMBridge, this framework uses electromyography (EMG) signals to achieve superior gesture recognition through cross-modal learning. The findings appear in a new study scheduled for the ICLR 2026 Conference.
Understanding EMG Technology
Electromyography (EMG) detects electrical activity produced by muscles during contraction. It supports medical diagnostics, physical therapy, prosthetic controls, and modern wearables plus AR/VR systems. Meta’s Ray-Ban Display glasses, for example, incorporate a wrist-based Neural Band that reads muscle signals to operate features, according to Meta’s product details.
Apple’s work employs EMG signals paired with hand pose data from established datasets, focusing on generalization beyond initial training.
How EMBridge Works
EMBridge aligns raw EMG data with structured hand pose representations. The process starts with separate pre-training of EMG and pose encoders. Researchers then synchronize these to let the EMG component learn from pose insights, capturing underlying gesture patterns.
Further refinement uses masked pose reconstruction: the model hides sections of pose data and rebuilds them using only EMG inputs. This boosts accuracy for novel gestures.
To minimize errors from similar gestures labeled as unrelated, the system identifies comparable hand shapes and applies soft targets. This organizes the model’s space, enhancing zero-shot performance on unfamiliar inputs.
Impressive Results on Benchmarks
Tests on emg2pose and NinaPro datasets show EMBridge outperforming competitors, particularly in recognizing never-seen gestures. It delivers strong results with only 40% of standard training data.
Challenges and Future Impact
The approach requires synchronized EMG and pose datasets, which are hard to gather. Yet, rising interest in EMG controls positions EMBridge for wearables like future smartwatches or glasses. Potential uses include seamless device interactions with Vision Pro, Macs, iPhones, and more, advancing accessibility and user experience.
