Researchers from Cornell University and KAIST have unveiled WatchHand, an innovative system that transforms ordinary smartwatches into real-time hand-tracking devices using AI-driven sonar. By leveraging built-in speakers and microphones, the technology emits inaudible sound waves that bounce off the user’s hand and return as echo signals. These signals are then processed by on-device machine learning algorithms to reconstruct detailed 3D hand poses, enabling precise tracking of finger and wrist movements without requiring cameras or external sensors.
This approach marks a significant shift in wearable interaction design. Unlike traditional hand-tracking systems that depend on bulky hardware or vision-based inputs, WatchHand works entirely through software, making it scalable across millions of existing devices. The system was tested on 40 participants and demonstrated reliable performance across various smartwatch models, hand positions, and environmental conditions. By eliminating the need for additional hardware, the technology lowers the barrier to entry for advanced gesture-based control.
