Hire MeProjectsContact

Sonar Based Gesture Recognition

2017

This project won first prize for Senior Projects at the University of Pennsylvania

This is a gesture recognition platform using sound processing. The goal of this project was to have IOT lights identify when a phone was gestured towards them while holding a phone.

To identify when a phone is gestured towards a particular device:

  1. We start playing a ping(linear chirp to be specific) at regular intervals on a phone
  2. All IOT lights in the room start listening
  3. The user then extends their arm in the direction of the light they wish to turn on
  4. The lights each identify how many meters the phone moved in their direction using the offset in the expected ping arrival times
  5. The light with the greatest gesture distance towards it turns on

Final Project Report

A sample collection of stacked pings from a receiver. We can make out the amplitude of the wave visually but there is lots noise due to reflections in sound other ambient noise.

We can use a hidden markov model to find the most likely path. The emission probabilty captures the tendency for the phone to not jump large distances in small periods of time and does a good job of creating an accurate measurement of the changing phone distance.