Leap Motion: The Remote Control of the Future

by cgrune

Recently, Vimeo integrated the Leap Motion controller with Couch Mode (our full-screen viewing experience) to enable the couch-iest experience possible. No need for a mouse — you can control Couch Mode with just a swipe of your finger in the air.

To make this integration possible, we had to build a few things. For starters, we made a wrapper around the Leap Motion JavaScript API to more easily understand different types of gestures, such as directional swiping (left, right, up, and down). The API wrapper also allows developers to associate callbacks with different periods of gestures (start, during, stop), and adds some simple helpers to easily debounce, throttle, and delay callbacks based on the gesture or event executed.

Additionally, the viewer needs to be notified of finger position in relation of the screen. We wrote a simple utility to take the positions of the viewer’s fingers and draw them on a full-screen, transparent canvas element. The code to render the canvas had to be as performant as possible. And with a full-screen video playing in the browser, there weren’t too many resources left to draw the position data. We used the requestAnimationFrame API, as well as object pooling to minimize garbage collection and keep interactions smooth and snappy.

The final step was to implement a way for Leap Motion to interact with existing video controls we have in place, which are standard DOM elements. We created a small library called EPD (Element Proximity Detection) that allows us to determine if normalized coordinates of the viewer’s fingers are within a set proximity or directly over a video control element. We then could associate a callback with the entry and exit over the element. Then when the viewer hovers his or her finger over an interactive element for long enough, EPD executes that element’s function, and still allows for separate control of the mouse in case they need to switch the tab to something less awesome.

We load the Leap Motion assets on command, so if you have a Leap Motion controller plugged in, there will be an asynchronous call to get the aforementioned files and initialize the controller. If not, Couch Mode will load as it does normally.

Leap Motion allows the viewers to forget about the computer even more. Imagine fast-forwarding and rewinding videos with a quick circle of your finger. Pause or play a video with a simple tap in the air. Just sit back, relax, and swipe away to see the latest and greatest videos on Vimeo Couch Mode.

And if you have a Leap Motion controller, snag Couch Mode in the Airspace Store!

Post author

cgrune //

Notes

  1. djflizzyflamesnotes reblogged this from vimeo
  2. vimeo reblogged this from makingvimeo and added:
    Some real Gryffindor wizard-level magic going on over at our Makers of the Vimeo Blog.
  3. pile reblogged this from makingvimeo
  4. okseanjay reblogged this from makingvimeo
  5. ridiculouslyawesome reblogged this from makingvimeo
  6. makingvimeo posted this