First impressions on UI design with Leap Motion

Reply
Avatar
by doctorwes

I got my Leap Motion last week and have spent a couple of days working with the Python API, implementing a Leap interface for an in-house 3D visualization tool. I have a pretty positive impression so far, with some reservations:

  • The "right" UI seems to be pretty application-specific. Ideas that work in one setting don't necessarily carry over well to another. I think it's going to be difficult to standardize for that reason.
  • Figuring out an approach that works well takes a lot of experimentation. I tried three or four different approaches before finding one that worked reasonably well and wasn't too flaky. It's still not perfect.
  • Even with an approach that works okay, interaction takes practice. The UI needs to be explained to a new user, since the movements they make intuitively aren't the ones that work.
  • The device is totally unsuitable as a replacement for mouse+keyboard, or even mouse/button interaction. On the other hand, it is a good fit for this particular application, though still not obviously better than using a XBox controller, except that the device is much less bulky, looks more professional and has more of a cool factor.

More specifics, which hopefully help explain some of my impressions:

  • Significant issues/instabilities: fingers appear/disappear; hand sphere radius is not useful; some hand rotations not reliably recognized; have to figure out how user can move hand into position without inadvertently triggering actions
  • Worked well enough for me: swipe and tap gestures (these take some practice, especially key taps; also, swipe gestures can get confused with circle gestures); two-finger swipes (horizontal ones are recognized pretty well, but not vertical ones)
  • Worked very well for me: circle gestures (these are recognized really well); single fingertip positions (these are tracked pretty reliably; the trick was to wait till the fingertip was stationary before interpreting further motions as actions)
  • Trying to use frame motions was a tantalizing dead end. Translations are recognized pretty well, but just did not work particularly well for the specific application. Rotations are not recognized as well, and I had even more trouble with scaling. I'm pretty optimistic these will be useful elsewhere.
  • I didn't try using the touch emulation.

My main takeaways were: be patient; don't just try to force your first approach to work, but keep trying other things; and don't assume you'll get satisfactory results mimicking someone else's UI if the application is different.

Avatar
by Grad_Student

Can you post some code from the UI you built? I'm just getting started and I'm trying to decide if I wanna use python or c#. I currently have no idea of how to go about doing this!