leapjs

Developer Guide

This is a terse guide to the classes and methods of the Leap javascript API. The Leap object is a singleton library, like jQuery's $ object. It is your bridge to the stream of data recieved from the Leap detector.

Methods

Classes

Do not taunt Happy Fun Ball! The only class you should manually instantiate is the Controller. (and for most applications you'll only want one controller instance.) None of the other classes should be manually created by you -- they should be retrieved by methods of the controller -- or methods of objects retrieved by controller methods ... and so on.

Getting Frame Data from your Leap Motion detector

In order to get information from your Leap Motion detector you need

  1. a Leap Motion Detector plugged into your computer
  2. the Leap Motion software installed.

Once this is established, you need to interact with the data coming from the hardware. This site covers interactions based on the javascript API library that binds Leap Motion data to a javascript code that listens for this data to stream to it.

There are three ways to access this data:

  1. Call Leap.loop, as shown in this jsfiddl.e
  2. Create a controller and passively listen for frame events.
  3. Create a controller and actively call controller.frame() when you want motion data.

Event listening from the static Leap class

Leap.loop(handler) allows you to skip creating a controller instance.

Leap.loop(function(frameInstance){
  ...
  });

Event listening from a Controller

if you listen to a controllers' events each and every frame will be broadcast from the Leap detector to your handler. 

  var my_controller = new Leap.Controller({enableGestures: true});
  // see Controller documentaion for option details
  my_controller.on('frame', function(frame_instance){ ... });
  my_controller.connect();

Polling a Leap.Controller

Polling will allow you to get one (or more) frames from the controller's stack, whenever and as often or rarely as you wish.

  var my_controller = new Leap.Controller({enableGestures: true});
  my_controller.on('connect', function(){
    setInterval(function(){
      var frame = my_controller.frame();
    }, 500);
  });
  my_controller.connect();

This has several potential advantages:

  1. You can control the number of measurements yourself; you might only need 10 frames of information per second, so why do six times as much processing? tying your polling to your rendering is more efficient since there may be no value to getting more than one frame per render cycle.
  2. You can get the best sample out of several measurements. You can sample the last 8 frames, and get the one with the most pointers/fingers/hands and use that to drive your application.
  3. You can choose whether or not to take signal data. After the user pushes a virtual button, you can stop listening while you resolve an action, then pick up listening later on.

Units of Measurement

All distances are expressed in millimeters, as a float. If you want to get relative (percent) measurements, use the InteractionBox.normalizePoint method. See the Leap.InteractionBox documentation for axis orientation.

All angles are measured in Radians. To convert to degrees, multiply by 180/Math.PI.

All time/timestamp measurements are given in milliseconds; 1000 milliseconds = 1 second. Timestamps are relative measures of time since the Leap Motion controller started.

Frame indexes reflect the LIFO nature of the Leap.Controller's frames collection.

  • my_controller.frames(0) == my_controller.frames() == the most recent frame.
  • my_controller.frames(1) is the previous frame
  • my_controller.frames(1) is two frames ago

and so on. Keep in mind that frames are constantly being pushed onto the stack, so my_controller(2) will be a different frame in a few milliseconds. When in doubt use frame.id to identify/compare frames.

All coordinates/positions are returned as an array of three distances (see above). See Leap.InteractionBox for methods on normalizing/denormalizing coordinates.

scaleFactor

Leap.Hands and Leap.Frames have scaleFactor functions. Scale factor indicates a change in distance between things over time. For hands, this means the change in distance between the fingers. the function has a parameter -- a baseline frame.

Scale factor for hands

Hands' scale factor reflects the relative "Jazz handiness" of your fingers, compared with the baseline spread in thei passed in frame.

  • If you pinch all your fingers together since the reference frame, your hand's scaleFactor will be < 1.
  • If you expand ("jazz hands") your fingers farther apart, your hands' scaleFactor will be > 1.

Scale factor for frames

When frame.scaleFactor(old_frame) is called, it reflects the relative distance between the hands -- the "fish measuring factor".

  • if you move your hands apart, the scale factor will be > 1
  • if you move your hands together, the scale factor will be < 1

If Leap cannot relate the baseline reference with the current frame/hand data, the scaleFactor will be 1.

Valid and Invalid function return results

Any function which returns an instance object (Pointable, Frame, Hand,...) will ALWAYS return an object, even under conditions where returning an object is impossible. (bad parameters, your hands aren't in front of the detector, etc.) Instead of returning false, null, or throwing an error, an invalid instance will be returned.

This means you will want to examine the valid property of all returned objects. Its a boolean property and all Leap instances have one.

Leap Classes

Leap.Frame

A collection of state information fed back from the Leap Motion hardware. A frame is the "root" data unit; it contains all the positional data that streams from the Leap Motion detector, Fingers/Tools/Pointables at a given instant in time.

See Getting Frame Data from your Leap Motion controller for documentation on getting frames.

A note on Frame.Fingers, Frame.pointables, and Frame.tools:

frame.fingers, frame.tools and frame.pointables are different collections of instances which are all instances of the Leap.Pointables base class. (there is no special Leap.Tools or Leap.Fingers classs -- just Leap.Pointable.)

  • The pointables collections contains all the pointables also found in tools and fingers.
  • The tools collections and the fingers collections are exclusive; there is no pointer which can be found in both the fingers collection and the tools collection
  • All pointables in these root collections can be found in the frame.hands properties..
  • Any of these collections can be empty, if the user's hands/fingers/tools aren't picked up by the Leap Motion detector.
  • All fingers/tools/pointables from both hands are stored with no ordering in these root level collections.

Properties

id
string A unique id of this Frame instance
pointables
[Leap.Pointable] array of all fingers and tools from both hands.
fingers
[Leap.Pointable] array of all fingers from both hands.
tools
[Leap.Pointable] array of all tools from both hands.
gestures
[Leap.Gesture] array of zero or more detected gestures. Requires configuration of the Leap.Controller to be present (see enableGestures.)
hands
[Leap.Hand] array of 0..2 Leap.Hands.
timestamp
int microseconds since the Leap detector started
valid
boolean indicates whether the frame is valid. An invalid frame contains no tracking data but does conform to the frame API.
Invalid
Leap.Frame an invalid instance of a frame.

Methods

dump
(): string returns a JSON-formatted string (not object) containing all the data of the frame.
finger
(id: string *): Leap.Pointable returns a finger from the fingers collection(see above).
pointable
(id: string *): Leap.Pointable returns a pointable from the pointables collection (see above).
hand
(id: string): Leap.Hand returns a hand from the hands collection (see above).
translation
(sinceFrame: int): [int/mm] The movement, in millimeters, of both hands since the passed-in timestamp,
scaleFactor
(sinceFrame: int): float The scale factor between both hands since the passed-in timestamp.
rotationAngle
(sinceFrame: int, axis: int): float (radians -- 0..π) The angle of rotation around the rotation axis (0, 1, or 2) of both hands since the passed-in timestamp The returned angle is expressed in radians measured clockwise around the rotation axis (using the right-hand rule).
rotationAxis
(sinceFrame: int): [int] the information described in rotationAngle, for all three axes, of both hands since the passed-in timestamp
rotationMatrix
(sinceFrame: integer): [int] The transform matrix expressing the rotation derived from the overall rotational motion, of both hands since the passed-in timestamp, The Leap derives frame rotation from the relative change in position and orientation of all objects detected in the field of view.
* Warning: IDs can change between frames -- there is no guarantee that an ID from a previous frame will work as a parameter for subsequent calls to frame.finger(). the IDs are only functional within the lifespan of a single frame.

Leap.Pointable

The Pointable class represents detected finger or tool **. Both fingers and tools are classified as Pointable objects. Use the Pointable.tool property to determine whether a Pointable object represents a tool or finger Note that Pointable objects can be invalid, which means that they do not contain valid tracking data and do not correspond to a physical entity. Invalid Pointable objects can be the result of asking for a Pointable object using an ID from an earlier frame when no Pointable objects with that ID exist in the current frame.

Properties

id
string
Invalid
Pointable an invalid instance of a Pointable.
length
number (mm)
tipVelocity
[number] the rate of change of the tip's position in mm/second.
tipPosition
[number/mm] the distance from the Leap Detector to the tip of the finger (raw data).
stabilizedTipPosition
[number/mm] the distance from the Leap Detector to the tip of the finger, stabilized
direction
[number] (three numbers) the direction the pointer is pointing.
width
number (mm) the est. diameter of the tool. Finger fatness.
tool
boolean whether the finger is classified as a finger or a tool. (see Leap.Frame).
valid
boolean

Methods

** The Leap classifies a detected entity as a tool when it is thinner, straighter, and longer than a typical finger.

Leap.Controller

Represents the connection to a single Leap detector. You can have multiple controllers in a single run time. Note that Leap.Controller is a Javascript class; whereas the Leap Motion controller refers to an actual piece of hardware -- the box.

the below code creates a Leap.Controller, listening to localhost.

  var controller = new Leap.Controller({
  host: '127.0.0.1',
  port: 6437,
  enableGestures: true,
  frameEventName: 'animationFrame'
});

Any or all these properties are optional; you can also create a controller with a simpler parameter set:

  var controller = new Leap.Controller({enableGestures: true});

Properties

connection
Leap.Connection the object maintaining communication between the Leap detector and the Controller instance.

Methods

connect
() initializes communication/events between the Leap Detector and the controller. (if it's plugged in...)
frame
(index: int?) Leap.Frame Index is a number from 0 to the number of frames retained in the "frame stack" (currently up to 60 frames are retained; however when the controller starts, the stack may have fewer than 60 frames. The most recent frame's index is 0 (zero); the frame before that has index 1, and so on. my_controller.frame() will return the most recent frame.***
on
(event: string, handler: function) binds a listener function to an event.
*** this method ALWAYS returns a frame, even if there is no frame at the stack matching the index you pass in. If you pass in an index for which there is no frame, an invalid frame will be returned.

Controller Events

connect
The client is connected to the websocket server
frame
A frame is finishing being processed by the controller. This event is either driven by the animationFrame or the deviceFrame event, depending which is the Leap.Controller was created with. The frame is passed as an argument to the event handler
ready
The protocol has been selected
disconnect
The client disconnects from the websocket server
focus
The browser received focus
blur
The browser loses focus
deviceConnected
A Leap device has been connected
deviceDisconnected
A Leap device has been disconnected
animationFrame
A frame being emitted in time with the animation loop. The frame is passed as an argument to the event handler
deviceFrame
A frame being emitted by the connection to the Leap device. The frame is passed as an argument to the event handler
protocol
The protocol has been selected for the connection. The protocol object is passed as an argument to the event handler

Here is a typical initialization cycle for interacting with a Leap Motion controller:

var controller = new Leap.Controller();
controller.on('connect', function() {
  console.log("Successfully connected.");
});
controller.on('deviceConnected', function() {
  console.log("A Leap device has been connected.");
});
controller.on('deviceDisconnected', function() {
  console.log("A Leap device has been disconnected.");
});
controller.connect();

Leap.Connection

@TODO: document

Leap.Hand

The Hand class reports the physical characteristics of a detected hand. Hands are accessed as properties of the Leap.Frame's hand array; at this point, up to two hands can be simultaneously tracked and returned with frame data. Hand tracking data includes a palm position and velocity; vectors for the palm normal and direction to the fingers; properties of a sphere fit to the hand; and lists of the attached fingers and tools.

See the Leap.Frame class documentation for the difference between Pointables, Fingers and Tools.

The below illustration shows the hands' palmNormal

Properties

id
string Note that hand IDs unlike many other IDs in the system, are relatively consistent from frame to frame, at least in the short term. When comparing two frame.hands arrays, there is no guarantee that the hands will be in the same order, but if both frames have two hands, you can likely match up hands by comparing their ids.
Invalid
Hand an invalid instance of a Hand.
direction
[number] (three numbers) the direction the hand is pointing; roughly speaking, a vector from the palm to the midpoint of the fingers.
palmNormal
[number] (three numbers) the direction the palm is pointing.
palmPosition
[float/mm] the location of the center of the palm, in millimeters.
stabilizedPalmPosition
[number] (three numbers) the location of the center of the palm, in millimeters -- stabilized
palmVelocity
[number] (three numbers -- mm/second) The rate of change of the palm position in millimeters/second.
sphereCenter
[number] (three numbers) the center of a "virtual sphere" that the hand is holding; see the illustration below. "cupping your fingers" makes the ball smaller, and therefore, brings the sphereCenter closer to your palm.
fingers
[Leap.Pointable] an array of the hands' Pointables which have been classified as fingers.
tools
[Leap.Pointable] an array of the hands' Pointables which have been classified as tools.
pointables
[Leap.Pointable] an array of all of the hands' Pointables.
valid
boolean whether or not the hand is valid.

Methods

finger
(id: string *): Leap.Pointable returns a finger from the fingers collection(see above).
pitch
number (-π...π) radians Rotation around the x-axis; or, the angle between the negative z-axis and the projection of the vector onto the y-z plane. If the vector points upward, the returned angle is between 0 and pi radians (180 degrees). If it points downward, the angle is between 0 and -pi radians.
roll
number (-π...π) radians rotation around the z-axis; or, the angle between the y-axis and the projection of the vector onto the x-y plane. If the vector points to the left of the y-axis, then the returned angle is between 0 and pi radians (180 degrees). If it points to the right, the angle is between 0 and -pi radians.
yaw
number (-π...π) radians rotation around the z-axis; or, the angle between the y-axis and the projection of the vector onto the x-y plane. If the vector points to the left of the y-axis, then the returned angle is between 0 and pi radians (180 degrees); if it points to the right, the angle is between 0 and -pi radians.
rotationAxis
(sinceFrame: Leap.Frame): [number] The transform matrix expressing the rotation derived from the change in orientation of this hand, and any associated fingers and tools, between the current frame and the specified frame. If a corresponding Hand object is not found in sinceFrame, or if either this frame or sinceFrame are invalid Frame objects, then this method returns an identity matrix.
rotationAxis
(sinceFrame: Leap.Frame): [number] The axis of rotation derived from the change in orientation of this hand, and any associated fingers and tools, between the current frame and the specified frame. The returned direction vector is normalized. If a corresponding Hand object is not found in sinceFrame, or if either this frame or sinceFrame are invalid Frame objects, then this method returns a zero vector.
rotationAngle
(sinceFrame: Leap.Frame, axis: [number]?): [number] returns the rotation relative to a reference Frame. If a second parameter is passed in, the angle is the reference angle to measure around.
scaleFactor
(sinceFrame: Leap.Frame): number: 0..? This is a "Pinchiness factor". scaleFactor for hands the change in the relative extent of all fingers of the hand between the reference frame and this frames.
translation
(sinceFrame: Leap.Frame): [float/mm] The relative position(movement) of the hand (palm) since the given frame, in millimeters If Leap cannot relate this hand with a hand in the passed-in frame, the scaleFactor will be 1.

Leap.InteractionBox

A representation of the "airspace" in which the Leap Motion controller can measure/see your hands and fingers. Note that the range of the interaction box may change with hardware or software advances. Use the interaction boxes' properties to scale your measurements -- don't hard code these values in your javascript.

the Axes' orientation

  • The x-axis goes to the right. 0 on the x axis is the middle of the Leap Motion controller.
  • The y-axis goes up, from the Leap Motion controller; you will likely have to negate it to make it consistent with screen/DOM graphic coordinates. 0 on the y axis is very close to the Leap Motion detector.
  • The z-axis points towards you. 0 on the z axis is right above the detector. Your monitor will be at -Z, your head will have a +Z measurement.

Properties

center
[float/mm] The center of the interaction box, in millimeters
depth
[float/mm] the z-size of the detection area.
height
[float/mm] the y-size of the detection area.
width
[float/mm] the x-size of the detection area.
valid
boolean whether the InteractionBox is valid
Invalid
InteractionBox an invalid InteractionBox instance.

Methods

normalizePosition
([number/mm): [number/-1..1] Scales the imput vector by the height/width/depth of the InteractionBox, returning a vector with values in the -1...1 range.
denormalizePosition
([number/-1...1]): [number/mm] takes a "normalized" vector array and returns a vector array in the units of the interaction box; the reverse of normalizePosition.

Gesture

The Gesture class represents a recognized movement by the user. The Leap watches the activity within its field of view for certain movement patterns typical of a user gesture or command. For example, a movement from side to side with the hand can indicate a swipe gesture, while a finger poking forward can indicate a screen tap gesture. When the Leap recognizes a gesture, it assigns an ID and adds a Gesture object to the frame gesture list. For continuous gestures, which occur over many frames, the Leap updates the gesture by adding a Gesture object having the same ID and updated properties in each subsequent frame. Important: Recognition for each type of gesture must be enabled; otherwise no gestures are recognized or reported. Subclasses of Gesture define the properties for the specific movement patterns recognized by the Leap.

The Gesture subclasses for include:

CircleGesture
A circular movement by a finger
SwipeGesture
A straight line movement by the hand with fingers extended.
ScreenTapGesture
A forward tapping movement by a finger.
KeyTapGesture
A downward tapping movement by a finger.

Number of gestures produced

Circle and swipe gestures are continuous and these objects can have a state of start, update, and stop.

The screen tap gesture is a discrete gesture. The Leap only creates a single ScreenTapGesture object appears for each tap and it always has a stop state.

Getting Gesture instances from a Frame object.

You can get a list of gestures from the Frame gestures array. You can also use the Frame gesture() method to find a gesture in the current frame using an ID value obtained in a previous frame. Gesture objects can be invalid.

Properties

id
number All gestures belonging to the same continuous gesture will have the same ID. Can be passed into my_frame.gesture(my_gesture.id) to get updated versions of a gesture.
duration
int the duration (in microseconds) of the gesture, as of the frame from which the gesture was retrieved.
pointableIds
[string] the ids of fingers associated with this gesture (if any)
handIds
[string] the ids of hands associated with this gesture (if any)
state
string/'start', 'update' or 'stop' where in the "clown car o' gesture" stream the current gesture is -- the first, the last, or somewhere in the middle. Tap gestures will always have a 'stop' state.
type
string/circle, 'swipe', 'screenTap', or 'keyTap'

© leapmotion 2013