The answer to your question might be kinda complicated. But also I believe the answer is in the Orion documentation.
You can fetch about 400 pieces of information from each frame. What I did was make a program to record frames and use the data for pattern matching. So a basic example I can record "hand horns" and name the pattern "Rock On" and my basic pattern matching algorithm will put "rock on" on the screen when I do hand horns. 
So I suggest you take a look at the included documentation in Orion and make a program to record the data so you can see everything that's happening.
