Kinect Hacking Tutorial: Begin by using FAAST

Maybe you are new to Kinect, like I was, and need help getting started. Here is a tutorial on how to use the FAAST to begin hacking and fiddling with your Kinect.

Introduction:

FAAST is middleware to facilitate integration of full-body control with games and VR applications. The toolkit relies upon software from OpenNI and PrimeSense to track the user’s motion using the Microsoft Kinect sensors. FAAST includes the body movements and the motion of at most 24 different body skeleton joints. Additionally, the toolkit can also emulate keyboard input triggered by body posture and specific gestures. This allows the user add custom body-based control mechanisms to existing off-the-shelf games that do not provide official support for depth sensors.

Download FAAST from here: Download FAAST 0.07

Kinect can be set up on a computer in two methods:

1. OpenKinect drivers
2. PrimeSense drivers

The OpenKinect driver set up is tricky and will be covered in later stages and hence the OpenNI set up will be explained here.

OpenNI is the Open Networked Interconnection tool-kit which is used to interface sensor devices like Kinect with the computer and read the readings coming from them.

For setting up FAAST, proceed as shown on the following steps:

Download and install each hyperlinks.

1. OpenNI Unstable Build for Windows v1.0.0.25

1. PrimeSense NITE Unstable Build for Windows v1.3.0.18 (works as the middleware) During NITE installation, enter the free license key from OpenNI: 0KOIk2JeIBYClPWVnMoRKn5cdY4=
2. Hardware drivers for your sensor: Microsoft Kinect: SensorKinect Module – download Win32 installer from Bin folder

After following the above steps, you can open the FAAST folder and run the executable file. It will run out-of-the-box. If not, the Microsoft Visual C++ 2008 SP1 Redistributable Package may be missing hence please install it to move ahead.

Opening the FAAST software, it looks like as shown below:

Image

The software has the button ‘connect’ to make it connect with your Kinect sensor. If you have installed the things correctly, after pressing connect, you will see the message as:

Quote:
Attempting to initialize OpenNI sensor… Sensor initialized successfully. Attempting to start VRPN server… Success!!

Now, seeing this, your Kinect is connected with the FAAST engine and has started looking for a human pose in front of it.

Image

For getting calibrated (locked) in the camera, one has to pose in front of the camera posing as the letter Psi.

As shown in the figure below:

Image

In a matter of a second or two, it will calibrate you as a human and start tracking your movement.

Once the tracking is done, you can perform any actions based on your movements. To write the action-movement set, one has to edit the Bindings file provided in the FAAST folder. The Bindings file contains the data of the one-to-one relationship with the gesture and its corresponding action.

The syntax to write one gesture-action set is given as:

Code:
action_name action_threshold virtual_event_type virtual_event_name

action_name can be anything from the list provided below.
action_threshold is the movement of the action in Inches.
virtual_event_type is the type of the event that you want to perform with the action.
virtual_event_name is the event that you want to perform with the action.

For example:

Code:
right_arm_forwards 10 key_press w

This will virtually press the key ‘w’ on the keyboard or type ‘w’ on the notepad whenever the right arm is moved forward by 10 inches with respect to the shoulder.

When you are done with writing the Binding file, Load it and then move to the ‘Action’ tab in FAAST and press ‘Start Input Emulation’.

This will start the execution according to the binding file you wrote and perform the virtual actions based on that. When not in use, press ‘Stop Input Emulation’.

The movements supported by FAAST are:

Action Name Type Threshold
lean_left skeleton angular body lean left (degrees)
lean_right skeleton angular body lean right(degrees)
lean_forwards skeleton angualr body lean forwards (degrees)
lean_backwards skeleton angular body lean back (degrees)
turn_left skeleton angular amount of left body turn (degrees)
turn_right skeleton angular amount of right body turn(degrees)
left_arm_forwards skeleton forward distance from left hand to shoulder (inches)
left_arm_down skeleton downward distance from left hand to shoulder (inches)
left_arm_up skeleton upward distance from left hand to shoulder (inches)
left_arm_out skeleton sideways distance from left hand to shoulder (inches)
left_arm_across skeleton sideways distance from left hand across body to shoulder (inches)
right_arm_forwards skeleton forward distance from right hand to shoulder (inches)
right_arm_down skeleton downward distance from right hand to shoulder (inches)
right_arm_up skeleton upward distance from right hand to shoulder (inches)
right_arm_out skeleton sideways distance from right hand to shoulder (inches)
right_arm_across skeleton sideways distance from right hand across body to shoulder (inches)
left_foot_forwards skeleton forward distance from left hip to foot (inches)
left_foot_sideways skeleton sideways distance from left hip to foot (inches)
left_foot_backwards skeleton backwards distance from left hip to foot (inches)
left_foot_up skeleton height of left foot above other foot on ground (inches)
right_foot_forwards skeleton forward distance from right hip to foot (inches)
right_foot_sideways skeleton sideways distance from right hip to foot (inches)
right_foot_backwards skeleton backwards distance from right hip to foot (inches)
right_foot_up skeleton height of right foot above other foot on ground (inches)
jump skeleton height of both feet above ground (inches)
crouch skeleton crouch distance, calculated as current height subtracted from standing height (inches)
walk skeleton height of each step above ground when walking in place (inches)

The corresponding Virtual event list is given as:
Virtual Event Type Virtual Event Name
key_press key to press when the action is first detected (either a single character or a special key from the table below)
key_hold key to hold down while the action is performed (either a single character or a special key from the table below)
mouse_click mouse button to click when the action is first detected (left_button, right_button, or middle_button)
mouse_double_click mouse button to double click when the action is first detected (left_button, right_button, or middle_button)
mouse_hold mouse button to hold down while the action is performed (left_button, right_button, or middle_button)
faast input emulator control event (pause, resume, or stop)

Hope this helps people out there like me!

is this primarily for games with the Kinect?

very interesting, I will try it once I get my new Kinect!

Great guide. I believe FAAST is more on key mapping certain gestures to existing programs. Games just get the bulk of the FAAST’s benefits because it already has programmed movements. All you need is to map gestures to the movements and you’re good to go.

But I’ve seen other uses of the FAAST and that involves starting programs from scratch. It takes a while but the results are great. An example is that you can also use FAAST to create a program for iTunes. That way, you can switch songs browse your favorite music with the Kinect. Try it, I’m already starting on that idea.

version 0.08 is available here : http://projects.ict.usc.edu/mxr/faast/

Words from the site itself:

“FAAST 0.08 has been released. This version upgrades to the latest versions of OpenNI and NITE. The sensor resolution settings have been removed from the GUI, since this is now set automatically by OpenNI. Also added capability to save/load calibration files, and added a key_type event. You will need to completely uninstall the older versions of OpenNI, NITE, and the sensor driver, and then install the new ones listed below.”

Since the OpenNI recently released an updated version of itself, FAAST was “fast” enough to update their own data base. Good job on their part. The feature of saving and loading calibration settings is great. Going to give it a spin later. Right now, I was wondering if the Kinect SDK will have the FAAST toolkit’s features.

If that happens, will this one will be outdated? Shiver shiver me timbers 😳 😳 😳

@ alphadaawg : since the new version of FAAST, the installation procedure goes like this (on their website)

I’m trying to get it to work… is this what you did? I have a 64 bit computer.
Thks!

Hey man, I’m having problems installing mine too. I have a 64 bit pc. what did you do?