Kinect 2 Tutorial on Tracking Strategies

Microsoft has published a set of great tutorials for making applications with Kinect 2. http://kinect.github.io/tutorial/ These detailed hands on Lab/Tutorials for V2 were published last month.

I highly recommend the tutorial on tracking strategies - for keeping the application accurate when multiple people are in the room.

http://kinect.github.io/tutorial/lab14/index.html

This lab will explain the following:

Tracking a user who is the first to be seen.
Tracking the user who is the closest to the Kinect 2.
Tracking a user who is the most active in the scene.
Tracking several people simultaneously.

Kinect SDK 1.5 is out

Slowly and steadily – the Kinect SDK is adding awesome new features. The latest release v 1.5 has seated mode skeletal tracking, improved skeletal tracking and the Kinect Studio (for easier record/playback). This release comes at a great time, even as LeapMotion is opening up another market in gesture technology. Just like Kinect did almost two years back, Leap Motion is changing the way people think about interfaces. I think both Kinect and Leap Motion target different markets, but if Microsoft is looking at using Kinect for precision hand tracking, Leap Motion is showing the way.

While Leap Motion is a few months away, Kinect SDK is getting great developer support. Carl Franklin has developed a Gesture recording and recognition toolkit – GesturePak (compatible with Kinect SDK V 1.5) – app that records you making your gestures and an SDK for .NET 4.0 that notifies your code when the user has made those gestures. This seems very useful as well.

Read the complete release by Criag Eisler - Kinect for Windows: SDK and Runtime version 1.5 Released

Seated mode skeletal tracking
Provides the ability to track users’ upper body (10-joint) and overlook the lower body if not visible or relevant to application. In addition, enables the identification of user when sitting on a chair, couch or other inanimate object.

Improved skeletal tracking
In near range, users who are seated or standing can be tracked within 40 cm (16 inches) of sensor. Plus, the skeletal tracking engine is now faster, making better use of the CPU and scaling of computer resources. In addition, newly added joint orientation information for skeletons is ideal for avatar animation scenarios and simple pose detection.

Face tracking capabilities
Makes it possible to fit a 3D mesh to users’ faces and track their facial features and head position in real time using components from the Developer Toolkit.

Developer toolkit
Provides components, libraries, tools, samples, Kinect Studio, and other resources to make it easier to develop applications using the Kinect for Windows SDK

Kinect Studio
Simplifies the review and testing of applications, using Kinect Studio to record, playback, and debug Kinect data.

Speech recognition options
Along with English, offers French, Spanish, Italian and Japanese speech recognition along with language packs to recognize the way a language is spoken in different regions: English/Great Britain, English/Ireland, English/Australia, English/New Zealand, English/Canada, French/France, French/Canada, Italian/Italy, Japanese/Japan, Spanish/Spain and Spanish/Mexico

Download the latest version from - http://www.microsoft.com/en-us/kinectforwindows/develop/developer-downloads.aspx

The Kinect with GP Demo at Convergence

2012 has been a very exciting year –

• Moving from the chilling cold to burning heat in Jan in Perth (and getting to meet the 'David Musgrave’)
• Getting to play a small part at the GP General session at Convergence 2012 (GP with Kinect Warehouse Demo)

There’s more coming up on the GP front and the Kinect front this year, but for now I just want to thank everybody involved in the Kinect with GP Demo. If you haven’t seen it yet – I recorded a part of it –

 

Kinect with GP Demo presented at Convergence 2012

 

You can watch the complete session and many other sessions at at Virtual Convergence

To read more about the GP General Session check out the following sessions -

• Microsoft Convergence 2012 Houston - Day 1: Dynamics GP General Session – David Musgrave
• Convergence 2012 Wrap Up -  Mark Polino
• The Dynamics GP Blogster: Microsoft Dynamics Convergence Houston 2012 - Day 1 – Mariano Gomez

Thank you the GP Product team -  Pam, Errol, Jay and Chad for letting me be a part of this, and Mariano and David for their help. This took up a large part of my weekends over the past 5 months – but was totally worth it. It was challenge considering we were using the Kinect SDK Beta – however, I think the idea was portrayed pretty well.

I have already had feedback from a lot of people about how something like this could be very helpful in the hazardous/chemical industries. With the Kinect SDK 1.0 and the commercial sensor now out – the possibilities are endless. I am very excited about the new Kinect + Windows 8 world – so if any of you are looking at doing something in this field and have any questions, feel free to give me a shout.

Using Gestures with Business Analyzer for Dynamics GP

When Kinect for XBOX came out last year, I immediately fell in love with it. Wii was good, but Kinect was a whole different ball game. All my friends wanted to have a go at each other in Boxing, Bowling and everything else.

Around the same time, I had acute back pain, which made me take a break from work. My awesome friends encouraged me to setup a small gaming zone (something I had always dreamed of). We all pooled in and put together some Foosball Tables, Gaming Systems, Racing Wheels – and well – that business is still going on.

We also put together a couple of Kinect Systems, so people could try them out, and buy them as well. At our Gaming Zone – I saw people fall in love with Kinect – fathers playing with their sons, families getting together. It was awesome!

I was dying to program with Kinect. I didn’t want to move away from GP either, but I didn’t see the two fitting together. Making a journal entry with Kinect seemed like a very bad idea. Then, earlier this year, I saw Business Analyzer (Available with Dynamics GP 2010 R2), and how it let you add beautiful KPIs, Charts and Graphs to your personal dashboard. What if we let people navigate Business Analyzer with their gestures. That seemed like a good fit – and could be used in meetings!

Microsoft has done a great job with the Kinect SDK and the Kinect ToolBox. Using the two I was able to track 4 gestures, and then send commands to Business Analyzer to navigate through the reports.

Check out the Video (only a couple of minutes long)

The video is available at http://youtu.be/ugJsdg4WWS8?hd=1

Gestures

I captured the following Gestures -

1. Swipe to Right – Moves to next Graph in Business Analyzer
2. Swipe to Left – Move back to previous Graph
3. Expand Gesture – Zooms into a graph
4. Contract Gesture – Zoom out

I lost a bit of weight while coding for this!

Initially while working on this I was constantly getting up from my chair to test out the gestures. Later on I figured out how you can record your gestures/sequence of gestures to avoid getting up or getting hold of somebody while you are are debugging. Kinect toolbox has a set of replay classes.

The Future

Kinect is extremely responsive, and only $150, and can be attached to any PC. As the technology matures, and maybe in the future Notebooks come with a Kinect like camera array, I don’t see any reason why you would be using a Presentation Pointer or Remote and not use your hand and natural gestures.

Resources

Kinect SDK is available at http://research.microsoft.com/en-us/um/redmond/projects/kinectsdk/

Kinect Toolbox is available at http://kinecttoolbox.codeplex.com/

My Favorite Kinect Games

• Kinect Sports – Everybody loves this. Had Football, Boxing, Bowling, Track and Field, Table Tennis
• Dance Central – Girls love this
• Child of Eden – Single Player – awesome game.
• Wipeout – Kids love this.
• Fighters Uncaged – Single Player, good workout. Controls are okay-ish.
• Body and Brain Connection – Good workout for the brain. Multiplayer is not as good.

There are about 20+ games available, but the rest were pretty average. I am looking forward to the new games coming out this holiday season.