This article was featuredin this month's IEEE Spectrum magazine:
We love Microsoft's Kinect 3D sensor, and not just because you can play games with it. At a mere $150, it's a dirt-cheap way to bring depth sensing and 3D vision to robots, and while open-source USB drivers made it easy, a forthcoming Windows SDK from Microsoft promises to make it even easier.

Kinect, which is actually hardware made by an Israeli company called PrimeSense, works by projecting an infrared laser pattern onto nearby objects. A dedicated IR sensor picks up on the laser to determine distance for each pixel, and that information is then mapped onto an image from a standard RGB camera. What you end up with is an RGBD image, where each pixel has both a color and a distance, which you can then use to map out body positions, gestures, motion, or even generate 3D maps. Needless to say, this is an awesome capability to incorporate into a robot, and the cheap price makes it accessible to a huge audience.

We've chosen our top 10 favorite examples of how Kinect can be used to make awesome robots, check it out:

1. Kinect Quadrotor Bolting a Kinect to the top of a quadrotor creates a robot that can autonomously navigate and avoid obstacles, creating a 3D map as it goes.



2. Hands-free Roomba Why actually vacuum when you can just pretend to actually vacuum, and then use a Kinect plus a Roomba to do the vacuuming for you?



3. iRobot AVA iRobot integrated two (two!) Kinect sensors into their AVA not-exactly-telepresence prototype: one to help the robot navigate and another one to detect motion and gestures.



4. Bilibot The great thing about Kinect is that it can be used to give complex vision to cheap robots, and Bilibot is a DIY platform that gives you mobility, eyes, and a brain in a package that costs just $650.



5. Gesture Surgery If you've got really, really steady hands, you can now use a Kinect that recognizes hand gestures to control a DaVinci robotic surgical system.

For images, see this post.

6. PR2 Teleoperation Willow Garage's PR2 already has 3D depth cameras, so it's kinda funny to see it wearing a Kinect hat. Using ROS, a Kinect sensor can be used to control the robot's sophisticated arms directly.



7. Humanoid Teleoperation Taylor Veltrop put together this sweet demo showing control over a NAO robot using Kinect and some Wii controllers. Then he gives the robot a banana, and a knife (!).



8. Car Navigation Back when DARPA hosted their Grand Challenge for autonomous vehicles, robot cars required all kinds of crazy sensor systems to make it down a road. On a slightly smaller scale, all they need now is a single Kinect sensor.



9. Delta Robot This Kinect controlled delta robot doesn't seem to work all that well, which makes it pretty funny (and maybe a little scary) to watch.



10. 3D Object Scanning Robots can use Kinect for mapping environments in 3D, but with enough coverage and precision, you can use them to whip up detailed 3D models of objects (and people) too.

Also, if you want to check the youtube videos, then here you go.
So cool. It seems Kinect is a big success, even if not for its originally intended purpose.

I gotta say though... Kinect controlled Da Vinci robot? not a good idea...
Can't anyone from Lebgeeks pull off a kinect hack ? It would be nice to hear of a Lebanese 'invention' - if you may say
It would not be characterized as an invention ! just because it needs a "computer" with the special software on it to run the system makes it impractical and "stupid" to use for anything outside the computer monitor.
J4D wroteIt would not be characterized as an invention ! just because it needs a "computer" with the special software on it to run the system makes it impractical and "stupid" to use for anything outside the computer monitor.
Hardly, they demo with the software available on the computer because they are prone to change it given its a work in progress. Can you imagine them compiling the entire thing into a device for every version of the software?

Nevertheless, those are still merely conceptual, hence the term "hacks".
X: Compiling the entire thing into a device
That's How robotics are worked :)
They cant 'but' run the software on a computer. Because the kinect can only be connected to a computer, other than of course the Xbox or perhaps the ps3 now. For example you cannot use it with a microcontroller.
Notice how even the small robots have a netbook on its back.

Blekh, That's not how i would deal with things.
@Bassem: Yeah, there's a great deal of delay and it needs alot of work, but it's still interesting to see.

@jadf24: Great minds think alike, MrClass was just discussing with me the potential of doing a DIY project which is based on a hacked Kinect sensor. I fail to see how I'm supposed to do it with my work schedule, and I agree with J4D on the idea of having to use a netbook to control it (I mean, is it really worth the trouble of accounting for the weight of the netbook as well as its size just to have a 3D mapping? I mean, 3D laser scanners can also generate a 3D map, just like the one from Lockheed Martin) However, not all the robots have the netbook on board (e.g. Roomba), so the real question is: is the 3D mapping generated by the Kinect sensor much better than that generated by 3D laser scanners that it's worth the trouble of using a netbook?
The ones not having a netbook onboard have a computer setup as a wireless processing and command relaying station.
You discuss this and fail to discuss the need for a netbookless hack off Kinect to interface it to any USB enabled microcontroller). That will turn attention to us more than anything else. I don't want to do any grunt work myself nonetheless.
The kinect is essentially Computer vision with depth perception. You cannot connect that directly to a simple microcontroller. you need a software to mediate between what's seen by the camera and the useful digits that it can spit out about the distance color....etc .

Walaw??!! :P
J4D wroteThe ones not having a netbook onboard have a computer setup as a wireless processing and command relaying station.
I'm very much aware of that, just stated that the netbook doesn't need to be on board.
arithma wroteYou discuss this and fail to discuss the need for a netbookless hack off Kinect to interface it to any USB enabled microcontroller).
As J4D so graciously pointed out (why can't you be less agressive when you comment, J4D?), you need a program to analyze what's percepted by the Kinect sensor in terms of machine vision and interpret it's "meaning" in terms of depth. On the other hand, do you need a "software" to interpret such a vision? No, take a look at the CMUcam from Seattle Robotics which is used with Parallax's Basic Stamp 2 microcontroller:



Ok, it's an RGB cam, but still you need a program that translates it's vision into usable data, and the program is simply a set of PBASIC 2.5 commands that are subroutines written by a bunch of university students at Carnigie Mellon. You don't need a software, just the cam (and it's controller board) interfaced with the microcontroller. Note that the cam itself has a microcontroller to process the video stream before it is translated to data for the main microcontroller which regulates the system. The latter versions CMUcam2 and CMUcam3 have better specifications, and hence require microcontrollers with larger EEPROM capacity.

So, what I really meant to ask, is it possible to hack the Kinect sensor and interface it to a USB microcontroller that is capable of generating the 3D mesh, which is basically what the netbook software is all about (depth perception too, but this can be done with an accelerometer)? What's a powerful enough processor to do the job?
I've seen the cmu cam before, its nice but its still a normal camera and lacks the depth perception of the kinect.
If you use a dual setup of cmucams you can simulate what the kinect really is. as for connecting the kinect directly to a microcontroller, i believe it can be done using one of those high end ARM processors that run .NET .
http://www.sparkfun.com/products/9797
But still, huge amounts of code necessary.

Aggressive ? :(
J4D wroteI've seen the cmu cam before, its nice but its still a normal camera and lacks the depth perception of the kinect.
If you use a dual setup of cmucams you can simulate what the kinect really is. as for connecting the kinect directly to a microcontroller, i believe it can be done using one of those high end ARM processors that run .NET .
http://www.sparkfun.com/products/9797
But still, huge amounts of code necessary.

Aggressive ? :(
Agreed, the CMUcam lacks the depth perception of the Kinect and a dual setup would do the job, but it's totally not worth the cost.

ARM processors, nice suggestion, frankly I didn't think of it till you brought it up. Excessive amount of coding is a given with such projects. Totally worth considering, I'll look it over the weekend.

As for the "aggressive" comment, unfortunately it's true most of the time. I love your passion towards electronics, and you have a vast experience with them, and this is why I enjoy discussing electronics with you (I don't really find electronic freaks frequently, they're rare). You're always on the same page when I discuss something, and your suggestions are amazing (you made me pick up Arduino => you rock!). On the other hand, some of your comments are aggressive (remember the "It's got stupid written all over it." comment, was that really necessary?). If you want to point out something, use phrases like "With all do respect, ...". This is a big deal, you're attitude influences people's perception of you, especially when you'll join the workforce. I'm not going to be an arm-chair critic, yes sometimes my comments hold aggression, but your aggressive comments are becoming more frequent. This can drive people off discussing stuff with you, and it would be a real loss for everyone. Just an advice, nothing more...
The kinect is essentially Computer vision with depth perception. You cannot connect that directly to a simple microcontroller. you need a software to mediate between what's seen by the camera and the useful digits that it can spit out about the distance color....etc .

Walaw??!! :P
This comment deserves an hour long ass spanking. I'll do it myself.
I never said a "simple microcontroller", did I? Software is not special to PCs. Suggesting that shows great twists in the ways you think. Someone can perhaps even experiment with a dedicated IC interfacing between the kinect and any other kind of device (be it a PC, an adruino, or another microcontroller).

@Jad: Don't actively try to out-jerk yourself. Be natural, you're already enough of a one :)
Love.
3 months later
Guys, the kinect is the future of robotics, and yes one of those small notebook pcs on board of a robot is the answer to all robot builder.

Think about it, why do i need to build a complex circuit with micro controllers and various electronic parts, program the micro controller in some low level languge, debug for errors in circuit and in program (trust me my friends, this is tedious work)
When i have a small wonder that we call asus or acer (10 inch Screen), weighing no more than 1 kg with a more powerful processor and memory than any available micro on the market (pic, atmel, basic stamp) , with so many input/output lines (usb), wifi,bluetooth and so much possibilities, programmable by a nice high level language like c# or java, equipped with its own rechargeable battery

Now we can focus on building the algorithm of our robot and forget about all the fuss of circuit soldiering, micros programming,......(some may argue that this is an expensive solution but i tell you it is well worth it , spend 200-300$ more and you will be free from all the fuss and headache)

To J4D i say, don't reinvent the wheel man, technology is your friend, make use of it.

To mesa177 i say, the kinect sensor is much easier to program than you think, with Microsoft releasing a beta sdk, the power of kinect lies in skeleton tracking of human body and joints and in depth sensing, skeleton tracking is out of the box in the sdk and is really easy to use

Finally, check the turtle bot, a very powerful robotic kit at a very affordable price, you could do wonders with this baby! Yeah!

http://www.willowgarage.com/turtlebot
Funny :) The only thing you would be doing here is code, And providing profit for other companies :P

I'd rather make it from scratch AND code it :)

Btw, i personally will not take you seriously unless you show us some of your "robots" and perhaps give us an insight about your age.