• Lobby
  • NAVI: A Kinect Hack That Helps Visually Impaired Navigate Indoors

So here's the abstract of the article:
Who knew that Microsoft's humble Kinect sensor could be hacked to do so much more than just impair your dancing skills? So far just in medicine, we've seen the Kinect used as a touch-free image browser, an augmented reality CT viewer, a force feedback sensor for robotic surgery, and a program to actually control these surgical robots, including the da Vinci robot.

The latest comes from the University of Konstanz in Germany. Engineers there have wired up a Kinect sensor to help improve indoor navigation for the visually impaired. The project, called NAVI (Navigational Aids for the Visually Impaired) consists of a helmet-mounted Kinect sensor connected to a computer in a backpack, a special belt containing vibration motors to warn the users of obstacles ahead and to the sides, and a Bluetooth headset to provide verbal feedback. Altogether, the device can help someone visually impaired navigate to a specific location, GPS-style, with tactile and verbal warnings of objects in their path. The system can even detect barcoded signs which could provide further information to the user.
Here are some images:

Helmet design:



NAVI System as worn by user - back view:



NAVI System as worn by user - frontal view:



Also, check out the video on this link.
It's quite nice to see it applied in the medical field, today at Arabnet was a session dedicated to hacking the kinect Samer attended that session I guess.
AymanFarhat wroteIt's quite nice to see it applied in the medical field, today at Arabnet was a session dedicated to hacking the kinect Samer attended that session I guess.
I did, it was demonstrated by some guys at cleartag that used the kinect to control a simple mouse pointer. It wasn't really impressive but it really triggered my interest as to what this piece of technology is capable of supporting.
Is the kinect going to be the new Wiimote? It looks amazing though :-)
samer wrote
AymanFarhat wroteIt's quite nice to see it applied in the medical field, today at Arabnet was a session dedicated to hacking the kinect Samer attended that session I guess.
I did, it was demonstrated by some guys at cleartag that used the kinect to control a simple mouse pointer. It wasn't really impressive but it really triggered my interest as to what this piece of technology is capable of supporting.
Exactly, me too. Once you realize that the Kinect is an affordable and mass produced 3d mapping (or whatever you call it) device, it opens up a whole new world of possibilities. (I know, that sounded corny).
Ok, so I found the article about hacking the Kinect sensor to control the well renowned surgical robot da Vinchi:
Pushing further the possibilities of what one can do with the $150 Microsoft's Kinect 3D controller, engineers from Johns Hopkins University managed to hook up a Kinect as an interface for a da Vinci surgical robot.

Nicolas Padoy, from the CIRL laboratory, has demonstrated in a youtube videohow to perform a needle insertion as well as other fine manipulation tasks using a kinect and a surgical da Vinci robot used here for non-clinical research.

The 3D positions of his hands are tracked using depth information obtained from the kinect and are used to control the 3D pose of the robotic tool. This approach allows to translate the 3 degrees of freedom of his hands into gestures which can control a device that has 6 degrees of freedom.

He uses the system to perform needle insertion on a suturing pod and also to grasp and transfer 6mm plastic rings between spikes.

All that is missing is a comfortable chair for the surgeon and a 3D screen that gives him full depth-perception of the surgical environment.

One can imagine that better human-machine interactions will arise with the combination of multiple kinects.
Some of the images:

da Vinchi surgical robot:





Kinetic Hacked for aiding in da Vinchi robot-controlled surgery:













Oh, and did I mention that the NAVI belt is controlled by an Arduino microcontroller? Gotta love Arduino ;)