Kinect used for robotic surgery

Researchers have adapted Microsoft’s Kinect technology for use with robotic tools in surgery, making remote treatment more straightforward and reducing costs.

The main function of the technology is to utilise the array of cameras and sensors – which allow videogame players to control their Xbox 360 without a controller – in order to provide surgeons with feedback when performing surgery.

According to Howard Chizeck, professor of electrical engineering at the University of Washington, robotic tools, which are commonly used for minimally invasive surgery, do not allow the surgeon any sense of touch when operating.

“What we’re doing is using that sense of touch to give information to the surgeon, like ‘You don’t want to go here,’” Chizeck says.

It is noted that tubes are often used with remotely controlled surgical instruments on the end which are then inserted into the patient in order to minimise scarring.

Surgeons are then able to control the instruments with input devices similar to complex joysticks, and use tiny cameras in the tubes to see inside the patient.

The problem for the surgeons is that they have no realistic feedback from the tools, so when they move a surgical instrument into something solid the instrument will be forced to stop, while the joystick is still able to move.

To resolve this graduate student Fredrik Ryden wrote code that allowed the Kinect to map the environment in three dimensions before relaying it back to the user. 

With this information it is then possible to restrict certain areas by stopping the joystick moving.  This could mean that the joystick would be able to follow along the line of a bone, or even block surgical tools from hitting vital organs inadvertently.

“We could define basically a force field around, say, a liver,” said Chizeck. “If the surgeon got too close, he would run into that force field and it would protect the object he didn’t want to cut.”

“It’s really good for demonstration because it’s so low-cost, and because it’s really accessible,” Ryden, said. “You already have drivers, and you can just go in there and grab the data. It’s really easy to do fast prototyping because Microsoft’s already built everything.”

Before the idea to use a Kinect for the system, which took just one weekend to complete, a similar alternative would have cost around $50,000, according to Chizeck.

Furthermore the system will allow greater reliability for long distance use, enabling doctors in major cities to perform long distanced surgery in remote areas.

This could also lead to applications for use in disaster relief or in battlefield use.

“Suppose there’s an earthquake somewhere,” Chizeck said. “First responders could get victims to a van with a satellite dish on top and the tools inside, and a surgeon somewhere else could perform the surgery.”

With a paper due to be published soon the researchers will now focus on scaling down the sensors to a size deemed appropriate for surgical use, and the resolution of the video will need to be increased before it is usable.