Modified Guitar Hero application gives amputees "Wii-habilitation"

From the Johns Hopkins University Gazette:

Wii-habilitation" — using Nintendo's Wii video game system in rehabilitation therapy — continues to be popular with health care workers looking to help patients get through what some see as the pain and torture of physical therapy. Now two engineers in APL's National Security Technology Department have cranked that concept up a notch. Bobby Armiger and Jacob Vogelstein have rewired Nintendo's Guitar Hero III: Legends of Rock game to allow amputees to rock out and get valuable training with prosthetic prototypes at the same time.

Their gaming is part of the APL-led Revolutionizing Prosthetics 2009 effort, funded by the Defense Advanced Research Projects Agency, to develop a prosthetic arm that will be controlled and also feel, look and perform like a natural limb. So far the project has produced two prototypes and has leveraged a surgical technique, developed at the Rehabilitation Institute of Chicago by Todd Kuiken, that reroutes the nerves that once controlled an amputee's arm to remaining muscles. These "re- innervated" muscles naturally amplify the nerve signals so that electrodes placed against the skin can detect activity and control the prosthetic arm.

The surgery has enabled patients to control the first prototype arm and will eventually be used to control individual fingers of the final prosthetic design. But for the arm to operate correctly using these rerouted nerves, the system must learn to interpret the patient's muscle signals. The process is similar to training voice-recognition software where, for example, you are prompted to read the Gettysburg Address, except that instead of analyzing tones, the APL system records and classifies muscle twitches.

This training takes place in a Virtual Integration Environment, another RP 2009 innovation. "In the VIE, an animated on-screen arm mimics the patient's intended movements in real time, based on inputs from the electrodes attached to the user's residual muscles," Vogelstein explains. "For the training, a patient sits in front of a computer and an on-screen prompt tells them to 'flex your wrist,' 'extend your wrist,' 'close your fist,' etc."

Calibrating the mechanical limbs to recognize and respond properly to electrical signals in an amputee's residual muscles is an exhausting and draining process, says Armiger, who has been on the front lines of training the system to respond naturally to the patient. "There is no real interaction and no feedback," he says. "And there is no encouragement for the patient to do more or do it better."

The idea to adapt the game for amputees came to Armiger while he was playing Guitar Hero at a party. He and Vogelstein borrowed a colleague's copy of the game and modified the controller with a soldering iron to allow it to be controlled by the VIE. Then, button clicks were substituted with muscle contraction signals as picked up by the electrodes.

Last fall, Armiger and Vogelstein traveled to RP 2009 partner Duke University to test the rigged system on Iraq veteran Jonathan Kuniholm, (pictured) who lost his right hand to shrapnel in 2005. With electrodes attached to what was left of his arm, Kuniholm was able to operate the frets using signals from his muscles.

The researchers found that playing a game is a far more intuitive way to speed up the tedious calibration process and make it more fun for volunteers. "It allows for large numbers of rapid, dynamic movements that are more natural and not stereotyped," Vogelstein says. "Ultimately we are going to get more input and longer training cycles out of patients — and that will translate to better and more natural limb control."

Armiger and Vogelstein next want to adapt a tennis game to train people with more radical amputations. "Eventually we want to make our software open source so that people can repurpose other games for rehabilitation," Armiger says.