New Tools to Help Patients Reclaim Damaged Senses
Correction Appended
Cheryl Schiltz vividly recalls the morning she became a wobbler. Seven years ago, recovering from an infection after surgery with the aid of a common antibiotic, she climbed out of bed feeling pretty good.
"Then I literally fell to the floor," she said recently. "The whole world started wobbling. When I turned my head, the room tilted. My vision blurred. Even the air felt heavy."
The antibiotic, Ms. Schiltz learned, had damaged her vestibular system, the part of the brain that provides visual and gravitational stability. She was forced to quit her job and stay home, clinging to the walls to keep from toppling over.
But three years ago, Ms. Schiltz volunteered for an experimental treatment - a fat strip of tape, placed on her tongue, with an array of 144 microelectrodes about the size of a postage stamp. The strip was wired to a kind of carpenter's level, which was mounted on a hard hat that she placed on her head. The level determined her spatial coordinates and sent the information as tiny pulses to her tongue.
The apparatus, called a BrainPort, worked beautifully. By "buzzing" her tongue once a day for 20 minutes, keeping the pulses centered, she regained normal vestibular function and was able to balance.
Ms. Schiltz and other patients like her are the beneficiaries of an astonishing new technology that allows one set of sensory information to substitute for another in the brain.
Using novel electronic aids, vision can be represented on the skin, tongue or through the ears. If the sense of touch is gone from one part of the body, it can be routed to an area where touch sensations are intact. Pilots confused by foggy conditions, in which the horizon disappears, can right their aircraft by monitoring sensations on the tongue or trunk. Surgeons can feel on their tongues the tip of a probe inside a patient's body, enabling precise movements.
Sensory substitution is not new. Touch substitutes for vision when people read Braille. By tapping a cane, a blind person perceives a step, a curb or a puddle of water but is not aware of any sensation in the hand; feeling is experienced at the tip of the cane.
But the technology for swapping sensory information is largely the effort of Dr. Paul Bach-y-Rita, a neuroscientist in the University of Wisconsin Medical School's orthopedics and rehabilitation department. More than 30 years ago, Dr. Bach-y-Rita developed the first sensory substitution device, routing visual images, via a head-mounted camera, to electrodes taped to the skin on people's backs. The subjects, he found, could "see" large objects and flickering candles with their backs. The tongue, sensitive and easy to reach, turned out to be an even better place to deliver substitute senses, Dr. Bach-y-Rita said.
Until recently sensory substitution was confined to the laboratory. But electronic miniaturization and more powerful computer algorithms are making the technology less cumbersome. Next month, the first fully portable device will be tested in Dr. Bach-y-Rita's lab.
The BrainPort is nearing commercialization. Two years ago, the University of Wisconsin patented the concept and exclusively licensed it to Wicab Inc., a company formed by Dr. Bach-y-Rita to develop and market BrainPort devices. Robert Beckman, the company president, said units should be available a year from now.
Meanwhile, a handful of clinicians around the world who are using the BrainPort on an experimental basis are effusive about its promise.
"I have never seen any other device do what this one does," said Dr. F. Owen Black, an expert on vestibular disorders at the Legacy Clinical Research and Technology Center in Portland, Ore. "Our patients are begging us to continue using the device."
Dr. Maurice Ptito, a neuroscientist at University of Montreal School of Optometry, is conducting brain imaging experiments to explore how BrainPort works.
Dr. Eliana Sampaio, a neuroscientist at the National Conservatory of Arts and Métiers in Paris, is using the BrainPort to study brain plasticity. Sensory substitution is based on the idea that all sensory information entering the brain consists of patterns carried by nerve fibers.
In vision, images of the world pass through the retina and are converted into impulses that travel up the optic nerve into the brain. In hearing, sounds pass through the ear and are converted into patterns carried by the auditory nerve into the brain. In touch, nerve endings on skin translate touch sensations into patterns carried into the brain.
These patterns travel to special sensory regions where they are interpreted, with the help of memory, into seeing, hearing and touch. Patterns are also seamlessly combined so that one can see, hear and feel things simultaneously.
"We see with the brain, not with the eyes," Dr. Bach-y-Rita said. "You can lose your retina but you do not lose the ability to see as long as your brain is intact."
Most important, the brain does not seem to care if patterns come from the eye, ear or skin. Given the proper context, it will interpret and understand them. "For me, it happened automatically, within a few minutes," said Erik Weihenmayer, who has been blind since he was 13.
Mr. Weihenmayer, a 35-year-old adventurer who climbed to the summit of Mount Everest two years ago, recently tried another version of the BrainPort, a hard hat carrying a small video camera. Visual information from the camera was translated into pulses that reached his tongue.
He found doorways, caught balls rolling toward him and with his small daughter played a game of rock, paper and scissors for the first time in more than 20 years. Mr. Weihenmayer said that, with practice, the substituted sense gets better, "as if the brain were rewiring itself."
Ms. Schiltz, too, whose vestibular system was damaged by gentamicin, an inexpensive generic antibiotic used for Gram-negative infections, said that the first few times she used the BrainPort she felt tiny impulses on her tongue but still could not maintain her balance. But one day, after a full 20-minute session with the BrainPort, Ms. Schiltz opened her eyes and felt that something was different. She tilted her head back. The room did not move. "I went running out the door," she recalled. "I danced in the parking lot. I was completely normal. For a whole hour." Then, she said, the problem returned.
She tried more sessions. Soon her balance was restored for three hours, then half a day. Now working with the BrainPort team at the University of Wisconsin, Ms. Schiltz wears the tongue unit each morning. Her balance problems are gone as long as she keeps to the regimen.
How the device produces a lasting effect is being investigated. The vestibular system instructs the brain about changes in head movement with respect to the pull of gravity. Dr. Bach-y-Rita speculated that in some patients, a tiny amount of vestibular tissue might survive and be reactivated by the BrainPort.
Dr. Black said he had seen the same residual effect in his own pilot study. "It decays in hours to days," he said, "but is very encouraging."
Blind people who have used the device do not report lasting effects. But they are amazed by what they can see. Mr. Weihenmayer said the device at first felt like candy pop rocks on his tongue. But that sensation quickly gave way to perceptions of size, movement and recognition.
Mr. Weihenmayer said that on several occasions he was able to find his wife, who was standing still in an outdoor park, but he admitted that he also once confused her with a tree. Another time, he walked down a sidewalk and almost went off a bridge.
Nevertheless, he is enthusiastic about the future of the device. Mr. Weihenmayer likes to paraglide, and he sees the BrainPort as a way to deliver sonar information to his tongue about how far he is from the ground.
Dr. Ptito is scanning the brains of congenitally blind people who, wearing the BrainPort, have learned to make out the shapes, learned from Braille, of capital letters like T, B or E. The first few times they wore the device, he said, their visual areas remained dark and inactive - not surprising since they had been blind since birth. But after training, he said, their visual areas lighted up when they used the tongue device. The study has been accepted for publication in the journal Brain.
Dr. Ptito says he would like to see if he could teach his subjects how to read drifting letters like those in advertising displays. Not seeing motion is a big problem for the blind, he said.
In another approach, Dr. Peter Meijer, a Dutch scientist working independently, has developed a system for blind people to see with their ears. A small device converts signals from a video camera into sound patterns delivered by stereo headset to the ears. Changes in frequency connote up or down. Changes in pixel brightness are sensed as louder or softer sounds.
Dr. Yuri Danilov, a neuroscientist and engineer who works with Dr. Bach-y-Rita, said the research team had thought of dozens of applications for the BrainPort, which he called a "USB port to the brain."
In one experiment, a leprosy patient who had lost the ability to experience touch with his fingers was outfitted with a glove containing contact sensors. These were coupled to skin on his forehead. Soon he experienced the data coming from the glove on his forehead, as if the feelings originated in his fingertips. He said he cried when he could touch and feel his wife's face.
The federal government has also shown interest in sensory substitution technology. The Navy is exploring the use of a tongue device to help divers find their way in dark waters at night, said Dr. Anil Raj, a research scientist at the Institute for Human and Machine Cognition at the University of West Florida in Pensacola.
The sensors detect water surges, informing Navy Seals if they are following the correct course. The Army is thinking about sending infrared signals from night goggles directly to the tongue, Dr. Raj said.
In another application, student pilots have been fitted with body sensors attached to aircraft instruments. When the airplane starts to pitch or change altitude, they can feel the movements on their chests.
Sensory substitution technology may eventually help millions of people overcome their sensory disabilities. But the devices may also have more frivolous uses: in video games, for example.
Dr. Raj said the tongue unit had already been tried out in a game that involved shooting villains. "In two minutes you stop feeling the buzz on your tongue and get a visual representation of the bad guy," he said. "You feel like you have X-ray vision. Unfortunately it makes the game boring."
Correction: December 20, 2004, Monday:
An article in Science Times on Nov. 23 about technology that substitutes one set of sensory information for another in the brain, helping people who have lost a sense like vision or touch, misstated the title of Dr. Anil Raj of the Institute for Human and Machine Cognition at the University of West Florida in Pensacola, who spoke about the technology. He is a research scientist, not the director.
Correction: December 22, 2004, Wednesday:
An article in Science Times on Nov. 23 about technology that substitutes one set of sensory information for another in the brain, helping people who have lost a sense like vision or touch, misstated the term for a device in aircraft that lets a pilot feel the movement of the plane without looking at instruments. It is a stimulator, not a sensor.