Post on 12-Oct-2020
A N E N E M Y I N O U R M I D S T
UNIVERSITY OF CHICAGO SUMMER 2006 BIOLOGICAL SCIENCES DIVISION
MIND OVER MATTER: MOVING OBJECTS
WITH YOUR MIND HAS LONG BEEN
FODDER FOR SCIENCE FICTION STORIES.
NOW, RESEARCH IS TURNING IT
INTO REALITY.By Kelli Whitlock Burton
MiNdMatter
icholas Hatsopoulos adjusts
the volume on his computer and looks
around. “Hear that?” he asks. There’s
a steady crackling noise coming from
the speakers, like hail on a tin roof.
Hatsopoulos smiles. The crackling
chorus is as satisfying to his ear as a
good guitar riff by Jimi Hendrix, one
of his favorite musicians.
What seems like a hail storm actually
is the sound of neurons firing inside
the motor cortex, the part of the brain
responsible for movement—a symphony
that just 15 years ago Hatsopoulos
could only dream of hearing, much
less composing.
Listening to the brain is no small
feat, and if that were the brightest note
in this little concert, it would be worth
an ovation. But Hatsopoulos, an assistant
professor of organismal biology and
anatomy at the University of Chicago,
can do more than eavesdrop. Along
with colleagues at Brown University,
he has developed a way to record signals
sent out by large groups of neurons—
commands telling the body how and
where to move—and to translate the
orders into a language a computer
understands and acts on.
The technology is called a brain-
computer interface—BCI for short—and
it’s not a new phenomenon. Scientists
have tinkered with BCI since the 1970s,
but it’s only in the past decade that the
technology’s true potential has been
realized. The main thrust today is devel-
oping BCI systems to aid people who are
paralyzed by injury or illness. While these
patients’ limbs may be stilled, studies show
that the motor cortex is not. Hatsopoulos’
team is one of only about half a dozen
university research groups working on
the problem in the United States.
Ten years ago, Hatsopoulos and
John Donoghue, his former postdoctoral
advisor at Brown University, became the
first scientists to teach monkeys how to
move a computer cursor with their minds.
Two years ago, they taught a person to do
it—a quadriplegic who was able to turn
on a television, check e-mail and wiggle
the fingers of an artificial hand, all with
his thoughts alone. The patient is part
of an FDA clinical trial of the BrainGate™
system, the product of a company
Donoghue and Hatsopoulos launched
in 2000.
In a nutshell, the researchers have
found a way to turn thought into action—
without moving a muscle.
WHEN SCIENCE TICKLES
If his dark curly hair, deep brown eyes and
distinctive surname aren’t clue enough to
his ancestry, the strains of Mediterranean
music coming from his office are a dead
giveaway. Hatsopoulos’ parents grew up in
Athens, Greece, but didn’t meet until the
N
19
Pho
to b
y D
an D
ry
Chicago researcher Nicholas Hatsopoulos
Medic
ineo
nth
eM
idw
ay
Sum
mer
200
6
20
early 1950s after each traveled separately
to Boston to attend college, his father at
MIT and his mother at Wellesley. With a
mechanical engineer and physicist for a
dad as well as a mathematician for a
mother, Hatsopoulos was surrounded
by science while he grew up in Lincoln,
Mass., just outside Boston. He even chose
to major in physics at Williams College.
Still, when it came time to make plans
after graduation, he wasn’t quite ready
to launch a scientific career. Instead,
Hatsopoulos decided to teach physics
and computers for a year in Greece.
When he wasn’t in class, he dabbled
with the bouzouki, an instrument similar
to a mandolin. He even toyed with the
idea of staying in his parents’ homeland
to become a professional bouzouki player.
But something about science tickled his
subconscious, so he returned to Boston
and took a job as a research assistant to a
Harvard mathematical psychologist. Soon
he enrolled in the psychology master’s
program at Brown. By 1992 he’d earned
a PhD in cognitive science. Skilled in
physics, psychology, cognition and the
bouzouki, Hatsopoulos headed to Caltech
for a postdoc with a scientist who studied
insects’ brains.
There, Hatsopoulos made his first
recording of a neuron—a single cell in a
locust’s brain. “As soon as I recorded my
first cell and saw the electrical spikes on the
oscilloscope and listened to the crackle of a
neuron spiking, I was hooked,” he said.
Hatsopoulos’ appreciation for science
is much like his love of music. While the
individual instruments are beautiful to
hear, it’s the combination of their sounds
that Hatsopoulos enjoys most. Similarly,
the actions of one brain cell may be
interesting, but the possibilities that
abound when cells work together are
downright fascinating.
Take the bouzouki, for example. “I
have always been enamored by the finger
work these musicians do and how they
coordinate their fingers that way,”
Hatsopoulos mused. How do the millions
of neurons in the motor cortex work
together to allow someone to strum sweet
music on a bouzouki?
“As soon as I recorded
my first cell and saw
the electrical spikes on
the oscilloscope and
listened to the crackle
of a neuron spiking, I
was hooked.”
Pho
to c
ourt
esy
of N
icho
las
Hat
sopo
ulos
The silicone chip—part of the brain-computer interface—
that’s implanted in the brain’s motor cortex. Photocourtesy of Nicholas Hatsopoulos
“We were looking at a group of neurons that
make up the neural ensemble that is responsible
for everything that we do as thinking beings.”
21
“What really turns me on in this
research is understanding how these large
groups of cells result in motor behavior
or thought,” Hatsopoulos said. Studying
single cells, as he was doing at Caltech,
would not help him find the answer.
So in 1995, Hatsopoulos returned
to Brown for a second postdoc with
Donoghue. The neuroscientist was bent
on figuring out how to record as many
neurons in the motor cortex as possible.
Hatsopoulos was more than happy to help.
THE RECORDINGINDUSTRY
Scientists had developed a way to listen
in on single brain cells years before
Donoghue began his studies. Many tech-
niques were employed, but one of the
most successful was a mechanism created
by Philip Kennedy, a scientist at Georgia
State University, who launched the first
human clinical trial of an implanted BCI
device. Consisting of glass cones with
two microelectrodes designed to pick up
neural impulses in the brain, the device
was implanted in the motor cortex of a
paralyzed patient. The woman was asked
to think about moving a computer cursor.
The device captured the cell signals and,
using a radio transmitter under the scalp,
sent them to a computer that decoded
them and moved the cursor.
But Kennedy could only record data
from one or two neurons with his device;
Donoghue’s sights were set much higher.
He wanted to capture signals from dozens,
perhaps hundreds, of brain cells. Monitor-
ing more neurons would provide a clearer
picture of brain activity, Donoghue believed.
All this and the researchers weren’t
even sure the device would collect the
multi-cell data they needed. They trained
a monkey to move a computer cursor by
maneuvering a joystick with its arm, set
up their system, and watched.
The array not only worked, it func-
tioned better than the scientists had
hoped. In 1996, the team recorded signals
from multiple brain cells in a monkey—
the first time anyone had collected data
from the monkey using this array. “I
remember that moment as really exciting
from more of a scientific point of view,”
Hatsopoulos said. “We were looking at
the brain on a more global level, looking
at a group of neurons that make up the
neural ensemble that is responsible for
everything that we do as thinking beings.”
The next step was to identify patterns
in neuronal activity related to movement.
Hatsopoulos’ job was to create algorithms
that translated the chatter between the
Things started to come together when
Donoghue met Richard Normann from
the University of Utah. Normann had
developed a sensor array—a silicone chip
about the size of a breath mint with 100
tiny electrodes, each capable of recording
neuronal impulses.
Donoghue and Hatsopoulos modified
the array, which had only been used in
cats and Petri dishes, for their studies
with monkeys. Each electrode on the
chip, which is implanted on the surface
of the motor cortex, picks up signals
from nearby neurons. Those impulses are
carried by gold wires that connect the
chip to a titanium pedestal that protrudes
about an inch from the monkey’s scalp.
A cord extends from the top of the
pedestal to a device that amplifies the
signals. A fiber-optic cable hooks the
amplifier to an acquisition system, which
captures the neural impulses and sends
them to a computer.
A titanium pedestal connects to the implanted chip.
Pho
to c
ourt
esy
of N
icho
las
Hat
sopo
ulos
22
neurons in the motor cortex into a
language the computer could understand
—language that conveys information
about movement.
That required Hatsopoulos to study the
monkey’s brain activity on a monitor that
displayed graphs of the neurons in action.
When active, neurons fire between 20 and
200 times a second, creating the spikes
on his computer. Hatsopoulos matched
neuronal signal patterns with each arm
movement. A computer program outfitted
with his algorithms could recognize those
patterns and move the cursor before the
monkey moved a muscle.
Eventually, the monkey figured out that
it didn’t have to move the joystick to make
the cursor move. These were exciting
developments for the researchers, but
there were annoyances, too. Hatsopoulos
couldn’t ask a monkey questions about its
thoughts or actions. The monkey couldn’t
provide any feedback. The researchers’
excitement soon grew to frustration as
they realized they’d taken the technology
as far as they could in animals.
In spring of 2000, Hatsopoulos and
Donoghue discussed their work’s potential
to help people. And then, Hatsopoulos
offered an idea that he never thought
would interest him. “What if we form
a company to do this?”
A RELUCTANTENTREPRENEUR
His father’s dream had been to come to
America and launch his own business. In
1956, the elder Hatsopoulos did, creating
a company to market a device he’d devel-
oped at MIT that converted heat energy
into electricity with no moving parts.
Entrepreneurship was not a dream
Hatsopoulos shared with his father. He
just wanted to be a scientist. But the work
he’d done with Donoghue changed all
that. Cyberkinetics was incorporated in
2001. The following January, Hatsopoulos
joined the faculty at Chicago and later
that year, the company merged with
Bionics, a business started by the Utah
scientist who’d developed the array
Hatsopoulos’ group modified for primates.
In August 2002, the company received
$5 million from a venture capital firm in
Boston and applied for FDA approval to
conduct a small clinical trial of their BCI
technology, which they named BrainGate.
“People have always believed this would
be possible,” said Tim Surgenor, chief
executive officer for Cyberkinetics. “We’ve
been reading about it in comic books
and science fiction our whole lives and
suddenly, someone comes along and says
it’s possible.”
A computer program
outfitted with
Hatsopoulos’
algorithms could
recognize neuronal
signal patterns in the
monkey’s brain activity
and move the cursor
before the monkey
moved a muscle.
The plan, Hatsopoulos said, wasn’t
just to create something that allowed
paraplegics to control electrical devices
with their minds. To set themselves apart
from similar entrepreneurial efforts in the
BCI field, they needed another “killer app.”
“We eventually came up with an idea
that is implicitly the mission of our
company: We want to provide the basic
operating system by which any brain-
computer interface system can work,” he
said. “We want to be the Microsoft of
communications between the brain and
the outside world.”
With a basic operating platform, tech-
nology could be developed to help people
with paralysis answer a phone, type a
letter, turn off a coffee maker—all at the
speed of thought. And scientists could
build on Cyberkinetics’ operating system
to develop therapeutic devices for neuro-
logical disorders: For example, a chip
implanted in the brain of an epileptic
that senses when a seizure is likely and
emits an electric jolt to stop it.
It took a year and five cases of supportive
documents to prepare the FDA application,
but by the middle of 2004, Cyberkinetics
received the go-ahead. The plan was to
enroll up to five paraplegics for a one-year
trial, implant the device, and teach them
to move a computer cursor with their
minds, a small step toward giving those
without the power to move the ability to
turn thought into motion.
“This technology is a whole new way
of dealing with neural repair,” Donoghue
said. “Instead of finding a substitute signal
for movement—like using the eyes to move
a cursor instead of a hand—Cyberkinetics
is trying to develop a system that takes the
brain’s own movement commands to the
outside world.”
ON TRIAL
Cyberkinetics received its first volunteer
for the trial in June 2004: Matthew Nagle,
a 25-year-old from Weymouth, Mass.,
whose spine had been severed at the neck
in a knife attack five years earlier.
Nagle underwent a five-hour surgery, in
which physicians made an 8-inch incision
in the scalp and bone, exposing the motor
cortex. They placed the silicone chip on the
brain surface with the electrodes penetrating
the cortex about 1 millimeter down. The
pedestal was positioned and the scalp
closed around it, leaving about 1 inch
protruding from the top of Nagle’s head.
Three times a week, Nagle was visited
by Maryam Saleh, a technician who used
to work with Cyberkinetics and now is a
doctoral student in Hatsopoulos’ lab. Saleh’s
first task was to teach Nagle to think
about moving. Before he was paralyzed, if
Nagle wanted to move a computer cursor,
he reached out his arm, grabbed a mouse,
and moved it. Without use of his
limbs, however, he must rely on
his mind. But how do you teach
someone to “think” about moving?
23
THE BRAINS BEHIND BRAINGATE
1 The person thinks about moving thecomputer cursor. Electrodes on a siliconechip implanted into the person’s braindetect neural activity from an array of neural impulses in the brain’s motor cortex.
2 The impulses transfer from the chip to a pedestal protruding from the scalp through connection wires.
3 The pedestal filters out unwanted signals or noise, then transfers the signal to anamplifier.
4 The signal is captured by an acquisitionsystem and is sent through a fiber-opticcable to a computer. The computer thentranslates the signal into action, causing the cursor to move.
Cyberkinetics is trying
to develop a system
that takes the brain’s
own movement
commands to the
outside world.
COMPUTER SYSTEM
PEDESTAL
SILICONE CHIP
AMPLIFIER
FIBER-OPTIC CABLE
ACQUISITION SYSTEM
Illus
trat
ion
by C
hris
tina
Ullm
an
24
Medic
ineo
nth
eM
idw
ay
Sum
mer
200
6
“Training patients to move things
with their minds is difficult and different
with each patient,” Saleh said. She had
Nagle visualize himself moving the cursor,
a simple-sounding command that took
him months to master. Then, during one
of their all-day sessions, Nagle did it.
He thought about moving the cursor—
and it moved.
“He was so excited, he told everybody,”
said Saleh, who now is working with a
patient in Chicago who enrolled in the
trial last year. “Most people involved in
the study think of themselves as pioneers.
Even though they’re told that this is
specifically geared toward controlling a
computer cursor, they see the prospects for
future applications. That’s why they do it.”
Indeed, study participants are told at
the outset that the trial’s goal is to help
scientists learn how to improve the
BrainGate system and how to train future
patients to use it. The system won’t
operate without a technician’s assistance
and when the yearlong trial is over, the
entire apparatus—pedestal, chip and all—
is removed.
“This isn’t being done for the patient’s
benefit; it’s being done for mankind’s
benefit,” said Richard Penn, professor of
neurosurgery at Chicago who operated on
another study participant with whom
Saleh now works.
“What’s so intriguing about this is
that we’re really listening in on what
neurons are doing,” Penn said. “This is
the strangest, most interesting surgery
I’ve ever done in that sense. Not the
technical stuff, but the data that we get
from the neurons firing in different
patterns when you’re thinking in different
ways. And seeing it is only the beginning.”
Researchers presented findings from
Nagle’s trial, which ended in 2005, at the
Society for Neuroscience annual meeting
later that year. Although the results are
preliminary, they suggest the BrainGate
system works.
So far, four volunteers have joined the
trial: two with spinal cord injuries, one
with amyotrophic lateral sclerosis (ALS,
also called Lou Gehrig’s disease) and a
stroke patient.
“Depending on their specific situation
and their specific interest, [people with
disabilities] may be interested in computers
turning text into speech, dialing telephones,
playing games, controlling their environ-
ment, moving their wheelchairs and
moving their arms,” Surgenor said. “For
each of these four participants, I think
each one is interested in something
different. We have to develop something
that’s flexible enough to work with a
number of different devices.”
SET IT AND FORGET IT!
An unabashed fan of infomercials,
Hatsopoulos draws his philosophy about
this project from a jingle he heard on a
promotion for a rotisserie oven. Baking a
chicken or roast in a conventional oven is
hard, messy work that requires constant
oversight, the infomercial host proclaimed.
Study participants are told at the outset that the
trial’s goal is to help scientists learn how to improve
the BrainGate system and how to train future
patients to use it.
NORMAL NEURAL ACTIVITY
CELL BODY OF MOTORNEURONSignals sent throughdendrites cause chemicalchanges that result in an electrical signal in thecell body.
AXONSNerve impulses are carried through axons away from the neuron’s cell body.
NEUROMUSCULAR JUNCTIONThe signal is passed by neuro-transmitters from synaptic bulbs onthe neuron to muscle fibers. Themuscle fibers then react to the signal.
Illus
trat
ion
by C
hris
tina
Ullm
an
DENDRITES
MUSCLEFIBERS
But with the rotisserie, you can “set it
and forget it!”
“That’s exactly my philosophy on
electrophysiology. You put the device
in and you forget it,” he said. “That’s
different than the way we used to do
recordings, with a single electrode that
you had to move around and spend
hours collecting data. Here, you just
‘set it and forget it!’”
At least, that’s the hope. BrainGate
has a long way to go before it functions
well without oversight. For example,
the cables and components patients use
now are fairly onerous and impractical.
Hatsopoulos and Donoghue want to
make the system wireless.
There’s also work to be done on the
array’s ability to capture neuronal impulses.
Now, the quality and number of signals
recorded are inconsistent, and Hatsopoulos
doesn’t know why. The team also wants
to improve the decoding software that
translates the brain’s electrical chatter
into movement commands a machine
can follow. That requires faster and
more accurate algorithms, a task
assigned to Hatsopoulos’ group.
Just what is the brain-computer interface going
to interface with? A robotic arm? A computer?
A wheelchair?
CHIP SENSOR PROCESSSensors, or electrodes, on thesilicone chip detect signalsfrom surrounding neuronsin the brain’s motorcortex. This area is highlysaturated with neurons,but each sensor onlyneeds to detect signalsfrom 10 to 50 neuronsto trigger the BrainGatesystem to move thecomputer cursor. Thesensors act as facilitatorsfor the message, which iscarried out by the computer.
MOTOR CORTEXThis neuron-rich area of the braininitiates body movement.
Illus
trat
ion
by C
hris
tina
Ullm
an
Actual size
4 MM
1 MM
PEDESTAL
SILICONE CHIPThe chip is implanted in thebrain’s motor cortex.
sensor onsilicone chip
signals
neuronsreleasesignals
motor cortexsurface
From a physiological standpoint,
researchers need to find out just how
long the chip can remain in the brain
and continue to function. Some of the
monkeys they’ve studied received implants
three years ago that still function with no
changes in brain physiology. But it’s yet
to be seen how a long-term brain implant
will affect a human.
Finally, the researchers need to focus
on the end target: Just what is the brain-
computer interface going to interface
with? A robotic arm? A computer? A
wheelchair? That may be the most
difficult question of all—and one that
the scientists must answer if they are
to prove that BrainGate is better than
what’s already out there.
“The kind of control that can be
obtained, I believe, will surpass these
other current methods,” Hatsopoulos said.
“We’re just scratching the surface now.”