I’ve been learning a lot about the neocortex lately, and have come across some really fascinating information that I’d like to share. What if I were to tell you I could turn your tongue into a third eye? Sound bizarre? You’d be surprised! Let’s talk a little about a concept neuroscientists call “pattern equivalency.”
The idea that patterns from different senses are equivalent inside your brain is quite surprising, and although well understood, it still isn’t widely appreciated. More examples are in order. The first one you can reproduce at home. All you need is a friend, a freestanding cardboard screen, and a fake hand. For your first time running this experiment, it would be ideal if you had a rubber hand, such as you might buy at a Halloween store, but it will also work if you just trace your hand on a sheet of blank paper. Lay your real hand on a tabletop a few inches away from the fake one and align them the same (fingertips pointed in the same direction, palms either both up or both down). Then place the screen between the two hands so that all you can see is the false one. While you stare at the fake hand, your friend’s job is to simultaneously stroke both hands at corresponding points. For example, your friend could stroke both pinkies from knuckle to nail at the same speed, then issue three quick taps to the second joint of both index fingers with the same timing, then stroke a few light circles on the back of each hand, and so on. After a short time, areas in your brain where visual and somatosensory patterns come together— one of those association areas I mentioned earlier in this chapter— become confused. You will actually feel the sensations being applied to the dummy hand as if it were your own.
Another fascinating example of this “pattern equivalency” is called sensory substitution. It may revolutionize life for people who lose their sight in childhood, and might someday be a boon to people who are born blind. It also might spawn new machine interface technologies for the rest of us.
Realizing that the brain is all about patterns, Paul Bach y Rita, a professor of biomedical engineering at the University of Wisconsin, has developed a method for displaying visual patterns on the human tongue. Wearing this display device, blind persons are learning to “see” via sensations on the tongue.
Here is how it works. The subject wears a small camera on his forehead and a chip on his tongue. Visual images are translated pixel for pixel into points of pressure on the tongue. A visual scene that can be displayed as hundreds of pixels on a crude television screen can be turned into a pattern of hundreds of tiny pressure points on the tongue. The brain quickly learns to interpret the patterns correctly.
One of the first people to wear the tongue-mounted device is Erik Weihenmayer, a world-class athlete who went blind at age thirteen and who lectures widely about not letting blindness stop his ambitions. In 2002, Weihenmayer summited Mount Everest, becoming the first blind person ever to undertake, much less accomplish, such a goal.
In 2003, Weihenmayer tried on the tongue unit and saw images for the first time since his childhood. He was able to discern a ball rolling on the floor toward him, reach for a soft drink on a table, and play the game Rock, Paper, Scissors. Later he walked down a hallway, saw the door openings, examined a door and its frame, and noted that there was a sign on it. Images initially experienced as sensations on the tongue were soon experienced as images in space.
These examples show once again that the cortex is extremely flexible and that the inputs to the brain are just patterns. It doesn’t matter where the patterns come from; as long as they correlate over time in consistent ways, the brain can make sense of them.
– Jeff Hawkins, from his book On Intelligence: How A New Understanding Of The Brain Will Lead To The Creation Of Truly Intelligent Machines
That blows my mind. If you vibrate your tongue in the same sorts of patterns and frequencies as visual images, it will become like an eye, and you will soon see images in space. You can make a blind man see by vibrating his tongue! I mean, what in the world! If you don’t understand how the brain works, that will sound almost unbelievable, but it’s true.
I suppose if you wanted a third eye, you could mount a camera on top of your head and start vibrating your tongue appropriately. Soon you would stop feeling “touch” and “taste” sensations, and would instead see from your tongue. You could probably do the same thing by wrapping a special device around your leg, vibrating your thighs and legs appropriately, and you would probably begin to see images from another fourth camera, which you could have pointing behind you. You’d be four eyed monster! Bwahahahaha!
That’s the very sort of mad scientist stuff I’m into. That is too cool!
Ok, so why does this work? How does this work? I’m glad you asked, my young Padawans. I will now teach you the dark art of sensory perception! Get ready!
In 1978, Vernon Mountcastle, a neuroscientist from John Hopkins University in Baltimore, published a paper entitled An Organizing Principle For Cerebral Function. He made the observation that no matter which area of the cortex he examined, whether it be Brocas area (responsible for our ability to speak language), auditory cortex (allowing us to hear), visual cortex (allowing us to see and perceive objects in space), or motor cortex (allowing us to control and move our bodies), they all had the same basic organizational structure. Sure, they varied a little bit, such as in thickness, length of horizontal connections, synapse density, and so forth, but overall, the sections were so similar that they had to be doing the same sort of operation. Every area of the cortex has the same layering, cell types, and connections.
Interesting. But what does that mean? Ever since we were small children, we’ve been taught that we have five senses – sight, sound, touch, hearing, and smell. But then you look at the brain areas in our cortex which we know produce these senses, and you see that they’re all basically structured the same. What’s going on?
We can’t experiment with human brains, but we have done some very interesting experiments on animals. For example, you can take a baby ferret and rewire its brain, sending its visual signals to the auditory sections of its brain, and the auditory signals to the visual area of its brain. What happens? The brain area that normally develops into auditory cortex turns into visual cortex, and vice versa. Scientists have done similar sorts of experiments on rats, running their sense of touch to their visual cortex, and as you’d probably guess, that area specializes for touch, not vision. The cortex of animals, including ourselves, is very plastic. It can be transformed into any sensory modality, depending on the inputs.
Our cortex is not specialized for any particular function at birth. If you’re born deaf, your brain will use the “auditory” cortex to process visual information. If you’re born blind, your “visual” cortex will be converted into a very sensitive area for touch, used primarily in reading braille. If a neocortical area is not receiving the normal inputs it’s “supposed” to receive, it will start to rewire itself, sending out connections, looking for inputs. So basically, these areas of the brain develop based on the inputs they receive.
So how do we turn your tongue into a third eye? We vibrate a 2D image onto its surface, having the various vibrational pressure intensities correspond to different colors. The brain will interpret the changes appropriately, and you will see from your tongue. Your cortex will begin to rewire itself, and turn your tongue sensory cortex into visual cortex. This will take a little time though. At first you’ll feel it as random vibrations on your tongue. But in time, you’ll become conscious of visual images. Then when you stop the vibrations and take off the device, it will, in time, revert back to a normal tongue. No permanent harm done.
I remember watching cartoons when I was a kid and there would be these evil mad scientists who would turn themselves into monsters. They’d have ten eye-balls, and wings, and a stinging tail, and the good guy would be like, “Dr. X, you monster! What have you done!” The villain would then cry out, “We’ll see if you have what it takes to stop me!” And then an epic battle would take place. As a kid I wondered how a human being could transform himself or herself into something like that. Now I kind of know. I somewhat understand the principles as to how to do that, but there are way too many technicalities. I’d have to pump blood to my wings, wiring in a vein structure, and linking muscles to motor cortex. I’d have to alter my heartbeat to accommodate the extra body mass, and so on. But I can see that if you were smart enough, and had the brain power, you could build yourself any sort of body, with vastly superior intelligence. When our entire cortex is unfolded and stretched out, it’s about the size of a dinner napkin. Imagine if you grew additional brain cortex, and wired it in appropriately as well. You could be ten times as smart.
The key point is “wired in appropriately”. Dolphins and whales have a lot more cortex than we do, though they’re not near as intelligent. Cortex alone isn’t all there is to it. It has to be wired into your sensory systems appropriately. Also, you can see that we already can build eyeballs out of cameras and make the blind see with them. With time, I wonder when we’ll move beyond biological bodies, and build new bodies out of nanotechnology, using robotic and computer parts. There’s still a lot of secrets as to why cortex vibrating according to various inputs creates conscious sensations. There’s very few things in this world which interest me more. To me, that’s the key to everything.
A while back, I think it was Everett who asked me what I thought about death, and what I believed happened when we died. This quest to understand consciousness, and what creates it, is the same problem as life and death. This matter organizes into these patterns, based on self-organizing systems (evolution, etc), and then when coming together into neural networks, we become conscious. I don’t think “I” die upon death. “Jason” will die. All my memories and past experiences, which are stored in my brain, will die. But the fundamental existence which “I” am is not this physical body. This physical body creates signals, which vibrates my cortex, which makes me conscious for the moment, but when this body dies and rots in the ground, I don’t see any reason to believe that some other matter, somewhere else in the universe, may not start vibrating, and I wake up as something or someone else. It sounds like Buddhist or Hindu reincarnation, but it’s what I believe in these days. I’ll probably wake up as something else, and not remember any of my past forms of existence. I may be some winged creature on another planet across the cosmos. Who knows. Question is, how does my personal subjective consciousness enter and work within my brain’s cortex? That’s what I need to know.