To what extent can the senses complement and replace
each other in processing information?
Can visual stimuli be picked up by the ears? Does what we see affect what we hear and how perceive smell? I have read this article a while back about the McGurk effect, in which the auditory component of a sound is paired with the visual component of another. McGurk published his findings in Hearing Lips and Seeing Voices", and he finds that often, the perceived phoneme is a third, intermediate phoneme. As an example, the syllable /ba-ba/ is spoken over the lip movements of /ga-ga/, and the perception is of /da-da/. This phenomenon is interesting, as it suggests that multiple sensory inputs are combined to provide a best guess especially when faced with conflicting information between sensory inputs. This phenomenon makes me wonder can this quirk of the human brain be exploited further? Can one sensory input do the job of another? In The Blind Man Who Taught Himself to See, Michael Finkel recounts his experiences with a visually impaired man, Kish, who taught himself to see through echolocation, much like the way bats navigate. Kish reports having vivid pictures of the world in his head, solely based on the auditory feedback he gets from constantly clicking his tongue. He describes his vision as similar to a sonar. Thus, he is able to locate buildings a thousand feet away and another person six feet away. In the 1980s, a group of neuroscientists developed a tactile display, in which 144 electrodes are place on a mouthpiece which can induce a current against the tongue. It was first used successfully as an aid for people with an impaired sense of balance due to damage of the semi-circular canal. From that onwards, the team decided to experiment with feeding other sensory data to the mouthpiece. It was found that visual input can also be converted into electrical impulses to the tongue, where it is processed as tactile input and converted back into visual information. Blind-folded test subjects report seeing high-contrast images of the surroundings and could navigate the world with that tactile information alone. Then I thought about my own experiences dealing with sensory information. As I have rather severe short-sightedness in both eyes, coupled with an equally bad case of astigmatism, the visual information I get without my glasses on is very noisy and blurry, much like an old TV with weak reception. One day, my glasses fell off while I was playing football and my teammate stepped on it, breaking it into two. As the game continued, I made many errors in tackling and found it hard to locate my teammates. In the time it took for a new pair of glasses to arrive, I found it
less disorienting to shut off my distorted vision completely at times and
focus on the rest of my senses. At first, I fooled around with my grandmothers walking stick at home, using it to locate obstacles to navigate around the house, with rather encouraging success. At the dining table, with my eyes closed, I become more aware of the different resistances provided by the different things on my plate. I could feel my fork going into the thick cuts of chicken, traversing its fibres and provided a soft landing for my fork while the long beans provided a more abrupt feedback response as it burst open. I was able to finish my meal without making too much of a mess, as while my fork moved around, I could feel its sides pushing against the vegetables, the sour cream, the chicken and the potatoes. A day later without glasses, I tried to rely on my sense of hearing to make sense of the world, without much success. That night, I stood on the pavement of a main road, with my father who was understandably frightened when we were about to cross the road. I could not see much at a distance all I could see was the glare of car lamps heading my way constantly, and I would have been stuck in my tracks for a long time if there wasnt anyone to guide me across. While crossing the road, I could hear the surroundings were slightly quitter than before, as there were less cars zooming by at that instant. After that, I became aware of the directional nature of the human hearing, and became more aware of the chaos in everyday life, where we are bombarded by thousands of information every instant. Thus, I wonder if we could live with less senses than five if we could still be as effective at processing information knowing as we are if we evolved with fewer senses than we have now. To what extent can the senses complement and replace each other in processing information?