Skip to Content

How People Process Auditory Information

by Steve Noble

The need for this information was identified during a work group discussion focusing on the “readability” of specialized media using audio and tactile means to relate information typically displayed in visual form. The premise of the work group’s understanding is that there may have been some research done to define the limits of audio, and in a more limited since tactile, information cognition. This report examines some of the major research areas of the past as well as some current projects in the field.

Sensory Substitution

A significant amount of psychophysics research along these lines was done during the 1950s, 1960s, and early 1970s. These studies examined how that the auditory and tactile sensory channels could be used to relate visual information, especially for persons who are blind. Larry Scadden (now at the National Science Foundation) and other researchers designed and tested experimental tactile image systems which converted optical images produced by television cameras into a dynamic tactile display, typically by way of vibrator pin array technology. Studies found that blind individuals could recognize common objects and describe their arrangements in physical space. The conclusion of these studies was that concerns over “sensory overload” were unfounded and that problems generally stemmed from the limitations of the system rather than the limitations of the perceiver.

Similar applications of substituting auditory analogs and cues for visual information have been formulated. As early as the 1940s research was conducted which showed that human subjects had the ability to perceive the location of solid objects by interpreting changes in ambient sound patterns as a person moves in proximity of the object. Still other studies confirmed that blind individuals could interpret binaural auditory information to such an extent as to be able to gauge the relative proximity of multiple solid objects within a controlled environment. Some extended applications of these findings have resulted in the creation of a number of binaural assistive navigation devices intended for blind users, although these devices have been found to have limited practical value.

More recent work has been done to utilize auditory analogs of two-dimensional printed information, such as mathematical equations and graphs. One experimental application developed by Dr. T. V. Raman (called ASTER) allows users to listen to a synthetic speech output where changes in speech pitch are used to depict vertical locations of equation components. For example, when reading algebraic equations superscript information is read in a higher pitch, while subscript information is read in a lower pitch. By using a variety of pitch encoding routines, a wealth of positional information can be communicated to the listener thus allowing independent reader access to higher level mathematical equations.

Other work has focused on the production of auditory analogs to communicate line graphs. Once again, this technique utilizes pitch encoding to produce an analog of visual information. In this instance, a constant tone is produces which rises in pitch as the graph line ascends the vertical axis, and lowers in pitch as the line descends.

Intelligibility and “Auditory Vigilance”

Psychoacoustical studies in auditory intelligibility include early work done by Dr. Emerson Foulke to explore increased reading speed with recorded materials. His work led to the development of TSM, time scale modification, where recorded information can be played back at a faster rate without causing pitch-shift distortion which commonly happens when recorded materials are sped up.

Along similar studies to examine intelligibility issues, research in auditory vigilance during the 1960s and 1970s focused on the ability of blind subjects to sort and interpret multiple auditory cues within both controlled and uncontrolled ambient environments. Results of these studies indicated that blind subjects were typically more aware of auditory environments and more adept at interpreting auditory information.

Virtual Reality

Most current research in psychoacoustics is driven by the desire to produce virtual reality environments. A significant amount of work is now being done in attempt to understand how the brain interprets subtle changes in auditory environments, and thus be able to create more “realistic” models of virtual reality.

Additional information and a selected bibliography is presented below.

Other Titles Suggested by the Author

Harris, J. Donald (John Donald), 1914-
Title: Psychoacoustics [by] J. Donald Harris.
Publisher: Indianapolis, Bobbs-Merrill, 1974.

Luce, R. Duncan (Robert Duncan)
Title: Sound & hearing: a conceptual introduction / R. Duncan Luce.
Publisher: Hillsdale, N.J.: L. Erlbaum Associates, 1993.

Warren, Richard M.
Title: Auditory perception: a new synthesis / Richard M. Warren.
Publisher: New York: Pergamon Press, c1982.

Thinking in sound: the cognitive psychology of human audition / edited by Stephen McAdams and Emmanuel Bigand.
Publisher: Oxford [England]; New York: Clarendon Press, 1993.

Information below this point does not originate with the author

.

From U. C. Berkeley, Department of Psychology,  Auditory Perception Lab

The research currently being conducted in the Auditory Lab is concerned primarily with issues involved in the higher-order processing of auditory information, especially as it impacts listening in real-world situations.

The first half of our research program concerns the localization of sounds in space. This includes the ability to integrate information collected from both ears, the phenomenon of “visual capture”, the ability to suppress echoes, and the grouping of sounds into perceptual objects.

Related to this is the second half of our research program, which involves the various types of attentional mechanisms available to listeners. These include the ability of listeners to focus attention on different auditory dimensions, the role this plays in distinguishing relations between elements in those dimensions, and once again the ability to group sounds into objects (this time attentionally rather than via spatial location).

Associations and Laboratories from UCB Hearing Sciences

Books from UCB Hearing Sciences

  • Blauert, J. (1997) Spatial Hearing. Cambridge: MIT Press.
  • Bregman, A.S. (1990) Auditory Scene Analysis. Cambridge: MIT Press.
  • Gilkey, R.H. and Anderson, T.R. (eds.), (1997) Binaural and Spatial Hearing in Real and Virtual Environments. New Jersey: Lawrence Erlbaum Associates.
  • Green, D. and Swets, J. (1966) Signal Detection Theory and Psychophysics. New York: John Wiley and Sons, Inc.
  • Green, D. (1988) Profile Analysis: Auditory Intensity Discrimination New York: Oxford University Press.
  • Hartmann, W.M. (1997) Signals, Sound and Sensation. Woodbury: AIP Press.
  • Macmillan, N.A. and Creelman, C.D. (1991) Detection Theory: A user’s guide. Cambridge: Cambridge University Press.
  • Moore, B.C.J. (1997) An Introduction To The Psychology Of Hearing. New York: Academic Press.
  • Parasuraman, R. and Davies, D.R. (1984) Varieties of Attention. New York: Academic Press.
  • Pashler, H.E. (1998) The Psychology of Attention. Cambridge: MIT Press.
  • Popper, A.N. and Fay, R.R. (1994) Mammalian Auditory Pathway: Neurophysiology. New York: Springer-Verlag.
  • Yost, W., Popper, A.N. and Fay, R.R. (1994) Human Psychophysics. New York: Springer-Verlag.