An Information Processing Approach to Mental Imagery

An intense debate has surrounded the relationship of pattern recognition and mental imagery for many years. This debate can be distilled into three related questions: first, what kinds of information processing might accomplish both pattern recognition and mental imagery? Second, to what extent are these functions located in distinct anatomical regions? Third, how is visual data actively represented, and does that differ between mental imagery and perception?

To tackle the first question, Kosslyn’s (1990) cognitive architecture posits that both imagery and pattern recognition can be supported by nearly identical information processing units. The only unique subsystem required by mental imagery is the “shape-shift” system, which can alter the perceived shape, location, or orientation of objects on the visual buffer, and may be capable of additional transformations (such as folding or rotation).

With the exception of this shape-shift subsystem, visual perception and mental imagery differ according to this information processing approach only in the dominant direction of data flow: this flow is reversed for imagery because pattern recognition is being driven by internally generated, as opposed to externally generated, patterns of neural activity.

In summary, then, the processes of pattern recognition and imagery appear very similar at an "information processing" level of analysis, as conducted by Kosslyn. However, this tight relationship between imagery and pattern recognition conflicts with more recent cognitive neuroscience evidence, which suggests that perception and imagery may use some very different mechanisms (and in some cases can be selectively impaired). In other words, some neuropsychological patients are unable to use mental imagery, but can yet perceive the visual world without problems. Yet other patients with visual agnosias are unable to name visually presented objects, but are still able to successfully imagine named items.

Tomorrow's post will review this neuropsychological evidence, the clever behavioral tests used to diagnose the patients, and the implications for the machinery supporting both perception and mental imagery.

Related Posts:
Dissociations between Imagery and Perception
Pictures in the Brain
An Informal Integration of Object Recognition Models
Kosslyn's Cognitive Architecture