Word Learning in Feature Space

What developmental mechanisms support the cognitive transition from baby to curious and inquisitive toddler? One period of rapid cognitive development is known as the "vocabulary spurt" at 18-22 months, during which children go from learning 2-3 words per week to learning around 8-9 words per week. Some have proposed that a new information processing mechanism may come online during this time, one that supports "referential" or symbolic processing as opposed to more "basic" associative processing.

In a 2005 article in Cognitive Science, Regier proposes that one need not assume such a mechanistic shift take place, and that the stage-like progression seen during this age can instead be a result of purely associative learning. He argues that the stage can be characterized on the basis of 4 qualities:

1) Ease of learning: associative learning becomes easier at this stage, as reflected by evidence that 13-15 month-olds can acquire object-word association in 9-12 training trials, while 2-3 year olds can learn within 1-3 trials.

2) Increased sensitivity to communicatively relevant sounds: whereas 14 month-olds can create unique sound-object associations only with fairly dissimilar sounds, 18-23 month-olds can associate even similar-sounding names with different objects. This increased sensitivity to communicatively relevant sounds is accompanied by decreased sensitivity to communicatively irrelevant sounds: 13-month-olds but not 20-month-olds are capable of learning an association between a sound made by a noisemaker and a target object. This evidence indicates that phonological processing in older children may have become more selectively "tuned," at the expense of flexibility.

3) Increased sensitivity to communicatively relevant semantics: after learning a new word-object association, 13-month-olds won't generalize the use of these words to new objects with similar shape but different color. However, older children will ignore color and size differences by generalizing words to new objects on the basis of shape alone.

4) Increased ability to cope with second labels: 16-month-olds have trouble learning a new word for an already-named object, whereas 24-month-olds can more easily acquire second labels for familiar objects

Looking at these qualitative shifts in behavior, it's not hard to see why some researchers believe this age shows a mechanistic shift to referential processing. Regier, however, implemented a neural network model which proves that similar phenomena can be observed without any such "symbolic" transition.

In the LEX (lexicon as exemplar) model that Regier proposes, word learning is based on the interaction of two aspects of language: word form, and word meaning. Each of these has its own layer, in which objects are presented to the network both phonologically and semantically, as is often the case in childhood ("see the car? that's a car!"). Two hidden layers connect these two outer layers, and through gradient descent, these layers extract the relationships between specific characteristics features of input. The training corpus consisted of 50 words.

The innovative part of the network is the use of two sets of weights: first, a standard set of "associative" weights between nodes, and a second set of "attentional" weights between nodes. While the associative weights merely pick out correlations among input patterns, the attentional weights are responsible for the further biasing of particular types of features. This is important because some "feature dimensions" assume more importance than others for communication - in the case of #3 above, shape would be more highly biased than size or color.

After some experience with the training input, LEX is also able to show increased ease of learning (#1). This occurs because training allows the attentional weights to be changed in such a way that they begin to define the "feature space" by emphasizing importance differences between items, and collapsing across irrelevant differences.

After some initial training, LEX also shows a stage-like capacity for second label learning (#4). Just as the interference between the representations of different objects is minimized by the distortion of feature space through learning of attentional weights, the interference between multiple representations for the same objects is minimized as well. The model suggests that the hardest words to learn are those which overlap maximally with either previously experienced word form, previously experienced word meanings, or both.