4/03/2006

Word Learning in Feature Space

What developmental mechanisms support the cognitive transition from baby to curious and inquisitive toddler? One period of rapid cognitive development is known as the "vocabulary spurt" at 18-22 months, during which children go from learning 2-3 words per week to learning around 8-9 words per week. Some have proposed that a new information processing mechanism may come online during this time, one that supports "referential" or symbolic processing as opposed to more "basic" associative processing.

In a 2005 article in Cognitive Science, Regier proposes that one need not assume such a mechanistic shift take place, and that the stage-like progression seen during this age can instead be a result of purely associative learning. He argues that the stage can be characterized on the basis of 4 qualities:

1) Ease of learning: associative learning becomes easier at this stage, as reflected by evidence that 13-15 month-olds can acquire object-word association in 9-12 training trials, while 2-3 year olds can learn within 1-3 trials.

2) Increased sensitivity to communicatively relevant sounds: whereas 14 month-olds can create unique sound-object associations only with fairly dissimilar sounds, 18-23 month-olds can associate even similar-sounding names with different objects. This increased sensitivity to communicatively relevant sounds is accompanied by decreased sensitivity to communicatively irrelevant sounds: 13-month-olds but not 20-month-olds are capable of learning an association between a sound made by a noisemaker and a target object. This evidence indicates that phonological processing in older children may have become more selectively "tuned," at the expense of flexibility.

3) Increased sensitivity to communicatively relevant semantics: after learning a new word-object association, 13-month-olds won't generalize the use of these words to new objects with similar shape but different color. However, older children will ignore color and size differences by generalizing words to new objects on the basis of shape alone.

4) Increased ability to cope with second labels: 16-month-olds have trouble learning a new word for an already-named object, whereas 24-month-olds can more easily acquire second labels for familiar objects

Looking at these qualitative shifts in behavior, it's not hard to see why some researchers believe this age shows a mechanistic shift to referential processing. Regier, however, implemented a neural network model which proves that similar phenomena can be observed without any such "symbolic" transition.

In the LEX (lexicon as exemplar) model that Regier proposes, word learning is based on the interaction of two aspects of language: word form, and word meaning. Each of these has its own layer, in which objects are presented to the network both phonologically and semantically, as is often the case in childhood ("see the car? that's a car!"). Two hidden layers connect these two outer layers, and through gradient descent, these layers extract the relationships between specific characteristics features of input. The training corpus consisted of 50 words.

The innovative part of the network is the use of two sets of weights: first, a standard set of "associative" weights between nodes, and a second set of "attentional" weights between nodes. While the associative weights merely pick out correlations among input patterns, the attentional weights are responsible for the further biasing of particular types of features. This is important because some "feature dimensions" assume more importance than others for communication - in the case of #3 above, shape would be more highly biased than size or color.

After some experience with the training input, LEX is also able to show increased ease of learning (#1). This occurs because training allows the attentional weights to be changed in such a way that they begin to define the "feature space" by emphasizing importance differences between items, and collapsing across irrelevant differences.

After some initial training, LEX also shows a stage-like capacity for second label learning (#4). Just as the interference between the representations of different objects is minimized by the distortion of feature space through learning of attentional weights, the interference between multiple representations for the same objects is minimized as well. The model suggests that the hardest words to learn are those which overlap maximally with either previously experienced word form, previously experienced word meanings, or both.

5 Comments:

Blogger BenP said...

Is he talking on the brain related part, to this neural network ?

4/03/2006 10:24:00 AM  
Blogger Chris Chatham said...

if I understand you correctly, the answer is no - he is not tying this network directly to neuroanatomy. It's a more abstract model. For example, the way he models "word form," it's actually an aglomeration of lots of kinds of phonological information.

I wish there were more neuroanatomical models of development, but we're just not there yet, unfortunately.

4/03/2006 10:28:00 AM  
Blogger Rutledge said...

Chatham, I wanted to let you know that I accidentally forgot to backup an important file to my old blog. I was screwing around with switching to a FTP server. But the good news is, I managed to send enough emails out to inform those who added to their blogroll.

I started a new blog, similar focus, and I hope to make it better...it'll also clinical cases that I come across while working at the Kettering Wallace Neuroscience Institute. But I was warned about patient confidentiality, so I'll have to be real careful with that.

here's the link....
http://biomedstudent.wordpress.com

4/03/2006 10:30:00 AM  
Blogger Chris Chatham said...

hey lavon - that's pretty funny because i just found your new blog (and linked to it) right before you posted. synchrony is weird.

good luck with the new blog; it looks great so far.

4/03/2006 10:34:00 AM  
Blogger Rutledge said...

that reminds me of "neural synchrony through controlled tracking." I watched a show on Dateline NBC about that...well, it could've been 20/20 for all I know.

These are crazy times chris, the craziest of times. off to another lecture...the saga continues

4/03/2006 11:53:00 AM  

Post a Comment

Links to this post:

Create a Link

<< Home