5 levels of Neural Theory of Language

5 levels of Neural Theory of Language

20 Minute Quiz For each of the two questions, you can use text, diagrams, bullet points, etc. 1) What are the main events in neural firing and transmission? 2) Describe the main events in neural development. How does activity lead to structural change? The brain (pre-natal, post-natal, and adult) exhibits a

surprising degree of activity dependent tuning and plasticity. To understand the nature and limits of the tuning and plasticity mechanisms we study How activity is converted to structural changes (say the ocular dominance column formation) It is centrally important for us to understand these mechanisms to arrive at biological accounts of perceptual, motor, cognitive and language learning

Biological Learning is concerned with this topic. Learning and Memory: Introduction Memory Declarative Episodic facts about a situation Non-Declarative Semantic general facts

Procedural skills Skill and Fact Learning may involve different mechanisms Certain brain injuries involving the hippocampal region of the brain render their victims incapable of learning any new facts or new situations or faces. But these people can still learn new skills, including relatively abstract skills like solving puzzles. Fact learning can be single-instance based. Skill learning requires repeated exposure to stimuli. Implications for Language Learning?

Models of Learning Hebbian ~ coincidence Recruitment ~ one trial Supervised ~ correction (backprop) Reinforcement ~ delayed reward Unsupervised ~ similarity Hebbs Rule The key idea underlying theories of neural

learning go back to the Canadian psychologist Donald Hebb and is called Hebbs rule. From an information processing perspective, the goal of the system is to increase the strength of the neural connections that are effective. Hebb (1949) When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that As efficiency, as one of the cells firing B, is increased

From: The organization of behavior. Hebbs rule Each time that a particular synaptic connection is active, see if the receiving cell also becomes active. If so, the connection contributed to the success (firing) of the receiving cell and should be strengthened. If the receiving cell was not active in this time period, our synapse did not contribute to the success the trend and should be weakened. LTP and Hebbs Rule

Hebbs Rule: neurons that fire together wire together strengthen weaken Long Term Potentiation (LTP) is the biological basis of Hebbs Rule Calcium channels are the key mechanism Chemical realization of Hebbs rule

It turns out that there are elegant chemical processes that realize Hebbian learning at two distinct time scales Early Long Term Potentiation (LTP) Late LTP These provide the temporal and structural bridge from short term electrical activity, through intermediate memory, to long term structural changes.

Calcium Channels Facilitate Learning In addition to the synaptic channels responsible for neural signaling, there are also Calciumbased channels that facilitate learning. As Hebb suggested, when a receiving neuron fires, chemical changes take place at each synapse that was active shortly before the event. Long Term Potentiation (LTP)

These changes make each of the winning synapses more potent for an intermediate period, lasting from hours to days (LTP). In addition, repetition of a pattern of successful firing triggers additional chemical changes that lead, in time, to an increase in the number of receptor channels associated with successful synapses - the requisite structural change for long term memory. There are also related processes for weakening synapses and also for strengthening pairs of synapses that are active at about the same time.

The Hebb rule is found with long term potentiation (LTP) in the hippocampus Schafer collateral pathway Pyramidal cells 1 sec. stimuli At 100 hz During normal low-frequency trans-mission, glutamate interacts with NMDA and nonNMDA (AMPA) and metabotropic

receptors. With highfrequency stimulation Enhanced Transmitter Release AMPA Early and late LTP (Kandel, ER, JH Schwartz and TM Jessell (2000) Principles of Neural Science. New York: McGraw-Hill.) A. Experimental setup for demonstrating

LTP in the hippocampus. The Schaffer collateral pathway is stimulated to cause a response in pyramidal cells of CA1. B. Comparison of EPSP size in early and late LTP with the early phase evoked by a single train and the late phase by 4 trains of pulses. Computational Models based on Hebbs rule The activity-dependent tuning of the developing nervous system, as

well as post-natal learning and development, do well by following Hebbs rule. Explicit Memory in mammals appears to involve LTP in the Hippocampus. Many computational systems for modeling incorporate versions of Hebbs rule. Winner-Take-All: Recruitment Learning

Units compete to learn, or update their weights. The processing element with the largest output is declared the winner Lateral inhibition of its competitors. Learning Triangle Nodes LTP in Episodic Memory Formation WTA: Stimulus at is presented 1 a

2 t o Competition starts at category level 1 a 2

t o Competition resolves 1 a 2 t o

Hebbian learning takes place 1 a 2 t o Category node 2 now represents at

Presenting to leads to activation of category node 1 1 a 2 t o Presenting to leads to activation of category node 1

1 a 2 t o Presenting to leads to activation of category node 1 1

a 2 t o Presenting to leads to activation of category node 1 1 a

2 t o Category 1 is established through Hebbian learning as well 1 a 2

t o Category node 1 now represents to Connectionist Model of Word Recognition (Rumelhart and McClelland) Triangle nodes and McCullough-Pitts Neurons? A

B C A B C Representing concepts using triangle nodes Feature Structures in Four Domains Makin

Ham dept~EE Color ~pink sid~001 Taste ~salty emp~GSI Container Push Inside ~region Schema ~slide Outside ~region Posture ~palm Bdy. ~curve Dir. ~ away Bryant

Pea dept~CS Color ~green sid~002 Taste ~sweet emp~GSI Purchase Buyer ~person Seller ~person Cost ~money Goods ~ thing Stroll Schema ~walk Speed ~slow

Dir. ~ ANY Distributed vs Localist Repn John 1 1 0 0 John

1 0 0 0 Paul 0 1 1

0 Paul 0 1 0 0 George

0 0 1 1 Georg e 0 0

1 0 Ringo 1 0 0 1 Ringo

0 0 0 1 What are the drawbacks of each representation? Distributed vs Localist Repn

John 1 1 0 0 John 1

0 0 0 Paul 0 1 1

0 Paul 0 1 0 0 George 0

0 1 1 Georg e 0 0 1

0 Ringo 1 0 0 1 Ringo

0 0 0 1 What happens if you want to represent a group? How many persons can you represent with n bits? 2^n

What happens if one neuron dies? How many persons can you represent with n bits? n Recruiting connections Given that LTP involves synaptic strength changes and Hebbs rule involves coincident-activation based strengthening of connections How

can connections between two nodes be recruited using Hebbss rule? X Y X Y Finding a Connection in Random Networks For Networks with N nodes and N branching factor, there is a high probability of finding good links. (Valiant 1995)

Recruiting a Connection in Random Networks Informal Algorithm 1. Activate the two nodes to be linked 2. Have nodes with double activation strengthen their active synapses (Hebb) 3. There is evidence for a now print signal based on LTP (episodic memory) Has-color Green

Has-shape Round Has-color GREEN Has-shape ROUND Hebbs rule is not sufficient

What happens if the neural circuit fires perfectly, but the result is very bad for the animal, like eating something sickening? A pure invocation of Hebbs rule would strengthen all participating connections, which cant be good. On the other hand, it isnt right to weaken all the active connections involved; much of the activity was just recognizing the situation we would like to change only those connections that led to the wrong decision. No one knows how to specify a learning rule that will change exactly the offending connections when an error occurs.

Computer systems, and presumably nature as well, rely upon statistical learning rules that tend to make the right changes over time. More in later lectures. Hebbs rule is insufficient tastebud tastes rotten eats food gets sick

drinks water should you punish all the connections? Models of Learning Hebbian ~ coincidence Recruitment ~ one trial Next Lecture: Supervised ~ correction (backprop) Reinforcement ~ delayed reward Unsupervised ~ similarity

Constraints on Connectionist Models 100 Step Rule Human reaction times ~ 100 milliseconds Neural signaling time ~ 1 millisecond Simple messages between neurons Long connections are rare No new connections during learning Developmentally plausible 5 levels of Neural Theory of Language Pyscholinguistic experiments

Spatial Relation Motor Control Metaphor Grammar Cognition and Language abstraction Computation Structured Connectionism

Triangle Nodes Neural Net and learning SHRUTI Computational Neurobiology Biology Neural Development Quiz

Midterm Finals Short term memory How do we remember someones telephone number just after they tell us or the words in this sentence? Short term memory is known to have a different biological basis than long term memory of either facts or skills.

We now know that this kind of short term memory depends upon ongoing electrical activity in the brain. You can keep something in mind by rehearsing it, but this will interfere with your thinking about anything else. (Phonological Loop) Long term memory But we do recall memories from decades past. These long term memories are known to be based on structural changes in the synaptic connections between neurons. Such permanent changes require the construction of new protein molecules and their establishment in the membranes of the synapses connecting neurons, and this can take several hours.

Thus there is a huge time gap between short term memory that lasts only for a few seconds and the building of long-term memory that takes hours to accomplish. In addition to bridging the time gap, the brain needs mechanisms for converting the content of a memory from electrical to structural form. Situational Memory

Think about an old situation that you still remember well. Your memory will include multiple modalities- vision, emotion, sound, smell, etc. The standard theory is that memories in each particular modality activate much of the brain circuitry from the original experience. There is general agreement that the Hippocampal area contains circuitry that can bind together the various aspects of an important experience into a coherent

memory. This process is believed to involve the Calcium based potentiation (LTP). Dreaming and Memory There is general agreement and considerable evidence that dreaming involves simulating experiences and is important in consolidating memory.

Recently Viewed Presentations

  • NYS 3-8 ELA and Math CBT Proctor/Teacher Training

    NYS 3-8 ELA and Math CBT Proctor/Teacher Training

    The Proctor should be familiar with all Nextera Test Delivery navigation features as well as how to tell if a student's testing device is online or offline while the student is testing. 5. ... the student must click "Play" on...
  • Introduction


    Physiology of Vision: a swift overview 16-721: Learning-Based Methods in Vision A. Efros, CMU, Spring 2009 Some figures from Steve Palmer Image Formation Monocular Visual Field: 160 deg (w) X 135 deg (h) Binocular Visual Field: 200 deg (w) X...
  • The Roman Empire - Weebly

    The Roman Empire - Weebly

    The Roman Empire The Result of the Assassination After Caesar's assassination, three men took control of Rome forming the 2nd Triumvirate. Mark Antony, a supporter of Caesar and a strong general. Octavian, Caesar's adopted son. Marcus Lepidus, a patrician and...


    Times New Roman Arial Calibri Wingdings Ribbons 1_Ribbons Slide 1 Slide 2 Slide 3 Slide 4 Slide 5 Slide 6 Slide 7 Slide 8 Slide 9 Slide 10 Slide 11 Slide 12 Slide 13 Slide 14 Slide 15 Slide 16...
  • Breeds and Classes of Horses

    Breeds and Classes of Horses

    A hand is equal to four inches. The height of the horse is measured from the top of the withers to the ground. A horse that is 15 hands is 60 inches or 5 feet at the withers. Light Horses...
  • Diapositive 1 - langues.ac-dijon.fr

    Diapositive 1 - langues.ac-dijon.fr

    "This sleek, sinuous, full-bodied animal, chasing and chuckling, gripping things with a gurgle and leaving them with a laugh, to fling itself on fresh playmates that shook themselves free, and were caught and held again. All was a-shake and a-shiver—glints...
  • Roanoke College Student Payroll

    Roanoke College Student Payroll

    Timesheets are available on the Payroll Office webpage. Complete timesheets each pay period, bi-weekly. Which timesheet should you complete? Student Employee Timesheet. Student Employee Timesheet (Federal Work Study) Timesheets must be completed in full. Timesheets must include signatures.
  • Cortical Vision Impairment: A Basic Understanding

    Cortical Vision Impairment: A Basic Understanding

    We hope you will join us again….blah, blah, blah for more …..to be continued. The WV SenseAbilities End slide - mentors at end Mentorship Program ….attachment Music again with more kid's pictures -show same kids Last words spoken should be:Each...