SEMMELWEIS UNIVERSITY, DIALOG CAMPUS PUBLISHER

Development of Complex Curricula for Molecular Bionics and Infobionics Programs within a consortial* framework**

Consortium leader

PETER PAZMANY CATHOLIC UNIVERSITY

Consortium members

SEMMELWEIS UNIVERSITY, DIALOG CAMPUS PUBLISHER

The Project has been realised with the support of the European Union and has been co-financed by the European Social Fund ***

**Molekuláris bionika és Infobionika Szakok tananyagának komplex fejlesztése konzorciumi keretben

***A projekt az Európai Unió támogatásával, az Európai Szociális Alap társfinanszírozásával valósul meg.

PETER PAZMANY CATHOLIC UNIVERSITY

SEMMELWEIS UNIVERSITY

Peter Pazmany Catholic University Faculty of Information Technology

MODELLING NEURONS AND NETWORKS

Lecture 12

Emergent simulation environment exercises, part 2

www.itk.ppke.hu

(Idegsejtek és neuronhálózatok modellezése)

(2. Emergent gyakorlat)

Szabolcs Káli

Overview

This lesson is the second part of the Emergent simulation environment exercises. You will learn about:

• The CPCA algorithm

• Hebbian learning simulations

• Renormalization, contrast enhancement

• Self-organizing model learning

• Pattern associators

• The GeneRec algorithm

Hebbian model learning simulations and CPCA algorithm

In this section, we will apply a version of the Hebb rule known as the conditional principal component analysis (CPCA) learning rule:

Where is the learning rate. The activation function is now simply:

The weights (w

) are adjusted to match the value of activation x

, weighted in proportion to the activation of the receiving unit (y

).

Thus if the receiving unit is not active, no weight change will occur.

As weight changes accumulate over time, the weight will approximate the expected value of the sending unit when the receiver is active.

In other words:

This approach does not work well in practice because of the low

Modified CPCA algorithm – contrast enhancement

To enhance the differences between weak and strong weights, a sigmoidal function is applied to the weights, so that the weights approach their limits slowly. The parameters of the function:

- Offset controls the function’s x offset (denoted by θ).

- Gain controls the steepness of the function(denoted by γ).

Figure: Example weight function with offset 1.3 and gain 6.

The linear weight value (w

) is used for computing weight changes,

and the effective weight value ( ) is used for computing the inputs to

units, and is also the wt parameter in the simulations.

Modified CPCA algorithm – Renormalization

If the sending layer has low average activity, the average conditional probabilities are low as well, and this violates the notion that a probability of 0.5 should represent lack of correlation. The weights are renormalized to prevent this:

Where m is the new maximum weight value, which is calculated as:

And the effective activity level is:

Where q

is the sending average activation correction (denoted by

savg_cor in the simulator). If q

=0, no renormalization occurs, and if

Obtaining Emergent

Homepage: http://grey.colorado.edu/emergent/index.php

• Available for Linux, Windows, OSX

• Follow instructions on webpage for installing

Note: there is a Windows bug in the current version (5.1.0): If you get a

“CSS_NUMBER syntax error on line 000001” when loading a project (and Emergent freezes), and you are using some non-English input formats (like Hungarian) do the following:

1.Go to the control panel, then regional and language options.

2.Click customize in the Regional options tab

3.Set the decimal symbol to a dot („.” character) in the Numbers tab.

4.Click Ok twice to close the regional options and the custom options tab, then restart Emergent.

Exercises – Hebbian correlation

Download the simulation file (Hebb_correl.proj) from

http://grey.colorado.edu/CompCogNeuro/index.php/CECN1_Hebbian_C orrelation, and open it (File – Open ). This exercise will follow the

instructions written at this website.

1. View the weights into the hidden unit in the HebCorrelNet window:

press r.wt and click on the hidden layer neuron. Switch to viewing activities, and run the simulation. Now go back to the weights, and run the simulation again. What happened to the weights during learning? Rerun the simulation with the weights displayed.

2. Click on the Input_Patterns tab in the 3D view. You will see the

stimuli that make up this environment (the default input is

OneLineEnv_spec). Click the control panel tab in the left frame to

change settings. Now set p_right to 0.7 in the control panel (hit

Apply and Init), and Run the simulation. Switch back to the

HebbCorrelNet tab to see the results.

3. Click on the EpochOutputData in the 3D frame to view the graph log. This plot displays the value of one of the weights from a unit in the right-leaning diagonal line (black line) and the value of another weight from a right-leaning diagonal line. Change the learning rate (lrate) in the Control panel to 0.1, and Run again. What difference does this make?

4. Set lrate back to 0.005. Set p_right to 0.5 (nonselective activation for the unit), and re-run the simulation. What do you see?

5. Let us now explore a more complex environment. Switch input_data to ThreeLinesEnv and do View EVENTS. Set p_right back to 0.7, and then Run. Do you see any problems with the resulting weights?

Exercises – Hebbian correlation

Download the simulation file (self_org.proj) from

http://grey.colorado.edu/CompCogNeuro/index.php/CECN1_Self_Orga nizing, and open it. This model uses the same CPCA learning rule, but there are multiple receiving units that compete with each other.

The grid view display on the left shows all of the network's weights.

The large 5x4 grid has the same arrangement as the hidden layer of the network. Each grid element shows the weights for a hidden layer unit. You can get the same pattern by clicking on a hidden layer unit and showing its weights (r.wt)

The plot on the right shows the results of the unique pattern statistic (shown as uniq_pats in the graph), which shows how many unique hidden activity patterns were produced by presenting individually every (10) horizontal and vertical line.

Exercises - Self-organizing model learning

6. View the weights in the network window. Click on the Lines_input_Data tab to see the training patterns. Switch to activations (act button) in the 3D window. First click Train:init in the control panel to initialize the network (Initialize network weights too), then step the simulation a few times with the Cycle button. Then run the simulation.

7. You can run a batch of 8 training runs starting with a different set of random initial weights each time by pressing Batch init then run in the Control Panel. You can manipulate the parameters wt_sig.gain, wt_sig.off, and savg_cor.cor (which implement renormalization and contrast enhancement of the weights during learning), and the learning rate (lrate). How do these changes affect the resulting representations (use batch runs)?

8. Reset the simulation and set the mean of the initial random weights (rnd.mean) to 0.5. Explain the results.

You can view an overview of the results in the BatchOutputData tab. The batch view shows summary statistics about the average (mean), maximum, and minimum of the unique pattern statistic at the end of each network training run. The last column contains a count of the number of times that a "perfect 10" on the unique pattern statistic was recorded.

Exercises - Self-organizing model learning

Download the simulation file (pat_assoc.proj) from

http://grey.colorado.edu/CompCogNeuro/index.php/CECN1_Pattern_A ssociator , and open it.

Restart Emergent and load pat_assoc.proj. Note: press the SetLearnRule button, if you have changed the learning rule!

View the EasyEnv tab to view the environment. Select DELTA as the learn_rule in the control panel. Hit Run to complete the training.

9. Switch env_type to HARD, and Run the training. Then change the learning rule (LearnRule) to Hebbian (HEBB), and compare the performance of the two rules.

10.Now set env_type to IMPOSSIBLE, and try training the network

(The impossibleEnv tab displays the task). Can the model solve this problem?

Exercises – Pattern associator

• There are no obvious biological mechanisms for the standard error backpropagation algorithm (because for example axons propagate signals only in one direction).

• Solution: Generalized Recirculation Algorithm (GeneRec): Rewrite the backpropagation algorithm, so that error propagation between

neurons takes place using standard activation signals.

The algorithm has two phases:

• Minus phase: outputs represent the response of the network as a function of the input

• Plus phase: The environment provides the target activations.

11.Restart Emergent and download generec.proj from

http://grey.colorado.edu/CompCogNeuro/index.php/CECN1_Generec Run this simulation with the default settings (View TrialOutputGrid and ImpossibleEnv). Can this network solve the “impossible”

problem?

The GeneRec algorithm

Solutions – Hebbian correlation

Exercise 1: Initial weights of the network

Exercise 1: Weights of the network after learning. The weights have changed according to the activation pattern (constant diagonal line input).

Solutions – Hebbian correlation

Exercise 2: Weights of the network after learning. The inputs are two diagonal lines with input probability 0.7 and 0.3.

Solutions – Hebbian correlation

Exercise 3:

Left: Weights of the network with learning rate 0.005 Right: the same with learning rate 0.1

The higher learning rate causes the network weights to oscillate strongly around the expected values.

Solutions – Hebbian correlation

Exercise 4: When both events have the same probability, the model behaves as an unconditional PCA model (activation is not input-

Solutions – Hebbian correlation

Exercise 5: If units represent categories of inputs, the weights are spread across multiple inputs because the individual probabilities for each input are relatively small.

Solutions – Hebbian correlation

Solutions - Self-organizing model learning

Exercise 7: The output of a batch run with default parameters.

Right: the uniq_count column shows that all 8 runs recognized all patterns.

Solutions - Self-organizing model learning

Exercise 7: The output of a batch run with gain set to 1.

There were no runs when all 10 patterns were recognized because of the low

Solutions - Self-organizing model learning

Exercise 7: The output of a batch run with the gain function offset (wt_sig_off) set to 0.8. Because of the shifting of the offset some runs could not recognize patterns.

Solutions - Self-organizing model learning

Exercise 7: The output of a batch run with the renormalization (savg_cor) set to 1. This makes the weights increase more rapidly, and reduces the level of

Solutions - Self-organizing model learning

Exercise 7: The output of a batch run with the learning rate set to 0.1.

Solutions - Self-organizing model learning

Exercise 8: The output with the mean of the initial weight randomization (rnd.mean) set to 0.5. Now every unit participates in coding each line, because when we start with larger weight values the amount of weight decrease will be large compared to the weight increase. Thus there is a decreased probability of a cell active in two consequent runs, so previously inactive units will be more

Solutions - Self-organizing model learning

Solutions – pattern associator

The output of the pattern associator in the easy task with Hebbian learning rules. The network can recognize all four inputs correctly.

Solutions – pattern associator

Exercise 9: Example output of the pattern associator in the hard problem following Hebbian learning. Left: target input-output matrix. Right: result input- output matrix.

Solutions – pattern associator

Exercise 9: Example output of the pattern associator in the hard problem after learning using the Delta learning rule. Left: target input-output matrix. Right:

result input-output matrix.

Solutions – pattern associator

Exercise 10: The output of the pattern associator in the “impossible” problem using Hebbian learning. Left: target input-output matrix. Right: result input-

Solutions – pattern associator

Exercise 10: The output of the pattern associator in the “impossible” task using the Delta learning rule. Left: target input-output matrix. Right: result input-output matrix. This problem can not be solved by a single layer of neurons, because the task is not linearly separable.

Solutions – pattern associator

Exercise 11: The weights adjusted by the GeneRec algorithm after learning.

Note the added hidden layer, which makes this network capable of solving non linearly separable problems too.

Solutions – pattern associator

Exercise 11: The output of the GeneRec algorithm. Left: target input-output matrix. Right: result input-output matrix.

Summary

In this lesson we learned about

• The conditional PCA (CPCA) learning algorithm, which is a version of the Hebbian learning rule with the following properties:

• The weights will approximate the expected value of the sending unit when the receiver is active and are constrained to the 0-1 range with a soft bound, meaning that they approach their limits according to a sigmoidal function.

• To explore the CPCA algorithm in detail we analyzed simulations:

• The Hebbian correlation simulation had a single receiving unit and a grid of input units. It could complete the task of learning single diagonal lines as inputs, but when the units represented categories instead of instances (three diagonal lines) the weights got diluted.

• Next we analyzed a more complex model where multiple receiving units competed with each other, and the task was to recognize 8 different

patterns. It turned out that configuring the weight function correctly plays an important role in recognition, and shifting or changing the steepness of the weight function causes some input patters not to be recognized correctly.

• Finally we explored the effectiveness of different algorithms for a pattern association task. It turned out that the delta rule performs better that the Hebbian rule, but there are tasks that neither of them can solve. We

introduced the GeneRec algorithm, which is a version of the backpropagation