Cellular Neural Network

Development of Complex Curricula for Molecular Bionics and Infobionics Programs within a consortial* framework**

Consortium leader

PETER PAZMANY CATHOLIC UNIVERSITY

Consortium members

SEMMELWEIS UNIVERSITY, DIALOG CAMPUS PUBLISHER

The Project has been realised with the support of the European Union and has been co-financed by the European Social Fund ***

**Molekuláris bionika és Infobionika Szakok tananyagának komplex fejlesztése konzorciumi keretben

***A projekt az Európai Unió támogatásával, az Európai Szociális Alap társfinanszírozásával valósul meg.

11/27/2011. TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 2

Cellular Neural Network

www.itk.ppke.hu

CNN – Celluláris neurális hálózat

J. Levendovszky, A. Oláh, K. Tornai

Digitális- neurális-, és kiloprocesszoros architektúrákon alapuló jelfeldolgozás

Digital- and Neural Based Signal Processing &

Kiloprocessor Arrays

Contents

• What is CNN?

• The neighboring topology

• The new computational paradigm

• Art of programming

• Applications

• Template analysis

• Stability – Fix point analysis

www.itk.ppke.hu

11/27/2011. TÁMOP – 4.1.2-08/2/A/KMR-2009-0006

www.itk.ppke.hu

4

Motivation

• Implementing HNN by analog circuitry

• Fully connected too many wires, and intersections

• Number of neurons

, small network

• Number of wires (connections, layers)

• Impossible to implement

• Idea

• Use only local connections less wire, less connection possible to implement

100

N = ^{O N (}

^{) → 10 00 0}

www.itk.ppke.hu

CNN – Topology

• Cellular Neural Networks

• Locally connected

• 2D arrangement of neurons: N columns, M rows

1

i

M

1 2 3 j N − 1 N

11/27/2011. TÁMOP – 4.1.2-08/2/A/KMR-2009-0006

www.itk.ppke.hu

Neighborhood

• Neighborhood around neuron C(i,j) with radius r

• Number of neuron in neighborhood

• Example: r=1, r=2

6

{ }

( , ) ( , ), ,

S i j

= k l i − ≤ k r j l − ≤ r

(2 1) (2 1)

r → r + × r +

Cells

• Regular cells

• C(i,j) is regular cell with respect to if and only if all neighborhood cells

exist.

• Boundary cells

• The non-regular cells

• Edge cells

• The outermost boundary cells

( , )

( , ) C k l ∈ S i j

( , )

S i j

Cells

• r=2

• Regular cells

• Boundary cells

• Edge cells

11/27/2011. TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 8

www.itk.ppke.hu

CNN – Connections

• Connections from the neighborhood

• Feedback – current state

• Feed forward – input image

State of CNN

Input Image

11/27/2011. TÁMOP – 4.1.2-08/2/A/KMR-2009-0006

www.itk.ppke.hu

• Feedback of state

• Feed forward of input image

10

www.itk.ppke.hu

Combined figure

State of CNN

Input Image Input

forward

State feedback Neighborhood

CNN – Definition

• In the following slides the standard cellular neural network architecture will be defined

• Standard CNN:

• M by N rectangular array of cells C(i,j) which cell is defined mathematically by the followings:

• State equation

• Output equation

• Boundary conditions

• Initial state

11/27/2011. TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 12

www.itk.ppke.hu

CNN – Definition

• 1. State equation

• Where are called:

state, output, input and threshold

• And where are called:

feedback and input synaptic operators (will be defined)

www.itk.ppke.hu

( , ) ( , ) ( , ) ( , )

( , ; , ) ( , ; , )

r r

C k l i j

ij ij kl k

C k

l ij

S l S i j

x x A i j k l y B i j k l u z

∈ ∈

= − + ∑ + ∑ +

ɺ

, , ,

ij

R y

R u

R

R

x ∈ ∈ ∈ z ∈

( , ; , ), ( , ; , )

A i j k l B i j k l

CNN – Definition

• 2. Output equation:

• Standard nonlinearity:

11/27/2011. TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 14

www.itk.ppke.hu

1 1

( ) 1 1

2 2

ij

f x

x

x

y = = + − −

y

1

− 1

− 1

1

CNN – Definition

• 3. Boundary conditions

• To specify the for cells belonging to the

neighborhood of the boundary and edge cells, but lying out of the array.

• 4. Initial state

• The initial x values for cells

www.itk.ppke.hu

kl

,

y u

M × N

www.itk.ppke.hu

CNN – Space and time invariance

• A and B operators:

• In general they are space and time variant

• If they are linear, space and time invariant:

• Therefore the CNN is a HNN with restricted connections, represented by A and B

11/27/2011. TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 16

( , ; , ), ( , ; , ) A i j k l B i j k l

( , ; , ) ( ; )

( , ; , ) ( ; )

A i j k l A i k j l B i j k l B i k j l

= − −

= − −

www.itk.ppke.hu

Boundary conditions

• Virtual cells

• Any cell C(k,l) in the neighborhood of cell C(i,j), where and/or

• Corresponding are called virtual state, virtual output, virtual input respectively.

• Any virtual variable must be defined via various boundary conditions.

• These conditions are most commonly used for 3x3 neighborhood

{ ^{1, 2,...} }

k ∉ M ^{l ∉} { ^{1, 2,... N} }

, ,

kl kl kl

x y u

www.itk.ppke.hu

Fixed boundary conditions

• Fixed (Dirichlet) boundary conditions

• Left virtual cells:

• Right virtual cells:

• Top virtual cells:

• Bottom virtual cells:

• The parameters are pre-defined, usually equals to zero

11/27/2011.

TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 18

.0 1 ,0 1

1, 2, ,

i i

y = α u = β i = … M

. 1 2 , 1 2

1, 2, ,

i N i N

y

= α u

= β i = … M

0,_j 3

u

j 1, 2, , N

y = α = β = …

1, 4 1, 4

1, 2 , ,

M j M j

N

y

= α u

= β j = …

i

,

α β

www.itk.ppke.hu

Zero-flux boundary conditions

• Zero-flux (Neumann) boundary conditions

• Left virtual cells:

• Right virtual cells:

• Top virtual cells:

• Bottom virtual cells:

• The corner virtual cells can be arbitrary defined.

.0 ,1 ,0 ,1

1, 2, ,

i i i

u

i M

y = y u = = …

. 1 . , 1 ,

1, 2, ,

i N i N i N i N

y

= y u

= u i = … M

0,_j 1,_j 0,_j

u

j 1, 2, , N

y = y u = = …

1, , 1, ,

1,2, ,

M j M j M j M j

y

= y u

= u j = … N

www.itk.ppke.hu

Periodic boundary conditions

• Periodic (Toroid) boundary conditions

• Left virtual cells:

• Right virtual cells:

• Top virtual cells:

• Bottom virtual cells:

• This is equivalent to fabricating a CNN chip on silicon torus as its substrate

11/27/2011.

TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 20

.0 , ,0 ,

1, 2, ,

i i N i i N

y = y u = u i = … M

. 1 .1 , 1 ,1

1, 2 , ,

i N i i N i

y

= y u

= u i = … M

0,_{j M j}, 0,_{j M j},

1, 2, , N

y = y u = u j = …

1, 1, 1, 1,

1,2 , ,

M j j M j j

y

= y u

= u j = … N

www.itk.ppke.hu

Boundary conditions

Fixed conditions

=0

Edge cells

Virtual cells

www.itk.ppke.hu

Boundary conditions

Zero-flux conditions

11/27/2011.

TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 22

Virtual cells

Edge cells

Boundary conditions

Toroid conditions

www.itk.ppke.hu

Virtual cells

Edge cells

www.itk.ppke.hu

Computational paradigm

• Given 2D array of locally interconnected cell (with radius r)

• Spatially and time invariant A and B operators

• Cloning templates, square matrices with size 2r+1

• One z threshold parameter

• Usually r=3 18+1 free parameters

• Nonlinear output function

11/27/2011.

TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 24

yij 1

−1

−1 1

1 1

( ) 1 1

2 2

ij

f x

x

x

y = = + − −

www.itk.ppke.hu

Computational paradigm

• Differential equation for each cell:

• CNN has the following computational model

• Given input and initial state,

• Given 19 free parameters =???

, ,

( , ) ( , )

k i r l j

ij ij

r k i r l

k j

kl l

r

x x A i k j l y B i k j l u z

− ≤ − ≤ − ≤ − ≤

= − + ∑ − − + ∑ − − +

ɺ

( ) ( ) ( ) l i m ( ( ) )

t t t t

→ → ∞ = → ∞

U X Y Y

www.itk.ppke.hu

Art of programming

• The task is to find the appropriate A and B template values and z threshold value

• (Engineering task)

• Special case :

• Time and space invariant CNN

• The differential equation for each cell:

• Where …

11/27/2011.

TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 26

ij ij ij ij

x ɺ = − + ⊗ x A Y + ⊗ B U + z

www.itk.ppke.hu

Special case

• If r=1

• Neighborhood of one cell C(i,j)

ij ij ij ij

x ɺ = − + ⊗ x A Y + ⊗ B U + z

1, 1 1,0 1,1 1, 1 1,0 1,1

0, 1 0,0 0,1 0, 1 0,0 0,1

1, 1 1,0 1,1 1, 1 1,0 1,1

a a a b b b

a a a b b b

a a a b b b

− − − − − − − −

− −

− −

   

   

=   =  

   

   

A B

1, 1 1, 1, 1 1, 1 1, 1, 1

, 1 , , 1

i j i j i j i j i j i j

ij i j i j i j ij i j i j i j

y y y u u u

y y y u u u

y y y u u u

− − − − + − − − − +

− + − +

+ − + + + + − + + +

   

   

=   =  

   

   

Y U

www.itk.ppke.hu

Special case

• The operator

11/27/2011.

TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 28

ij ij ij ij

x ɺ = − + ⊗ x A Y + ⊗ B U + z

1, 1 1,0 1,1 1, 1 1, 1, 1

0, 1 0,0 0,1 , 1 , , 1

1, 1 1,0 1,1 1, 1 1, 1, 1

1, 1 1, 1 1,0 1, 1,1 1, 1

0, 1 , 1 0,0 , 0,1 , 1

1

i j i j i j

i j i j i j

i j i j i j

i j i j i j

i j i j i j

a a a y y y

a a a y y y

a a a y y y

a y a y a y

a y a y a y

a

− − − − − − − − +

− − +

− + − + + +

− − − − − − − − +

− − +

 

 

 

 

⊗   =

 

   

   

⋅ ⋅ ⋅

= ⋅ ⋅ ⋅

, 1₋

y

a

y

a

y

 

 

 

 ⋅ ⋅ ⋅ 

 

⊗

www.itk.ppke.hu

Special case

• Both template can be decomposed as follows

• Thus the following equation will be used

ij ij ij ij

x ɺ = − + ⊗ x A Y + ⊗ B U + z

1, 1 1,0 1,1

0

0,0 0, 1 0,1

1, 1 1,0 1,1

0

0 0

0 0

0 0

0 0 0

a a a

a a a

a a a

− − − −

−

−

 

 

 

 

= =   +  

   

   

+

=

+ A A

B B

A

B

00

ij ij ij ij ij

x ɺ = − + x a ⋅ + ⊗ y A Y + ⊗ B U + z

www.itk.ppke.hu

Edge detection

• Global task

• Given: static binary image P

• Input: U(t) = P

• Initial state: arbitrary

• Boundary conditions: fixed type, =0

• Output: Y( ∞ ) = Binary image showing all edges of P in black

11/27/2011.

TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 30

www.itk.ppke.hu

Edge detection

• Example

• White pixel: –1 , Black pixel: +1

www.itk.ppke.hu

Edge detection – rules

• Local rules

• White pixel white pixel, independent of neighbors

• Black pixel white, if all nearest neighbors are black

• Black pixel black, if at least one nearest neighbor is white

11/27/2011.

TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 32

if u

= − 1, then ( ) y

∞ = − 1

if u

= ∧ ∀ 1 ( , ) l k ∈ S i j

( , ) : u

=1, then ( ) y

∞ = − 1

if u

= ∧ ∃ 1 ( , ) l k ∈ S i j

( , ) : u

= − 1, then ( ) 1 y

∞ =

www.itk.ppke.hu

Solution

1

0 0 0

0 0 0

0 0 0

1 1 1

1 8 1

1 1 1

1 r

z

=

 

 

=  

 

 

− − −

 

 

= −  − 

 − − − 

 

= − A

B

• Radius of locality

• Only the closest neighbors are needed

• This A template is a null template providing a CNN without feedback connections

• This B template contains threshold values and it is constructed using the previous local rules

• Will be analyzed

www.itk.ppke.hu

Edge detection

• Remember

• Some notation

• This case

11/27/2011.

TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 34

00

ij ij ij ij ij

x ɺ = − + x a ⋅ + ⊗ y A Y + ⊗ B U + z

( ) 0

( )

0

( )

ij ij

g x w

ij ij ij ij ij

t

x ɺ = − + x a ⋅ f x + ⊗ A Y + ⊗ B U + z

( , ) 1( ( , ( ,

) ) , )

8 1

ij ij ij ij ij kl

S i j

k l i j

k l

x x z x u u

∈≠

= − + ⊗ ^{B U} + = − + − − ∑

ɺ

www.itk.ppke.hu

Edge detection – analysis

• Rule 1: ( )

• The rule has been satisfied

( , ) 1

( , ) ( , ) ( , )

8 1

ij ij kl

S

k l i j

k l i j

w u u

∈≠

= − − ∑

if u

= − 1, then ( ) y

∞ = − 1

8 every pixel is white in the neighborhood 8 1 8 every pixel is black in the neighborhood

1 17 w

white white

−

= − − −   =



= − →

−

www.itk.ppke.hu

Edge detection – analysis

• Rule 2: ( )

• Rule 3: ( )

• These rules have been satisfied

11/27/2011.

TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 36

( , ) 1

( , ) ( , ) ( , )

8 1

ij ij kl

S

k l i j

k l i j

w u u

∈≠

= − − ∑

8 1 8 (every pixel is black in the neighborhood) 1

w

= − − = − = white

if u

= ∧ ∀ 1 ( , ) l k ∈ S i j

( , ) : u

=1, then ( ) y

∞ = − 1

if u

= ∧ ∃ 1 ( , ) l k ∈ S i j

( , ) : u

= − 1, then ( ) 1 y

∞ = 8 1 ( 6, at least one pixel is white in the neighborhood)

1

w

n n

black

= − − ≤ =

= + =

www.itk.ppke.hu

Previous example

1

0 0 0

0 0 0

0 0 0

1 1 1

1 8 1

1 1 1

1 r

z

=

 

 

=  

 

 

− − −

 

 

= −  − 

 − − − 

 

= − A

B

• Consider a previous edge detection template and example

• The stability of the state equation can be observed plotting the

driving-point (DP) plot

• In order to do this the following rules of thumb have to be

considered

www.itk.ppke.hu

DP – Plot (uncoupled case)

• Uncoupled In the A template only the center element is nonzero

11/27/2011.

TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 38

( ) ( )

00

( )

ij ij

g

ij i

t ij

w

j j

x

x ɺ = − + x a ⋅ f x + ⊗ B U

+ z

xɺij

−1 1 x^ij

P^- left breakpoint P⁺ right breakpoint

Slope = -1

(left saturation region)

Slope = -1

(right saturation region) Slope = s₀₀=a₀₀-1

(central linear region)

x_ij = w_ij

DP – Plot (uncoupled case)

• Stability

www.itk.ppke.hu

( ) ( )

00

( )

ij ij

g

ij i

t ij

w

j j

x

x ɺ = − + x a ⋅ f x + ⊗ B U

+ z

xɺij

−1

1

Q^- stable point Q⁰ stable point Q⁺ stable point

www.itk.ppke.hu

11/27/2011.

TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 40

( ) ( )

00

( ) 1

ij ij

ij

g x w

ij i ij ij

t

x ɺ = − + x a ⋅ f x

+ ⊗ B U + z w ≤ −

xɺij

−1 1

xij

W_ij<-1

Edge

Q stable point

www.itk.ppke.hu

( ) (

00

)

( ) 1 1

ij ij

g x

ij ij ij ij

w

ij t

x ɺ = − + x a ⋅ f x + ⊗ B U + z − < w <

xɺij

−1 1

xij

w_ij

Edge

Q stable point

www.itk.ppke.hu

11/27/2011.

TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 42

( ) (

0

)

0

( ) 1

ij ij

ij ij ij ij ij

g x w t

x ɺ = − + x a ⋅ f x + ⊗ B U + z ≤ w

xɺij

−1 1

xij

w_ij

Edge

Q stable point

www.itk.ppke.hu

Logical not template

• Global task

• Given: static binary image P

• Input: U(t) = P

• Initial state: arbitrary

• Boundary conditions: fixed type, =0

• Output: Y( ∞ ) = Negated image

• Local rules

• White pixel black pixel, independent of neighbors

• Black pixel white pixel, independent of neighbors

www.itk.ppke.hu

Solution

11/27/2011.

TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 44

1

0 0 0

0 1 0

0 0 0

0 0 0

0 2 0

0 0 0

0 r

z

=

 

 

=  

 

 

 

 

=  − 

 

 

= A

B

• Rule 1

• Rule 2

• In case of rules the g() function:

• The state equation

• Depending on the color of input pixel 2

2

ij ij

w w

= +

= −

(

)

g x = − + x a ⋅ y = − + x y

ij ij ij

2

x ɺ = − + x y ∓

www.itk.ppke.hu

DP – Plot

xɺij

−1

1 ^xij

w_ij=2

xɺij

−1 1 x_ij

w_ij=-2

yij 1

−1

−1 1

yij 1

−1

−1 1

www.itk.ppke.hu

Simulations – Edge template Binary Edge template on binary picture

11/27/2011. TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 46

Simulations – Edge template Binary Edge template on a grayscale picture

www.itk.ppke.hu

www.itk.ppke.hu

Simulations – Edge gray template

11/27/2011.

TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 48

Edgegray template on

Gray scale images

www.itk.ppke.hu

Simulations – Other tasks with one template Finding closed

shapes in a picture

Modified diffusion template to find

dense clusters

www.itk.ppke.hu

Complex task with multiple templates

11/27/2011.

TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 50

Finding watermark on picture

(Further step,

remove watermark

from the picture)

www.itk.ppke.hu

Summary

• Cellular Neural Network

• It is impossible to implement a HNN

• Idea: local connections only

• Structure

• Local connections, Input connections, Templates

• Differential equation as state transition rule

• 19 free parameters

• Definition of CNN

• Templates, boundary conditions, initial state

www.itk.ppke.hu

Summary

• Image processing, task definition

• Local rules

• Constructing the templates

• Analysis, convergence

• DP plot, Edge template analysis

• Real examples applications

• EDGE, EDGEGRAY

• Logical functions

• Complex tasks

11/27/2011.

TÁMOP – 4.1.2-08/2/A/KMR-2009-0006 52