Analysis of myoelectric signals using a Field Programmable SoC

(1)

Analysis of myoelectric signals using a Field Programmable SoC

Bence J. Borb´ely

Interdisciplinary Technical Sciences Doctoral School Pázmány Péter Catholic University, Faculty of Information Technology

September 27, 2013

September 27, 2013 B. J. Borb´ely, PPCU-FIT

(2)

Outline

Background Our goals Algorithm Implementation Results

Future work

(3)

The ultimate goal

Star Wars Episode V: The Empire Strikes Back

(4)

What do we need?

1. properhardware that is able to restore hand functions (e.g.

different grip patterns)

2. real-time and precise control for dynamic and life-like operation

(5)

What do we need?

different grip patterns)

(6)

What’s available - hardware (on the market)

i-limbfrom Touch Bionics

http://www.touchbionics.com/products/active- prostheses/i-limb-ultra-revolution/

(7)

What’s available - hardware (on the market)

bebionic3from RSLSteeper

http://futuristicnews.com/bebionic3-new-robotic-hand/

(8)

What’s available - hardware (on the market)

Michelangelofrom Otto Bock

http://www.living- with- michelangelo.com/uploads/pics/anwender_hand_002_barebone.png http://www.oandp.com/articles/images/2011- 04_12/04_12-02.jpg

(9)

What’s available - hardware (on the market)

Common characteristics:

I various grip patterns

I various grip forces

I opposable thumb

I mostly used withconventional myoelectric control

(10)

What’s available - hardware (in research)

Modular Prosthetic Limbfrom Johns Hopkins University (APL)

http://www.sciencedaily.com/releases/2010/08/100804081227.htm http://www.jhuapl.edu/newscenter/stories/images/st120524_arm3.jpg

(11)

What’s available - hardware (in research)

Main characteristics:

I state of the art(DARPA project,∼$30.5 million)

I full kinematic control of 26 degrees of freedom

I mass and size of an average female arm

I being tested with targeted muscle reinnervation

(12)

What do we need?

different grip patterns)X

(13)

What do we need?

different grip patterns)X

(14)

What’s available - control (on the market)

Conventional myoelectric control

I based on surface electromyogram (sEMG): the electrical activity recorded from the covering skin of skeletal muscles during contraction

(15)

What’s available - control (on the market)

I 2 EMG recording sites

I direct control of only one DoF (e.g. hand open/close)

I mode switching

I velocity is constant or proportional to a chosen signal metric (e.g. RMS)

(16)

What’s available - control (in research)

Pattern recognition based control

I based on forearm sEMG

I multiple recording sites (≥4)

I direct control of multiple degrees of freedom (e.g. hand open/close, wrist flexion/extension, wrist rotation, etc.)

I velocity is proportional to a chosen signal metric (e.g. RMS)

(17)

What’s available - control (in research)

Targeted Muscle Reinnervation with pattern recognition

http://www.rehab.research.va.gov/jour/11/486/page643.html

(18)

Background Our goals Algorithm Implementation Results Future work

Main goals - focus on the control part

1. implementation of the standard pattern recognition method (based on forearm EMG) in hardware

1.1 offlineand

1.2 online(real-time) processing

2. development of new strategies for dynamiccontrol 3. integrate the above into a working prototype (hardware +

algorithm)

(19)

Main goals - focus on the control part

1.1 offlineand presented at ECCTD 2013 1.2 online(real-time) processing

2. development of new strategies for dynamiccontrol 3. integrate the above into a working prototype (hardware +

algorithm)

(20)

Why do we need pattern recognition?

Surface EMG is a good source...

I non-invasive

I ”easy” to measure

I can be related to muscle contraction and exerted force

I stochastic signal summed in time and space

I the relation is usually time variant and highly non-linear

(21)

Why do we need pattern recognition?

I non-invasive

...but it is not perfect.

(22)

Why do we need pattern recognition?

I non-invasive

I not really selective

(23)

Why do we need pattern recognition?

I non-invasive

(24)

Why do we need pattern recognition?

I non-invasive

(25)

The standard pattern recognition algorithm

Offline training

Windowing (150 ms) Offline input EMG signals

Preprocessing

Storage

Best separating hyperplane

Class projections Predefined

kinematic positions (classes)

Dimension reduction (LDA)

(26)

The standard pattern recognition algorithm

Offline training Real-time

classification

Preprocessing

Storage

Class projections

Windowing (150 ms) Real-time input

EMG signals

Preprocessing

Dimension reduction

Classification Predefined

Actual movement

intent

Actuator movement Average accuracy

is above 90%.

(27)

The standard pattern recognition algorithm

classification

Preprocessing

Storage

Class projections

EMG signals

Preprocessing

Dimension reduction

Actual movement

intent

is above 90%.

(28)

The standard pattern recognition algorithm

classification

Preprocessing

Storage

Class projections

EMG signals

Preprocessing

Dimension reduction

Actual movement

intent

is above 90%.

(29)

Preprocessing

I EMG signals are transformed into a feature space using a 150 ms long moving window and various possible window shift values (1, 5, 10, 25 and 50 ms were tested)

I examples of time-domain features:

I Mean Absolute Value (MAV)

I Number of Zero Crossings (NZC)

I Number of Slope Sign Changes (NSSC)

I Waveform Length (WL)

time (ms)

EMG activity (mV)

0 50 100 150

-500 0 500

(30)

Preprocessing

time (ms)

EMG activity (mV)

0 50 100 150

-500 0 500

(31)

Preprocessing

time (ms)

EMG activity (mV)

0 50 100 150

-500 0 500

(32)

Preprocessing

time (ms)

EMG activity (mV)

0 50 100 150

-500 0 500

(33)

Preprocessing

time (ms)

EMG activity (mV)

0 50 100 150

-500 0 500

(34)

Dimension reduction and classification

I dimension reduction usingLinear Discriminant Analysis (LDA) involving numerical calculation of:

I covariance matrices

I inverse matrices

I eigenvalues and eigenvectors

x₁ x2

-10 -5 0 5 10

I classification using nearest centroid method based on Eucledian or Mahalanobis distance

(35)

Hardware design considerations

I Goals:

I Real-life hardware implementation of the standard EMG pattern recognition method.

I Scalable test environment for advanced myoelectric control algorithms.

I Basis for ahigh performance, embeddedsystem that can be integrated into a forearm socket.

I The used hardware should have:

I high computational potential, including parallel processing

I low power consumption

I small area

I possibility of reconfiguration

(36)

Xilinx Zynq-7000 family: PS + PL

http://www.xilinx.com/technology/roadmap/zynq7000/features.htm

(37)

Reference implementation in ANSI C

I offline classification

I the system is prepared for real-time operation

I numerical implementation of all functions involving matrix algebra

I test environments:

I Intel Core i5-540M (@ 2.53 GHz, TDP∼40 W) running Ubuntu Linux 12.04 LTS

I ARM Cortex-A9 of the Zynq SoC (@ 667 MHz, TDP<4 W) running Digilent’s embedded Linux OS (Digilent Zedboard prototyping platform with Zynq-7020)

(38)

System design and partitioning on the Zynq-7020

ARM Processor Core Preprocessor

unit

Vector Processor

Sensor Interface

PL (FPGA fabric)

Memory Controller Sensor

data

AXI Bus AXI Bus AXI Bus

AXI Bus

PS (Hard Processor System)

External Memory(sensor datastored)

(39)

Architecture of the implemented units

ABS Unit

MAV Unit

MAVS Unit

RMS Unit

WAMP Unit

NZC Unit

NSSC Unit VAR Unit

WL Unit Sample AbsDiff Unit

MAV out MAVS out

RMS out

AVG

Unit VAR out

WAMP out

NZC out

NSSC out

WL out

Samples from AXI Bus

Preprocessor

64×64 bit Vect_0

× +

D

A DI

DO

ReadAddr WriteAddr

64×64 bit Vect_1

A DI

DO

64×64 bit Vect_n A DI

DO

Scratchpad memory

DI DO

A DI DO

PS domain

Vector Processor

(40)

Resource utilization on the FPGA

Resources

Requirement

General 53200 1161 2820 3981 7.5%

Flip-Flops 106400 1548 3002 4550 4.3%

Slices 13300 371 1118 1489 11.2%

Dedicated

DSP48 Slice 220 21 16 37 16.8%

140 0 4 4 2.9%

Available on the

Zynq-7020 Resource

utilization Preprocessor

(PP) Vector Processor

(VP) Total

(PP + VP) 6-input LUTs

36Kb BlockRAM

asd

(41)

Resource utilization on the FPGA

Resources

Requirement

General 53200 1161 2820 3981 7.5%

Flip-Flops 106400 1548 3002 4550 4.3%

Slices 13300 371 1118 1489 11.2%

Dedicated

DSP48 Slice 220 21 16 37 16.8%

140 0 4 4 2.9%

Available on the

Zynq-7020 Resource

utilization Preprocessor

(PP) Vector Processor

(VP) Total

(PP + VP) 6-input LUTs

36Kb BlockRAM

asd

We have a lot of free space!

(42)

Execution times (4 channels, 150 ms EMG window)

Classifier training

Processing window shift

Processing time (ms)

50 ms 25 ms 10 ms 5 ms 1 ms 10¹

10² 10³ 10⁴ 10⁵

PC ARM VEC

Classification

Processing window shift

Processing time (ms)

50 ms 25 ms 10 ms 5 ms 1 ms 10^-2

10^-1 10⁰

PC ARM

(43)

Main goals - focus on the control part

1.1 offlineand presented at ECCTD 2013 1.2 online(real-time) processing

2. development of new strategies for dynamiccontrol

to be presented at BioCAS 2013

3. integrate the above into a working prototype (hardware + algorithm)

(44)

Main goals - focus on the control part

1.1 offlineand presented at ECCTD 2013

1.2 online(real-time) processing proposal to ISCAS 2014

(45)

Main goals - focus on the control part

1.1 offlineand presented at ECCTD 2013

1.2 online(real-time) processing proposal to ISCAS 2014

to be presented at BioCAS 2013

(46)

Estimating the instantaneous wrist flexion angle from multi-channel surface EMG of forearm muscles

Classification results

Time (s)

Wrist flexion angle (degree)

1 2 3 4 5 6 7

-80 -60 -40 -20 0 20 40 60

80 Original Without the constraint With the constraint

(47)

Summary

I offline implementation of the standard pattern recognition method is done

I online implementation is in progress

I new directions towards dynamic control are being investigated and the first steps show promising results

(48)

Acknowledgement

ECCTD co-authors: Zoltán Kincses(University of Szeged),Zsolt Vörösházi(University of Pannonia),Zoltán Nagy(PPCU-FIT, MTA-SZTAKI),Péter Szolgay(PPCU-FIT, MTA-SZTAKI) This research project was supported by the OTKA Grant No.

K84267. The publication/research has been supported by the European Union and Hungary and co-financed by the European Social Fund through the projects:

I T´AMOP-4.2.1.B-11/2/KMR-2011-0002

I T´AMOP-4.2.2.C-11/1/KONV-2012-0004

(49)

Thank you for your kind attention!

borbely.bence@itk.ppke.hu