HAND-DRAWN CHARACTERS SENSING WITH ACCELEROMETER
Márton L. Kiss
Ph.D. student, assistant lecturer
Institute of Automation and Infocommunications, University of Miskolc 3515 Miskolc, Egyetemváros
ABSTRACT
This paper is introducing a methodology for motion sensing. The method must is appropriate to sens the targetted character based on Poisons chi-square statistics. Our aim was that to create a controller, what looks like a magic wand and it can sens the simple air drawings. This paper presents the structure of the controller and the algorithm of data processing and the result of the first test with the algorithm.
INTRODUCTION
This paper aim is to introduce the method which can sens the simple air drawings shape.
Basically, I choose the numbers from 0 to 9 for sensing. I used the simple accelerometer and the STM32 microcontroller board. Basically, the accelerometer has a lot of measurement error, therefore I try to find a method which not require accurate measurement. Firstly, I excluded the methods based on vector analysis. I should choose the method to form artificial intelligence or statistical methods. I will use this method on an embedded system, so my other important point is the performance requirement.
Finally, I find the Pearson's chi-squared method. I used these to classify the shapes.
In this paper, I will introduce the structure of the prototype. I give the funny name about that „magic wand”. After that, I will introduce the Pearson's chi-squared mathematical implications. I’m writing about the teaching phase of the algorithm and the fist measurement result.
TASK TO BE IMPLEMENTED
Measuring orientation data from the "magic wand" and the two-way rotation around the axis is very important! The data is recorded with an inertial sensor. It is imperative that I can send an appropriate real-time control signal from the gestures.
Furthermore, our goal is to recognize simple shapes (eg. digits, characters) drawn in the air using the inertial sensor which placed in the "magic wand". For remote controllers operating on a similar principle [3], they take control by recognizing the shapes shown on the following figure (Figure 1).
MultiScience - XXXIII. microCAD International Multidisciplinary Scientific Conference University of Miskolc, 23-24 May, 2019, ISBN 978-963-358-177-3
DOI: 10.26649/musci.2019.045
Figure 1 Targeted shapes
During the review of the literature, I met several times with gesture recognition, and there are many researchers around the world deal with the topic, which has confirmed the topicality of our research. There are many methods of implementation. I tried to solve the problem with the help of mathematical statistics methods.
The following sections describe the sensor used and the process of sensing and processing of the signal.
PROTOTYPE STRUCTURE
I need to build a “magic wand” prototype because I was able to make the measurements for the test phase of the algorithm. In the prototype, I used the STM32F407 discovery microcontroller card and an MPU6050 accelerometer. The accelerometer and the card were attached to an aluminum tube. The accelerometer is positioned on the “magic wand” tube so that the y-axis of the sensor is parallel to the tube. I have already been able to make measurements with this preformed product. The axes of the mounted accelerometer and the orientation angles (α, β) are illustrated in the following figure (Figure 2).
Figure 2
„Magic wand” orientation
Where RxRyRz are the registers of MPU650 accelerometer.
DETECTING ORIENTATION OF MAGIC WAND
Since Ry is parallel to the axis of the wand, its value increases proportionally by the tilt of the wand.
𝑅𝑦 ≈ 𝛽 𝛽 = 0°; 𝑅𝑦 = 0 𝛽 = 90°; 𝑅𝑦 = 𝑔 (1) This relationship exists because the accelerometer has a special feature that its axis pointing in the same direction as gravity acceleration measures the acceleration of gravity. In the vertical position of the wand, the y-axis points towards gravity, while in the horizontal position, the z-axis points towards gravity.
It is harder to calculate the rotation around the y-axis. The z-axis of the accelerometer is set to 0°, so it starts at 90° and increases to the left when rotated, while turning the right to decrease the angle. Rotation around the axis was calculated with the relation (2).
𝛼 = 90° + 𝑎𝑟𝑐𝑡𝑎𝑛Rz
Rx (2)
where α is the rotation angle around the axis of the wand.
METHOD OF RECOGNITION OF GESTS
Gesture recognition applications have been made by more people, there are user specifics and general ones that can be used by anyone. There are also pre-taught and self-learning systems [3] [4].
The application will be user-specific. Control commands can be predefined, so the system can be taught before the first use. I tried to simplify it because the algorithm will be run by a small microcontroller with finite memory and processor performance.
The process of training was done by drawing the shapes corresponding to the control signals into the air ten times in succession, and saving the measurement data to the computer, then averaging it into the benchmark, for which I compared the later data.
The following figure (Figure 3) illustrates a series of data recorded by the prototype while drawing the “1” digit into the air. The prototype reads the registers of accelerometer every 10 milliseconds.
Figure 3
”1” digit measurement
The green line is the z-axis and the blue is the data measured along the x-axis. On the horizontal axis of the coordinate system the number of samples, on the vertical axis the values of the register can be read. During gesture recognition, data measured on the y- axis of the accelerometer are neglected.
In order to automate the measurement, somehow, I have to decide where is starting the useful data series (drawing a shape into the air). I was able to infer this problem with the help of the Euclidean distance calculation. Calculation (3) is applied to each successive tenth element along the axis z and x.
𝐷 = √(𝑥𝑘 − 𝑥𝑘−10)2+ (𝑧𝑘 − 𝑧𝑘−10)2 (3) Pearson's chi-squared test and applications
Once I know where from to start saving the accelerometer data I can turn to the presentation of the applied statistical procedure.
Let 𝐴1, 𝐴2, … , 𝐴𝑘 be a complete event system. Make n number Bernoulli trials to monitor the events and Xi marks the frequency of Ai then
(𝑋1, 𝑋2, … , 𝑋𝑘) (4)
vectors are polynomial distribution. Write down the following hypotheses.
𝐻0 ∶ 𝑝𝑖 = 𝑝𝑖(𝜗), 𝑖𝑓 𝜗 ∈ 𝜃0 (5)
𝐻1 ∶ 𝑝𝑖 𝑜𝑝𝑡𝑖𝑜𝑛𝑎𝑙. (6)
Then
2 ln 𝐿(𝐻0, 𝐻1) = 2 ∑ 𝑌𝑖ln 𝑝̂𝑖
𝑘
𝑖=1
− 2 ∑ 𝑌𝑖ln 𝑝𝑖(𝜗̂)
𝑘
𝑖=1
= 2 ∑ 𝑌𝑖ln ( 𝑝̂𝑖 𝑝(𝜗̂))
𝑘
𝑖=0
, (7)
-2000 -1000 0 1000 2000 3000 4000 5000
1 36 71 106 141 176 211 246 281 316 351 386 421 456 491 526 561 596 631 666 701 736 771 806 841 876 911 946 981 1016 1051 1086 1121
Drawing "1" digit
where, 𝑝̂𝑖 =𝑌𝑖
𝑛 and 𝜗̂ is the estimate of 𝜗 maximum likelihood if 𝐻0 is fulfilled. After the derivation (see [1]), the following formula is obtained:
.2 ln 𝐿(𝐻0, 𝐻1) = ∑ (𝑜𝑖−𝑒𝑖)2
𝑒𝑖
𝑘𝑖=1 (8)
Pearson's chi-squared test can be used for testing optional distribution, that is fit testing.
In our case, oi is the i-th value of the distribution function of the benchmark data series and ei is the i-value of the measured distribution function (ei =0 in case of 2 ln 𝐿(𝐻0, 𝐻1) = ∑𝑘𝑖=1(𝑜𝑖)2).
To apply the above method, the measuring range should be divided into equal intervals, in the course of our application, I divided the measuring range into ten parts. On the basis of the division, from the measurement lines have to form a distribution function, with which the calculation can be performed.
At fit testing, the distance must be calculated for each benchmark and the one with the lowest value is most similar to the shape.
CONCLUSION
I have created the prototype of the "magic wand" with which I made test measurements.
Pearson's chi-squared test was used to detect air-drawn shapes. When comparing the measured data by the prototype with the benchmark, 95% of the respondents came to the right conclusion. 60 different comparisons were made during the evaluation. The error probably caused some kind of measurement error because it appeared on a certain data line. The great advantage of the method described above is that it calculates based on the distribution function, which means that a ten-element eight-bit integer array can store a benchmark or a measured data set so it takes relatively little memory to store and evaluation is a real-time evaluation due to the simplicity of the calculation.
ACKNOWLEDGMENTS
The research work described in this article is realized part of the EFOP-3.6.1-16- 00011, Rejuvenating and Renewable University - Innovative Knowledge City - University of Miskolc's Smart Specialization Institutional Development” project – in Széchenyi 2020 - with the support of the European Union, co-financed by the European Social Fund LITERATURE
[1] Dr. Fegyverneki Sándor, Dr. Raisz Péter (2009). Sztochasztikus modellezés. Nemzeti tankönyvkiadó.
[2] Sipos Attila, „Magyarok a Marson,” [Online]. Available:
http://www.magyarokamarson.hu/weblap2017/index_elemei/kuldetes2017.html. [Hozzáférés dátuma: 01 11 2015].
[3] J. K. O. W.-C. B. W. C. E. C. Y. J. J. C. Y. K. Sung-Jung Cho, „Magic Wand: A Hand-Drawn Gesture Input Device on 3-D Space with Inertial Sensors,” in Samsung Advanced Institute of Technology, Korea, 2004.
[4] L. B. S. Honor, „RBF háló alapú kérmozdulat felismerés,” in Erdélyi Magyar Műszaki Tudományos Társaság, Kolozsvár, 2016.
