Control of Articulated Mechanical Prosthetics...Control of Articulated Mechanical Prosthetics...

Control of Articulated Mechanical Prosthetics Federico Parra De Nova1, Samuel Katz2, David Miller3, Chris Eads4, Daren Wilcox5

Abstract - In this paper, the development of a prototype biomimetic hand prosthetic device controlled by Myoware sensors and a brain-machine interface (BMI) is presented. The use of both the BMI and Myoware sensors create a reliable system so that the prosthetic does not operate without intention from the user. The prototypes BMI is to analyze EEG signals from the prefrontal cortex of the brain and translate them into useful information for the prosthetic's control system. The goal is to create a user-friendly, cost efficient, reliable, responsive, BMI-enhanced prosthetic. As a result, the device could be used in day to day activities whilst giving the patient a sense of control and individuality. This project and research uses signal processing knowledge to filter out noise and identify the bio-signals obtained. The work is ongoing research at Kennesaw State University.

Keywords: Prosthetic, Control, Brainwave, Myoelectric, BMI, Signal Processing, Bio-sensors, Bio-signals.

1 Introduction Each year, an estimated 185,000 amputations occur in the United States, with an estimated 2 million people in the United Sates living with limb loss. The main cause of limb loss is vascular disease at 54 percent, followed by trauma, then cancer. (2) Due to prosthetic devices, loss of a limb does not have to be the end of the amputee’s ability to complete their day-to-day activities. At present, most prosthetics are custom-made to fit a specific patient’s need, thus electromechanical prosthetics are immensely expensive, often comparable to the price of a new vehicle, because of the manufacturing specificity and required durability of the materials. Furthermore, industry standards for the prosthetics only include limited movement and poor dexterity. Amputees yearn for a reliable device that will give them enough dexterity to complete their day to day activities without a hassle. The goal of this research is to find a new, cost-effective way to control a prosthetic device, as well as prototyping a cost-effective prosthetic. A combination of brainwave mapping and myoelectric signal control will be researched and applied to a prosthetic hand prototype in order to achieve better control and dexterity in this project. This will prove that a BMI could be used to achieve dexterity through integration of muscle signals in a package that is affordable to the public. Presented in this paper is the current state of the KSU research project.

2 System Level Design The design of the prosthetic hand includes a biomimetically designed mechanical assembly of a hand, 5

1 Kennesaw State University, 1100 S. Marietta Parkway, Marietta, GA 30060-2896, [email protected] 2 Kennesaw State University, 1100 S. Marietta Parkway, Marietta, GA 30060-2896, [email protected] 3 Kennesaw State University, 1100 S. Marietta Parkway, Marietta, GA 30060-2896, [email protected] 4 Kennesaw State University, 1100 S. Marietta Parkway, Marietta, GA 30060-2896, [email protected] 5 Kennesaw State University, 1100 S. Marietta Parkway, Marietta, GA 30060-2896, [email protected]

micro servo motors as actuators, a MindFlex Sensor, Myoware Sensors, and an Arduino UNO microcontroller board. The design decisions are clarified below: 2.1 Biomimetic Hand Assembly:

• Cost efficient • Low risk assessment (highly replaceable with small

turnaround time) • No assembly required after purchase • Micro servo slots allow for simple motor

replacement • Opposable thumb to permit grasping

Modifications were made to this assembly to make it more efficient and fit the needs of the project. For instance, the assembly is made from acrylic pieces, so the micro servos did not complete the full turn at times due to resistance in the joints. Therefore, instead of lubricating the mechanical joints with grease or a petroleum base lubricant it had to be coated with graphite to ensure a smooth finger movement.

Figure 1. Robotic Hand with Servos

2.2 Micro Servo SG90 • Cost Efficient ($10 or less) • Low risk assessment (highly replaceable with small

turnaround time) • Simple to use and common design • Small enough so five of these can fit inside a small

assembly. (22.2mm by 11.8mm by 31mm) • Strong enough to move finger assembly and grasp.

(1.8kgf•m) • Fast response (60º turn in 0.1s)

Int'l Conf. Biomedical Engineering and Sciences | BIOENG'18 | 7

ISBN: 1-60132-472-3, CSREA Press ©

2.3 Arduino UNO

• Cost Efficient (less than $50) • Low risk assessment. • Small enough to fit in a forearm environment. • Simple coding environment. • Compatible code for a smaller Arduino (Pro-Mini) if

a smaller footprint is needed. • Small and common power supply.

2.4 Myoware Sensor

• Cost Efficient (less than $50) • Low risk assessment • Conveniently small (all-in-one packet). • Flexibility on choice of button pad electrodes

2.5 Mindflex Controller

• Cost Efficient (less than $100) • Medium level risk assessment (replaceable with

medium turnaround time) • All-in-one packet electronics and sensors • Common replaceable battery power supply

3 Control System Integration 3.1 Myoware Sensor

The Myoware sensor detects voltage difference between two points in the arm, due to a muscle being flexed or relaxed. This voltage difference is then rectified and converted to a smooth wave that an analog-to-digital converter (ADC) can process. The Myoware sensor because it possesses signal processing capabilities integrated in a small package. In the case of electro-cardio-myography (EKG), the raw data is necessary to investigate the complex waves when the heart's ventricles contract and expand. For this project, however, that level of detail is not needed. So much detail on the detected signal of a fibrous muscle's flexion or extension will render oscillations due to the sodium potassium imbalances in the muscle coming into balance once the muscle finishes the contraction. Figure 2 illustrates the difference.

The signal coming out of the Myoware makes it simple for the Arduino to interpret when connected to one of the analog pins. Once the signal is connected to an analog pin, one can give it any designation: voltage if conversion is done, or the raw signal as an integer (to have a more precise threshold). Figure 2 illustrates the difference.

Figure 2. Myoware User Manual Signal Explanation (1)

3.2 MindFlex Controller (EEG Sensor)

The MindFlex controller is a headband sensor that detects and interprets brainwave signals. The MindFlex sensor is comprised of a signal filtering board, a Neurosky chip that interprets the brainwave signals, and a 2.4 GHz transmitter. The proprietary Neurosky chip interprets the signals sensed and turns them into a CSV that is sent to the transmitter approximately every second. (9) The signals sent in the array are as follows: 1. Connection quality: A value from zero (best connection)

to two-hundred (no connection). Any number in between indicates a low-quality connection.

2. Attention: Value interpreted directly by the Neurosky chip that determines how much attention the subject is paying based on the brainwave signals.

3. Meditation: Value interpreted by the Neurosky chip that determines how relaxed the person wearing the headband is.

4. Delta Waves: Brainwave signals (BWS) associated with deep sleep and dreaming

5. Theta Waves: BWS associated with a comatose or dazed state

6. Low Alpha: BWS associated with a state of relaxation or meditation

7. High Alpha: BWS associated with a state of relaxation or meditation

8. Low Beta: BWS associated with alertness or decision making

9. High Beta: BWS associated with alertness or decision making

10. Low Gamma: BWS associated with a state of focus or alertness.

11. High Gamma: BWS associated with multiple parts of the brain firing simultaneously, interpreted primarily as noise.

The electromechanical prosthetic, made of acrylic plastic and

light metal, resembles a human hand with micro servo-controlled joints. An Arduino UNO will be used to control the micro servos and to comply with design simplicity. For brainwave signal acquisition, a MindFlex controller containing a signal probe, noise reduction circuitry, and a Neurosky chip

8 Int'l Conf. Biomedical Engineering and Sciences | BIOENG'18 |


(which detects and interprets the signal) will be used. The Neurosky chip contains four pins that connect it to the noise reduction and power circuit board inside the MindFlex. The transmit pin is connected via cable to pin 0 (or RX) of the Arduino. The data transmitted is processed by the Arduino to make decisions according to the code used. The Myoware sensors will be connected to the 5V power supply of the Arduino according to a recommended setup from the Myoware manual (1), then the Myoware signal pins will be connected to four of the six the analog pins of the Arduino. 4 Power Distribution The hand’s servo motors are on their own rail powered by 4 AA batteries (a 7.4V lithium polymer battery could be used, allowing better space efficiency and rechargeability). Power distribution to the Myoware sensors utilize a power rail coming off of the UNO’s 5V pin. Power for the MindFlex is supplied by 3 AAA batteries that are placed inside the enclosure that sits on the right side of the controller. All the devices share the Arduino ground, ensuring better signal quality from the MindFlex. 5 Sensor Implementation & Placement 5.1 Mindflex

Placement of the mind flex controller is critical in the case of this project because of the area of the brain that needs to be targeted might not be the most efficient. In other words, the signals are obtained from an area superior to the left eye, distal to the forehead, and anterior to the head. This placement targets brainwaves originating from the left frontal lobe (attributed with motor function and movement)(5), making placement a double edge sword because movement from the body could be of any kind, not only the one targeted for the hand. Trials were run on each team member and compared using MATLAB software to correlate the signals, in order to find one-dimension patterns within the eight brainwaves signals captured to train the microcontroller on how to deal with the brainwave captured.

5.2 Myoware

This sensor reads the voltage difference between two points via electrodes consisting of metal buttons attached to a hydro-gel adhesive. These electrodes are to be placed on the skin on top of the muscle which the signal is desired to obtain. The Advancer Technologies Myoware manual states that "the correct placement is between an innervation zone and a myotendon junction”. (1) As with the placement of the MindFlex, the placement of the Myoware is critical because the voltage difference is greater in the actual contraction zone, thus producing a more reliable signal for the Arduino UNO. In this project, the sensors will be placed in the external cutaneous zone on top of the extensor and flexor digitalis muscles located on the forearm to target the below elbow amputee population. The second placement will be on the biceps and triceps brachii muscles to target above elbow amputees.

6 Noise Reduction

Noise reduction played an important part in this project. The MindFlex operates based on signal quality: the Neurosky chip only transmits values to the Arduino if it detects a strong quality signal. The controller is made to send signals via 2.4GHz to waves to a base. Thus, the transmitter is sitting right next to the Neurosky chip. In order to run the trials without affecting the device, a ferrite core was placed around a reinforced USB cable with the computer unplugged from its power source to increase the chance for a better signal quality. This core will not affect the signal, since the power to the Arduino will come from a 9V battery with no signal jitter to consider and no potentially ungrounded power sources.

7 Control System Implementation The control System was implemented as follows:

Figure 3. Closed Loop Control System Design

8 Software

Software used for this project was created using the Arduino IDE, with several open source libraries integrated in the main code. The most crucial libraries used in the code are "VarSpeedServo.h" and "brain.h". Use of the Processing 3 program allows for easy visualization of the EEG waves for the trial runs, permitting immediate feedback to be seen if this program is used in conjunction with the MindFlex controller.

8.1 VarSpeedServo.h

The VarSpeedServo library allows the use of any Arduino output pin as a PWM by utilizing interrupts and other methods included in the "readme.txt" file. The library also allows the speed of the servos to vary by utilizing specific functions. For example, if the fingers are to move slower a value (between 1 and 255) can be chosen to adjust speed. This library also allows the microcontroller to wait on the servo to complete the necessary movement before anything else happens. The code includes two diagnostic functions, "void openf2f ()" and "void closef2f()", which are called back to back to ensure proper servo movement. While using the VarSpeedServo library, the thumb movement commands are reversed due to its opposable design, necessitating altering the range value from 180-0 instead of 0-180 (range values for the other fingers).



8.2 brain.h The brain.h library allows for the interpretation and usage of

the CSV data array within the Arduino IDE. As mentioned before the data comes in in a continuous array of 11 different values. The library makes the data extraction of the brainwaves as simple as possible from the array.

8.3 Processing 3 IDE Trials were done using a modified version of the open source code "Brain-Master Grapher.pde", running on the Java-based program Processing 3, which allows for the visualization of data incoming from the Arduino. Additionally, the Processing program allows for the incoming data to be saved as either a TXT or CSV file. Since the data incoming from the Neurosky chip is already in a CSV format, the Brain-Master Grapher code just needs to be modified to save the file in the sketch folder. Figure 4 displays a sample EEG signal from the MindFlex as shown in the processing program.

Figure 4. Example EEG Capture in Processing IDE

9 Results Integrating the Myoware sensors with the Arduino and mechanical hand proved to be a challenging but achievable task. The integration happened on an early stage of the project. Since it happened on such an early stage it allowed us to get within the prosthetic industry standard: opening and closing the prosthetic hand using electrical signals from the forearm and/or arm muscles. Figure 5 displays a sample signal obtained from the Myoware sensor.

Figure 5. Signal Capture from Myoware on Bicep Brachii

The integration of more Myoware sensors also allowed for more control and a possible control advantage to the industry standard. By introducing another sensor, the mechanical prosthetic was able to perform finger flexion and extension based on the signals recorded. However, the goal to add dexterity proved to be a signal processing challenge in such an early stage of the system development. Nevertheless, this is where the MindFlex was introduced in the system.

The MindFlex trials (three no action and six action) ran successfully. Amongst the action trials, the movements consist of opening the hand, closing the hand, biceps flexion, and eye movement, with some of the actions being repeated in different trials. In order to make data statistically significant, the trials were run for approximately 500 samples each run, producing 5500 data points each. These data points were extracted from the CSV file created in the Processing 3 program, then imported into MATLAB as a vector array. The trials were separated into actions performed during the 500 samples. For example, below are the results obtained from the trials with no action. The waveforms represented in the next figures are (top to bottom): Connection, Attention, Meditation, Delta, Theta, Low Alpha, High Alpha, Low Beta, High Beta, Low Gamma, High Gamma.

Figure 6. Trial 1 Signals Discerned



Figure 7. Trial 2 Signals Discerned

Figures 6 and 7 are critical, since these will be the noise reduction signals. Furthermore, if one-dimensional signal matching is achieved these signals will be compared in the Arduino instead of just using thresholds and the brain.h library. The signals were correlated to each other using the cross-correlation function "xcorr" in MATLAB. Since the no action signals are being studied, figures with the correlation results between them are shown below. While MATLAB was used for proof-of-concept, the cross-correlation will be implemented in the Arduino signal processing.

Figure 8. Correlation Result Trial 2 vs. Trial 1

Figure 9. Correlation Results Trial 2 vs. Trial 3

Figure 10. Correlation Result Trial 1 vs. Trial 3

10 Conclusion Cost-effective, reliable and efficient prosthetics are

becoming achievable. A simple design with the interests of the patient in mind should standardize the prosthetic industry, thus making it cost efficient. Different materials could be used to revolutionize the standards. For example, carbon fiber does not have to be the standard for a mechanized prosthetics. A much less expensive enhanced acrylic with a reliable rigidity and grip where needed could be used. Alternatively, the control system should not make the prosthetic device more expensive. As demonstrated above basic conditions based on muscle signals are simple to integrate in a system. Furthermore, a prosthetic with a more complex microcontroller could handle the EEG data along with the muscle signals to differ between movements at the same cost. Undoubtedly the design of any of the ideas mentioned should not overwhelm the cost planned. The industry standard should be raised electronically and controls wise whilst keeping the cost effectiveness low and reliability high. As seen above in the results signal processing can be used to achieve a higher understanding and interpretation of biological signals. Prototyping a small BMI with muscle sensor integration proved to be challenging. However, the goals of cost-effectiveness and dexterity can be achieved. Therefore, by addressing the needs of the patients a new and more efficient prosthetic industry becomes less of a far-fetched concept.



11 References [1] Myoware datasheet. sparkfun.com Web site. https://cdn.sparkfun.com/datasheets/Sensors/Biometric/MyowareUserManualAT-04-001.pdf. Accessed Mar 12, 2018.

[2] Limb loss statistics. Amputee Coalition Web site. https://www.amputee-coalition.org/limb-loss-resource-center/resources-filtered/resources-by-topic/limb-loss-statistics/limb-loss-statistics/#1. Accessed Mar 8, 2018.

[3] J Katona, I Farkas, T Ujbanyi, P Dukan, A Kovari. Evaluation of the NeuroSky MindFlex EEG headset brain waves data. The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Conference Proceedings. Jan 1, 2014:91.

[4] ThinkGear serial stream guide. neurosky.com Web site. http://developer.neurosky.com/docs/doku.php?id=thinkgear_communications_protocol. Accessed 1/29/, 2018.

[5] EEG-based brain controlled prosthetic arm. 2016 Conference on Advances in Signal Processing (CASP), Advances in Signal Processing (CASP), Conference on. 2016:479. http://proxy.kennesaw.edu/login?url=http://search.ebscohost.c

[6] om/login.aspx?direct=true&db=edseee&AN=edseee.7746219&site=eds-live&scope=site. doi: 10.1109/CASP.2016.7746219.

[7] Spike detection in electroencephalogram (EEG) using Arduino UNO. 2015 International Conference on Control, Instrumentation, Communication and Computational Technologies (ICCICCT), Control, Instrumentation, Communication and Computational Technologies (ICCICCT), 2015 International Conference on. 2015:629. http://proxy.kennesaw.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=edseee&AN=edseee.7475354&site=eds-live&scope=site. doi: 10.1109/ICCICCT.2015.7475354.

[8] Mahajan R, Bansal D. Real time EEG based cognitive brain computer interface for control applications via Arduino interfacing. Procedia Computer Science. 2017;115(-2017):812-820. http://proxy.kennesaw.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=edselp&AN=S1877050917319919&site=eds-live&scope=site. doi: 10.1016/j.procs.2017.09.158.

[9] Tatum WO. Handbook of EEG interpretation, second edition. Vol Second edition. New York: Demos Medical; 2014. http://proxy.kennesaw.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=nlebk&AN=741445&site=eds-live&scope=site.

[10] Evaluation of the NeuroSky MindFlex EEG headset brain waves data. 2014 IEEE 12th International Symposium on Applied Machine Intelligence and Informatics (SAMI), Applied Machine Intelligence and Informatics (SAMI), 2014 IEEE 12th International Symposium on. 2014:91. http://proxy.kennesaw.edu/login?url=http://search.ebscohost.c

om/login.aspx?direct=true&db=edseee&AN=edseee.6822382&site=eds-live&scope=site. doi: 10.1109/SAMI.2014.6822382.

[11] Siswoyo A, Arief Z, Indra AS. Application of artificial neural networks in modeling direction wheelchairs using neurosky mindset mobile (EEG) device. Emitter: International Journal of Engineering Technology, Vol 5, Iss 1 (2017). 2017(1). http://proxy.kennesaw.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=edsdoj&AN=edsdoj.0618eaddcc9a420fa0922960a3d3e0b0&site=eds-live&scope=site. doi: 10.24003/emitter.v5i1.165

[12] Speed control of festo robotino mobile robot using NeuroSky MindWave EEG headset based brain-computer interface. 2016 7th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), Cognitive Infocommunications (CogInfoCom), 2016 7th IEEE International Conference on. 2016:000251. http://proxy.kennesaw.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=edseee&AN=edseee.7804557&site=eds-live&scope=site. doi: 10.1109/CogInfoCom.2016.7804557.



Control of Articulated Mechanical Prosthetics...Control of Articulated Mechanical Prosthetics...

Documents

Transcript of Control of Articulated Mechanical Prosthetics...Control of Articulated Mechanical Prosthetics...