Consistency and Discrepancy Analysis of Human Walkingresentation of human bipedal behavior. The...

Consistency and Discrepancy Analysis of Human Walking

by

Chen-Yu Chan

A dissertation submitted in partial satisfaction of the

requirements for the degree of

Doctor of Philosophy

in

Engineering - Mechanical Engineering

in the

Graduate Division

of the

University of California, Berkeley

Committee in charge:

Professor Masayoshi Tomizuka, ChairProfessor Andrew PackardProfessor Edward A. Lee

Fall 2017


Copyright 2017by

Chen-Yu Chan

1

Abstract


by

Chen-Yu Chan

Doctor of Philosophy in Engineering - Mechanical Engineering

University of California, Berkeley

Professor Masayoshi Tomizuka, Chair

As the trend for wearable bioinformatic sensors continues to increase, researchers pursueintegrating knowledge of the human body into the design of a simpler sensory system. Byobserving the consistency and discrepancy of human behavior, insight can be gained onhow to design a monitoring architecture and what content to monitor. Because no twohuman individuals are identical, it is important to capture consistency among people basedon an all-inclusive understanding. The first step of monitoring is to quantify the behavior.Because there is a limitation to a commercial motion capturing system, a durable sensorysystem, the Smartshoe, is designed to remotely record and measure ground contact force(GCF). The recorded data is locally processed by the hardwares computing unit and a non-blocking software architecture regulates the data rate of the recording. The accuracy andrepeatability of GCF measurements with the piezoelectric sensor are demonstrated. Thedetailed schematic design is also explained in the text. A graphical user interface (GUI) isdesigned to show the real-time applicability of the software.

An optimal sensor location consistently differentiates distinct human movements andrequires a minimal usage of the computational resource. Drawing inspiration from L1 regu-larization in classification problems, the use of feature selection results is an optimal sensorlayout for gait phase estimation. The method reveals the significant sensor location involvingin the classification and filters out the redundant locations. As a consequence of this opti-mization, the input complexity is greatly reduced. The results resemble some of the previouschoices but also eliminate some redundancy. The estimators’ performance of the optimizedlayout and the anatomic layout are compared and there is no significant disagreement.

Finally, a two-level hierarchical decision framework is proposed for a comprehensive rep-resentation of human bipedal behavior. The method decouples behavior identification to twolevels, discrete modes and mode-dependent special tasks. The double-peaked GCF patternis used as a metric to distinguish walking mode from running mode. Support vector machine(SVM) with a moving window is implemented to identify the discrete modes in the first level.In the second level, the mode-dependent tasks are defined under single or multiple modes. Inwalking mode, a set of two-sided fuzzy logic rules is utilized for robustly recognizing the gait

2

phases. The speed of walking is also estimable by observing the peak-to-valley ratio in theGCF pattern. The diagnosis of touch-based symptoms, on the other hand, are dual-modespecific tasks. Plantar Fasciitis causes a smaller Mid-Stance to Terminal Stance ratio incombination with asymmetric running GCF pattern in the affected side. And finally, thecombined performance using the framework shows a robust and accurate analysis of humanwalking behavior.

i

To my parents,

ii

Contents

Contents ii

List of Figures iv

List of Tables vi

1 Introduction 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 State of the Art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Problem Definition and Contribution . . . . . . . . . . . . . . . . . . . . . . 51.4 Dissertation Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Crafting of the Smartshoe 72.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.2 Sensory system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.3 Data Acquisition Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 132.4 Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3 Optimal Force Sensor Layout for Gait Estimation 163.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.2 Gait Cycle and Gait Phase . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.3 Optimal Layout by Feature Selection . . . . . . . . . . . . . . . . . . . . . . 193.4 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.5 Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

4 Hierarchical Decision Framework for Bipedal Behavior 354.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.2 Hierarchical Decision Framework . . . . . . . . . . . . . . . . . . . . . . . . 354.3 Classification of Discrete Modes . . . . . . . . . . . . . . . . . . . . . . . . . 364.4 Gait Estimator using Two-Sided Fuzzy Logic . . . . . . . . . . . . . . . . . . 404.5 Walking Speed Estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444.6 Diagnosis of Touch-based Symptoms . . . . . . . . . . . . . . . . . . . . . . 474.7 Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

iii

5 Conclusions and Open Problems 505.1 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505.2 Open Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

A Detail of test subjects 53

Bibliography 72

iv

List of Figures

1.1 IMU-only systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Instrumented shoes with IMU systems . . . . . . . . . . . . . . . . . . . . . . . 31.3 Instrumented shoes with only force sensor. . . . . . . . . . . . . . . . . . . . . . 4

2.1 A glance of the Smartshoe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.2 Amplifying circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.3 Calibration mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.4 Calibration curve of the piezoelectric sensor . . . . . . . . . . . . . . . . . . . . 102.5 Data acquisition PCB with amplifying circuit . . . . . . . . . . . . . . . . . . . 112.6 Validation of GCF on one side . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.7 Finite state machine of one shoe . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.8 GUI plotting force data from both sides . . . . . . . . . . . . . . . . . . . . . . 15

3.1 Gait phases (skeletal diagram cited from [28]) . . . . . . . . . . . . . . . . . . . 183.2 Comparison of feature vector . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.3 Comparison of estimated feature under different noise level . . . . . . . . . . . . 233.4 Comparison of estimated feature under different sample size . . . . . . . . . . . 243.5 F-Scan system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.6 Feature vector (a) and its mapping to a footprint (b). The features are drawn

as a contour map and most of the features are zero or close to zero. Only thehighly contrasted color dots (pointed with arrows) are the significant featureswhich used to determine the gait. . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3.7 Gait phases estimator of subject 1 test . . . . . . . . . . . . . . . . . . . . . . . 293.8 Gait phases estimator of subject 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 303.9 Gait phases estimator of subject 3 . . . . . . . . . . . . . . . . . . . . . . . . . . 313.10 Comparison between footprints . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.11 Performance of gait phase estimation with different sensor layout. . . . . . . . . 34

4.1 Modes of bipedal behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364.2 The location of sensors for classifying discrete modes. . . . . . . . . . . . . . . . 374.3 Comparison of GCF pattern at different bipedal modes. . . . . . . . . . . . . . . 38

v

4.4 The moving window method. Each input feature is a snapshot of neighboringmeasurement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.5 Error rate of the classification for different size of moving window . . . . . . . . 404.6 A hyperbolic tangent membership function . . . . . . . . . . . . . . . . . . . . . 414.7 Performance comparison of the two-sided fuzzy logic. . . . . . . . . . . . . . . . 434.8 Gait phases estimator with naive fuzzy logic . . . . . . . . . . . . . . . . . . . . 444.9 Analysis of the experiment using the hierarchical decision framework . . . . . . 454.10 Comparison of GCF pattern at different walking speed. . . . . . . . . . . . . . . 464.11 GCF ratio vs treadmill speed. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

A.1 Feature map for “Initial Contact”phase, subject 1 . . . . . . . . . . . . . . . . . 54A.2 Feature map for “Loading Response”phase, subject 1 . . . . . . . . . . . . . . . 55A.3 Feature map for “Mid Stance”phase, subject 1 . . . . . . . . . . . . . . . . . . . 56A.4 Feature map for “Terminal Stance” phase, subject 1 . . . . . . . . . . . . . . . . 57A.5 Feature map for “Pre-Swing” phase, subject 1 . . . . . . . . . . . . . . . . . . . 58A.6 Feature map for “Swing” phase, subject 1 . . . . . . . . . . . . . . . . . . . . . 59A.7 Feature map for “Initial Contact”phase, subject 2 . . . . . . . . . . . . . . . . . 60A.8 Feature map for “Loading Response”phase, subject 2 . . . . . . . . . . . . . . . 61A.9 Feature map for “Mid Stance”phase, subject 2 . . . . . . . . . . . . . . . . . . . 62A.10 Feature map for “Terminal Stance” phase, subject 2 . . . . . . . . . . . . . . . . 63A.11 Feature map for “Pre-Swing” phase, subject 2 . . . . . . . . . . . . . . . . . . . 64A.12 Feature map for “Swing” phase, subject 2 . . . . . . . . . . . . . . . . . . . . . 65A.13 Feature map for “Initial Contact”phase, subject 3 . . . . . . . . . . . . . . . . . 66A.14 Feature map for “Loading Response”phase, subject 3 . . . . . . . . . . . . . . . 67A.15 Feature map for “Mid Stance”phase, subject 3 . . . . . . . . . . . . . . . . . . . 68A.16 Feature map for “Terminal Stance” phase, subject 3 . . . . . . . . . . . . . . . . 69A.17 Feature map for “Pre-Swing” phase, subject 3 . . . . . . . . . . . . . . . . . . . 70A.18 Feature map for “Swing” phase, subject 3 . . . . . . . . . . . . . . . . . . . . . 71

vi

List of Tables

3.1 Ratio of data points (shown in %) . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4.1 Performance of different classification method . . . . . . . . . . . . . . . . . . . 384.2 Two-sided Fuzzy rules for gait estimation using the three optimal sensors . . . . 424.3 Ratio of data points (shown in %, right limb) . . . . . . . . . . . . . . . . . . . 484.4 Ratio of peak force (shown in % of body weight) . . . . . . . . . . . . . . . . . . 49

A.1 Detail physical condition of the test subjects . . . . . . . . . . . . . . . . . . . . 53

vii

Acknowledgments

My journey in Berkeley started with an admission on April 1st and ended with a bullet.This has so far been the most dramatic story in my life. Throughout my whole stay on thismemorable campus, Professor Tomizuka has always been there to provide the most criticalsupport that I need, whether academically or emotionally. His devotion to academic researchand passion on advising junior researchers reminds me why Berkeley is the best school. Hiskindness and caring attitude to us, long away from home, always make MSC lab so homey.To this, I am sincerely grateful to be advised by Professor Tomizuka.

I would also like to thank Professor Packard for leading me to the teaching career inBerkeley. His enthusiastic lecture and “fairly short” homework definitely set a role modelfor me. Hopefully I can be as good lecturer as him. Professor Lee’s lectures are anotherpart of Berkeley that I will always remember, and I’m deeply thankful for his help with thisdissertation.

My dearest appreciation also goes to Miwako-san for taking care of me during my recoveryafter the gunshot. To Dr. Ambarish Goswami for sharing his vision and the resource ofHonda Research Institute with my project. To all the colleagues that I taught with, Prof.Roberto Horowitz, Prof. Jose Carmena, Prof. Panos Papadopoulos, Prof. Seth Sanders andProf. Ron Fearing.

To my dear MSC lab members, thank you for being part of my family in Berkeley. Specialthanks to my fellow students Prof. Minghui Zheng, Dr. Junkai Lu, Dr. Chung-Yen Lin,Omar Abdul-Hadi and Hiroshi Niki for the lecture slides and homework solution that weshared. Thanks to the human mechatronics group members, Prof. Joonbum Bae, Dr. KanKanjanapas, Prof. Wenlong Zhang, Dr. Robert Matthew, Dr. Kevin Haninger, and DaisukeKaneishi for all the discussion and inspiration. Thanks to all my respected veterans of MSClab, Dr. Sanggyum Kim, Dr. Wenjie Chen, Dr. Evan Chang-Siu, Prof. Xu Chen, Dr. MikeChan, Dr. Pedro Reynoso, Prof. Cong Wang, Dr. Chi-Shen Tsai, Dr. Yizhou Wang, Dr.Raechel Tan, Dr. Shiying Zhou and Dr. Changliu Liu for all the little tips that you sharedwith me. And to the ongoing MSC members, Liting, Xiaowen, Yu-Chu, Hsien-Chung, Te,Yu, Shuyang, Wei, Cheng, Yongxiang, Zining, Jianyu, Kiwoo, Yujiao, Jiachen, Yeping, Chen,Zhuo, Hengbo and Jessica, good luck with your stay and think twice if you are consideringtaking my seat :P.

To my fellow ME friends, James, Cheng-Ju, Yiwen, Chang, Fang-Chieh, Eric, Lily, Nick,Sony, Yawo, Shareena, and Isabel, I’m glad that I have your help during the time in the MEdepartment. To all my BATS friend, thank you for having fun with me.

And lastly to my family, Mr. Ming-Hsien Chan, Ms. Li-Mei Lee, Mr. Shih-Yu Chan, Ms.Huei-Min Huang for always willing to wait for me. Thanks to my girl Ci Zhang for being sosupportive during my rough hours.

Bye, Berkeley. And Go Bears!

1

Chapter 1

Introduction

1.1 Motivation

The development of wearable technology has flourished in recent year. The wearable bioin-formatic system reflects the real-time health status and keeps track of the daily body per-formance. When more pioneering sensor unit is commercially available, a comprehensiveinterpretation of the measurement is necessary. The acceleration of a wristband [11] is inter-preted as the pace of walking. The height of a leap is calculated by the levitation time of aninstrumented shoe [7]. However, there still exists a gap between the sensor technology andan integrated sensory system due to the lack of evidential interpretation. In this dissertation,more quantitative consistency of walking behavior will be presented and discrepancy amongindividuals are extracted for abnormality recognition.

There are several issues that cause walking problems. First, the fatigue of muscle, joint,ligament, fascia in the lower extremity affects the functionality of the limbs, and an inefficientwalking pattern continuously causes more fatigue to the body tissue. Furthermore, incorrectwalking may cast damage to other functionality of the body. When the contact impulse fromthe ground is not fully absorbed by lower limbs, the shock propagates along the spine andmay eventually affect brain function. Lastly, diagnosis of the early stage walking problem isimportant for elderly home care. Imbalanced lower extremity may result in falling and moreserious injury follows.

Among different types of sensors that capture walking problem, the touch-based trans-ducer is widely used for measuring the ground contact force (GCF). The GCF is a directreflection of the functional change of the lower limbs and there exist consistency of GCFpattern among different individuals. By examining and quantifying the consistency, the gaitphases in a walking cycle are defined [28] and being recognized as a characteristic attribute ofwalking. Analyzing the gait phases determines whether the limbs are performing the correctfunction. Short of certain gait phases means the possible loss of limb function. Moreover, thetemporal ratio of each phase also reflects the normality of lower extremity. So, the need ofestimating gait phases in real-time stimulates this research about a fast and robust gait phase

CHAPTER 1. INTRODUCTION 2

estimation system. In the following section, several state-of-the-art gait phase estimationsystems and further discussion about the limitation of these works will be presented.

1.2 State of the Art

Motion capture system

The first type of works for quantifying waking is the commercial motion capture system[41, 29], which track multiple optical beacons that are attached to a human body. Thebeacons are placed at some locations of interest so that they can be associated with jointsor bones. Linking the pieces together became a snapshot estimate of the skeletal structure.By concatenating these snapshots, the system is able to track behaviors and store them fora playback in order to check repeatability and consistency. Although the motion capturesystems are widely used for behavioral research, there are some limitation listed as follows,

• The optical beacon may not be seen at all time.Since human has all the flexibility, optical beacons are sometimes occluded from thesight of the cameras. This raises the uncertainty on tracking performance. Somealgorithms [6] are developed to solve this issue but these methods do not perform wellin real-time.

• The recorded information are all indoor behavior.A motion capture system has to be set up in an indoor space, where the relative positionof cameras are known and calibrated. Sometimes, types of equipment like a treadmillor some suspended ropes are added to the space for more variety of behaviors. Still, amotion capture system is not a suitable measuring tool for outdoor activities.

• Contact dynamics cannot be interpreted from kinematics.The interacting force and torque cannot be measured by the system because the massinformation of the body segments is missing or inaccurate. The system only tracks theposition of the markers, and differentiating the measurements results in the velocityand acceleration of body segments. To measure interacting force, some other types ofsensors need to involve.

Several pieces of work proposed some alternative sensor to solve the limitation of a motioncapture system. For example, when the inertial measurement unit (IMU) is attached to abody segment, the kinematic information of the attached segment can be estimated. Asshown in Fig. 1.1a, the SHIMMER system [27] sets several IMUs at different parts of thebody and use them to monitor the symptoms of the Parkinson Decease (PD). In Fig. 1.1b,the Physilog module [22] is put on top of a shoe to capture the abnormal walking trajectory ofPD patients. Both of the work successfully solve the occlusion problem of an optical motioncapture system. However, because an IMU-only system is still not adequate to measurethe contact dynamics, the idea to integrate GCF measurement with the IMUs is proposed.


(a) The SHIMMER system. [27] (b) The Physilog module. [22]

Figure 1.1: IMU-only systems

(a) [4] (b) [14] (c) [44]

Figure 1.2: Instrumented shoes with IMU systems

Morris et al. creates an instrumented shoe [4] with five force sensors and IMUs to estimatethe whole kinematic and contact dynamics. Fig. 1.2a depicts the type and location of touch-based sensors embedded in the shoe. Jung et al. further expand the idea to a fully mobilemotion capture system [14] which include the Smart Shoes and fifteen IMUs attached to thebody. Fig. 1.2b shows the configuration of the Smart Shoes, which contains four pneumaticsensors each link with an elastic tube coil. And finally, Zhang et al. use the integratedsystem to provide real-time visual feedback to physical therapists (PT) and patients forrehabilitation practices [44]. As shown in Fig.1.2c, a displaying tablet is suspended at theside of the waist for the user to read the real-time information of foot touch, step length,stride width and toe out angle. It renovates the rehabilitation process and PTs are able toassess the patients’ performance in real-time.


(a) [26] (b) [36] (c) [2]

Figure 1.3: Instrumented shoes with only force sensor.

Instrumented shoe system

The motion capture system is a good start to quantify walking. However, the system is not aslightweight to record off-the-grid walking. First, the computation needed for fusing multipleIMUs is heavy, which lowers the feasibility of a stand-alone execution. The experimentsare still only available in the laboratory. Second, the IMUs are secured to the human bodyusing Velcro straps or fitted clothing. The tactile of the IMUs affects the feeling of walkingwhich biases the measurement. Lastly, the full body IMU suits are still infeasible to wearat any time. As such, further development of instrumented shoes explores more capabilityof a shoe-only sensory system. To build the instrumented shoe, several types of sensor aresuitable to be embedded in the insole. The GCF is measurable by air pressure sensor [16,2, 14], fabric sensor [36], strain gauge [20, 43, 21], or piezoelectric material [4, 26]. Fig. 1.3shows that the placement of the force sensors are determined either by the most touchedarea [26, 36] or the anatomic structure [2]. Although the choice of the locations is logicalto extract the most of the GCF, thorough discussion about the optimality of these choicesis not covered in the literatures. Finally, the gait phases are identified by threshold-basedmethod [16] or classification method [1, 44]. Both of which achieve reasonable accuracy ongait phase estimation yet the real-time performance is not discussed.


1.3 Problem Definition and Contribution

Despite the previous work of the alternative motion capture system and the instrumentedshoes, little effort is devoted to creating a lightweight monitoring system for walking behav-ior. The system should robustly estimate gait phases in real-time and utilizes several metricto assess the walking behavior. To keep the computation light, the redundancy of sensorunits will be omitted yet the effectiveness and accuracy of the estimation must match theperformance of the existing choice of sensor location. Furthermore, with the new layout,more consistency and discrepancy of the measured GCF will be extracted for a more com-prehensive interpretation. The dissertation will introduce a behavior quantifying frameworkthat encapsulates several metrics of walking. The major contribution of this framework isas follows,

• Optimal feature maps for gait phase estimation.It is later proven in the text that three sensors are adequate for estimating gait phases.A two-sided fuzzy logic is developed for a more robust performance of gait phaseestimation.

• A lightweight wireless sensory system.The hardware and the software design of the sensory system are presented in detail.A non-blocking software architecture regulates the data rate of the recording whichcontributes to a rapid gait estimation algorithm executing on a microcontroller.

• Injury diagnosis by metrics extracted from consistency in walking.Through the sensory system, more of the consistency in the GCF measurement becomemetrics of assessing the walking behavior. Touch-based symptoms can be identified bythe abnormality in the GCF.

1.4 Dissertation Outline

The content of this dissertation is organized as follows. Chapter 2 will introduce the craftingdetail of the hardware system, which includes characteristics of the contact force sensor andthe state machine of data acquisition architecture. Chapter 3 discusses the optimal layoutof the sensor location from the perspective of data complexity. An l1-norm based method isderived in this chapter to refine the optimal location of the sensors for the purpose of gaitestimation. Some consistency and discrepancy that we discover among different subjects arealso discussed. Chapter 4 composes a hierarchical framework that interprets and analyze themeasurements. The method decouples behavior identification into two levels, discrete modesand mode-dependent special tasks. Several machine learning methods are implemented andcompared for classifying discrete mode in real-time. In the second level, a two-sided fuzzylogic table is constructed for a robust gait phase estimation. Several other metrics arealso demonstrated including walking speed estimate and touch-based symptom diagnosis.


Chapter 5 summarizes the conclusion and lays the possible direction of making more use ofour findings.

7

Chapter 2

Crafting of the Smartshoe

2.1 Introduction

As mentioned in chapter 1, one of the main goal of this dissertation is to quantify bipedalbehavior. Since the contact dynamics is not measurable by a motion capture system, somemethod needs to be developed to measure the contact. Fortunately, shoes are generally wornby people for outdoor activities and shoes are the medium between human body and theenvironment. Measuring the contact force applied to the shoes quantifies the interaction.Some commercial products [25, 39] already prove the feasibility of capturing ground contactforce (GCF) from an insole sensor. In this chapter, a sensory system, Smartshoe, is developedfor a real-time behavior quantization.

The first prototype shoe [16] developed by Kong uses an air coil with an air pressuresensor as a transducer unit to measure GCF. The air coil is an elastic tube that deformswith the applied force. By winding a tube into the coil shape, the cumulated compressionof air inside the tube is measured by the air pressure sensor. However, there are severaldrawbacks of these silicon tube transducers. First, aging of the tube changes elasticity andthus affects the repeatability. Second, frequent usage of sensors causes the measurementdeviates from the true value. The sensor unit has to be re-calibrated as it ages, which is notsuitable for long-term usage. Lastly, the sensor unit needs to be sealed tight. The sealingmaterial also ages and causes leakage of air. This is undesirable and new sensor option issought. In the following sections, details of the sensory system will be shown as well as thedata acquisition architecture.

2.2 Sensory system

The Smartshoe system contains two separated sides, both are equipped with its own com-puting unit and sensors. Fig. 2.1 shows part of the details. Force transducers are put in theinsole layer of shoe connected with the signal wires. The insole is padded with cushion forcomforts, and the design hides the unevenness of wires and minimally disturbs the wearers’

CHAPTER 2. CRAFTING OF THE SMARTSHOE 8

Figure 2.1: A glance of the Smartshoe

tactile feeling. All signal wires are bundled into the black plastic project enclosure. Thecomputing unit organizes the collective sensor signal as a character string and transmits thedata through the Bluetooth module. Notice that the usage of a wireless transmitting channelenables the shoe to work on its own. No extra wire is connected to the shoe, which improvesthe mobility of the sensory system. On top of the enclosure, a module is designed to stablyprovide power to the entire circuit. In the following paragraph, the piezoelectric force sensoris introduced, which is a more reliable substitute for the elastic tube.

Piezoelectric force sensor with amplifying circuit

A piezoelectric sensor is a device that uses the piezoelectric effect to measure changes inpressure, acceleration, strain or force by converting them to an electrical charge. It has thebenefits of low cost and good linearity. However, the voltage difference generated by thesensor is relatively small. Some amplifying circuit is necessary for tuning the sensitivity ofthe output measurement. Fig. 2.2 shows two amplifying circuits that are commonly used.

It is relatively simple to implement the voltage divider in Fig. 2.2a. Resistance of thesensor is inversely proportional to the force being applied to it and can be expressed asRps = k

fwhere k is a constant and f is the force. By putting the sensor as Rps in Fig. 2.2a,

the output is

Vout = VinRf

Rps +Rf

= VinRf

kf

+Rf

= Vinf

kRf

+ f(2.1)


Vin

Rps

Rf

Vout

(a)

Vin

Rps Rf

Vout-

+

(b)

Figure 2.2: Amplifying circuit

where Rf is a fixed value resistor. Although the output increases as the force increases, thenon-linear relation is complex to calibrate. So, the operational amplifier in Fig. 2.2b is abetter solution to the linearity problem. With this circuit, the relation between applied forceand the output voltage is

Vout = −VinRf

Rps

= −Vinf ·Rf

k∝ f (2.2)

To operate the circuit correctly, Vin must be negative. The method to creat a negativevoltage will be discussed in the following paragraph of the schematic design. Since Vout in(2.2) is proportional to the force, the calibration problem is simpler to solve.

As shown in Fig. 2.3, a load-cell is used as a reference to calibrate the sensor. The c-clampapplies force directly to the stack of the sensor and the load-cell. A wide range of forces canbe applied by adjusting the tightness of the c-clamp. Fig. 2.4a shows several data points ofthe measured voltage. The relation is indeed proportional as derived in (2.2).

To solve the parameter of the linear relation between f and Vout, a linear regression isutilized to fit the data. Fig. 2.4b shows the calibrated result of the piezoelectric sensor. Theblack solid line is with 1:1 slope and the curve closely aligned to the solid line. Notice thatthere is a non-responsive region in the curve. When the input force is less than 10lb, thesensor is outputting the same value. This is a natural characteristic of the sensor and somepreload is added to overcome this problem.

Schematic design

On top of the amplifying circuit, there is a microcontroller that records the output voltagemeasurement of all the sensors. The mbed [23] is used as the local processor on both sidesof the shoes. mbed has a high-performance ARM Cortex M3 core with the support of a


Figure 2.3: Calibration mechanism

(a) Voltage measurement v.s. load-cell (b) Calibrated sensor measurement

Figure 2.4: Calibration curve of the piezoelectric sensor


Figure 2.5: Data acquisition PCB with amplifying circuit

variety of communication channels. The six analog-to-digital converters (ADCs) are utilizedfor the data acquisition purpose. With a core of 96MHz, the performance of sampling these6 channels is aimed at 100Hz. Furthermore, the microcontroller provides UART and I2Cinterface which can be transmitted wirelessly. The feature of a wireless communicationis very beneficial since the natural walking behavior should not be affected in any sense.Having wire hanging on the shoe causes the user to walk in an abnormal way and deformsthe baseline walking. A two-layered printed circuit board (PCB) is designed to integrate allthe components needed in the operational amplifier circuit. The size of PCB is 2.5” by 1.55”and it accepts up to six piezoelectric sensors.

As shown in Fig. 2.5, there are two major components on the board. One is the amplifyingcircuit and the other is the microcontroller. As mentioned in the previous section, theamplifier needs a negative driving voltage to operate. An SRS-0505 power regulator is usedto achieve such requirement. The board is mounted outside of the shoes, so the size of thecircuit board must be limited. The 3D spacing of the board is also carefully considered.Heat generated by the regulator is a potential threat to the circuit. Since physically themicrocontroller is stacked on top of the amplifying circuit, adequate spacing is necessary.Finally, the plastic project enclosure box that covers all the components improves hardwaredurability. Some potential upgrade to shrink the size of the circuit are to use surface mountdevices (SMD) or to use a 4-layer PCB manufacturing process.


Figure 2.6: Validation of GCF on one side

Qualitative performance

The pair of shoes records GCF from both side. The quality and repeatability of the sensormust be verified. Perry et al. [28] propose the nominal vertical GCF pattern of a healthywalking cycle. The magnitude of the vertical force varies with the changing limb position. Inthe sagittal plane, the sequence of the normal forces forms two peaks separated by a valley.At walking frequency of 0.8Hz, each peak approximates 110% of body weight while the valleyforce is about 80% of body weight. The first peak magnitude is increased above body weightby the addition of acceleration from the rapid drop of the body mass and muscular forcesstabilizing the joint. The second peak happens by the push of ankle muscles against thefloor in addition to the acceleration of body mass fall forward to the toe.

Fig 2.6 is the measured GCF for a test subject of 130lb. The walking frequency is 0.8Hzand there are four piezoelectric sensors in the insole. We observe the desired double peakpattern with some variation in each step. The GCF drops to zero when the shoe is nottouching the floor, and repeatedly reaches back to the peak if touching. This verifies the


repeatability of the sensors and responsiveness of the circuits.

2.3 Data Acquisition Architecture

The shoe is shown to work on its own for recording the measurement. Yet, the sensorysystem is merely a component of the whole measuring architecture. Since the entire systemis designed for further understanding of behaviors, inevitably, the system must present therecording in real-time. As such, the signal priority should be carefully designed. A successfulprotocol protects the completeness of the data and avoids congestion in the transmittingchannel. In this section, some cyber-physical system approach [18] is utilized to verify theeffectiveness of the software system.

Software System Analysis

The finite state machine in Fig. 2.7 shows the discrete dynamics of the shoe’s function. Thereare three states in the system namely Idle, Record and Transmit(TX). The softwarealways starts with the Idle state for initialization of communication channel and loadingcalibration parameter for the transducers. The Record state is triggered when input key is“R”. The Record state keeps writing the sensor data to local memory and “data count” isthe counter to keep track of the amount of data being stored. The possible maximum numberof “data count” depends on the memory size of the microcontroller. The role of state TX isto take the data from memory and push it to the Bluetooth serial communication channel oneline at a time. “data count” descends as the data being transmitted. Since the Bluetoothprotocol is port dependent, data from each side are separated by the port number. Thereceiving computer contains a mutex object that sequentially reads both Bluetooth channelsand stores data separately. The graphical user interface (GUI) in Fig. 2.8 shows the resultof data acquisition. The data rate of the channel is maintained at 30Hz from both sides.However, the plotting windows refresh the frame every time when a new data is pushed tothe channel. This causes a lagging effect in the visualization of the data. When plotting isnot involved in the process, the data rate is better guaranteed.

2.4 Chapter Summary

This chapter presented the first step of understanding bipedal behavior - by fundamentallybuilding a sensory system. The Smartshoe in which the force sensors are utilized overcomesthe limitation of a motion capture system. The accuracy and repeatability of the GCFmeasure are demonstrated with the new choice of piezoelectric sensor. The detail schematicdesign is also explained in the text. To verify the effectiveness of the sensory architecture, afinite state machine is used to analyze the discrete dynamics of the software system. A GUIis designed to show the performance of the architecture.


inputs: key : {D,I,R}states: data count : Zoutput: transmit : pure

Idle

Record

TX

key = R /

key = I /

/data count := data count + 1

key = D /

data count >0 / transmitdata count := data count -1

data count = 0 /

key = D /

/data count := 0

Figure 2.7: Finite state machine of one shoe


Figure 2.8: GUI plotting force data from both sides

16

Chapter 3

Optimal Force Sensor Layout for GaitEstimation

3.1 Introduction

Among all the bipedal behaviors, walking is most common for traveling. It uses a sequenceof limb motion to simultaneously move the body forward while also maintaining stancestability. There are a variety of studies focusing on walking behavior. Westervelt et al.[42] studied the locomotion of human and applied feedback control to a bipedal test bedRABBIT. The robot performed stable waking over a wide range of speeds with significantrobustness to model error and external perturbations. From the clinical perspective, walkingassessment [3, 15, 28] is a key component for diagnosing pathological behavior. Althoughthere are still debates whether walking assessment results in consistent diagnosis, this doesnot make people question whether PTs should have used walking assessment as part ofthe decision-making process. However, in current walking assessment practice, there existsminor discussion about the role of an instrumented shoe since such equipment is yet widelyavailable.

In this chapter, the Smartshoe is introduced to estimate a fundamental feature of walkingassessment, gait phases. The definition and necessity of gait phases will be introduced insection 3.2. The estimation is formulated as a classification problem in which the forcemeasures are the inputs and gait phases are the outputs. A feature extraction techniqueintroduced in section 3.3 optimally determines the layout of force sensors for gait phaseestimation. The estimation performance of the optimal layout is compared with conventionalsensor location. And lastly in section 3.4 some consistency and discrepancy are shown amongdifferent test subjects.

CHAPTER 3. OPTIMAL FORCE SENSOR LAYOUT FOR GAIT ESTIMATION 17

3.2 Gait Cycle and Gait Phase

A single sequence by one limb is called a gait cycle. With one action flowing smoothly intothe next, there is no specific starting or ending point. The moment of floor contact is themost significant event, so this action generally has been selected as the start of a gait cycle.Each pattern of motion relates to a different functional demand. The Rancho Los AmigosGait Analysis Committee [35] concluded that each stride contains two periods, Stance andSwing. A total of eight functional phases are categorized into these two periods, namely“Initial Contact”, “Loading Response”, “Mid Stance”, “Terminal Stance” and “Pre-Swing”for Stance period, and “Initial Swing”, “Mid Swing” and “Terminal Swing” for Swingperiod.

Fig. 3.1 shows the eight gait phases and the associated insole force distribution. Thefirst row depicts the left foot and the second row is the right foot. Each footprint pairs issynchronous in time. The limb postures associated with each gait phases are also shown atthe bottom. Notice that the distinction of three swing phases is defined by limbs or pelvisorientation. Since the insole is not pressed during Swing period, it does not distinguish thethree sub-swing phases.

The function of “Initial Contact” and “Loading Response” are accepting the weighttransition using the just-contacted limb. Once the limb stably supports the body weight,the main function immediately switches to single leg support. In this second task only onelimb touches the floor and collectively acts like an inverted pendulum. Once the opposite limbtouches the floor, the third task begins with “Pre-Swing” which lifts the heel pivoting fromthe front foot. It is the proceeding of functional demands that defines the transition of gaitphases. The timing is important for these transition but gait phases do not necessarily showtemporal symmetry. Such asymmetry is natural [31] even for people without impairments.The gait analysis focuses more on correlating the simultaneous actions of the individualtouch into functional significance.

Gait Phase Estimation

The clinical walking assessment considers various of kinematic indexes [33, 34, 5] in which gaitphase estimation play a crucial role. However, most of the conventional assessment methodrecords the walking data and estimates gait phases off-line. This misses instantaneous insightfor PTs to correct the gait on-site. To solve this issue, several pioneering research hasutilized instrumented shoes as part of the clinical assessment. Zhang et al. [44] createsvisual feedback interface for PTs and patients to observe during a rehabilitation process.The gait phases determine whether the limbs are performing the correct function in a gaitcycle. Short of certain gait phases means the possible loss of limb function. Moreover, thetemporal ratio of each phase also reflects the normality of the lower extremity. With a robustand real-time gait phase estimator, the correctness of walking can be nicely quantified.

The application of a real-time gait phase estimator is not limited to clinical rehabilitation.For daily walking behavior, the gait phases information reflects the performance of limb


Figure 3.1: Gait phases (skeletal diagram cited from [28])


function. Some early-stage walking problem such as fatigue in the muscle, imbalanced gait, ortouch-based foot symptoms can be captured by estimating the gait phases. This greatly helpsthe correction of walking behavior and is specifically beneficial for self-health monitoring.

Lastly, gait phase estimation is an instantaneous feedback of how each limbs involvein the walking, which is critical to operating an exoskeleton. The exoskeleton’s machineryis designed to follow or augment the movement of a human body. When the function ofeach limb segments is estimated, the exoskeleton aims to contribute more of the function.To control an exoskeleton, one of the main issues is to identify the intention of the userand follow the needs. With a real-time gait phase estimator, the exoskeleton is able toinstantaneously respond to the user’s need.

Discussion on Sensor Location

A major debate for the instrumented shoe is where to put the sensor. An optimal choiceof sensor location consistently differentiates distinct moves yet requires a minimal usage ofthe computational resource. From the notion of gait phases, it is very intuitive to put thesensors at the most touched area [26, 36]. However, the shape of the footprint is uniquefor different individuals. People with over-pronation tends to touch the intermediate areabetween the heel and the frontal area. There is no clear evidence whether laying sensor in theintermediate area helps to estimate gait phases. Some other research [1, 14, 44] arranges thesensor according to the anatomic structure of the foot, where the location is the heel, 1st and2nd metatarsal joints, 4th and 5th metatarsal joints, and the hallux (toe). The estimationresult is reasonably well but there is no discussion whether four sensors are the optimalnumber or if there is some missing information.

In the following section, the problem of optimal layout of the embedded force sensors inan instrumented shoe is solved from the inspiration of an L1 regularization in a classificationproblem. A conclusion of optimal sensor layout is drawn from the feature selection result.The method highlights the significant sensor location involving in the classification and filtersthe irrelevant ones. Since only the data measured from the highlighted location needs to beconsidered, the input complexity is optimally simplified. The results resemble some of theprevious works but also eliminate some redundancy.

3.3 Optimal Layout by Feature Selection

The insole force distribution is directly induced by the functional changes in a gait cycle. Itis feasible for Smartshoe to estimate these change of distribution and infer the gait phasesfrom the GCF. However, the location of force sensors is a critical factor yet to be deter-mined. An optimal choice will increase the accuracy of gait phase estimation and lower thecomputing resources on data processing. As the estimation problem is formulated as solvinga classification problem, some insight is gained from the perspective of data complexity.


Classification is a widely discussed topic and one main challenge of efficient classificationis improving the sparsity of the input features. A sparse feature vector indicates the mostdiscriminative information in the data and L1 regularization (often refer to as LASSO [40,30]) helps improving sparsity. When this method is applied to gait estimation, it removesirrelevant sensor location for estimating gaits and finds the optimal layout in the insole. Thischaracteristic from machine learning solves some long-time debate by PTs. Given there are6 gait phases to be identified, a full insole sensor is redundant. In the following paragraph,the details of the L1-norm based feature selection method that picks the optimal layout offorce sensors will be introduced.

L1 Regularization

To show the validity of L1 regularization, the concept is first implemented on a linear re-gression problem. Suppose the input/output pairs are generated by

yi = θ>xi + ω (3.1)

where xi ∈ Rn×1 is the input, yi is the output scalar, i is the sample index, θ ∈ Rn×1 is themodal parameter and ω is a scalar zero-mean noise. Sample the model m times and formthe vectors as

x =[x1 x2 . . . xm

]>(3.2)

y =[y1 y2 . . . ym

]>(3.3)

the least square problem for solving parameter θ is formulated as

θ∗ = arg minθ‖y− x · θ‖2 (3.4)

and the explicit least square solution is

θ∗ =(x>x

)−1x>y (3.5)

Fig. 3.2a shows the solution of (3.5). The y axis in the plot is the weight that each of thefeature represent. There are 50 samples drawn from the linear model with noise level of 0.1times input magnitude. The feature vector has 100 features but only 10 of which (non-zeroblue stars) are used to generate the outputs. The estimated feature vector are shown in redcircle and there is a significant mismatch with the blue stars. Beyond the mismatch, there isalso an overfitting problem. Since the sample size drawn from the model is larger than thefeature complexity, it is a overdetermined problem and the redundant features will try to fitthe noise in training data. This jeopardies robustness of the model and regularizing L1-normof feature vector helps to solve the sparsity problem. To introduce the L1 regularization term,(3.4) is further modified as

θ∗ = arg minθ

(‖y− x · θ‖2 + λ |θ|1) (3.6)


(a) Solution without L1 regularization

(b) Solution with L1 regularization

Figure 3.2: Comparison of feature vector


where the L1-norm of the feature vector θ is added in the minimization problem with a scalingfactor λ. The magnitude of λ decides how much the term is penalized in the optimization.Larger λ results in a sparser solution of θ. There will not be an explicit solution as (3.5)but the objective function is still convex. To solve problem (3.6), CVX, a package forspecifying and solving convex programs [9, 13] is introduced. Fig. 3.2b depicts the solutionof (3.6). Comparing the results with Fig. 3.2a, the method catches all the involving featuresand rejects most of the irrelevant features. Although the weight of some of the featuresare inaccurately estimated, the key contribution of the method is to filter out the criticalfeatures’ index.

Another benefit of this method is its robustness to noise. Fig. 3.3 compares the estimatedfeatures under different level of noise contamination. Fig. 3.3a shows that more features willbe activated when the noise level is larger. In this simulation the noise level is five timeslarger than Fig. 3.3b. Still, the key features remain robust with higher weight. Finally, inFig. 3.4 the problem of noise is tackled by adding more samples. As seen in Fig. 3.4b, whenthe sample size is five times larger, the overfitting problem is solved.

From the simulation shown in this section, the following benefits of L1 regularization areconcluded. First, it recognizes the most significant features that contribute to the modeland rejects unrelated features. Second, it is robust to the noise contamination. And lastly,the sparsity of feature reduces the computation load solving the inference. Although thereis some inaccuracy of the estimated features’ weights, it can be corrected by retraining themodel with the sparse features. In the next paragraph, the L1 regularization will be used ina classification to optimally determine the force sensor location.

Logistic Regression with L1 Regularization

The effectiveness of L1 regularization acting on a linear regression simulation is verified.However, linear regression is not an ideal method for classification. The outcome of linearregression is continuous yet a classification result must be discrete. Second, linear regressionmay predict the posterior probability that is over 1 or lower than 0. As such, the logisticregression is introduced as the classifier of gaits.

Logistic regression is a classification tool that has been widely used in machine learning[10, 19, 24]. As oppose to linear regression model, logistic regression does not require theoutcome to be continuous. Instead, it predicts a posterior probability p(y = 1|x; θ) of asample pair. Take binomial logistic regression as an example,

p(y = 1|x; θ) =1

1 + exp (−θ>x)(3.7)

where x ∈ Rn×1 is an input vector, y ∈ {0, 1} is the outcome category, and θ ∈ Rn×1 is amodal feature vector. The θ vector is of the same length as x, and the inner product θ>xcan be viewed as a score. The score is then mapped by the logistic function to an outputprobability. If the probability is larger than 0.5, the outcome is category 1, otherwise it iscategory 0. To train the modal parameter θ, the maximum likelihood method is utilized.


(a) Noise level = 0.5

(b) Noise level = 0.1

Figure 3.3: Comparison of estimated feature under different noise level


(a) Noise level = 0.5 with 100 samples

(b) Noise level = 0.5 with 500 samples

Figure 3.4: Comparison of estimated feature under different sample size


Assume there are M ’s (x,y) pairs been measured and M = M1 + M0, where M1 is thenumber of pairs that are category 1 and M0 is the number of pairs that are cateogry 0. Thelikelihood function of M trials is

L(θ|x) = p(y = 1|x1; θ) . . . p(y = 1|xM1 ; θ) · p(y = 0|xM1+1; θ) . . . p(y = 0|xM ; θ) (3.8)

=

M1∏i=1

[p(y = 1|xi; θ)] ·M∏

j=M1+1

[p(y = 0|xj; θ)] (3.9)

The log likelihood is

L(θ|x) =

M1∑i=1

log(p(y = 1|xi; θ)) +M∑

j=M1+1

log(p(y = 0|xj; θ)) (3.10)

=

M1∑i=1

log(1

1 + exp (−θ>xi)) +

M∑j=M1+1

log(exp (−θ>xj)

1 + exp (−θ>xj)) (3.11)

= −M∑i=1

log(1 + exp(−θ>xi))−M∑

j=M1+1

θ>xj (3.12)

=

M1∑i=1

θ>xi −M∑i=1

log(1 + exp(θ>xi)) (3.13)

Based on (3.13), the only parameter to be determined is the feature vector θ. There areseveral methods that solve the maximum likelihood problem. For the purpose of adding theL1 regularization term into the log likelihood function, the CVX tool is applied to numericallysolve for the optimal value.

θ∗ = arg maxθ

(M1∑i=1

θ>xi −M∑i=1

log(1 + exp(θ>xi))− λ |θ|1

)(3.14)

The regularization term λ |θ|1 is added to the log likelihood function with negative sign. Since(3.14) is a maximum likelihood formulation, the regularization term should be penalized withnegation.

The showed derivation is a binomial case, which the posterior is a probabilistic estimateof how likely the input is a hit. However, there are six gait phases distinguishable by forcedistribution and a classifier for multinomial cases is needed. In the next section, somemodification the method for estimation purpose will be discussed.

3.4 Experimental results

To further use the logistic regression as a multinomial classifier, some modification is madefor training the gait phase estimators. Each gait phase is associated with one binomial


(a) Recording graphical user interface (b) Insole sensor array

Figure 3.5: F-Scan system

Table 3.1: Ratio of data points (shown in %)

Subject 1 Subject 2 Subject 3Initial Contact 2.06 1.83 2.11

Loading Response 9.50 10.56 6.94Mid Stance 11.89 16.33 19.67

Terminal Stance 33.94 28.00 25.33Pre-Swing 10.56 12.28 11.78

Swing 32.06 31.00 34.17

classifier. The classifier is trained by taking the corresponding samples as “hit” and othersamples as “miss”. The posterior probability of a new datum will be evaluated by all the sixclassifiers and the one with largest posterior is the outcome category.

The walking data of three healthy subjects is collected with the full insole measuringsystem, F-Scan [39]. The insole sensor array shown in Fig. 3.5b has 21× 60 = 1260 sensorypoints which take the instantaneous snapshot of force distribution. The sensor value arereshaped to a one dimensional vector x ∈ R1260×1 and the feature vector is θ ∈ R1260×1.The experiments are recording at 30Hz with a total of 900 frames per trial. Each frame isassociated with one gait phase label. Table 3.1 shows the number of sample points in eachphase. Notice the discrepancy among different subjects yet they are all healthily walking.More characteristics will be discovered in a later paragraph.

Fig. 3.6 shows the feature vector θ of one trial. This is the optimal layout for recognizinggait phase “Initial Contact” for subject 1. There are several key points to interpret thefeature map:

• The clusters with highly contrasted color indicate the locations of important features.However, the color of these clusters is not as meaningful because the solution of featurevector can be either positive or negative.

• The solution by CVX suffers minor numerical error. It causes some theoretical zero


(a) An “Initial Contact” classifier (b) Reference footprint

Figure 3.6: Feature vector (a) and its mapping to a footprint (b). The features are drawn asa contour map and most of the features are zero or close to zero. Only the highly contrastedcolor dots (pointed with arrows) are the significant features which used to determine thegait.


solution to be nonzero. These errors reflect as a biased color the contour map. Allthese points with biased color should be viewed as zero.

• The reference footprint is drawn by averaging the force measures of one gait cycle.Warmer color indicates larger cumulated force.

Consistency in the Pattern

Fig. 3.7–3.9 show all the six gait estimators arranging from left to right. (Please refer toAppendix A for larger figures and details of subjects). The seventh subplots are the footprintsfor reference. There exist some consistency of the feature maps among different users.

• For “Initial Contact”, the classifier focuses on the location where the heel touches.

• Activated features of “Loading Response” resembles the location of “Initial Contact”but covers larger area. Some minor points in the frontal area are also activated butoutnumbered by the heel region.

• “Mid Stance” is the beginning of single leg stance and all of the body weight aresupported by one limb. Weight begins to widely spread on touched area. However,there is no significant difference of activated features comparing to those of “LoadingResponse”.

• More sensor points in the frontal area are activated in “Terminal Stance”. This indi-cates the shift of body weight and preparation for heel rising.

• The heel rises in “Pre-Swing” phase and now only the frontal area is pressed. Noticethe sparsity of the activated features remains.

• Lastly, no sensor is touched in the “Swing” phase so any location can be an effectiveindicator. The method chooses several points in the frontal area.

The concluded consistency holds among all three test subjects. Footprints of these subjectsare different in size and shape. Yet, they all share the same regions of interest when esti-mating gait phases. Moreover, the peaks in the force distribution map do not necessarilyimply criticalness. The hallux’s touch, for example, is unrelated to estimating gait phases.As such, when dealing with the complexity of recognizing six gait phases, a full insole sensoris unnecessary. The optimal location for laying out the force sensors are the heel region andthe metatarsal region.

Discrepancy among the Subjects

Besides the consistency discovered in estimating gait phases, there is some discrepancy inFig. 3.10 worth the attention.


Figure 3.7: Gait phases estimator of subject 1 test


Figure 3.8: Gait phases estimator of subject 2


Figure 3.9: Gait phases estimator of subject 3


• Subject 2 activates some intermediate area when shifting its weight between the heeland the frontal area, yet this method does not consider the area to involve in estimatinggaits.

• Subject 1 does not touch the inner arch area while the other two subjects touch. Thisis a possible indication of a flat foot but the gait estimators are independent from thissymptom.

• Subject 3 is lighter in body weight than the other two subjects (Detail weights arerecorded in Appendix A). The peaks are less significant but remain the main con-tributing features.

• The occurrence ratio of each gait phase shown in Table 3.1 is different among subjects.This also means that the temporal ratio is not identical, which concludes that temporalinformation are is not critical to gait phase estimation.

Validity of the Location Choice

The optimal solution of the sensor layout requires three sensors. Reducing the number ofsensors risks the loss of information that other literatures preserved. The performance of thenew layout must be comparable to the other literatures. To compare the performance of thepropose location and the conventional choices, the one-sided fuzzy logic [16] is implementedas a baseline for comparison. As shown in Fig. 3.11, the raw GCF measurement is plottedon the top where the new method contains only three channels of signal. In the lower partof the figure, the dashed lines indicate the ground truth gait phases and the solid lines arethe estimated phases by the two method. The comparable points is where the “TerminalStance” phase transits to “Pre-Swing”. The new method does not detect any of the “Pre-Swing” phases while the previous method detects only a small leap. The results showthat the reduced-number estimator are maintaining the same performance as the previouswork. Although both of the method lacks a good recognition on the “Pre-Swing” phase, theestimation on other gait phases are accurate.

The results propose new answers to some yet settle debates when laying out the forcesensor. As most of the literature [12, 38, 43, 36, 4] determines the sensor location byanatomy or clinical experiences, solving the optimality by data complexity justifies someconventional choices. When the findings are associated with the skeletal structure, the heelregion matches the lowest end of the calcaneus and the frontal area matched with the headof the five metatarsal bones. The choices are intuitive to explain and indeed necessary forgait phase estimation. As to the hallux, it is redundant for classifying gait phases. Theproblem of missing the “Pre-Swing” phase is a general behavior of using just one side of themeasurement to estimate gait. In section 4.4, a new two-sided fuzzy logic will be introducedto solve this problem.


(a) Subject 1 (b) Subject 2 (c) Subject 3

Figure 3.10: Comparison between footprints


(a) Estimator with three sensors. (b) Estimator with four sensors. [16]

Figure 3.11: Performance of gait phase estimation with different sensor layout.

3.5 Chapter Summary

In this chapter, the critical role of walking assessment as a foundation for understanding thehuman bipedal behavior is explained. Gait cycle is introduced as a repeated and effectivesequence of actions that maintains the balance of the limbs and moves forward. Thereare eight gait phases defined by the Rancho Los Amigos Gait Analysis Committee whichconcluded from clinical and anatomical observation. To estimate gait phases, a classificationmethod with an attribute of feature selection called L1 regularized logistic regression isproposed. The result of feature selection helps to determine the optimal sensor location forestimating gait phases from the perspective of data complexity. The conclusion drawn fromobserving the consistency and discrepancy among test subjects justifies some optimality ofprevious literatures and eliminates some redundant choices of sensor location.

35

Chapter 4

Hierarchical Decision Framework forBipedal Behavior

4.1 Introduction

Walking is one fundamental bipedal behavior and the sensor location has been optimallychosen to quantify it. As more knowledge is extracted about other bipedal behavior, theparameters learned from chapter 3 do not necessarily hold the optimality. For example, theheel region is less touched during faster running and should be treated as a less importantfeature of estimating running gaits. A set of optimal estimators for one special purpose isaccurate yet inflexible. Moreover, the assessment tool for forward-walking is not as usefulwhen dealing with other modes [8]. The complexity and flexibility of human create morebehavior to be quantified. As there are multiple modes of bipedal behavior, the optimal gaitestimators are contributing limited role.

As a result, a more comprehensive perspective is needed on human bipedal behavior. Inthis chapter, a framework is proposed to encapsulate more bipedal modes and deal with thecomplexity of identifying modes. A two-level hierarchical decision framework is developedfor Smartshoe. At the first level, the framework classifies the discrete modes of rest, stand-ing, and walking. Several classification methods are compared [32, 17] for correctness andapplicability on Smartshoe’s microcontroller. The second level of framework utilize the pre-trained parameters for instantaneous state estimation. For different mode, the parametersare distinctive. Experimental result of a robust walking gait estimator will be shown in thelast section.

4.2 Hierarchical Decision Framework

The idea of hierarchical decision framework originates from observing a human’s behavior.Other than walking, human also rest, stands, jumps, runs, marches and squat. These movestransit to each other and conceptually they are distinct from each other. This discrete

CHAPTER 4. HIERARCHICAL DECISION FRAMEWORK FOR BIPEDALBEHAVIOR 36

Figure 4.1: Modes of bipedal behavior

characteristic is an inspiration to decouple the bipedal behavior to two levels. The first levelis the discrete modes which are associated with the distinct moves. The second level is somemode-dependent special tasks for different purposes. For example, gait phase estimation isa special purpose task that reflects the quality of walking mode. Walking speed estimation[44] is another special purpose walking assessment tool. And these two special tasks are bothonly related to the walking mode.

In the decision framework shown in Fig. 4.1, the white blocks represent the discrete modeswhile the red blocks are the special tasks associated with the mode. The arrows indicate thepossible mode transition. It is clearly shown that modes do not necessarily share the samedefinition of special tasks. Through this framework, the special tasks only operate whenthe associated mode is activated. The result of the special tasks is now protected by themode transition guards and will reveal more robust performance. In addition to robustness,all the previously design state estimation algorithm [16, 1] can be accommodated in. Inthe following section, different methods will be introduced separately in both levels of theframework.

4.3 Classification of Discrete Modes

Classifying between modes plays an important role in this framework. The experimental datais recorded by the Smartshoe with three sensors lay out at the location shown in Fig. 4.2. Theheel sensor is located at the lowest end of the calcaneus. meta12 and meta45 are referring tothe frontal area of the foot. The two sensors are placed at the head joints of the 1st, 2nd, 4th

and 5th metatarsal bones. These are the location concluded from section 3.4 that optimallyestimate gait phases.


Figure 4.2: The location of sensors for classifying discrete modes.

Identifying the modes with instantaneous data is difficult. At every instance, there aremerely three values measured from both sides of the shoes. The instantaneous data doesnot reflect the temporal dependency between adjacent measurement. Because the GCF aretemporal series, the time dependency should be included in the classification problem. Asshown in Fig. 4.3, the GCF pattern of different bipedal modes is very distinctive. For thewalking modes in Fig. 4.3a and Fig. 4.3b, there are two peaks in the GCF measurement.Although the peaks reach different percentage of body weight at different walking speed, thedouble-peak pattern remains. Moreover, for the running mode depicted in Fig. 4.3c, thereexist only one peak in the GCF pattern. This explains the time dependency of the measuredGCF and the dependency should be considered in classifying modes.

Before feeding the force sensor data as inputs to the classification algorithm, some ex-pansion of the the input dimension is expanded with the temporal neighboring sensor data.This helps to encapsulate the temporal dependency between instantaneous measurement.As shown in Fig. 4.4, a moving window sweeps along a series of data to stack up multipleframes and treat it as a huge feature vector. The input vector size become 3× n which n isthe window size.

As the modes are discrete, several suitable classification methods are investigated. Thesupport vector machine (SVM) [32], logistic regression, and decision tree [17] are imple-mented to evaluate the performance. Table 4.1 reveals the resultant performance of each


0 20 40 60 80 1000

50

100

150

200

250

% of stance phase

% o

f bod

y w

eigh

t

Average GCF at 2 mile/hr

Subject 1Subject 2Subject 3

(a) GCF at 2 mile/hr (Walking)

0 20 40 60 80 1000

50

100

150

200

250

% of stance phase

% o

f bod

y w

eigh

t



(b) GCF at 3 mile/hr (Walking)

0 20 40 60 80 1000

50

100

150

200

250

% of stance phase

% o

f bod

y w

eigh

t



(c) GCF at 5 mile/hr (Running)

Figure 4.3: Comparison of GCF pattern at different bipedal modes.

Table 4.1: Performance of different classification method

Method Accuracy Execution Time FeasibilitySVM 86.87% 11ms +

Logistic Regression 85.10% 10ms +Decision Tree 75.12% 54ms -


Figure 4.4: The moving window method. Each input feature is a snapshot of neighboringmeasurement.

method. There are several benchmarks to be considered. A) The accuracy. B) The runningtime of each method. Notice this does not include the training effort. C) The applicabilityof these methods on Smartshoe. The mbed [23] has limited C-library support and some ofthe classification methods are too complicated to be implemented. All of the methods aretrained offline and implemented on the microcontroller. The execution time is recorded tocompare the computation cost. Based on the performance of Table 4.1, the SVM is chosenfro better refinement.

The size of the window largely affects the performance. A smaller size window lowers thepossible time dependency of the measurement and relies more on instantaneous informationwhile a larger window requires more memory to save earlier measurement. To determine thesize of the window, a cross-validation is implemented to test the size from 0 to 100. Theresult of cross-validation is shown in Fig. 4.5. The error rate of the SVM drops to 5% whenthe window size is around 60 frames. A size larger than 60 causes overfitting. Since the datarate of the experiment is 20Hz, 60 frames indicate 3 seconds of data.

In conclusion, an SVM based method is chosen to classify discrete modes. The methodrequires only 11ms to run on the mbed microcontroller once the parameters are pre-trained.Three sensors in the insole are utilized as the fundamental component of the input vector.To include the effect of time dependency in the sensor data, the moving window methodexpands the input vector by stacking temporal neighboring measurement. With the helpof cross-validation, the window size is chosen to be 3 seconds of data. Some buffer array isimplemented on the mbed microcontroller for storing the 3-second historical data, and themethod has a 95% accuracy.


Figure 4.5: Error rate of the classification for different size of moving window

4.4 Gait Estimator using Two-Sided Fuzzy Logic

The second level of the framework is the special tasks of each mode. For the walking mode,estimating gait phases is the focused task with the Smartshoe. Unlike the F-Scan [39]system, the three sensors are optimally placed at the heel, meta12, and meta45 locationshown in Fig.4.2. To run the estimation algorithm on the Smartshoe, the L1 regularizedlogistic regression from section 3.3 is not applicable due to the heavy computational load.Instead, a set of fuzzy logic rules is chosen to accomplish the task.

The fuzzy logic requires the lower level of computational power as the SVM. It is athreshold based method that uses a continuous membership function to map the input toa categorical output but with ambiguity. The smooth transition of a membership functionallows multiple if-then-else rules to combine as collective classifier for determining the output.

Recalling from section 3.4, the identification of “Pre-Swing” phase is difficult if the logictable considers only one side of the measurement. However, some of the gait phases are


0

0.2

0.4

0.6

0.8

1

f larg

e (x)

x0

x largex smaller

s increases

Figure 4.6: A hyperbolic tangent membership function

defined by the opposite limb. As [28] described,

“Pre-Swing”. Terminal double limb support is initiated by floor contact of theother limb. The reference limb responds to the initial weight transfer with in-creased ankle plantar flexion (PF), knee flexion, and a reduction of hip extension.The opposite limb is in loading response.

A more robust gait estimation performance will be achieved with both side of the measure-ment. In the following paragraph, the details of the fuzzy rule table and how to choose theparameters for the gait phase estimator will be discussed.

Fuzzy Rules on Walking Behavior

A membership function defines the boundary of logical transition between distinct categories.For our purpose, a smooth hyperbolic membership function is chosen for “large” as follows,

f large(x) =1

2[tanh (s · (x− x0)) + 1] (4.1)

where x is the measurement, x0 is the threshold and s is the sharpness of the transitionedge. The value of f large is 0 when x is much smaller than x0 and f large is 1 when x is muchlarger than x0. As depicted in Fig. 4.6, the choice of s affects how wide the transition bandis and the choice of x0 determine where the transition centered at. A “small” membershipfunction is just the contra function of (4.1) and can be expressed as,

f small(x) = 1− f large(x) (4.2)


Table 4.2: Two-sided Fuzzy rules for gait estimation using the three optimal sensors

Right Left

Right Gait Indicator µLarge Small - Large Initial ContactLarge Large - Large Loading ResponseLarge∗ Large∗ - Small Mid StanceLarge∗∗ Large∗∗ Small - Terminal Stance

- Large Large - Pre-SwingSmall Small Small - Swing

*: The metatarsal sum is smaller than the heel measurement.**: The metatarsal sum is larger than the heel measurement.

By multiplying the membership functions of each of the related sensor points, the inferencerules of all gait phases is now constructed. Table 4.2 is all the resultant inference rules whereeach of the gait phases has its own definition of involving sensors. Take “Pre-Swing” as anexample, the indicator function µPre−Swing is defined as

µPre−Swing,right = f large(xmeta45,right)× f large(xheel,left) (4.3)

and (4.3) is interpret as “If right meta45 is large and left heel is large, then the rightpre− swing is 1”

The performance of the fuzzy logic on gait estimation is shown in Fig. 4.7. The upperpart of the figure is the raw measurement of each of the sensors. The lower part of thefigure shows the µ indicator for the gait phases where the dashed lines are the reference gaitphases and the solid lines are the estimated gait phases. The estimator captures the correctorder of each phase and there is no abrupt jump in the phases’ sequence. Furthermore,with the information of the opposite limb, the identification of all the gait phases is moreaccurate and robust. However, there is some latency in the result and even some acausalpredicting behavior. This is caused by the choice of threshold x0. A moving average methodis used to keep updating the threshold values so that the estimator update the parametersautonomously. Although this method makes the Smartshoe less dependent on the user’sweight, it also makes the system less responsive. There a trade-off between the user depen-dency and the accuracy of the estimator. The overall estimation accuracy achieved is 82.7%and will be higher if the thresholds are tuned according to a distinct user.

Combined Experimental Result

In this section, the performance of the entire framework will be verified by experimentalmeasurement. First, when the force data is naively fed with only the fuzzy logic, the estima-tor acts nicely in the walking segment but generate some meaningless result in non-walkingmode. Fig. 4.8 is an extended version of Fig. 4.7b with the walking modes are at the right


(a) Double-sided estimator (b) Single-sided estimator

Figure 4.7: Performance comparison of the two-sided fuzzy logic.

portion of the plot. During segment 1 of Fig. 4.8, the user is merely putting on the Smartshoeand do not belong to any of the pre-defined bipedal behavior in Fig. 4.1. In segment 2 ofFig. 4.8, the user is in stand mode yet the gait indicator is trigger by some steady force inthe heel sensor. These undesired results are noisy and waste the computational power of theSmartshoe.

The two-level decision hierarchy for gait phase estimation is to use SVM to isolate thewalking mode at the first level and use the fuzzy logic table for estimation. The fuzzy logicis only triggered when the bipedal mode is recognized as walking. With such mechanism,the microcontroller on the Smartshoe can switch to another set of parameter or even otherfunction in another mode.

Fig. 4.9 reveals the comprehensive result of the framework. The mode classifier shownin the second row recognizes two bipedal modes, Stand and Walk. Since the SVM proposedrequires a 3-second window to determine the modes, there exists some latency in the classi-fication result. Yet this latency does not jeopardize too much of the performance since thebipedal behavior does not change abruptly. The lower part of the figure shows the indicatorsof the five gait phases. It is clear that the gait phase estimation is now more robust underthe hierarchical decision framework.


Figure 4.8: Gait phases estimator with naive fuzzy logic

4.5 Walking Speed Estimator

Another mode-specific task encapsulated under the hierarchical framework is the estimationof walking speed. Conventionally, the speed of walking is not estimable with only the instru-mented shoe. All of the previous literatures [4, 14, 44] require an IMU signal to estimate thekinematics of the lower limbs. However, there is a pattern in the GCF that changes withthe speed of walking. By extracting the information from the pattern, a consistency sharedamong different users is observed. In this section, the detail of the metric for estimating thewalking speed by GCF pattern will be presented with a statistical result.

When the GCF is measured as the summation of all the contact sensors, there exists twopeaks and one valley in the pattern. The first peak is contributed by the weight acceptancefunction which the body weight transfer abruptly onto a limb. The second peak is causedby the preparation of swing limb advancement, where a force larger than the body weightis exerted to boost the limb for a swing. Walking at a slower speed reduces the verticalacceleration, which results in a smaller amount in the height of the peaks and the depth ofthe valley.

Fig. 4.10 illustrates a comparison between the GCF of walking at 2 miles per hour and 3miles per hour. Seven test subjects with normal walking gait are recorded by the Smartshoeon a treadmill. The figure is only showing three representatives for clarity. The GCF is


Figure 4.9: Analysis of the experiment using the hierarchical decision framework


0 20 40 60 80 1000

20

40

60

80

100

120

140

% of stance phase

% o

f bod

y w

eigh

t



(a) GCF at 2 miles/hr

0 20 40 60 80 1000

20

40

60

80

100

120

140

% of stance phase

% o

f bod

y w

eigh

t



(b) GCF at 3 miles/hr

Figure 4.10: Comparison of GCF pattern at different walking speed.

normalized by the body weight of each subject and the time axis is normalized by thetemporal length of a stance phase. As shown in the Fig. 4.10a, the GCF is very consistentamong the three shown subjects. Both the peaks reach about 110% of the body weight andthe valley drops to 90%. For Fig. 4.10b, the subjects are walking at 3 miles per hour andthe two peaks reach over 120% of the body weight and the valley drops to 80%.

Based on this phenomena, define the GCF peak-to-valley ratio as,

R = (p1 + p2

2− v)/v (4.4)

where p1 and p2 are the two peak GCF values of each stance and v is the valley GCF value.In the experiment, the peaks and the valleys are obtained by taking the derivatives of themeasured GCF. The local maximum and minimum points are where the derivatives changesign. The ratio value R is calculated in real-time for each step and there is no significantasymmetry between the two limbs.

Fig. 4.11 shows the ratio value among three test subjects. The experiment is recordedat five distinct treadmill speed which is evenly spread between 2 miles per hour to 3.6 milesper hour. The figure is only showing three test subjects for clarity but the same trend holdsfor all the test subjects. There is a monotonic increasing trend between the GCF ratio andthe treadmill speed. At the lower speed, the ratio value stays consistently at 20% with smallvariance. When the walking speed increases, the mean of the ratio among different subjectraises and starts to deviate more from the mean. At 3.6 miles per hour, the ratio become70% and the variance become 15%. The deviation is caused by the distinct foot mechanismof each subject. When walking at a higher speed, the vertical speed increases and will takethe foot more effort to absorb the shock. The stiffness of flesh greatly affects the range ofthe ratio value. A good absorption will flatten the pattern and lower the ratio value.

Despite the increased variances, the trend of the mean is a suitable index for estimating


0.8 1 1.2 1.4 1.6 1.80.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Treadmill velocity (m/s)

GC

F r

atio

GCF ratio v.s. Walking velocity


Figure 4.11: GCF ratio vs treadmill speed.

the walking speed. This breaks the conventional opinion about the capability of a force-sensor-only instrumented shoe.

4.6 Diagnosis of Touch-based Symptoms

Except for the single-mode specific tasks, other tasks require a joint evaluation by multiplemodes. Some touch-based symptom is not as obvious in single mode but will be recognizablewith metrics in several modes. In this section, the diagnosis of Plantar Fasciitis [37] will bediscussed.


Table 4.3: Ratio of data points (shown in %, right limb)

Subject 1 Subject 2 Subject 3 Subject 4 Subject 5 Subject 6 Subject 7IC 2.06 1.83 2.11 1.95 2.03 1.51 1.76LR 9.50 10.56 6.94 8.06 7.83 8.55 10.06MS 11.89∗ 16.33 19.67 17.77 15.75 18.11 12.23∗∗

TS 33.94∗ 28.00 25.33 27.06 29.11 26.18 32.06∗∗

PS 10.56 12.28 11.78 13.01 13.45 13.54 11.83S 32.06 31.00 34.17 32.15 31.83 32.11 32.06

*,**: both Subject 1 and Subject 7 have a larger temporal ratio on “Terminal Stance”.

Plantar Fasciitis’ Symptom in Walking Mode

The plantar fasciitis is a foot symptom that is causing heel pain. The pain is most significantat the first step in the morning. As the patient moves more in the day, the pain normallydecreases. A patient suffering from the plantar fasciitis usually adapts their gait by avoidingtoo much weight laying on the heel. From the perspective of gait phases, the patient tendsto have,

• Shorter “Initial Contact” and “Loading Response”

• Prefer “Terminal Stance” than “Mid Stance”

However, from the statistical data shown in Table 4.3, this trend is not obvious. Althoughsubject 1 is the patient that actually have plantar fasciitis, subject 1 and subject 7 bothhave a significant longer “Terminal Stance” than “Mid Stance”. This is not qualified as aclear evidence of the existence of the symptom. Furthermore, the pain in the heel decreasesthroughout the day. There might be other subjects that are irritated by the symptomyet cannot be captured at the moment of the experiment. A more solid metric should beconsidered to diagnose plantar fasciitis.

Plantar Fasciitis’ Symptom in Running Mode

Diagnosing plantar fasciitis by the gait phases’ temporal ratio at walking is not evidential.In order to be more deterministic, the GCF pattern of the running mode is taken intoconsideration. As shown in Fig. 4.3c, the GCF is concentrated to a single peak at therunning mode. The peak of each stance can be detected by the same method described insection 4.5. In running mode, the peak value of a healthy GCF reaches up to 200% of thebody weight and there should be no significant asymmetry. However, in Table. 4.4 the onlysubject has an asymmetry is subject 1 where the left foot is taking 168.81% of the bodyweight and the right foot is only taking 92.96% of the body weight. The GCF for subject7 is better symmetric which justifies the healthiness. As such, the asymmetry in runningGCF together with the temporal ratio of “Mid Stance” phase and “Terminal Stance” phaseworks as the diagnostic indicator for plantar fasciitis.


Table 4.4: Ratio of peak force (shown in % of body weight)

Subject 1 Subject 2 Subject 3 Subject 4 Subject 5 Subject 6 Subject 7Left 168.81 171.95 183.00 201.95 172.03 181.51 161.76

Right 92.96∗ 167.05 171.89 208.06 177.83 178.55 170.06∗∗

*: A significant drop of peak value at the affected foot.**: No significant difference.

4.7 Chapter Summary

A hierarchical decision framework is proposed in this chapter. The main contribution of thisframework is a comprehensive interpretation and analysis of bipedal measurement. Suchmethod decouples behavior identification to two levels, discrete modes and mode-dependentspecial tasks. The modes in the first level interact as distinct moves where the differencein recognizable by observation. To classify the modes, several machine learning methodsare trained off-line, implemented on the microcontroller and compared for the applicabilityin our system. The SVM is chosen for our usage and the measurement is expanded witha temporal moving window for more degree of freedom. A cross-validation of window sizeshows the best accuracy are reached at 3 seconds of input measurement. At the second level,a fuzzy logic table is constructed for gait phase estimation. We choose a hyperbolic tangentfunction as a smooth membership function to distinguish large measurement with smallmeasurement. The three sensor location, heel, meta45, and meta12 are chosen according tothe optimal layout learned from chapter 3. The estimator reaches 82.7% of accuracy andfine-tuning the thresholds for a specific user will improve the result. Finally, the combinedinterpretation using the framework shows a robust and accurate analysis of human walkingbehavior. Several other metrics encapsulated by the framework are also shown with thestatistical result.

50

Chapter 5

Conclusions and Open Problems

5.1 Concluding Remarks

This dissertation studies the consistency and discrepancy of the human bipedal behaviorby designing a buttom-up integrated system. There are several major achievements by thiswork listed in the following paragraph.

A Robust and Flexible Framework for Recognizing BipedalBehavior

The Smartshoe is an integrated sensory system in which the force sensors is utilized toovercome the limitation of an optical motion capturing system. As there are different movesof the human bipedal behavior, the Smartshoe needs to achieve different tasks of quantifyingthe moves. A hierarchical decision framework is proposed in this dissertation and the maincontribution of this framework is a comprehensive interpretation and analysis of bipedalmeasurement. The behavior identification is decoupled to two levels, discrete modes andmode-dependent special tasks. Such separation of tasks makes the framework flexible to addany new activities and it also preserves better robustness on the identification performance.

Optimal Layout of Force Sensor for Gait Estimation

There has been a long debate on where to put the sensors in the shoe for estimating gaitphases. From the perspective of data complexity, a method is developed to optimally lay thesensors. The algorithm is inspired by the L1 regularization that performs feature selectionwhen solving a classification problem. The result of feature selection helps to determine theoptimal sensor location for estimating gait phases. The conclusion drawn from observingthe consistency and discrepancy among test subjects justifies some optimality of previouswork and eliminates some redundant choices of sensor location. The gait phase estimationperformance is also compared.

CHAPTER 5. CONCLUSIONS AND OPEN PROBLEMS 51

Rapid Gait Phase Estimation and Comparison of Applicability

The accuracy and repeatability of GCF measurement are demonstrated with the new choiceof piezoelectric sensor. Following that, a two-sided fuzzy logic table is constructed for arobust and accurate gait phase estimation. The fuzzy logic requires low computationalresource and can be implemented on the microcontroller. It is a threshold based methodthat uses a continuous membership function to map the input to a categorical output butwith ambiguity. The smooth transition of a membership function allows multiple if-then-elserules to combine as a collective classifier for determining the output. The accuracy of themethod reaches 82.7% and will be higher if the thresholds are tuned according to a distinctuser.

5.2 Open Problems

Several open issues were raised during the course of this study. Some potential directionsare listed for future work.

Diagnosis of Physical Condition

One promising application of the Smartshoe is the use of sensor data in clinical diagnosis.Usually walking assessment is coordinated by a physical therapist. One of the most impor-tant processes in the gait rehabilitation treatment is observing walking to estimate severalattributes such as step and stride length, weight shift, trunk alignment, pelvic rotation, reci-procity, symmetry, and stability as the patient walks on a stable or unstable surface overthe ground or on a treadmill. This gait rehabilitation treatment, however, has several draw-backs such as A) the effect of rehabilitation treatment depends on the physical therapistsknowledge, the accuracy of clinical observation and experience matching the impairments tothe appropriate intervention to minimize limitations and disability, B) effectiveness of theintervention strategy to remediate the impairment is difficult to document objectively, andC) the rehabilitation treatment consists of manual, verbal and visual feedback (mirrors andvideotapes) while working one on one with a therapist in a healthcare setting [2]. The mon-itoring device may overcome these limitations of the current practice by providing objectivedata sets as well as consistent analysis of the data.

Self Health Monitoring

Human is more comfortable with wearable devices now. One extended usage of the Smartshoeis a wearable sensor for health monitoring. The information of force distribution reflect somesymptoms that are not easily seen with bare eyes. Take plantar fasciitis as an example, thepain originated from the inflamed tissue cause a patient to avoid an abrupt touch of theheel. This phenomenon can be easily captured if the user walks wearing the Smartshoe.Moreover, the pain of plantar fasciitis relieves as the daily walk proceeds. The symptom

CHAPTER 5. CONCLUSIONS AND OPEN PROBLEMS 52

is most significant right off the bed in the morning and is best captured at such moment.Thus, the Smartshoe has great potential in the application of personal health monitoring.

Athletic Research

Athletic research is the science dealing with improving the athletic performance and moni-toring body status for injury prevention. In some of the athletic moves, the ground contactforce (GCF) is a crucial index. Take marathon running for example, as the runner accu-mulates more fatigue in the muscles while running, the stride-wise average of GCF patterndeviates. The Smartshoe can precisely capture the pattern deviation with its local memoryand record for a decent amount of time. Furthermore, the computation power carried on theSmartshoe may be used as a real-time analyzer. The Smartshoe can process the data andfeed correcting instruction back to the runner immediately. The mobility of the Smartshoemakes an instantaneous running pattern analysis possible.

53

Appendix A

Detail of test subjects

This appendix presents additional results of the optimal layouts. As the graphs are magnified,more detail of the contour map are shown. Notice that most of the feature maps containonly very few significant features. The colorfulness is cause by the numerical solution ofCVX that we have explained in section 3.4. The physical measures of three test subject arelisted in Table A.1.

Table A.1: Detail physical condition of the test subjects

Body Weight Age Height Shoe Size GenderSubject 1 145lb 30 5’7” US 9 MSubject 2 200lb 27 6’2” US 13 MSubject 3 120lb 25 5’3” US 7.5 F

APPENDIX A. DETAIL OF TEST SUBJECTS 54

5

10

15

20

25

30

35

40

45

50

55

60−10 −5 0 5 10 15 20 25 30

Figure A.1: Feature map for “Initial Contact”phase, subject 1


5

10

15

20

25

30

35

40

45

50

55

60−10 −5 0 5 10 15 20 25 30

Figure A.2: Feature map for “Loading Response”phase, subject 1


5

10

15

20

25

30

35

40

45

50

55

60−10 −5 0 5 10 15 20 25 30

Figure A.3: Feature map for “Mid Stance”phase, subject 1


5

10

15

20

25

30

35

40

45

50

55

60−10 −5 0 5 10 15 20 25 30

Figure A.4: Feature map for “Terminal Stance” phase, subject 1


5

10

15

20

25

30

35

40

45

50

55

60−10 −5 0 5 10 15 20 25 30

Figure A.5: Feature map for “Pre-Swing” phase, subject 1


5

10

15

20

25

30

35

40

45

50

55

60−10 −5 0 5 10 15 20 25 30

Figure A.6: Feature map for “Swing” phase, subject 1


5

10

15

20

25

30

35

40

45

50

55

60−10 −5 0 5 10 15 20 25 30


72

Bibliography

[1] Joonbum Bae and Masayoshi Tomizuka. “Gait phase analysis based on a HiddenMarkov Model”. In: Mechatronics 21.6 (2011), pp. 961–970.

[2] Joonbum Bae et al. “A mobile gait monitoring system for gait analysis”. In: Rehabili-tation Robotics, 2009. ICORR 2009. IEEE International Conference on. IEEE. 2009,pp. 73–79.

[3] Richard W. Baker. Measuring Walking: A Handbook of Clinical Gait Analysis. Wiley,2013. isbn: 978-1-908316-66-0.

[4] Stacy J Morris Bamberg et al. “Gait analysis using a shoe-integrated wireless sensorsystem”. In: IEEE transactions on information technology in biomedicine 12.4 (2008),pp. 413–423.

[5] Sarah Beynon et al. “Correlations of the gait profile score and the movement analysisprofile relative to clinical judgments”. In: Gait & posture 32.1 (2010), pp. 129–132.

[6] M Burke and Joan Lasenby. “Estimating missing marker positions using low dimen-sional Kalman smoothing”. In: Journal of biomechanics 49.9 (2016), pp. 1854–1858.

[7] Robert Capps. REVIEW: NIKE HYPERDUNK+ BASKETBALL SHOES. Ed. byWIRED.com. [Online; posted 28-August-2012]. 2012. url: https://www.wired.com/2012/08/nike-plus-basketball/.

[8] Prudhvi Tej Chinimilli, Sangram Redkar, and Wenlong Zhang. “Human activity recog-nition using inertial measurement units and smart shoes”. In: American Control Con-ference (ACC), 2017. IEEE. 2017, pp. 1462–1467.

[9] Inc. CVX Research. CVX: Matlab Software for Disciplined Convex Programming, ver-sion 2.0. http://cvxr.com/cvx. Aug. 2012.

[10] Scott A Czepiel. “Maximum likelihood estimation of logistic regression models: theoryand implementation”. In: Available at czep. net/stat/mlelr. pdf (2002).

[11] fitbit. fitbit. url: https://www.ftibit.com/.

[12] A Gefen et al. “Biomechanical analysis of the three-dimensional foot structure duringgait: a basic tool for clinical applications”. In: Journal of biomechanical engineering122.6 (2000), pp. 630–639.

BIBLIOGRAPHY 73

[13] M. Grant and S. Boyd. “Graph implementations for nonsmooth convex programs”. In:Recent Advances in Learning and Control. Ed. by V. Blondel, S. Boyd, and H. Kimura.Lecture Notes in Control and Information Sciences. http://stanford.edu/~boyd/graph_dcp.html. Springer-Verlag Limited, 2008, pp. 95–110.

[14] Pyeong-Gook Jung et al. “A mobile motion capture system based on inertial sensorsand smart shoes”. In: Journal of Dynamic Systems, Measurement, and Control 136.1(2014), p. 011002.

[15] Christopher Kirtley. Clinical gait analysis: theory and practice. Elsevier Health Sci-ences, 2006.

[16] Kyoungchul Kong and Masayoshi Tomizuka. “A gait monitoring system based on airpressure sensors embedded in a shoe”. In: Mechatronics, IEEE/ASME Transactionson 14.3 (2009), pp. 358–370.

[17] Chi-Chun Lee et al. “Emotion recognition using a hierarchical binary decision treeapproach”. In: Speech Communication 53.9 (2011), pp. 1162–1171.

[18] Edward A. Lee and Sanjit A Seshia. Introduction to embedded systems: A cyber-physicalsystems approach. MIT Press, 2017. isbn: 978-0-262-53381-2.

[19] Su-In Lee et al. “Efficient l˜ 1 regularized logistic regression”. In: Proceedings of theNational Conference on Artificial Intelligence. Vol. 21. 1. Menlo Park, CA; Cambridge,MA; London; AAAI Press; MIT Press; 1999. 2006, p. 401.

[20] Tao Liu, Yoshio Inoue, and Kyoko Shibata. “A wearable ground reaction force sensorsystem and its application to the measurement of extrinsic gait variability”. In: Sensors10.11 (2010), pp. 10240–10255.

[21] Tao Liu et al. “A mobile force plate and three-dimensional motion analysis system forthree-dimensional gait assessment”. In: IEEE Sensors Journal 12.5 (2012), pp. 1461–1467.

[22] Benoit Mariani et al. “On-shoe wearable sensors for gait and turning assessment ofpatients with Parkinson’s disease”. In: IEEE transactions on biomedical engineering60.1 (2013), pp. 155–158.

[23] Mbed. Mbed. url: https://www.mbed.com/.

[24] Lukas Meier, Sara Van De Geer, and Peter Buhlmann. “The group lasso for logisticregression”. In: Journal of the Royal Statistical Society: Series B (Statistical Method-ology) 70.1 (2008), pp. 53–71.

[25] Novel. novel. url: http://www.novel.de/novelcontent/.

[26] Ion PI Pappas et al. “A reliable gait phase detection system”. In: IEEE Transactionson neural systems and rehabilitation engineering 9.2 (2001), pp. 113–125.

[27] Shyamal Patel et al. “Monitoring motor fluctuations in patients with Parkinson’sdisease using wearable sensors”. In: IEEE transactions on information technology inbiomedicine 13.6 (2009), pp. 864–873.

BIBLIOGRAPHY 74

[28] Jacquelin Perry and Judith M. Burnfield. Gait Analysis: Normal and PathologicalFunction. 2nd ed. Delmar Learning, 1992. isbn: 978-1556427664.

[29] Phasespace. PhaseSpace Motion Capture. url: http://phasespace.com/.

[30] Volker Roth. “The generalized LASSO”. In: IEEE transactions on neural networks15.1 (2004), pp. 16–28.

[31] Heydar Sadeghi. “Local or global asymmetry in gait of people without impairments”.In: Gait & Posture 17.3 (2003), pp. 197–204.

[32] Christian Schuldt, Ivan Laptev, and Barbara Caputo. “Recognizing human actions: alocal SVM approach”. In: Pattern Recognition, 2004. ICPR 2004. Proceedings of the17th International Conference on. Vol. 3. IEEE. 2004, pp. 32–36.

[33] LM Schutte et al. “An index for quantifying deviations from normal gait”. In: Gait &posture 11.1 (2000), pp. 25–31.

[34] Michael H Schwartz and Adam Rozumalski. “The Gait Deviation Index: a new com-prehensive index of gait pathology”. In: Gait & posture 28.3 (2008), pp. 351–357.

[35] Rancho Los Amigos Medical Center. Pathokinesiology Service, Rancho Los AmigosNational Rehabilitation Center, and Rancho Los Amigos Medical Center. PhysicalTherapy Department. Observational Gait Analysis. Los Amigos Research and Ed-ucation Institute, Rancho Los Amigos National Rehabilitation Center, 2001. isbn:9780967633510. url: https://books.google.com/books?id=sZdMPgAACAAJ.

[36] Lin Shu et al. “In-shoe plantar pressure measurement and analysis system based onfabric pressure sensing array”. In: IEEE Transactions on Information Technology inBiomedicine 14.3 (2010), pp. 767–775.

[37] Mohammad Ali Tahririan et al. “Plantar fasciitis”. In: Journal of research in medicalsciences: the official journal of Isfahan University of Medical Sciences 17.8 (2012),p. 799.

[38] Weijun Tao et al. “Gait analysis using wearable sensors”. In: Sensors 12.2 (2012),pp. 2255–2283.

[39] Tekscan. F-Scan System. url: https://www.tekscan.com/products-solutions/systems/f-scan-system.

[40] Robert Tibshirani. “Regression shrinkage and selection via the lasso”. In: Journal ofthe Royal Statistical Society. Series B (Methodological) (1996), pp. 267–288.

[41] Vicon. Motion Capture Systems. url: http://www.vicon.com/.

[42] Eric R Westervelt et al. Feedback control of dynamic bipedal robot locomotion. Vol. 28.CRC press, 2007.

[43] Hui Yu et al. “A walking monitoring shoe system for simultaneous plantar-force mea-surement and gait-phase detection”. In: Advanced Intelligent Mechatronics (AIM),2010 IEEE/ASME International Conference on. IEEE. 2010, pp. 207–212.

BIBLIOGRAPHY 75

[44] Wenlong Zhang, Masayoshi Tomizuka, and Nancy Byl. “A wireless human motionmonitoring system for smart rehabilitation”. In: Journal of Dynamic Systems, Mea-surement, and Control 138.11 (2016), p. 111004.

Consistency and Discrepancy Analysis of Human Walkingresentation of human bipedal behavior. The...

Documents

Transcript of Consistency and Discrepancy Analysis of Human Walkingresentation of human bipedal behavior. The...