Human Behavior Modeling in a Cursor Tracking Game · Human Behavior Modeling in a Cursor Tracking...

Human Behavior Modeling in a Cursor Tracking Game

A Thesis Presented

by

Yijie Lu

to

The Department of Mechanical and Industrial Engineering

in partial fulfillment of the requirements

for the degree of

Master of Science

in

Mechanical Engineering

Northeastern University

Boston, Massachusetts

August 2019

ACKNOWLEDGMENTS

I would like to thank my thesis advisor Prof. Rifat Sipahi for his guidance and persistent

help on my research, which greatly helped me to finish this thesis.

I would also like to thank my thesis reader, Prof. Tansel Yucelen, who provided me

valuable advice on improvements to my work.

In addition, I would like to express my deepest appreciation to my parents. I am grateful

for their generous support for me pursuing the Master degree.

At last, I am also thankful for my pets: Fatty, Boss, and PangPang. Their companionship

helped me overcome the obstacles of the study and also brought precious joys to my graduate student

life.

i

TABLE OF CONTENTS

LIST OF FIGURES iv

LIST OF TABLES vi

NOMENCLATURE vii

ABSTRACT viii

1 Introduction 11.1 Human Behavior Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Model Reference Adaptive System . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2.1 System Equations [1] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2.2 Reference Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.3 Experiments with Human Subjects . . . . . . . . . . . . . . . . . . . . . . . . . . 81.3.1 Reaction Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.3.2 Control Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.3.3 Collected Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2 Extended State Observer 122.1 Active Disturbance Rejection Control [2] . . . . . . . . . . . . . . . . . . . . . . 122.2 Formulations of ESO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.3 Implementation of ESO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.3.1 Second-Order States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.3.2 Third-Order System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.3.3 Other Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3 Modeling and Validation 233.1 Fit Percentage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243.2 Linear Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.2.1 Linear Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243.2.2 Validation of the Linear Model . . . . . . . . . . . . . . . . . . . . . . . . 25

3.3 Nonlinear Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.3.1 Nonlinear Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.3.2 Validation of the Nonlinear Model . . . . . . . . . . . . . . . . . . . . . . 29

ii

4 Prediction of Human Behavior 324.1 Direct Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324.2 Indirect Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

4.2.1 Data Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384.2.2 Prediction of the Linear Model . . . . . . . . . . . . . . . . . . . . . . . . 394.2.3 Prediction of the Nonlinear Model . . . . . . . . . . . . . . . . . . . . . . 42

5 Conclusion 455.1 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

References 48

iii

LIST OF FIGURES

1.1 Human-in-the-loop MRAC System [3] . . . . . . . . . . . . . . . . . . . . . . . . 21.2 The Coordinate System Used in the Aircraft Motion Control (The Picture is Modi-

fied based on [4]) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.3 The Graphical User Interface of Human-involved MRAC Experiment . . . . . . . 91.4 Example of MRAC System Output with Human Subject’s Cursor Input . . . . . . 111.5 Example of MRAC System Output with Unfrozen Game Setting . . . . . . . . . . 11

2.1 The Architecture of Original ADRC System with ESO . . . . . . . . . . . . . . . 122.2 The Architecture of Human Dynamics which Applied with ESO [5] . . . . . . . . 142.3 Plot of Tracking Variable with respect to Time . . . . . . . . . . . . . . . . . . . . 162.4 Phase Plot of z3 with respect to z1 (Subject #9) . . . . . . . . . . . . . . . . . . . 172.5 Phase Plot of z3 with respect to z1 (All Subjects, Second-Order System) . . . . . . 182.6 Comparison between Actual Position x1 and Estimated Position z1 Obtained Based

on the Three-Order System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.7 Phase Plot of z3 with respect to z1 Obtained Based on the Three-Order System . . 202.8 Phase Plot of z3 with respect to z1 (All Subjects, Third-Order System) . . . . . . . 21

3.1 Block Diagram of Human Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . 243.2 Example of Linear Identification Results . . . . . . . . . . . . . . . . . . . . . . . 253.3 Block Diagram of the Structure of the Nonlinear ARX Model [6] . . . . . . . . . . 283.4 The Effect of Applied Filter to Nonlinear Identification Result . . . . . . . . . . . 29

4.1 Command Error Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324.2 System Output and Output Error Comparison . . . . . . . . . . . . . . . . . . . . 344.3 Error Comparison between the Linear Model and the Nonlinear Model (Case 1) . . 354.4 Error Comparison between the Linear Model and the Nonlinear Model (Case 2) . . 364.5 Results of System Output with Two Different Input Provider Cases . . . . . . . . . 374.6 Comparison of Control Commands from Linear Identification (From left to right:

Case 1 and Case 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384.7 Derivative of Commands and Its +1/-1 Plot (Case 1) . . . . . . . . . . . . . . . . . 394.8 Optimization Result for +1/-1 Plot of Derivatives . . . . . . . . . . . . . . . . . . 404.9 Indirect Prediction Results of Linear Models (Case 1 and 2) . . . . . . . . . . . . . 414.10 Indirect Prediction Results of Linear Models (Case 3 and 4) . . . . . . . . . . . . . 42

iv

4.11 Indirect Prediction Results of Nonlinear Models (Case 1 and 2) . . . . . . . . . . . 434.12 Indirect Prediction Results of Nonlinear Models (Case 3 and 4) . . . . . . . . . . . 43

v

LIST OF TABLES

1.1 Parameters Used in the Nonlinear Dynamical System [7] . . . . . . . . . . . . . . 61.2 Parameters Used in the Neal-Smith Pilot Model [7] . . . . . . . . . . . . . . . . . 71.3 Stability for Different Reference Model Design Methods [1] . . . . . . . . . . . . 8

3.1 Best Linear Fit Result for Each Subject . . . . . . . . . . . . . . . . . . . . . . . 263.2 Best Fit Result for 5, 10, 50 Poles and Zeros . . . . . . . . . . . . . . . . . . . . . 273.3 Best Fit Result for Each Subject as Model Provider . . . . . . . . . . . . . . . . . 303.4 Nonlinear Fit Results for a Given Delay in Timesteps . . . . . . . . . . . . . . . . 303.5 Each Subject’s Total Nonlinear Fit Numbers for a Given Delay in Timesteps . . . . 31

vi

NOMENCLATURE

MRAC Model Reference Adaptive Control

RMDP Rightmost Dominant Pole

RMNP Rightmost Non-Dominant Pole

LQR Linear-Quadratic Regulator

ADRC Active Disturbance Rejection Control

ESO Extended State Observer

vii

ABSTRACT

The growing demand for human-machine system applications makes it more frequent todeal with human-in-the-loop control problems. The Model Reference Adaptive Controller (MRAC)can adaptively handle uncertainties in dynamical systems and can be useful in human-in-the-loopsettings. But once a human becomes a command provider of a control system, human reaction delaycan make the system unstable.

In this thesis, through the experimental data of the human-in-the-loop MRAC system ap-proved by the IRB, the control behaviors of human subjects are investigated. By using the ExtendedState Observer proposed in the Active Disturbance Rejection Control system, this thesis finds that allthe human subjects in the study have a similar control pattern when participating in the experiment,which indicates that there is a potential generalized behavior model for human subjects.

Then, the human-in-the-loop MRAC experimental data were subjected to both linear andnonlinear identification through the System Identification Toolbox provided by Matlab. The resultsshow that only using the modeling tools provided by Matlab is not enough to directly match all thedata, therefore these models cannot be used to predict the control behavior of most of the subjects.However, these models can indirectly predict the human behavior in the experiment, that is, predictthe intention of human as to whether the human will control the cursor up or down in the near future.Throughout the test for different trials, the best predictive model is found to be the nonlinear model,which has an accurate prediction of human intentions in most cases. This promising result showsthat it is possible to utilize this model in the future to prevent the human subject’s excessive controlin the control loop.

viii

Chapter 1

Introduction

1.1 Human Behavior Modeling

In recent years, from the professional machine to the common automobile and aircraft,

a variety of concept systems that do not require real-time supervision and manipulation from the

human operator are emerging one after another. However, due to the immaturity of the current

technology, we still have a long way to go before a completely fully automatic era has arrived.

Therefore, engineers, for now, have to consider the effect of human-in-the-loop when designing an

interactive engineering system. Due to human’s natural imperfections such as the reaction delay,

knowledge level, and decision accuracy, both the performance and stability of the controlled dynam-

ical system will be impacted because of the involvement of the human operator. Therefore, how to

better integrate the control effort of human into the control system so that the human-in-the-loop

system can be more efficient is a common topic investigated by researchers in various fields [8–24].

One optimization method of the human-in-the-loop system is to treat the human operator’s

command input as a disturbance. The human input will be processed through the controller of the

human-in-the-loop system in real-time and then utilized to generate the actual system input [13, 25].

In real life situation, humans inevitably encounter some unpredictable incidents. Taking driving as

an example, if a driver witnesses a sudden deceleration of the front car, an unexpected lane changing

or an obstacle on the road, although the driver may think it is intuitive and easy to response, it is

difficult for the vehicle to predict exactly how the driver may respond even if the vehicle can sense

upcoming incidents. Therefore, another method to optimize the human-in-the-loop system is to

establish the behavior model of the human operator through a large amount of data, so that the

system can predict the operator’s intention in advance and that the corresponding control algorithm

1

CHAPTER 1. INTRODUCTION

can be employed according to the prediction [26–34].

It can be said that human models play a vital role in the reliability of human-in-the-loop

systems. Hence researchers are paying more attention to find suitable human behavior models when

designing human-in-the-loop systems. As early as the last century, to better understand the control

behaviors of pilots in flying, the researchers proposed a variety of simplified pilot models, such

as the Crossover and Precision pilot behavior model [35, 36], and also the Tustin pilot model in

[37]. However, such behavior models are more of a linear model, while the human model in the

real world should contain a certain level of nonlinearity. Therefore, these models tend to be used

to replace the real human only in the system design phase for the sake of convenience. To meet

with the actual control system requirements, researchers have proposed various behavior modeling

methods, including the modified linear model [38–44], fuzzy interface system (FIS) model [45–47],

and the model simulating human’s cognitive process [48, 49].

1.2 Model Reference Adaptive System

Human Dynamics

Reference Command

Reference Model

ControllerUncertainDynamical

System

−

ParameterAdjustment

Mech.

System Error

Outer Loop Inner Loop

Figure 1.1: Human-in-the-loop MRAC System [3]

Adaptive control can be divided into three types: open-loop adaptive control, direct adap-

tive control, and indirect adaptive control [50]. The Model Reference Adaptive Controller (MRAC)

system, here, is an direct adaptive control method. By comparing the output values of the reference

model and uncertain dynamical system or plant, the difference will be calculated, and then the corre-

sponding adjustment will feedback to the controller itself through closed-loop. Figure 1.1 shows the

2


block diagram of the human-in-the-loop MRAC system, given by [3]. The control system and ex-

perimental data obtained via simulations here inspired by this figure. Compared with the traditional

MRAC system, the input to the MRAC system is called the Outer Loop here, which represents the

human itself who is operating the system. Based on the fact that human fully understands the objec-

tives, through a series of observations, reflections and execution processes, etc., the human operator

can use a variety of methods to input control commands to the robotic system in the Inner Loop

of MRAC. In the Inner Loop, there is no difference from the traditional MRAC architecture. The

parameters in the controller will change in real-time according to the output of the system, so the

accuracy of the control results can be guaranteed to the utmost. Here, the mathematical modeling

of the various parts of the Inner Loop comes from [3].

1.2.1 System Equations [1]

The uncertain dynamical system in the MRAC system can be conveniently represented by

the equation shown below

xp(t) = Apxp(t) +BpΛu(t) +Bpδp(xp(t)), xp(0) = xp0 , (1.1)

where xp represents the accessible state vector in uncertain dynamical system. Both Ap and Bp are

assumed to be system matrices in known models, Ap ∈ Rnp×np , Bp ∈ Rnp×m. Λ ∈ Rm×m+ ∩Dm×m is an unknown control effectiveness matrix, u(t) ∈ Rm is the input to the system. δp is an

uncertainty function, here defined as

δp(xp) = WTp σp(xp), xp ∈ Rnp , (1.2)

where Wp ∈ Rs×m stands for a unknown weight matrix, σp : Rnp → Rs stands for a known basis

function, defined as

σp (xp) = [σp1 (xp) , σp2 (xp) , . . . , σps (xp)]T . (1.3)

Here, xc(t) ∈ Rnc is the integrator state and satisfies

xc(t) = Epxp(t)− c(t), xc(0) = xc0 , (1.4)

where Ep ∈ Rne×np is used to select a subset from xp(t), and c(t) is the command generated by

human dynamics in the outer loop. By combining Equation 1.1 and Equation 1.4 together, we have

x(t) = Ax(t) +BΛu(t) +BWTp σp (xp(t)) +Brc(t), x(0) = x0 (1.5)

3


where

A ,

Ap 0np×nc

Ep 0nc×nc

∈ Rn×n

B ,[BT

p , 0Tnc×m

]T ∈ Rn×m

Br ,[0Tnp×nc

,−Inc×nc

]T∈ Rn×nc

and the augmented state vector is given by x(t) , [xTp (t), xTc (t)]T ∈ Rn, and x0 , [xTp0 , xTc0 ]T ∈

Rn,n = Np + nc.

According to the feedback control law, the signal in the inner loop is

u(t) = un(t) + ua(t), (1.6)

where un(t) ∈ Rm is nominal control input, and ua(t) ∈ Rm is adaptive control input. Assume that

the nominal control input is

un(t) = −Kx(t), (1.7)

where K ∈ Rm×n, Ar , A−BK. Substituting, one obtains

x(t) = Arx(t) +Brc(t) +BΛ[ua(t) +WTσ(x(t))], (1.8)

where WT , [Λ−1WTp , (Λ

−1 − Im×m)K] ∈ R(s+n)×m is an unknown weight aggregated matrix.

σ is a known aggregated basis function and defined as

σT(x(t)) , [σTp (xp(t)) , xT(t)] ∈ Rs+n. (1.9)

Assume next that the adaptive control input is in the following form

ua(t) = −WT(t)σ(x(t)), (1.10)

where W (t) ∈ R(s+n)×m is an estimation of W and satisfies the following relationship

˙W (t) = γσ(x(t))eT(t)PB, W (0) = W0, (1.11)

where γ ∈ R+ is called the learning rate. System error is then given by

e(t) , x(t)− xr(t), (1.12)

where xr(t) ∈ Rn is the reference state vector and satisfies the reference system

xr(t) = Arxr(t) +Brc(t), xr(0) = xr0 . (1.13)

4


In addition, P ∈ Rn×n+ ∩ Sn×n is a solution to the Lyapunov equation

0 = ATr P + PAr +R, (1.14)

where R ∈ Rn×n+ ∩ Sn×n. For a given R, there is always a unique P ∈ Rn×n+ ∩ Sn×n.

In order to simulate the human-in-the-loop MRAC system in a real-world situation, here

Mr. Yanchao Wang [1] selects a well-developed Boeing 747 aircraft model from [7] as the uncertain

dynamical system. When the pilot performs a pitching motion by using the joystick, the pitch

angle of the aircraft will be subjected to change. To simplify the simulation, only the longitudinal

movement of the aircraft will be simulated.

Pitch Angle

Center of Gravity

Figure 1.2: The Coordinate System Used in the Aircraft Motion Control (The Picture is Modified

based on [4])

The mathematical model of the aircraft, in this case, can be found in [7], as shown below

for the equation of motion of the aircraft at a height of 40,000 ft and cruising at 0.8 Mach.

x = Apx(t) +Bpu(t) +WTσ(x(t)), x(0) = x0, (1.15)

where x(t) = [u,w, q, θ]′, u and w represent the velocities along the x-axis and y-axis respectively,

while q represents the angular velocities along y-axis. θ is the pitch angle in the Euler coordi-

nate system established by the aircraft fuselage, as shown in Figure 1.2. Here, W is the unknown

weighting matrix. σ(x(t)) = [1, u(t), w(t)]T is the known basis function.

Since the human subjects experiments were performed in Matlab on the PC platform, the

above continuous-time equation needs to be converted into the discrete-time form. By performing

5


the discrete-time conversion on Equation 1.8, the following equation can be easily obtained:

x[n] = x[n− 1] + (Arx[n− 1] +Brc[n]

+BΛ[(−WT[n] +WT)σ(x[n− 1])])∆t.(1.16)

According to the parameters given in [7], the parameters used in the simulation are shown in the

following table. Here simulation sample time was selected as ∆t = 0.005 seconds.

Table 1.1: Parameters Used in the Nonlinear Dynamical System [7]

Ap

−0.003 0.039 0 −0.322

−0.065 −0.319 7.74 0

0.0201 −0.101 −0.429 0

0 0 1 0

Bp

[0.01 −0.18 −1.16 0

]TW

[0.1 0.3 −0.3

]TΛ 1

γ 50

Ep

[0 0 0 1

]

1.2.2 Reference Model

It was shown in [1] that the output of a nonlinear MRAC closed-loop system will be

stable if the reference model meets some LMI conditions. Thus it can be much easier to design a

human-in-the-loop MRAC system by separating the problem into two parts. In the first part, the

control problem is designing a zero-delay dynamical system with uncertainties and nonlinearities.

The second part, however, relates to a linear reference model and human linear model with human

reaction delay. Therefore, the only challenging problem will be to stabilize the linear combination

model in part two.

Since there exist various K that can make Ar = A−BK Hurwitz matrix in the reference

model, which is necessary for a stable control system, then how to find the most appropriate value

of K among those is critical. In order to understand the impact of different K values on human

performance, a variety of methods can be used to design the reference model. In [1], a total of three

6


designing methods for reference model were taken into consideration, namely: Classical Linear-

Quadratic Regulator (LQR), Modified LQR, and Pole Placement.

Here, for the convenience of comparing the results, the Neal-Smith model was used to

represent the behavior of real pilot controlling the pitch angle of the aircraft. The Neal-Smith model

is given by [51]

H(s) = kpTps+ 1

Tzs+ 1e−τs (1.17)

where kp is the model gain. Tp is the lead time constant. Tz is the lag time constant, and τ is the

reaction delay of the pilot itself. These parameters are selected as shown in the below Table.

Table 1.2: Parameters Used in the Neal-Smith Pilot Model [7]

kp 4.5

Tp 1

Tz 7.4

τ 0.5

In brief, Classical LQR cannot choose the real part of the rightmost dominant pole (RMDP).

For Modified LQR, the RMDP cannot easily shift to left of the complex plane. In Pole Placement

method, it was shown that rightmost non-dominant pole (RMNP) can impact the peak time of the

stable reference model.

Since the focus of the experiment is to find the impact of time-domain step response and

the reference model poles positions on human dynamics performance, the pole placement method

is more suitable with the experimental requirements. Thus, the pole placement method was chosen

to design the reference model in this human-in-the-loop MRAC experiment.

By fixing the RMDP at -0.02 and setting the peak value to 1.7, the following pole sets can

be obtained and utilized in the experiment

P1 = −[0.02, 0.31, 2, 2.2, 2.3],

P2 = −[0.02, 0.35, 0.9, 1.3, 1.7],

P3 = −[0.02, 0.5, 0.6, 0.7, 0.8].

Although the peak values of these three pole sets are very close, the peak time is quite different.

The peak time for pole set 1 is 4.004 seconds, while pole set 2 and pole set 3 are 6.014 seconds and

8.176 seconds respectively. See details in [1].

7


Table 1.3: Stability for Different Reference Model Design Methods [1]

Methods RMDP RMNP Stability

Classical LQR

-0.0117 -0.2943 Stable

-0.0117 -0.1907 Stable

-0.0117 -0.1249 Stable

-0.0117 -0.1064 Stable

Modified LQR

-0.0118 -0.1741 Stable

-0.0121 -0.1147 Stable

-0.0123 -0.0642 Stable

-0.0125 -0.0203 Stable

+0.0187 -0.0126 Unstable

Pole Placement

-0.0126 -0.2139 Stable

-0.0126 -0.0817 Stable

+0.0076 -0.0126 Unstable

1.3 Experiments with Human Subjects

The human-in-the-loop experiment described herein with volunteers participated was led

by Mr. Yanchao Wang and approved by the Institutional Review Board (IRB) with IRB#: 18-04-

05, see [1]. In experiments, a total of 16 subjects were recruited, but only 14 of them actually

participated, 11 of whom were male and 3 were female. None of the recruited subjects encounter

health issues and learning disabilities as self-reported on the pre-experiment survey.

1.3.1 Reaction Time

It is obvious that the reaction time varies through different people. Although running

simulation game itself will not require obtaining the subjects’ reaction time, having the reaction

time of subjects has a great effect on the modeling analysis after experiments. The reaction time

test was performed through online website Human Benchmark [52]. In the test, the subject needs to

click the mouse whenever he or she saw the screen’s color was changed. Such a reaction time test

is merely a simple test and does not represent the actual situation. In the real situation, the aircraft

pilot’s tasks are much more complex, which usually cost more times to observe, decide, and execute.

8


However, the experiment is also a simple task, since the subject only needs to move the mouse to

achieve the goal (see next section for details of control game). Therefore, it can be assumed that

the subject in these two experiments has a similar reaction time. According to the obtained reaction

time test data, the average reaction time of all 14 subjects is 0.355 seconds.

1.3.2 Control Game

The human-in-the-loop MRAC system mentioned above shows that human dynamics or

the outer loop should have 2 separate input signal; one is the reference signal, while another one

is the output signal of uncertain dynamical system or plant. As for the output of human dynamics,

there should be only one command signal. Based on these fundamental requirements, an interactive

Matlab game with the following user interface can be designed, as seen in Figure 1.3. The main body

Visual cue

Human mouse input

Fixed setpoint

Output of nonlinear dynamics

Time

Po

siti

on

Red: The human starts the game at time = 3 sec

Figure 1.3: The Graphical User Interface of Human-involved MRAC Experiment

of this interactive game was developed by Prof. Tansel Yucelen (University of South Florida). The

modification including the reference model, graphical user interface, and experiment preparation

was all developed by Mr. Yanchao Wang, see [1].

The objective of this game is to make the blue line as close as possible to the green line

as well as maintain it until the end of the experiment. The human subject needs to use a mouse

to control the vertical slider on the left. Whenever the mouse drags the slider to a different value,

9


the blue line will then have a related movement, namely, move upward or downward. It is worth

mentioning that the slider bar does not control the blue line directly. There exists a nonlinear dy-

namics which represents the ”aircraft dynamics”. The blue line’s value here represents the aircraft’s

pitch angle, while the green line’s value represents the desired pitch angle which the aircraft needs

to reach (Here assuming the reference value of pitch angle is one). The slider bar is an abstracted

aircraft joystick.

During the game, the first three seconds are called the ”frozen time”, the subject will have

three seconds to get ready before the actual starting. Theoretically, the blue cursor will constantly

remain zero for the first three seconds until, when the blue cursor touches the red line, which is the

actual starting line. Then the subject was allowed to use the mouse to achieve the objective. How-

ever, due to the experiment setting problem, the first four subjects’ gaming set allows the subjects to

move the mouse cursor during the first three seconds. This problem was fixed for the subjects No.6

to No.16.

For each subject, three different pole sets for the reference model were utilize. Each was

played for 5 times. However, the order of pole sets in each game group is randomized to make

the subject behave as consistent as possible. Between each game group, there will be a one minute

break. It is expected that each subject’s entire control game playtime is about 12 minutes.

1.3.3 Collected Data

A total of 210 experimental results were obtained from 14 subjects. What can be found

from the data is that although subjects have different ways to handle the control task, not a single

subject can fully guarantee to stabilize the uncertain dynamical system with the help of the MRAC

controller. A sample of the result obtained in the experiments is shown in Figure 1.4, the green line

and blue curve have the same meaning mentioned in the last section, while the red wave represents

the actual input value through the slider. Although the red wave here is presented and shown directly,

only the green line and blue curve are seen by the subjects during the gaming process.

As described in the last section, there exist two initial conditions for 14 subjects’ experi-

ments. For the first four subjects’ experiments, although the slider bar was fixed, the mouse cursor

could still move to the slider bar and may inadvertently apply a small disturbance to the slider. As

can be seen from the collected data, the input commands for the inner loop given by these people

have some initial value in the first few seconds, as shown in Figure 1.5. Hence, in this thesis, the

first four subjects’ recording data are not used for analysis and modeling.

10


Figure 1.4: Example of MRAC System Output with Human Subject’s Cursor Input

Figure 1.5: Example of MRAC System Output with Unfrozen Game Setting

11

Chapter 2

Extended State Observer

2.1 Active Disturbance Rejection Control [2]

TransientProfile

Generator�1𝑏𝑏0

𝑣𝑣 𝑦𝑦𝑣𝑣1

−𝑣𝑣2

Nonlinear Weighted

Sum−

𝜃𝜃1

𝜃𝜃2𝑢𝑢0

−

𝑢𝑢Plant

𝑏𝑏0

Extended State

Observer

𝑧𝑧3𝑧𝑧2𝑧𝑧1

Figure 2.1: The Architecture of Original ADRC System with ESO

The Extended State Observer (ESO) was originally an important part of Han’s Adaptive

Disturbance Rejection Control theory [2], the ADRC framework is shown in Figure 2.1. In the fig-

ure, v is the desired value for y, while v1 and v2 are desired trajectory and its derivative respectively.

u is the input to plant after adjusting. Han’s ADRC control is a good alternative to PID control since

the input is adjusting based on the observed results from ESO. Besides, regardless of the form of

nonlinearities in the dynamical system, it can be estimated through ESO and then returned to the

controller for appropriate control optimization.

Although the human itself cannot be brought into any control method since the human

is not an engineering system, it is still possible to utilize ESO’s characteristics, in which the non-

linearity of the system can be estimated without knowing the explicit dynamical system model, to

12

CHAPTER 2. EXTENDED STATE OBSERVER

estimate the nonlinearities of human dynamics.

2.2 Formulations of ESO

Suppose there is a second-order system with the following equation:x1 = x2

x2 = F + bu

y = x1

(2.1)

where u is the input, while y is the output. F is an unknown function which could be a multivariable.

Assume F is an additional state variable, which is F = x3. Then introduce another denotation G,

let G = x3. The equation above can be rewrite as

x1 = x2

x2 = x3 + bu

x3 = G

y = x1

(2.2)

which is easy to build the following extended state observer. By denoting the xi with zi as an

estimation of the system states, the function can be further redescribed asz1 = z2 − β01ez2 = z3 + bu− β02h (e, 0.5, T )

z3 = −β03h (e, 0.25, T )

(2.3)

where β01, β02, and β03 are nothing but a group observer gains, which can be selected as wish. e

is the representation of the tracking error, where e = z1 − y. However, h, as shown below, is a

denotation of the nonlinear coefficient function proposed by Han (see details in [2]).

h(e, α, δ) =

eδ1−α , |e| ≤ δ,|e|α sign(e), |e| > δ.

(2.4)

Based on the equation above, the discrete-time formulation of the ESO can be easily

obtained: z1(n+ 1) = z1(n) + Tz2(n)− β01ez2(n+ 1) = z2(n) + T (z3(n) + bu(n))− β02h(e, 0.5, T )

z3(n+ 1) = z3(n)− β03h(e, 0.25, T ).

(2.5)

13


The observer gains can be selected in many ways. For convenient, the observer gains here

are also selected based on Han’s method:

β01 = 1 β02 =1

2T 0.5β03 =

2

52T 1.2(2.6)

where T is the sampling period. In this thesis, the sampling period is set at 0.005 seconds.

2.3 Implementation of ESO

It is difficult to assert how many orders in human dynamical system. However, given the

fact that in this experiment, the subject observes the blue cursor’s position and then manipulates the

mouse through force, it can be inferred that human dynamics has at least two states, namely x1 and

x2. For the sake of simplicity, only second-order and third-order system were tested in this paper.

2.3.1 Second-Order States

ReferenceAngle

HumanDynamics

Extended State Observer

𝑥𝑥1𝑥𝑥2𝑥𝑥3 +

−

𝑒𝑒1𝑧𝑧1𝑧𝑧2𝑧𝑧3

−

+

𝑢𝑢PitchAngle

Figure 2.2: The Architecture of Human Dynamics which Applied with ESO [5]

Assuming the human dynamics with the second-order system has the architecture as Fig-

ure 2.2 shown, then the corresponding equations can be derived asx1(t) = x2(t)

x2(t) = F (x1(t), x2(t), t) + bu(t)

y(t) = x1(t)

(2.7)

where F is the unknown nonlinear human dynamics function, and u(t) is the input that subject

observers from the computer screen. The coefficient b is assumed to be one (b = 1), with appropriate

14


units. As mentioned above, ESO’s formulation for the second-order system isz1(t) = z2(t)− β01ez2(t) = z3(t) + u(t)− β02h(e, 0.5, 005)

z3(t) = −β03h(e, 0.25, 0.005).

(2.8)

In this setting, since the actual mouse position x1 is known through the collected data, the

tracking performance can be measured through the difference between x1 and z1. An assumption is

made here that if the error e = x1 − z1 for the first state goes to zero, the tracking results for other

states are also reliable. Therefore, the reliability of z3 depends on steady-state error, and as long as

e→ 0, the unknown function F can be estimated from the state x3.

Figure 2.3a shows a comparison of actual position x1 and tracking position z1 with respect

to time t, and each game set has a trial selected. It can be seen from the figure that only a minor

difference exists in between the orange line (x1) and the blue line (z1). Besides this example figure,

the average error across the 10 subjects is 0.0123, while the maximum and minimum are 0.0333

and 0.0019 respectively. Hence it can be proved that ESO’s tracking is reliable for x1 as well as for

unknown function F .

Figure 2.3b are the time plots of z3, and each plot corresponds to Figure 2.3a. The un-

known function F as estimated by z3 is clearly varying in each trial. Based on the fact that many

variables exist in each trial, such as subjects might take a different strategy in each trial, or subjects

might take time to familiarize the game settings, these difference in F is not only conceivable but

also acceptable. However, only obtain a couple of time plots of z3 cannot prove that there exists

a generally linear or nonlinear model across all subjects. Another perspective of view should be

considered to analyze the data.

15


(a) Estimated Position z1 Compare to Actual Position x1

(b) Estimated Result of Nonlinearity z3 with respect to Time t

Figure 2.3: Plot of Tracking Variable with respect to Time

16


(a) Original Phase Plot (b) 3D View of Phase Plot with Respect to Time t

Figure 2.4: Phase Plot of z3 with respect to z1 (Subject #9)

Here, considering human subject’s control effort z1 has a relationship with z3, since the

human nonlinearity might have a contribution to each change of mouse position, hence it is nec-

essary to investigate how G = dFdt ≈ z3 varies with respect to z1. As shown in Figure 2.4a, data

from subject #9’s multiple trials are plotted in one figure for a better comparison. Although it is

somewhat difficult to distinguish each trial from the figure, it is still visible from the figure that the

trials are similar in shape. To make the figure clearer and easier to recognize, the original phase

plot is converted into 3D view, as shown in Figure 2.4b. From this figure, it can be clearly seen that

for the derivatives of z1 and z3, even it is in different time, the ring shape patterns always have a

same size for the same derivative value of z1. This feature proves that z3 must be a function of z1.

Although it is not showing here, the points on the figure always begin with the origin and then over

time rotates clockwise from either left or right of the origin to form an infinitely shaped graph on

the plot.

Figure 2.5 shows all (except #9) subjects’ phase plot. It needs to be noted that the first

trial of all game set was deleted from the plot since the human subject may not be familiar with

the game setting and produce much more noise. Although the first trials are deleted and only one

subject’s data were showed here, this similar pattern actually exists among all trials and all subjects.

This similar pattern proves that human subjects may have a similar strategy when trying to control

the mouse to stabilize the pitch angle around the desired value. In other words, this indeed suggests

that a generalized model could potentially express the control intention of a set of subjects.

17


(a) Sub.6 (b) Sub.7 (c) Sub.8

(d) Sub.11 (e) Sub.12 (f) Sub.13

(g) Sub.14 (h) Sub.15 (i) Sub.16

Figure 2.5: Phase Plot of z3 with respect to z1 (All Subjects, Second-Order System)

18


2.3.2 Third-Order System

Similar to the second-order system, for human dynamics with the third-order system, the

equations are

x1(t) = x2(t)

x2(t) = x3(t)

x3(t) = F (x1(t), x2(t), x3(t), t) + u(t)

y(t) = x1(t).

(2.9)

The corresponding ESO’s formulation isz1(t) = z2(t)− β01e(t)z2(t) = z3(t)− β02h(e(t), 0.5, 005)

z3(t) = z4(t) + u(t)− β03h(e(t), 0.25, 005)

z4(t) = −β04h(e(t), 0.125, 005).

(2.10)

It should be noted that β04 here has no specific formulation. Its value needs to be tested

and selected according to the tracking result. Here, for the sake of simplicity in implementation,

β04 was selected as same as β03.

Figure 2.6: Comparison between Actual Position x1 and Estimated Position z1 Obtained Based on

the Three-Order System

Figure 2.6 shows the tracking mouse position results obtained through the three-order

state-space system. The plots were using the same data as Figure 2.3a. It may not easy to identify

19


the difference between Figure 2.3a and Figure 2.6. However, for the three-order system, the average,

maximum, and minimum error across the 10 subjects are 0.0150, 0.0454, and 0.0020 respectively.

The error of the tracking position becomes much larger for all trials. And it is also can be proven that

if increasing the value of β04, the error will also increase at the same time. However, if decreasing

the value of β04, the error will decrease to as same as second-order system.

Figure 2.7: Phase Plot of z3 with respect to z1 Obtained Based on the Three-Order System

Similar to the second-order system, Figure 2.7 shows the plot z3 varies with respect to z1

in third-order ESO system. It can be seen that in the third-order system, the pattern still exists but on

a larger scale. This is because of the value selection of β04. The scale will change as β04 changes,

but the shape of the pattern will always remain the same. Figure 2.8 is a similar phase plot to 2.5

except obtained based on the third-order system.

20


(a) Sub.6 (b) Sub.7 (c) Sub.8

(d) Sub.11 (e) Sub.12 (f) Sub.13

(g) Sub.14 (h) Sub.15 (i) Sub.16

Figure 2.8: Phase Plot of z3 with respect to z1 (All Subjects, Third-Order System)

21


2.3.3 Other Systems

Although only two systems’ results were presented here, other orders of systems were

also tested for the collected data. It can be concluded that with the orders of the system increased,

the scale of z3 will become larger, while z1 remains the same. Therefore, the pattern will always

be the ∞-like shape. However, one major problem for higher-order state-space system is that the

observer coefficients for ESO are harder to determine. As stated in the previous section, the error is

related to the coefficients.

22

Chapter 3

Modeling and Validation

Despite this human subjects experiment is based on a basic computer game, in which

the subject only needs to move the mouse in two directions, it can be seen from the collected

results that even for the same person with the same set of conditions, his or her performance will

be very different. However, in the previous chapter, it was shown that not only the same subject

can have the same control logic across all trials but also all subjects may have a similar control

strategy, such as subjects might be taking similar actions to prevent overshoot when the system

output is close to the reference line. No matter what it is, since human executes an action by taking

information and processing it, it can be assumed that the human brain is an advanced controller

which includes a linear part and a nonlinear part similar to engineered controllers. However, how to

obtain a mathematical model for such a controller is a very complicated problem.

Matlab provides a handful tool called “System Identification Toolbox”, which could easily

estimate a linear or nonlinear model from measured data. As shown in Figure 3.1, it is assumed that

human subject in the outer loop of MRAC system only receives a reference signal and a feedback

signal from uncertain dynamical system as inputs and outputs one command signal to the inner loop

after the reaction. Since collecting data from previous experiments includes all of these informa-

tion, it is possible to create a black-box model of human dynamics characteristics with the System

Identification Toolbox. However, it is unknown at this point whether the linear model or nonlinear

model could best fit human subjects. Therefore, it is necessary to check all data in order to obtain a

reasonable human model.

23

CHAPTER 3. MODELING AND VALIDATION

HumanDynamics(With Delay)

Reference InputControl Command

System Output

(Feedback from Uncertain Dynamical System)

Figure 3.1: Block Diagram of Human Dynamics

3.1 Fit Percentage

In order to compare and evaluate the human model obtained through the toolbox, a scoring

standard is needed. Here, the similarity between the output from a model and the output from its

estimated model is called the goodness of fit [53]. The goodness of fit is defined as follow

Fit = 1− ‖xo − x‖‖xo − xo‖

(3.1)

where xo is the measured data, and x is the estimated data. If the estimated data and the measured

data are the exact same, then the goodness of fit will be 1. However, if the estimated data and the

measured data are not similar, then the goodness of fit can reach as low as −∞.

3.2 Linear Identification

3.2.1 Linear Modeling

There are a couple of modifications needed on the data before using System Identifica-

tion Toolbox to create the linear model. The first one is choosing the input data for the Toolbox.

Although the architecture of MRAC indicates that the human subject should have two inputs, the

estimated transfer function in the linear model should be single-input/single-output. And it is con-

ceivable that the human subject will also comprehensively process the two inputs received on the

screen. It is assumed here that human subject unconsciously calculates the difference between the

green line and the blue curve, and uses this difference as the input signal to affect the subsequent

action. Hence, the real input data here is the difference obtained by subtracting the system output

from the reference.

Besides this, the input and output were also selected from three seconds to twenty seconds

of every single experiment, which is the actual playable time for human subjects. The delay effect

24


for human subjects was set to 0.355 seconds, which is the average reaction delay of all subjects. The

initial condition for all subjects was also set to be zero. A total of seven different transfer function

models were generated for comparison of data. By importing the experimental data to the toolbox

to generate the transfer function, each subject gets a total of 105 (15 experiments multiplied by 7

linear models) transfer functions. Then we input the experimental data to the transfer functions and

perform the simulation to get the estimated command in each game of each subject. This makes it

easy to compare the accuracy of each transfer function using the goodness of fit function, hence the

best transfer function can be found. It should be noted that the number of poles and zeros of the

seven transfer functions used in linear identification are 1, 2, 3, 4, 5, 10 and 50.

3.2.2 Validation of the Linear Model

(a) Identification Result: 0.858243 (b) Identification Result: -3.608000

Figure 3.2: Example of Linear Identification Results

In Figure 3.2, two linear identification examples are displayed. It is more consistent with

common sense that linear identification can have a better performance only when the original sub-

ject’s command does not change drastically. For each subject, Table 3.1 presents the best goodness

of fit result. It can be seen from the Table 3.1 that any kind of transfer function setting cannot

perform high-accuracy identification for all subjects and game sets.

As is shown in Table 3.1, most of the best results are with 10 poles and zeros in the transfer

function. However, due to the limitations of the identification setting, it is not clear that the 10 poles

and zeros are the best linear identification parameters. It only shows that larger the number of poles

and zeros, the better the result will be. But there still exists an upper limit. For example, the fit

25


Table 3.1: Best Linear Fit Result for Each Subject

Subject# Game Set# Poles and Zeros# Fit Result

6 2 10 0.858243

7 3 10 0.821355

8 3 10 0.827832

9 2 10 0.662257

11 2 10 0.713371

12 1 10 0.808103

13 3 5 0.732491

14 3 5 0.688763

15 2 10 0.778785

16 2 10 0.685117

results with 50 poles and zeros are very poor, as shown in the Table 3.2. Although some subject’s

best fit results still have very high reliability in 50 poles and zeros, most of the best fit results are far

below zero. This indicates that linear identification did not play any roles at all for most subjects.

In addition, in the case of 5 poles and zeros, the model fit results are more balanced, most of the

trials show relatively good fits, while the model fit results of 10 poles and zeros are very good for a

few trials, but others are much worse compared to 5 poles and zeros. However, this raises another

question: if keeping the transfer function exactly the same, even the parameters are not changed, can

these transfer function, which performs very well in some specific game set and subject situations,

also maintain the same good fit results when inputting the data of other trials?

Here, since the linear model must be obtained from any subject’s one of 15 trials by using

the toolbox, the trial that provides the linear model is referred to as the model provider; and the trial’s

data used as inputs to the linear model to obtain the estimated output are referred to as input provider.

So for the data obtained from the experiment, there will be a total of 150 model providers and 150

input providers. For each linear identification setting, there will be a total of 22500 results (150

model providers multiply 150 input providers). By testing all seven linear identification settings,

157,500 results were obtained. It can be shown that most linear models cannot provide convincing

results by using other input data, so it is impossible to find a specific one from these linear models

that can be used as a general model for all subjects. It can be said that linear identification has great

limitations since the linear identification can only provide some good results with the same model

26


Table 3.2: Best Fit Result for 5, 10, 50 Poles and Zeros

Subject#Game Set 1 Game Set 2 Game Set 3

5 10 50 5 10 50 5 10 50

6 0.607851 0.840400 0.707800 0.804027 0.858243 -0.145560 0.078609 -0.305280 -0.429790

7 0.455898 0.557423 -0.166580 0.632803 0.656694 0.026586 0.765277 0.821355 0.363655

8 0.654744 0.591755 -0.523110 0.630255 0.756752 -0.111200 0.648606 0.827832 0.165349

9 0.487156 0.542807 -0.174740 0.577812 0.662257 0.022285 0.622201 0.635379 -0.337100

11 0.302698 0.400903 0.088544 0.581586 0.713371 -0.139520 0.435145 0.162498 0.150495

12 0.598148 0.808103 0.048967 0.716204 0.756947 -0.290170 0.286177 0.482929 0.452977

13 0.581897 0.717434 -0.628730 0.648952 0.580645 -0.098760 0.732491 0.308928 0.375717

14 0.620784 0.651688 -0.008180 0.459837 0.322682 -0.006160 0.688763 0.597345 0.281751

15 0.609446 0.738331 -0.442210 0.673742 0.778785 -0.358170 0.723705 0.761559 0.100832

16 -0.017970 -0.140540 -0.206400 0.412845 0.685117 0.048715 0.492024 0.665914 -0.148340

provider and input provider.

3.3 Nonlinear Identification

3.3.1 Nonlinear Modeling

Matlab provides three nonlinear model structures: Nonlinear ARX models, Hammerstein-

Wiener models, Nonlinear grey-box model, see [54] for details. Hammerstein-Wiener models are

used for a system which has static nonlinearities. Nonlinear grey-box models are used for the

system which has known difference equations but without parameters. Nonlinear ARX models,

however, have the widest range of applications because the system will be modeled through dynam-

ics networks. Since the nonlinear identification case here has no known function or form to express,

therefore the only way to identify is utilizing the nonlinear ARX model.

In Matlab, the structure of a nonlinear ARX model is shown in Figure 3.3. The regressors

block and nonlinearity estimator block will act together as a model for simulating the subjects

decision process. The regressors for this experiment’s identification process are selected based on

the average reaction of subjects. Since the average reaction time is 0.355 seconds and the collected

data is in discrete time ∆t = 0.005 seconds, the regressors first considered are based on u(t− 71),

u(t − 72), y(t − 1), and y(t − 2), where u is the input to the subjects and y is the output of the

model. However, other possible delay timesteps from 0 to 500 are also been tested.

27


Regressors Output 𝑦𝑦Input 𝑢𝑢 Nonlinear

Function

LinearFunction

Nonlinear Estimator

Figure 3.3: Block Diagram of the Structure of the Nonlinear ARX Model [6]

For nonlinear estimator, its equation is given by [55]

F (x) = LT(x− r) + d+ g(Q(x− r)) (3.2)

where LT(x−r)+d represents linear function block. x is the vector of regressors, in our case here,

x = [u(t − 71), u(t − 72), y(t − 1), y(t − 2)], while r is the mean of the vector x. L is a linear

coefficient, and d is a scalar output offset. g(Q(x − r)) represents the nonlinear function block,

where g is the nonlinear function, while Q is a projection matrix. F (x) is the output of nonlinearity

estimator, and it is actually the same as y(t).

In Matlab, nonlinearity estimator block estimates the nonlinear model through many dif-

ferent estimator networks. Here, the wavelet network is selected to create the nonlinear model. The

wavelet network is described as the following function expansion [56]:

F (x) = (x− r)PL+ as1f (bs1 ((x− r)Q− cs1)) + . . .

+ asnsf (bsns ((x− r)Q− csns))

+ aw1 g (bw1 ((x− r)Q− cw1 )) + . . .

+ awnwg (bwnw ((x− r)Q− cwnw)) + d

(3.3)

where f(z) = e−0.5zzT

is the scaling function, and g(z) =(m− zzT

)e−0.5zz

Tis the wavelet

function, while z is a vector input to the function. P stands for a different projection matrix than Q.

as and bs are scaling coefficients, while cs is the scaling vector. Similarly, aw, and bw are wavelet

coefficients, while cw is the wavelet vector. Depending on the settings of the nonlineariry estimator,

the number of scaling and wavelet coefficients can reach tens or even hundreds. Here, the total

number of scaling and wavelet coefficients are selected as 100. The rest of the symbols have the

same meaning as (3.2).

28


3.3.2 Validation of the Nonlinear Model

Figure 3.4a shows an example of the estimated output (blue) from nonlinear identification

and the real output (grey) from a human subject, as well as the goodness of fit result with percentage

unit. It can be seen from the figure that the raw nonlinear model output contains a lot of oscillations.

Therefore, to reduce these high frequency oscillations in the output, an extra filter is needed to

process the output of the nonlinear model. Here, a discrete-time low-pass filter is introduced as

y[n] = y[n− 1] +(x[n− 1]− y[n− 1])∆t

T(3.4)

where T = 0.025s and ∆t = 0.005s.

(a) Before Filtering (b) After Filtering

Figure 3.4: The Effect of Applied Filter to Nonlinear Identification Result

Figure 3.4b is the identification result after filtering. It can be seen that after filtering, the oscillations

reduced and the goodness of fit results are also improved.

When estimating the nonlinear model of subjects’ decisions, it can be shown that the result

of the goodness of fit becomes poor compared to the linear identification, if the model provider and

the input provider are the same subject. Table 3.3 shows the best goodness of fit of nonlinear

identification for each subject as the model provider under the settings of 71 timesteps delay. Here

for the sake of convenience, the data of game set 1 are called as the 1st to the 5th trials in the order

of recording, and the data of game set 2 are called the 6th to 10th trials. The data of set 3 are called

the 11th to 15th trials. It can be seen that as a model provider, the best fit results in most cases are

not produced with itself as input provider. And the model, for example, comes from game set 1 does

29


Table 3.3: Best Fit Result for Each Subject as Model Provider

Model ProviderFit Result for

Model ProviderInput Provider

Fit Result for Input

Provider

Sub.6 9th trial −0.804400 Sub.8 9th trial 0.399306




Sub.11 1st trial 0.472376 Sub.11 1st trial 0.472376



Sub.14 13th trial 0.318348 Sub.14 11th trial 0.406755

Sub.15 1st trial −0.759880 Sub.7 15th trial 0.553569


not necessarily have a good estimation result only when the input also comes from the same game

set, but the input from other game sets can also simulate a good result.

This is a very interesting phenomenon since the estimation result shows that importing the

input data to another model rather than itself will get a better result. Recall the linear identification’s

result, which fits well when the model provider and input provider are the same. The results of

nonlinear identification indicate that the nonlinear model obtained through System Identification

Toolbox actually contains more common factors that exist among all subjects, rather than each

subjects’ unique factors.

Table 3.4: Nonlinear Fit Results for a Given Delay in Timesteps

DelaySubject #

6 7 8 9 11 12 13 14 15 16

0 0.459561 0.375461 0.414429 0.398204 0.218329 0.334941 0.429534 0.524615 0.392828 0.278882

25 0.376060 0.396565 0.398901 0.366676 0.275598 0.470890 0.400084 0.534543 0.404611 0.244987

50 0.430955 0.416826 0.382706 0.287123 0.265373 0.394275 0.485902 0.429032 0.439614 0.253626

60 0.489254 0.397344 0.383052 0.351220 0.244491 0.438561 0.385082 0.392779 0.512777 0.392096

71 0.399306 0.439101 0.411243 0.401610 0.472376 0.412803 0.459194 0.406755 0.553569 0.364914

80 0.477761 0.456510 0.376196 0.412458 0.297346 0.425203 0.495331 0.421498 0.481258 0.361516

100 0.395634 0.365415 0.411243 0.401610 0.321410 0.406848 0.494069 0.406755 0.403182 0.394753

125 0.474224 0.365600 0.371908 0.450870 0.343916 0.383740 0.483663 0.431198 0.459535 0.386639

150 0.418020 0.400183 0.407227 0.420004 0.287910 0.409706 0.470098 0.415430 0.301897 0.267269

175 0.650060 0.486649 0.444235 0.315393 0.277429 0.374445 0.437013 0.412787 0.276025 0.458324

200 0.376537 0.329295 0.401495 0.294265 0.315413 0.366868 0.471194 0.399183 0.417840 0.397248

300 0.285547 0.498537 0.326338 0.226003 0.190628 0.374089 0.339326 0.353936 0.206401 0.275470

500 0.251686 0.180851 0.253274 0.208480 0.155381 0.379296 0.149751 0.194417 0.130350 0.177291

30


Table 3.5: Each Subject’s Total Nonlinear Fit Numbers for a Given Delay in Timesteps

DelaySubject #

6 7 8 9 11 12 13 14 15 16

0 5 19 22 14 0 19 14 21 5 3

25 6 10 21 9 1 10 16 37 11 0

50 9 17 17 4 1 19 31 39 12 1

60 13 19 14 18 0 22 34 28 13 6

71 20 37 28 26 3 25 34 33 14 8

80 20 38 24 22 1 18 43 39 10 12

100 22 23 26 26 3 28 37 33 4 13

125 23 33 19 26 4 22 35 35 13 17

150 26 21 20 32 4 32 27 21 3 3

175 17 27 15 6 1 13 18 13 1 17

200 9 7 15 1 1 11 19 12 5 6

300 1 6 1 0 0 6 8 3 0 1

500 1 0 1 0 0 2 0 0 0 0

The first attempt of setting for nonlinear identification was using 71 samples of delay

as regressors. However, this setting shows that the goodness of fit results are not good enough

compared to linear identification. Since the delay setting was based on the average reaction time, it

is a question that whether if this poor fit could be solved by using other settings of delay timesteps.

Table 3.4 shows the highest goodness of fit results for different subjects with various of different

delay timesteps. The first two high of fit for each subject are highlighted. The highest fit result in

the table is 0.650060, which is 175 delay samples from 11th trial of subject 6 (same model provider

and input provider). It can be seen from these results that the overall estimation performance of

nonlinear identification is indeed much lower than linear identification. Table 3.5 shows the total

number of good fit results with different delay samples, and it includes both the same provider

situation and the different provider situation. Similary, the first two high of fit numbers for each

subject are also highlighted. The results in Table 3.5 indicates that nonlinear identification’s models

indeed reflected the real condition of human subjects. In other words, the peak of the total fits

numbers occur between approximately 71 and 150 delay samples for each subject suggest that when

subjects played the Matlab game, the actual reaction delay for subjects should also be around 71 to

150 samples, which is 0.355 to 0.750 seconds. However, due to the poor performance of nonlinear

identification through System Identification Toolbox, the fit results are still not a good indication

that these nonlinear models can be used reliably.

31

Chapter 4

Prediction of Human Behavior

4.1 Direct Prediction

Here, direct prediction means comparing both estimated human commands and estimated

uncertain dynamical system output directly with the original output from the experiment.

(a) Linear Model (b) Nonlinear Model

Figure 4.1: Command Error Comparison

Previous chapter showed that some of the linear models and nonlinear models have pretty

good fit results for subjects’ command under specific conditions. Figure 4.1 shows the error of

command from two models obtained through System Identification Toolbox. The model in Figure

4.1a is a linear model from the 8th trial of subject 6. The model in Figure 4.1b is a nonlinear model

32

CHAPTER 4. PREDICTION OF HUMAN BEHAVIOR

from the 11th trial of subject 6. Both of them are two best-performing models in their respective

identification categories.

Figure 4.2a and 4.2b are the system output obtained by applying two models into the

human-in-the-loop MRAC system, and the corresponding error of output. The linear model’s system

output seems promising since the error is relatively low (less than 0.025). However, for the nonlinear

model, the system output becomes unstable. As shown in Figure 4.2b, the output value exceeds +5/-

1 and rapidly reached negative infinity after 14 seconds.

The results in Figure 4.2 are achieved by using the same model provider and input provider.

If we change the input provider to other trials or subjects, the goodness of fit results will be no doubt

affected. The errors in Figure 4.3 and 4.4 are the results obtained through different input providers.

It should be noted that the two input providers are randomly chosen, and the estimated commands

are then imported to the human-in-the-loop MRAC system to substitute into human subjects and

obtain the system output results. The errors in these two figures, especially command errors suggest

that the difference between the real and estimated subject’s command are quite large. The errors

can even reach 3 unit at some point, knowing that the reference and input difference at the start of

the game is only 1 unit, therefore 3 unit difference is a huge error. The system’s output errors also

indicate that when importing other inputs, identification models are difficult to match the results of

the system output obtained by the original command. Figure 4.5 presents the comparison of sys-

tem outputs. It can be seen that the estimated output either has an overshoot or becomes unstable

over time. Therefore, it is impossible to predict the output of the human-in-the-loop MRAC system

directly by applying linear or nonlinear models.

33


(a) Linear Model

(b) Nonlinear Model

Figure 4.2: System Output and Output Error Comparison

34


(a) Linear Model Result

(b) Nonlinear Model Result

Figure 4.3: Error Comparison between the Linear Model and the Nonlinear Model (Case 1)

35


(a) Linear Model

(b) Nonlinear Model

Figure 4.4: Error Comparison between the Linear Model and the Nonlinear Model (Case 2)

36


Figure 4.5: Results of System Output with Two Different Input Provider Cases

37


4.2 Indirect Prediction

4.2.1 Data Processing

In contrast to the direct prediction, the indirect prediction means estimated output from

models need to be processed before comparing with the original output from the experiment. Figure

4.5 in the last section indicates that direct prediction may not be the best way to generate the model

of human subjects. But from the figure, one interesting pattern can be observed that the estimated

outputs seem to have similar growth and decrease tendency compared to the original outputs. So

it is worthwhile to calculate the derivative of the estimated and subject’s real command, and then

compare them.

Figure 4.6: Comparison of Control Commands from Linear Identification (From left to right: Case

1 and Case 2)

As shown in Figure 4.6, the real and estimated control command from linear identification

are drawn in the same figure for the sake of comparison. Clearly from Figure 4.6 the estimated com-

mand and the actual command from the subject do have a nearly simultaneous increase and decrease

tendency. However, for the actual derivative of command, since the human subject was moving the

mouse in a highly nonlinear regime during the play process, the absolute value of derivatives can

be extremely large. Therefore, both the real and estimated derivatives here should be normalized:

when the derivative is positive, set its value to 1, and when the derivative is negative, set its value to

38


Figure 4.7: Derivative of Commands and Its +1/-1 Plot (Case 1)

-1. To simplify, here only one case is demonstrated. The derivatives of actual human commands and

its +1/-1 plot are shown in Figure 4.7. Due to the fact that the human subject motion was recorded

with some discontinuity during the simulation game, the corresponding derivatives are also affected

and became discontinuous. To further eliminate the discontinuous of +1/-1 plot of derivatives, an-

other optimization attempt should be made. Here, the basic idea is to try to minimize the oscillation

number of the derivative value between zero and positive or negative in a specific period of time

(such as the oscillation between 2s to 4s in the right figure). To do this, all the zero between this

specific period should be set to equal to its adjacent value. As shown in Figure 4.8, the thick blue

line is the +1/-1 plot of derivatives after processing, while the slim orange one is the original. It can

be seen that the thick blue line is basically the approximate outer contour of the orange one after

ignoring the small disturbance.

4.2.2 Prediction of the Linear Model

With the modified positive/negative derivative plot of subject’s command, the estimated

command can be added without further process and is easily compared with, as shown in Figure 4.9.

The meaning of the blue line is still the same, which represents the modified subject’s command

derivative, while the red dot alongside the blue line is the estimated command derivative from linear

identification. In the prediction result figure, if the segment of red dots and the blue-line segment

39


Figure 4.8: Optimization Result for +1/-1 Plot of Derivatives

are highly coincident, then it could be said that this model has a perfect estimation ability for the

derivative of the command. As for the example case in Figure 4.9a, although red-dot segments are

about a second earlier than the blue-line segment, red-dot segments still match the change of signs

in the blue-line segments, which is still a promising result for prediction.

Figure 4.9b shows the prediction result of Case 2. As it can seen from the figure, Case 2’s

subject commands are significantly different from Case 1. Case 1’s subject commands are almost

alternately varying through entire playing time, and after each change of derivative’s sign, the sign

will remain the same for a while (normally larger than 1 second). Meanwhile, for Case 2, the sign

only changes in the first half playing time, and also almost every change is instantaneous. Still, the

prediction result for case 2 is convincing since the estimated signs of derivative matches the actual

data in the first half.

To further investigate the results of case 1 and 2, another two input providers were used

to obtain the prediction and compare with case 1 and case 2, as shown in Figure 4.10. Also as

mentioned in the last section, case 1 is selecting 15th trial data of subject 7 as input provider, while

case 2 is selecting 2nd trial of subject 8 as input provider. Here case 3 is derived from the 15th

trial data of subject 8, while case 4 is derived from the 2nd trial data of subject 7. From Figure

40


(a) Case 1 (b) Case 2

Figure 4.9: Indirect Prediction Results of Linear Models (Case 1 and 2)

4.10 it can be seen that both prediction results in case 3 and 4 are still convincing enough. Besides,

the derivatives of actual control commands also share the same patterns in case 1 and 3, as well as

case 2 and 4. The first two have patterns of alternating sign change, and changes last for at least

a second, while the last two both have a much smaller time of duration of the derivative changes.

After more comparisons, it can be concluded that these similarities were caused by the difference

in the game set condition. Since the game set 1 is actually more sensible than the set 3, subjects

will have two control strategy in set 1, namely: either excessively and frequently moving the slider

bar during the entire playtime, or slowly moving the slider bar to a balanced position where system

output is close to the reference line and then no longer intervening. However, for the game set 3,

because the response is much slower, the subjects will put more effort to move the slider through

time. Also, due to the slow response of set 3, the control effort of subjects often leads to a continuous

overshooting, which is the cause of alternating commands, therefore it is difficult to reach the stable

states for dynamical system output at the end. As for prediction purposes, it can be seen that the

more alternating the subjects’ commands are, the higher the accuracy of the identification model’s

prediction.

41



Figure 4.10: Indirect Prediction Results of Linear Models (Case 3 and 4)

4.2.3 Prediction of the Nonlinear Model

Since the high fit result of the nonlinear model does not necessarily mean the best predic-

tion, after comparison, the model obtained from the 12th trial of Subject 7 under the setting of 80

delay samples is selected as the indirect prediction’s nonlinear model.

Similar to the steps in the previous section, the prediction results in Figure 4.11 are ob-

tained by calculating the derivative of the estimated commands through the nonlinear model. It can

be seen from Figure 4.11a that the derivatives of estimated commands not only match the patterns of

actual data but also change signs at the same time with the actual situation, which is a much better

result compared to the linear model. However, as seen in Figure 4.11b, for the cases which subjects

manage to achieve the stable system output, the accuracy of prediction can be poor.

Figure 4.12 shows the nonlinear model’s prediction result for case 3 and 4. Unsurpris-

ingly, both cases with alternating characteristics show better prediction results than using linear

model: estimated and actual’s sign of derivative not only has a great match for most of the positive

or negative value but also predicts the right time period for each sign’s change. It needs to be noted

that because of the initial condition setting for the estimation process, the actual starting point for

nonlinear estimation is beginning from 1 second later of the game start. Therefore all estimated

values are set to zero before 1 second.

42



Figure 4.11: Indirect Prediction Results of Nonlinear Models (Case 1 and 2)


Figure 4.12: Indirect Prediction Results of Nonlinear Models (Case 3 and 4)

43


Although not all prediction results are shown, it is certain that this nonlinear model pre-

diction method can provide similar prediction results as in Figure 4.11 and 4.12, if it meets the

conditions of 1) dynamical system does not stabilize yet; 2) the sign changes of commands deriva-

tive are nearly alternating. Except for the accurate prediction of the nonlinear model, another major

pro of the nonlinear model is that it can easily be transformed for use in real-time prediction. Since

the nonlinear model is actually a function of complex finite number of coefficients and it only takes

regressors as inputs, the function can be added to the human-in-the-loop MRAC system and acquire

the human subject’s intention of action 70 to 100 samples in advance. Therefore, it is conceivable

that if there is a suitable algorithm that can combine other characterization data with this prediction

model and calculate the possibility of the subject’s excessive control behavior, then the additional

methods can be used to either remind the human subject to be careful of the excessive action or take

a rapid action to assist the subject to stabilize the dynamical system.

44

Chapter 5

Conclusion

5.1 Concluding Remarks

The main contribution of this thesis is a series of analysis of the data obtained from pre-

vious human-in-the-loop MRAC experiments. It can be found from the obtained data that subjects

have different ways to handle the control task, and not all subjects can guarantee to stabilize the

uncertain dynamical system within 20 seconds even with the help of the controller. Three sets of

poles setting used in the reference model of MRAC in the experiment make the dynamical system

response from more sensitive to less sensitive. Although the sensitive controller setting is more

likely to form a stable result, on the other hand, it will also cause the highest overshoot. Similarly,

although a slower controller setting is more difficult to control, if enough control effort is put into

it, the system output will gradually approach the stable status. Therefore, it can be seen from the

experimental data that only relying on the adjustment of the controller setting does not guarantee

that the system has the best control effect for the human-in-the-loop situation.

In Chapter 2, the idea proposed in the ADRC system was adopted. By assuming that the

subjects’ control behavior can be represented by a set of state-space equations, then the unknown

function F, which consists of both linearity and nonlinearity of human behavior, can be estimated

according to ESO. In this paper, both second-order and third-order state space are tested respectively,

and the corresponding estimated variables are obtained. By analyzing the estimated variables, it can

be found that there is a certain pattern between the derivative of the estimated value of the unknown

equation F , z3, and the derivative of estimated mouse position, z1. Since this pattern is ubiquitous

between all trials of all subjects, a generalized model could potentially express the control intention

of a set of subjects.

45

CHAPTER 5. CONCLUSION

In Chapter 3, a set of linear models and nonlinear models were obtained through the

System Identification Toolbox provided in Matlab, based on the collected data. Although linear

models have the highest fit results compared to nonlinear models, this can only prove that the linear

models will have a good match if the changes in the cursor’s position are relatively small. In

addition, the modeling results also show that using different modeling parameters have a great

impact on the fit results. Since the average reaction time among all subjects is measured as 0.355

seconds, this number is set as the delay parameter of the model in the modeling settings. However,

the existing pattern in both the highest fit results table and total fit numbers table in nonlinear

validation indicate that the reaction time of subjects in this experiment should be actually roughly

between 0.355 and 0.750 seconds.

Chapter 4 uses the best linear and nonlinear model obtained in Chapter 3 to perform

human behavior predictions on experimental data from other trials or subjects. The results show

that neither linear nor nonlinear model can directly predict human behavior since the estimated

value is far from the actual value. At the same time, the best performing nonlinear model in Chapter

3 could generate too many oscillations in direct prediction. Therefore, another nonlinear model

should be adopted and used for further prediction attempt. In the indirect prediction, it is found that

if we take the derivative of both actual and estimated cursor position and then convert them into

+1/-1 forms, the prediction result can be much more accurate. Both linear and nonlinear models

can better predict the future control behavior of subjects, while the prediction of nonlinear model is

more synchronous with the actual behavior change. Therefore, it is worthy of being selected as a

human behavior prediction model for future optimization of the human-in-the-loop MRAC system.

5.2 Future Work

Due to the limitation of time, this thesis mostly focused on the effort of the acquisition

and verification of the model. However, there is still a lot of work to be investigated, such as:

• To show that there exists a generalized model between all the subjects in the experiment, a

simplified state-space system is assumed, and the corresponding variables are estimated by

using ESO. Although the result of estimate proves that z3 should be at least a function of z1,

and it is obvious that z3 should also have other independent variables excepts z1. Since this

thesis did not explore other possible independent variables for z3, the subsequent work can

start with the ESO estimation results and further analyze the characteristics of z3.

46

CHAPTER 5. CONCLUSION

• For human behavior modeling, the model obtained in this article comes from the System

Identification Toolbox of Matlab. Although it has a good prediction result to some extent,

due to the wavelet function, the stability of the model is difficult to be calculated. Therefore,

these models are limited to predict the possibility of the control behavior of the subjects under

certain conditions. So other mature methods of obtaining the behavior models should also be

considered.

• In this thesis, only some promising prediction models are obtained and used to predict the

control behavior of the subjects, while no specific quantitative standard is given to measure

the accuracy of the prediction. Although it can be seen that the accuracy is good enough by

visual observation of the comparison, an accuracy score of the model should be provided,

if the model needs to be substituted into the actual control system, so that the system can

distinguish different states of estimated human behavior.

• Based on the prediction model obtained in this paper, a comprehensive algorithm can be

developed after the fusion of other characteristic features of the system, specifically for iden-

tifying the excess behavior of the subjects. According to the calculated possibility of the

future excessive behavior, controller switching may be adopted to prevent the incident so that

the uncertain dynamical system is as stable as possible.

47

REFERENCES

[1] Y. Wang, “Human performance on model reference adaptive control systems with human re-

action delay,” Master’s thesis, Northeastern University, 2018.

[2] J. Han, “From pid to active disturbance rejection control,” IEEE transactions on Industrial

Electronics, vol. 56, no. 3, pp. 900–906, 2009.

[3] T. Yucelen, Y. Yildiz, R. Sipahi, E. Yousefi, and N. Nguyen, “Stability limit of human-in-

the-loop model reference adaptive control architectures,” International Journal of Control, pp.

1–18, 2017.

[4] Federal Aviation Administration, “Illustration of pitch effects,” accessed: 2019-

08-05. [Online]. Available: https://lessonslearned.faa.gov/ChinaAirlines140/Pitch Effects

Illustration.jpg

[5] Y. Lu, R. Sipahi, Y. Wang, and T. Yucelen, “Observer-based design to mathematically model

human behavior in a cursor tracking game,” 2019, submitted.

[6] The MathWorks, Inc., “Structure of nonlinear arx models,” accessed: 2019-08-05. [Online].

Available: https://www.mathworks.com/help/ident/ug/nonlin nlarx.png

[7] A. E. Bryson, Control of spacecraft and aircraft. Princeton university press Princeton, New

Jersey, 1993.

[8] L. Xiong, G. Teng, Z. Yu, W. Zhang, and Y. Feng, “Novel stability control strategy for dis-

tributed drive electric vehicle based on driver operation intention,” International journal of

automotive technology, vol. 17, no. 4, pp. 651–663, 2016.

48

https://lessonslearned.faa.gov/ChinaAirlines140/Pitch_Effects_Illustration.jpg

https://lessonslearned.faa.gov/ChinaAirlines140/Pitch_Effects_Illustration.jpg

https://www.mathworks.com/help/ident/ug/nonlin_nlarx.png

REFERENCES

[9] L. Feng, C. Wiltsche, L. Humphrey, and U. Topcu, “Synthesis of human-in-the-loop control

protocols for autonomous systems,” IEEE Transactions on Automation Science and Engineer-

ing, vol. 13, no. 2, pp. 450–462, 2016.

[10] P. Van Overloop, J. Maestre, A. D. Sadowska, E. F. Camacho, and B. De Schutter, “Human-

in-the-loop model predictive control of an irrigation canal [applications of control],” IEEE

Control Systems Magazine, vol. 35, no. 4, pp. 19–29, 2015.

[11] T. Wenqian, L. Zongyuan, Q. Xiangju, W. Yanyang, and S. Liguo, “Aircraft-pilot system mod-

eling and pilot control behavior research for airdrop task,” IFAC-PapersOnLine, vol. 49, no. 32,

pp. 177–182, 2016.

[12] O. Celik and S. Ertugrul, “Predictive human operator model to be utilized as a controller using

linear, neuro-fuzzy and fuzzy-arx modeling techniques,” Engineering Applications of Artificial

Intelligence, vol. 23, no. 4, pp. 595–603, 2010.

[13] J. Tang, Q. Zhao, and R. Yang, “Stability control for a walking-chair robot with human in the

loop,” International Journal of Advanced Robotic Systems, vol. 6, no. 1, p. 3, 2009.

[14] R. A. Hess, “Modeling human pilot adaptation to flight control anomalies and changing task

demands,” Journal of Guidance, Control, and Dynamics, vol. 38, no. 6, pp. 655–666, 2015.

[15] T. Hatanaka, N. Chopra, and M. Fujita, “Passivity-based bilateral human-swarm-interactions

for cooperative robotic networks and human passivity analysis,” in 54th IEEE Conference on

Decision and Control (CDC), Osaka, Japan, 2015.

[16] D. Lee and M. W. Spong, “Passive bilateral teleoperation with constant time delay,” IEEE

transactions on robotics, vol. 22, no. 2, pp. 269–281, 2006.

[17] M. Xia, A. Rahnama, S. Wang, and P. J. Antsaklis, “On guaranteeing passivity and perfor-

mance with a human controller,” in IEEE 23rd Mediterranean Conference on Control and

Automation (MED), Torremolinos, Spain, 2015.

[18] J. B. Witte, “An investigation relating longitudinal pilot-induced oscillation tendency rating to

describing function predictions for rate-limited actuators,” Air Force Institute of Technology,

Wright-Patterson Air Force Base, Ohio, Tech. Rep., 2004.

[19] A. J. Thurling, “Improving UAV handling qualities using time delay compensation,” Air Force

Institute of Technology, Wright-Patterson Air Force Base, Ohio, Tech. Rep., 2000.

49

REFERENCES

[20] E. Yousefi, Y. Yildiz, R. Sipahi, and T. Yucelen, “The effect of remote-signal feedback archi-

tectures on the stability of human-in-the-loop telerobotics in the presence of time-delays,” in

IEEE Conference on Control Technology and Applications (CCTA), Kohala Coast, Hawai’i,

2017.

[21] S. Horl, “Agent-based simulation of autonomous taxi services with dynamic demand re-

sponses,” Procedia Computer Science, vol. 109, pp. 899–904, 2017.

[22] R. E. Yagoda and M. D. Coovert, “How to work and play with robots: an approach to modeling

human–robot interaction,” Computers in Human Behavior, vol. 28, no. 1, pp. 60–68, 2012.

[23] G.-J. M. Kruijff, M. Janıcek, S. Keshavdas, B. Larochelle, H. Zender, N. J. Smets, T. Mioch,

M. A. Neerincx, J. Diggelen, F. Colas et al., “Experience in system design for human-robot

teaming in urban search and rescue,” in Field and Service Robotics. Springer, 2014, pp.

111–125.

[24] M. Saida, J. R. Medina, and S. Hirche, “Adaptive attitude design with risk-sensitive optimal

feedback control in physical human-robot interaction,” in 2012 IEEE RO-MAN: The 21st IEEE

International Symposium on Robot and Human Interactive Communication. IEEE, 2012, pp.

955–961.

[25] T. Yucelen, Y. Yildiz, R. Sipahi, E. Yousefi, and N. Nguyen, “Stability limit of human-in-the-

loop model reference adaptive control architectures,” International Journal of Control, vol. 91,

no. 10, pp. 2314–2331, 2018.

[26] V. A. Shia, Y. Gao, R. Vasudevan, K. D. Campbell, T. Lin, F. Borrelli, and R. Bajcsy, “Semiau-

tonomous vehicular control using driver modeling,” IEEE Transactions on Intelligent Trans-

portation Systems, vol. 15, no. 6, pp. 2696–2709, 2014.

[27] A. Efremov, M. Tjaglik, U. Tiumentzev, and T. Wenqian, “Pilot behavior modeling and its

application to manual control tasks,” IFAC-PapersOnLine, vol. 49, no. 32, pp. 159–164, 2016.

[28] G. Zhao and C. Wu, “Mathematical modeling of driver speed control with individual differ-

ences,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 43, no. 5, pp.

1091–1104, 2013.

50

REFERENCES

[29] M. Li, H. Cao, X. Song, Y. Huang, J. Wang, and Z. Huang, “Shared control driver assistance

system based on driving intention and situation assessment,” IEEE Transactions on Industrial

Informatics, vol. 14, no. 11, pp. 4982–4994, 2018.

[30] R. B. Warrier, “Inferring human intent in novice human-in-the-loop control tasks,” Ph.D. dis-

sertation, 2018.

[31] D. Lee, A. Hansen, and J. K. Hedrick, “Probabilistic inference of traffic participants’ lane

change intention for enhancing adaptive cruise control,” in 2017 IEEE Intelligent Vehicles

Symposium (IV). IEEE, 2017, pp. 855–860.

[32] R. Li, Y. Li, S. E. Li, E. Burdet, and B. Cheng, “Indirect shared control of highly au-

tomated vehicles for cooperative driving between driver and automation,” arXiv preprint

arXiv:1704.00866, 2017.

[33] L. Li, Z. Zhu, Y. Chen, K. He, X. Li, and X. Wang, “Engagement control of automated clutch

for vehicle launching considering the instantaneous changes of driver’s intention,” Journal of

Dynamic Systems, Measurement, and Control, vol. 139, no. 2, p. 021011, 2017.

[34] S. Bae, S. M. Han, and S. Moura, “Modeling & control of human actuated systems,” IFAC-

PapersOnLine, vol. 51, no. 34, pp. 40–46, 2019.

[35] D. T. McRuer and H. R. Jex, “A review of quasi-linear pilot models,” IEEE transactions on

human factors in electronics, no. 3, pp. 231–249, 1967.

[36] D. T. McRuer and E. S. Krendel, “Mathematical models of human pilot behavior,” Advisory

Group for Aerospace Research and Development, Paris, Tech. Rep., 1974.

[37] A. Tustin, “The nature of the operator’s response in manual control, and its implications for

controller design,” Journal of the Institution of Electrical Engineers-Part IIA: Automatic Reg-

ulators and Servo Mechanisms, vol. 94, no. 2, pp. 190–206, 1947.

[38] G. Markkula, “Modeling driver control behavior in both routine and near-accident driving,”

in Proceedings of the human factors and ergonomics society annual meeting, vol. 58, no. 1.

SAGE Publications Sage CA: Los Angeles, CA, 2014, pp. 879–883.

[39] R. A. Hess, “A model for pilot control behavior in analyzing potential loss-of-control events,”

Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engi-

neering, vol. 228, no. 10, pp. 1845–1856, 2014.

51

REFERENCES

[40] ——, “Simplified approach for modelling pilot pursuit control behaviour in multi-loop flight

control tasks,” Proceedings of the Institution of Mechanical Engineers, Part G: Journal of

Aerospace Engineering, vol. 220, no. 2, pp. 85–102, 2006.

[41] A. J. Pick and D. J. Cole, “A mathematical model of driver steering control including neuro-

muscular dynamics,” Journal of Dynamic Systems, Measurement, and Control, vol. 130, no. 3,

p. 031004, 2008.

[42] T. Qu, H. Chen, D. Cao, H. Guo, and B. Gao, “Switching-based stochastic model predictive

control approach for modeling driver steering skill,” IEEE Transactions on Intelligent Trans-

portation Systems, vol. 16, no. 1, pp. 365–375, 2014.

[43] R. Mori and S. Suzuki, “Modeling of pilot landing approach control using stochastic switched

linear regression model,” Journal of Aircraft, vol. 47, no. 5, pp. 1554–1558, 2010.

[44] R. Hess and A. Modjtahedzadeh, “A control theoretic model of driver steering behavior,” IEEE

Control Systems Magazine, vol. 10, no. 5, pp. 3–8, 1990.

[45] Y. Zhang, H. Li, and R. Huang, “Investigation into pilot control model base on anfis for

teleoperation rendezvous and docking,” in 2016 IEEE Advanced Information Management,

Communicates, Electronic and Automation Control Conference (IMCEC). IEEE, 2016, pp.

1921–1926.

[46] C. Chen, W.-Q. Tan, X.-J. Qu, and H.-X. Li, “A fuzzy human pilot model of longitudinal

control for a carrier landing task,” IEEE Transactions on Aerospace and Electronic Systems,

vol. 54, no. 1, pp. 453–466, 2017.

[47] Y. Tan, H. Shen, M. Huang, and J. Mao, “Driver directional control using two-point preview

and fuzzy decision,” Journal of Applied Mathematics and Mechanics, vol. 80, no. 6, pp. 459–

465, 2016.

[48] L. Bi, J. Shang, G. Gan, and Y. Liu, “Modeling driver vehicle control in a cognitive archi-

tecture,” in 2011 Third Pacific-Asia Conference on Circuits, Communications and System

(PACCS). IEEE, 2011, pp. 1–4.

[49] R. Mori, “Pilot control modeling with stochastic periodical discrete movement,” IEEE Trans-

actions on Aerospace and Electronic Systems, vol. 54, no. 2, pp. 912–922, 2017.

52

REFERENCES

[50] I. D. Landau, R. Lozano, M. M’Saad, and A. Karimi, Adaptive control: algorithms, analysis

and applications. Springer Science & Business Media, 2011.

[51] D. Schmidt and B. Bacon, “An optimal control approach to pilot/vehicle analysis and the neal-

smith criteria,” Journal of Guidance, Control, and Dynamics, vol. 6, no. 5, pp. 339–347, 1983.

[52] “Reaction Time Test - Human Benchmark,” https://www.humanbenchmark.com/tests/

reactiontime, [Online; accessed 10-June-2019].

[53] The MathWorks, Inc., “goodnessOfFit,” https://www.mathworks.com/help/ident/ref/

goodnessoffit.html, [Online; accessed 09-Aug-2019].

[54] ——, “Nonlinear Model Identification,” https://www.mathworks.com/help/ident/

nonlinear-model-identification.html?s tid=CRUX lftnav, [Online; accessed 09-Aug-2019].

[55] ——, “What are Nonlinear ARX Models?” https://www.mathworks.com/help/ident/ug/

what-are-nonlinear-arx-models.html, [Online; accessed 09-Aug-2019].

[56] ——, “wavenet,” https://www.mathworks.com/help/ident/ref/wavenet.html, [Online; accessed

09-Aug-2019].

53

https://www.humanbenchmark.com/tests/reactiontime

https://www.humanbenchmark.com/tests/reactiontime

https://www.mathworks.com/help/ident/ref/goodnessoffit.html

https://www.mathworks.com/help/ident/ref/goodnessoffit.html

https://www.mathworks.com/help/ident/nonlinear-model-identification.html?s_tid=CRUX_lftnav

https://www.mathworks.com/help/ident/nonlinear-model-identification.html?s_tid=CRUX_lftnav

https://www.mathworks.com/help/ident/ug/what-are-nonlinear-arx-models.html

https://www.mathworks.com/help/ident/ug/what-are-nonlinear-arx-models.html

https://www.mathworks.com/help/ident/ref/wavenet.html

Human Behavior Modeling in a Cursor Tracking Game · Human Behavior Modeling in a Cursor Tracking...

Documents

Transcript of Human Behavior Modeling in a Cursor Tracking Game · Human Behavior Modeling in a Cursor Tracking...