System Identification: Black Box
Robert GreplInstitute of Solid Mechanics, Mechatronics and Biomechanics
Faculty of Mechanical Engineering, Brno University of Technology
2010 - 2015
Black box - agenda
1. Prerequisities: Least squares.
Discrete dynamic system.
2. How to use LS to estimate parameters of DDS? Example: mechanical oscillator
3. Problem: Noise.
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
2
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
3
Methods: Least Squares 1/2
generally: LS is Linear Optimization method
assume function
(linear)(ordinary) LS requires linear parametrization:example:
consider m measurements... written in matrix form:
if m=n, then Problem: Find parameters of
line if 2 points are given.
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
4
Methods: Least Squares 2/2
in real apps, the measurement is not ideal
task is: how to solve this overdetermined problem? (m equations and n unknowns)
answer: minimize criterion
result:
Note: Penrose pseudoinversion
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
5
LS example: regression 1D function
fit 1D function
(this task = the fitting of static model linear in parameters)
noise added to measurement
regression matrix:
[E015]
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
t
y
noisy data
original
fit
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
6
Recursive Least Squares (RLS) (Plackett's algorithm)
online estimation
gives identical result as OLS
assume model
in matrix form:
Black box - agenda
1. Prerequisities: Least squares.
Discrete dynamic system.
2. How to use LS to estimate parameters of DDS? Example: mechanical oscillator
3. Problem: Noise.
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
7
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
8
OLS & RLS example: Estimation of mass+spring+damper system 1/3
1) Discretization
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
9
OLS & RLS example: Estimation of mass+spring+damper system 2/3
2) Create matrix X and vector Y
Y(k) = y(k);
X(k,:) = [y (k-1), y(k-2), u(k-2)];
{E003}
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
10
OLS & RLS example: Estimation of mass+spring+damper system 3/3
3) Solve LS/RLS problem– both LS and RLS provide the same results
– convergence of RLS/precision of LS depends on excitation signal
{E003}0 2 4 6 8 10
-10
0
10
t [s]
Oscilator position
0 2 4 6 8 10-50
0
50
t [s]
inpu
t u
Input excitation force [N]
0 2 4 6 8 10-2
0
2convergence of Recursive LS (RLS)
t [s]
continuous sim
ARX sim
a1
a2
b2
But: there is no noise
considered in our
simulation!
Black box - agenda
1. Prerequisities: Least squares.
Discrete dynamic system.
2. How to use LS to estimate parameters of DDS? Example: mechanical oscillator
3. Problem: Noise.
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
11
Problem = noise: simple example study – setup 1/4
consider first order system
discretization:
for simplicity, consider NO INPUT
TASK: estimate one parameter a.
simulation data :
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
12
Ts
1k kx ax
0 2 4 6 8 100
0.5
1
1.5
2
tx
-1 0 1 2 3
0.5
1
1.5
x(k)
x(k
+1
)
Problem = noise: simple example study – without noise 2/4
Consider ini value x_0 and simulate equation:
On lower Fig. we see the plot x(k) vs. x(k+1).From the equation, it is obvious, that it should be a line.
The slope of this line is a.
With data WITHOUT noise, the estimation of a is precise.
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
13
1k kx ax
0 2 4 6 8 100
0.5
1
1.5
2
t
x
-1 0 1 2 3
0.5
1
1.5
x(k)
x(k
+1
)
Problem = noise: simple example study – noise on output 3/4
Consider further the noise on output (e.g. noise of sensor). Equation: (w is random number with normal distribution (Gauss)
We can observe a problem with estimation of parameter a.
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
14
1k k
k k
x ax
y x w
0 2 4 6 8 10-2
-1
0
1
2
3
4
5
6
t
x
-1 0 1 2 3 4 5
-1
0
1
2
3
4
5
x(k)
x(k
+1
)
Example:
SI 004
% simulation
x(1) = x0;
for i = 2:length(t)
x(i,1) = a*x(i-1);
y(i,1) = x(i) + randn*0.2;
end
0 2 4 6 8 10-2
-1
0
1
2
3
4
5
6
t
x
-1 0 1 2 3 4 5
-1
0
1
2
3
4
5
x(k)
x(k
+1
)
Problem = noise: simple example study
Next, consider noise „inside“ the systemwhich modifies the state of x(k).Equation:
This noise can be considered as a randomdisturbance (input) influencing the system.
Using the simulation, we can observe, that the estimation ofparameter a is perfect.
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
15
1k k
k k
x ax w
y x
Example:
SI 004
% simulation
for i = 2:length(t)
x(i,1) = a*x(i-1) + randn*0.2;
y(i,1) = x(i) ;
end
0 2 4 6 8 10-8
-6
-4
-2
0
2
4
6
8
10
12
t
x
-5 0 5 10
-6
-4
-2
0
2
4
6
8
10
x(k)
x(k
+1
)
0 2 4 6 8 10-8
-6
-4
-2
0
2
4
6
8
10
12
t
x
-5 0 5 10
-6
-4
-2
0
2
4
6
8
10
x(k)
x(k
+1
)
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
16
Linear Models for System Identification
q = forward shift operatore.g.: q-1 x(k) = x(k-1)
linear models = relation between y(),u(), and noise v(k) through transfer functions
general form:
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
17
Linear Models for System Identification
AR – autoregressive model
MA – moving average
example: smoothing
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
18
Linear Models for System Identification
ARMA
ARX
ARMAX
OE
BJExample: system completely deterministic,
noise added only to output (e.g. position
sensor = potentiometer)
Very popular – linear in parameters
OLS
Black-box identification- the use of a tool System Identification/GUI
Robert GreplInstitute of Solid Mechanics, Mechatronics and Biomechanics
Faculty of Mechanical Engineering, Brno University of Technology
2010 - 2015
System Identification Toolbox - GUI
System Identification Toolbox very (very) complex tool, author L. Ljung
you can work using
– command line
– GUI
how to run it?: ident
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
20
System Identification Toolbox – GUI: Import data
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
21
System Identification Toolbox – GUI: Preprocessing
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
22
preprocessing of the data check data
select estimationand verification data
Check
imported data
divide
Estimation and
Verification
data
System Identification Toolbox – GUI: Select data
Select data for Estimation and for Verification
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
23
Data for
estimation
Data for
verification
System Identification Toolbox – GUI:
Estimate order of the system
You can estimateseveral ARX models atonce.
Think about noise...
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
24
Example:
ARX
Example:
OE
Exercise
S005
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
25
Dodatky
• Offline vs. online odhad parametrů
• IV (Instrumental variable) method
• Nelineární nejmenší čtverce
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
34
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
35
Offline vs. online odhad parametrů
batch estimation– measurement in time (0,T)
– data processing
– parameter estimation using all data
online estimation– parameter estimation during system operation
– recursive algorithm
– advantages:
» memory requirements (often) reduced – OLS vs. RLS
» the slow changes of parameters can be captured
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
36
Recursive Least Squares (RLS) (Plackett's algorithm)
online estimation
gives identical result as OLS
assume model
in matrix form:
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
37
OLS & RLS example: Estimation of mass+spring+damper system 3/3
Solve LS/RLS problem– both LS and RLS provide the same results
– convergence of RLS/precision of LS depends on excitation signal
{E003}0 2 4 6 8 10
-10
0
10
t [s]
Oscilator position
0 2 4 6 8 10-50
0
50
t [s]
inpu
t u
Input excitation force [N]
0 2 4 6 8 10-2
0
2convergence of Recursive LS (RLS)
t [s]
continuous sim
ARX sim
a1
a2
b2
No noise considered in our
simulation!
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
38
OLS & RLS example: Estimation of mass+spring+damper system with noise
Assume noise added to measurement (e.g. if potentiometer used)
noise_stdvar = 0.1;
(later, we name this noise model “Output Error” = OE model)
{E003} 5.5 6 6.5 7 7.5 8
-6-4-202
t [s]
Oscilator position
0 2 4 6 8 10-50
0
50
t [s]
inpu
t u
Input excitation force [N]
0 2 4 6 8 10
-1
0
1
2
convergence of Recursive LS (RLS)
t [s]
continuous sim
ARX sim
a1
a2
b2
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
39
Estimation of OE: Instrumental Variable (IV) Method
solution for consistency problem
1. Estimate ARX
2. Simulate model with estimated parameters
3. Create instrumental matrix Z
4. Estimate
repeat steps 2-4 (usually upto 3-4 steps) [E009]
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
40
Nonlinear Least Squares 1/2
motivation: need to find parameters for functions which are nonlinear in parameters
assume vector field
goal: minimize criterion
Taylor series
– gradient:
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
41
Nonlinear Least Squares 2/2
all NLS methods are iterative
methods for computation of h:– Steepest descent
– Newton’s method
– Gauss-Newton
– Levenberg-Marquardt
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
42
Example NLS: Solution of overdetermined system
Assume vector field
clearly, the system is overdetermined
x = 1 / 0.4115
Task: Find solution which has minimal mean square error.
[E002]0 2 4 6 8 10 12 14 16 18 20
0
0.2
0.4
0.6
0.8
1
convergence of iteration method
iterations
valu
es o
f x
0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.30
0.2
0.4
0.6
0.8
variable x
Plot of square error
LM lambda = 2
GN alpha = 0.5
solution of individual equations
f12
f22
f12 + f
22
www.mechlab.czISMMB, Faculty of Mechanical Engineering, Brno University of Technology
43
References
[Iserman 2011] Isermann, R., Muenchhof, M.: Identification of Dynamics Systems, Springer 2011
[Nelles, 2001] O. Nelles: Nonlinear system identification: from classical approaches to neural networks and fuzzy models, Springer-Verlag, 2001google books
[Noskievič, 1999] P. Noskievič: Modelování a identifikace systémů, Montanex, 1999
[Ljung 2008] L. Ljung: Perspectives on System Identification, Seoul 2008, http://www.control.isy.liu.se/~ljung/
[Denery 2005] T. Denery: Modeling Mechanical, Electric and Hydraulic Systems in Simulink, Matlab webinars, 2005
[Blaha] doc. Ing. Petr Blaha, PhD., FEKT, CAKhttp://sites.google.com/site/modelovaniaidentifikace/
[Gander, 1980] Gander, W.: Algorithms for QR-Decomposition, research report, 1980
http://www.emeraldinsight.com/journals.htm?articleid=1943785&show=html
http://courses.ae.utexas.edu/ase463q/design_pages/fall02/AWE_web/secondorder.htm
http://www.ni.com/white-paper/4028/en/
Kontakt: www.mechlab.czÚstav mechaniky těles, mechatroniky a biomechaniky
Fakulta strojního inženýrstvíVysoké učení technické v Brnědoc. Ing. Robert Grepl, Ph.D.
Top Related