Two Discrete Optimization Problems Problem #2: The Minimum Cost Spanning Tree Problem.
Minimum Norm State-Space Identification for Discrete-Time Systems
-
Upload
rosalyn-britt -
Category
Documents
-
view
30 -
download
2
description
Transcript of Minimum Norm State-Space Identification for Discrete-Time Systems
![Page 1: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/1.jpg)
Minimum Norm State-Space Identification for Discrete-Time Systems
Zachary FeinsteinAdvisor: Professor Jr-Shin Li
![Page 2: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/2.jpg)
Agenda
•Goals•Motivation•Procedure•Application•Future Work
![Page 3: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/3.jpg)
Goals
•Find a linear realization of the form:
•To solve:
![Page 4: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/4.jpg)
Goals
•In the case of output-data only, create realization of the form:
•This is called historically-driven system
![Page 5: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/5.jpg)
Motivating Problem
•Wanted to find a constant linear realization to approximate financial data
•Use for 1-step Kalman Predictor on historically-driven system:
![Page 6: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/6.jpg)
Motivating Problem
•The specific problem being addressed initially was analysis of the credit market
•Try to do noise reduction and prediction of default rates
![Page 7: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/7.jpg)
Motivation
•Why do we need a new technique?▫Financial Data does not follow any clear
frequency response▫Cannot use any identification technique
that finds peaks of transfer function (e.g. ERA or FDD)
![Page 8: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/8.jpg)
Procedure: Agenda
•Background•Find Weighting Pattern•Find Updated Realization•Find Optimal Delta Value•Discussion of Output-Only Case
![Page 9: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/9.jpg)
Procedure: Background
•Let A, B, C have elements which lie in the complex plane.
•Let p = length of output vector y(k)•Let n = length of state vector x(k)•Let m = length of input vector u(k)
![Page 10: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/10.jpg)
Procedure: Background
•For simplification assume x0 = 0•Want to solve:
•Remove all points at the beginning such that u(k) = 0 for all k = {0,…,M}
![Page 11: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/11.jpg)
Procedure: Find Weighting Pattern•Discrete time weighting pattern:
•We can write:
![Page 12: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/12.jpg)
Procedure: Find Weighting Pattern•Our minimization problem can now be
rewritten as:
•Want to solve for optimal Fk for all k
![Page 13: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/13.jpg)
Procedure: Find Weighting Pattern•Want an iterative approach•Since each norm in the sum only has Fl
for l ≤ k we can solve find such a formula•Solving each as a minimum norm
problem:
![Page 14: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/14.jpg)
Procedure: Find Realization
•Given that we have weighting pattern
•Now we have an objective function:
•Again want an iterative approach to solve
![Page 15: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/15.jpg)
Procedure: Find Realization
•Would use Convex Combination of previous best solution and optimal case for next norm:
![Page 16: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/16.jpg)
Procedure: Find Realization
•For the kth update solving for minimum norm:
•These values solve:
![Page 17: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/17.jpg)
Procedure: Find Realization
•Choose to update the matrices in the order:
![Page 18: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/18.jpg)
Procedure: Find Realization
•This update order was chosen since:
•Let C be a constant then from F0 we can find optimal B
•Using this optimal B and C then use F1 we can find optimal A
•Logical to update C next
![Page 19: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/19.jpg)
Procedure: Find Optimal Delta
•Want to solve for the optimal delta values such that:
![Page 20: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/20.jpg)
Procedure: Find Optimal Delta
•First discuss how to solve for δB
•Then discuss δC since it is similar to δB
•Finally, discuss δA because this case has higher order terms
![Page 21: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/21.jpg)
Procedure: Find Optimal δB
•For simplification rewrite optimization problem to be:
•Through use of counterexample, it can be seen that δB ≥ 0
![Page 22: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/22.jpg)
Procedure: Find Optimal δB
•Using property of norms, mainly the triangle inequality
![Page 23: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/23.jpg)
Procedure: Find Optimal δB
•Using these inequalities it can be seen that:
![Page 24: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/24.jpg)
Procedure: Find Optimal δB
•Therefore we can find upper and lower bounds for δB:
![Page 25: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/25.jpg)
Procedure: Find Optimal δB
•Using these bounds use a search algorithm to find optimal δB
•Evaluate at 2 endpoints and 2 interior points• If value at endpoint is smallest recursively
call again with new endpoints of that endpoint and the nearest interior point
•Otherwise choose the 2 points surrounding that minimum as the new endpoints and call recursively
•Terminate if interval is below some threshold
![Page 26: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/26.jpg)
Procedure: Find Optimal δC
•Analogous to δB
▫Rewrite the objective function as:
▫Can use same properties to find an upper-bound on this objective function
![Page 27: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/27.jpg)
Procedure: Find Optimal δC
•We can use same properties as before to find bounds on δC:
•Therefore we can use the same search algorithm as in the δB case to find the optimal δC
![Page 28: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/28.jpg)
Procedure: Find Optimal δA
•To simplify we first want to find a linear approximation in δA for:
•Using knowledge of exponentials, we can say:
![Page 29: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/29.jpg)
Procedure: Find Optimal δA
•Using this linear approximation, we can rewrite the minimization problem to be:
![Page 30: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/30.jpg)
Procedure: Find Optimal δA
•Given the linearization in δA we can use the same properties as with δB to find bounds on δA
•Using these bounds, we can run the same search algorithm as given for δB
•This search will run on the full objective function, not the linearized version
![Page 31: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/31.jpg)
Procedure: Output-Only Case
•More important case for us given the motivating problem of financial data▫Input for financial markets is unknown
•Same procedure as given before•In finding the optimal weighting pattern:
let u(k) = yact(k) for all k
![Page 32: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/32.jpg)
Application
•Implemented in MATLAB with a few additions to the Procedure
•Tried on test input-output system•Discussion of the unsuccessful results for
the test case
![Page 33: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/33.jpg)
Application: Implementation• MATLAB chosen due to native handling of matrix
operations• Few differences in implementation and procedure given
before▫ Initial choice of C matrix is chosen as a random matrix
with elements between 0 and 1▫ If δ drops below some threshold, stop updating the
corresponding matrix▫ After calculation, if A is an unstable matrix (i.e. |λmax| >
1) then restart with new initial C matrix▫ At end of implementation compare new value of
objective function to previous one If better by more than ε, iterate through again If better by less than ε, stop and choose new realization If worse by any amount, stop and choose old realization
![Page 34: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/34.jpg)
Application: Input-Output Test
•Run MATLAB code on well-defined state-space system:
![Page 35: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/35.jpg)
Application: Input-Output Test
•The resulting calculated realization was:
![Page 36: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/36.jpg)
Application: Input-Output Test
•The objective function had a value of 37.7 for this calculated realization
•Easier to see in plots on next 3 slides.▫Value of with x-axis of k▫Output of the test system (first output only)▫Output of the calculated system (first
output only)
![Page 37: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/37.jpg)
Application: Objective Value Plot
![Page 38: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/38.jpg)
Application: Actual Output Plot
![Page 39: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/39.jpg)
Application: Calculated Output
![Page 40: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/40.jpg)
Application: Discussion• As shown, these results show this technique to
be unsuccessful, this can be due to:▫ It is assumed that the δ values are small, which is
not necessarily true▫ It is assumed that the convex combination will
bring us towards a better solution, which is seen to not be the case
▫ Changing from the initial minimization problem to finding the best approximation for the weighting pattern means that some of the relationships between the elements of [A,B,C] could be lost
![Page 41: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/41.jpg)
Future Work
•There are 2 types of techniques that may be useful to solve this problem and find a better solution than the shown solution:▫Gradient Descent Method▫Heuristic Approach
![Page 42: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/42.jpg)
Future Work: Gradient Descent
•Advantage:▫Mathematically Robust▫Proven that it will find a local minimum
•Disadvantage:▫Given m*n+n2+n*p variables, this will take
a long time to solve▫The objective function (as a sum of norms)
is large, therefore the gradient may take an incredible amount of computational power and memory to compute and store
![Page 43: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/43.jpg)
Future Work: Heuristic Approach•Example: Genetic Algorithm, Simulated
Annealing•Advantage:
▫Can somewhat control level of computational complexity
•Disadvantage▫Only finds a “good” solution
![Page 44: Minimum Norm State-Space Identification for Discrete-Time Systems](https://reader036.fdocuments.in/reader036/viewer/2022062407/56812b0c550346895d8ef63e/html5/thumbnails/44.jpg)
Thank you
Questions?