Intelligent Embedded Systems

Juraj Števek∗

Institute of Applied InformaticsFaculty of Informatics and Information Technologies

Slovak University of Technology in BratislavaIlkovicova 3, 842 16 Bratislava, Slovakia

juraj.stevek@stuba.sk

AbstractThis work describes a modelling method for a non-linearsystem which is based on a multi-point linear approxi-mation for a model predictive control (MPC) purpose.The method is derived from artificial neural network tech-niques and exploits good properties of a Orthogonal Ac-tivation Function based Neural Network (OAF-NN). Inthis work, we describe a technique of a Explicit-MPC(EMPC) which performance, ease of implementation andextension for the hybrid system gives promising directionof a algorithm design for the embedded systems (ES).Proposed modelling procedure is characterized by fasttraining property and wide applicability in medical sys-tems, automotive, power-electronics industry etc.. Lin-earity of the model allows to transform into PiecewiseAffine (PWA) system structure and to exploit existing al-gorithms for a explicit model predictive control design.The applicability of the technique was evaluated on a hy-brid system by means of existing software tools for MPCdesign.

Categories and Subject DescriptorsG.1.2 [Approximation]: Linear approximation; G.1.6[Optimization]: Integer programming, Constrained op-timization; G.4 [Mathematical Software]: Algorithmdesign and analysis; I.6.7 [Simulation and Modelling]:Simulation support systems; I.6.4 [Simulation and Mod-elling]: Model validation and analysis; I.2.6 [ArtificialIntelligence]: Parameter learning

Keywordsembedded system, neural network, orthogonal polyno-mials, multi-parametric programming, non-linear system,PWA approximation, hybrid system, model predictive con-trol

∗Recommended by thesis supervisor: Prof. Stefan KozakDefended at Faculty of Informatics and Information Tech-nologies, Slovak University of Technology in Bratislava onFebruary 16, 2012.c© Copyright 2012. All rights reserved. Permission to make digital

or hard copies of part or all of this work for personal or classroom useis granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies show this notice onthe first page or initial screen of a display along with the full citation.Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy other-wise, to republish, to post on servers, to redistribute to lists, or to useany component of this work in other works requires prior specific per-mission and/or a fee. Permissions may be requested from STU Press,Vazovova 5, 811 07 Bratislava, Slovakia.Števek, J. Intelligent Embedded Systems. Information Sciences andTechnologies Bulletin of the ACM Slovakia, Vol. 4, No. 1 (2012) 20-28

1. IntroductionAn embedded system is, in recent time, inherent com-ponent of many tech gadgets and appliances which servepeople in everyday life, i.e. washing machine, digital cam-era, cell phone, mp3 player, vehicle, thermostat etc.. Theembedded system is a computer system designed to per-form one or a few dedicated functions [11] often withreal-time computing constraints. In the washing machine,the embedded system controls washing routines with con-straints of water volume and washing temperature. Ina vehicle, the embedded systems control a engine perfor-mance, a air-conditioning, a stabilization and many otherfunctions. Such an electronic system is embedded as partof a complete device often including hardware and me-chanical parts.The program for embedded system is usually developedwith low level programming language (assembler) but somecommercial systems offer special programming languagesor even higher programming languages. The speed of pro-gram execution depends on the programming languageand a computational performance of the embedded sys-tem. The computational ”power” of the embedded systemis the main limitation for using of advanced control algo-rithms.Model predictive control (MPC) is one of the widespreadmodern control principle in area of applied informatics. Inrecent years, the range of applications of MPC has beenextended from constrained linear time-invariant system(LTI) to non-linear, so-called hybrid systems. A hybridsystem is a dynamic system that exhibits both continuousand discrete dynamic behavior - a system that can bothflow and jump. When this extension is made, binary vari-ables are introduced in the problem. As a consequence,the Quadratic programming (QP) problem has to be re-placed by a far more challenging Mixed Integer QuadraticProgramming (MIQP) problem. Generally, for this typeof optimization problems, the computational complexityis exponential in the number of binary optimization vari-ables. There are many problems related to algorithm de-sign for a hardware platform with limited resources, i.elow frequency (tenths of Mhz) and limited program mem-ory (kilobytes). Such limitations disqualify the platformfrom use of a sophisticated and a complex MPC based al-gorithms. In literature, many advance control algorithmswere developed that accept the limitations. In this work,we describe a technique of a Explicit-MPC (EMPC) whichperformance, ease of implementation and extension forthe hybrid system gives promising direction of a algorithmdesign for the embedded systems (ES).A linear model can be obtained by linearizing the originalnon-linear system around operating point, Fig. (1a). The

Information Sciences and Technologies Bulletin of the ACM Slovakia, Vol. 4, No. 1 (2012) 20-28 21

x

f(x)

D1

(a) Linearization in a single operat-ing point

x

f(x)

D2D1 D3

(b) A piecewise affine multiple-point linearization

Figure 1: The figure illustrates increasing accuracy of a linearization with number of linearization points

linear model has been widely used for prediction of thesystem evolution in MPC concept. The main drawbackof the linear model is its decreasing accuracy when systemoperates in distant from linearization point. To overcomethis handicap, the concept of a Piecewise Affine (PWA)system was introduced [25], Fig. (1b). A next-state anda output of the PWA system are both described by piece-wise linear maps, i.e., by maps which are affine on eachof the components of a finite polyhedral partition.Unfortunately, to the best of our knowledge, there are fewresults on system identification and explicit controller de-sign for neural network based models. Modeling frame-works mentioned above belong to deterministic modelingtechniques based on differential or difference equations,typically derived from physical laws. Most important as-pect of created model is its linearity. On the other hand,neural network belongs to black box modeling techniqueswhich exploit non-linear relations in neural network struc-ture to model non-linear characteristics of real processes.To overcome this problem we propose original method forlinearizing special case neural network, well known as theOrthogonal Activation Function based Neural Network(OAF-NN). The linearized neural network is equivalentto PWA system and all derived theoretical properties andexisting tools for analysis and control design can be bringto bear.

2. Explicit Model Predictive ControlIn [4] was shown that solution of a multi-parametric prob-lem can be expressed as a piecewise affine function overconvex polyhedral region set. This result allows to ex-press solution of the MPC problem as a explicit state-feedback regulator. This state-feedback regulator is a lin-ear piecewise affine function of the state variables withdefined polyhedral partitioning of the state space . Theexplicit-MPC brings new possibilities for MPC implemen-tation. All computations are moved off-line and solutionis formulated as a lookup table, Fig. (2). The necessaryon-line implementation effort thus reduces to a identifica-tion of a active polyhedral set based on the current stateand evaluation a associated affine feedback law. The con-troller inherits all the stability and performance proper-ties of the classic MPC but can be implemented withoutany involved on-line computations. Therefore it is pos-sible to use hardware platform with lower computationpower and with lower amount of a available storage space,and decrease a price of control system hardware.The explicit model predictive control (EMPC) was pre-

sented as suitable control scheme for high-sampling fre-quency operation [18, 19, 32, 10]. The problem con-

straints are naturally respected by including them intothe optimal control problem. At run time, the EMPCalgorithm iterates through a binary search-tree (BST) tofind a active region in the state space and selects the as-sociated feedback control law to apply in order to obtainthe optimal control action. In order to reduce controlcomputation time, the trend is today to move controllerimplementation from micro-controllers or DSP to FPGAor dedicated silicon devices.

From the hardware synthesis viewpoint, [14] showed thatexplicit MPC solutions can be implemented in an applica-tion specific integrated circuit (ASIC) with about 20,000gates, leading to computation times in order of 1µs.Table 1 shows the progress of the MPC in high-samplingtime applications. The fastest EMPC application is op-erating with nanoseconds sampling time. In the applica-tion the temperature regulation of a multi-core proces-sor is performed by minimizing power usage subject totemperature limitations. The storage of the control lawrequired 71kB and the online computation using BST re-quired 10731 FLOPS.

2.1 Implementation of the control lawThe explicit control law is given by

u(k) = Krx(k) +Gr (1)

where the index r denotes the active region, i.e. the re-gion which contains the given state x(k). The evaluationof explicit MPC controller turns into simple procedurewhere simple membership-test returns active control lawfor given state x(k) and control action is calculated ac-cording (1) and applied to the system [16].

Table 1: The table shows the progress of the MPCin high-frequency applications

Sample time Application Institute18 ns Multi-core thermal manage-

ment [32]EPFL

10 µs Voltage source inverters [20] ETHZ25 µs Direct torque control [10] ABB50 µs AC / DC converters [22] ETHZ20 ms Traction control [5] FORD30 ms Ball and plate [16] ETHZ500 ms Building climate control us-

ing weather forecast [21]Siemens

22 Stevek, J.: Intelligent Embedded Systems

010

2030

40

0

50

100

0

1000

2000

3000

4000

5000

vω

u* f(v,ω

)

Figure 2: A example of the control law of explicit-MPC

3. A hybrid systemThe label Hybrid system , in recent time, represents non-linear dynamical system of heterogenous continuous anddiscrete character. The Hybrid system switches amongmany operating modes, where each mode is governed byits own characteristic dynamical laws.Mode transitionsare triggered by variables crossing specific thresholds (stateevents), by the elapse of certain time periods (time events),or by external inputs (input events)[2]. We consider dy-namical systems whose process model is defined in thediscrete-time domain by a state-update equation

x(t+ 1) = f(x(t), u(t)). (2)

Here, x(t) ∈ Rn is the state vector at time instance t ∈N+, x(t + 1) denotes the successor state, u(t) ∈ Rm isthe vector of control inputs, and f(·) is the state-updatefunction. It is assumed that states and inputs are subjectto constraints

x(t) ∈ X ⊆ Rn, (3a)

u(t) ∈ U ⊆ Rm, (3b)

where these constraints should hold ∀t ≥ 0.In the most general setup the state-update functionf(x(t), u(t)) can be arbitrary, e.g. linear, piecewise lin-ear, nonlinear, discontinuous, non-differentiable, etc.

3.1 A Piecewise Affine SystemThe accuracy of the approximation of f(·) can be signifi-cantly increased if the function is linearized around mul-tiple distinct linearization points. This methodic leads toa Piecewise Affine (PWA) linear system, Fig. (1b). ThePWA system offers framework for modelling hybrid orcommon non-linear systems. Dynamical behavior of suchsystems is captured by relations of the following form:

x(k + 1) = Aix(k) +Biu(k) + fi (4a)

y(k) = Cix(k) +Diu(k) + gi (4b)

subjet to: ymin ≤ y(k) ≤ ymax (4c)

umin ≤ u(k) ≤ umax (4d)

∆umin ≤ u(k)− u(k − 1) ≤ ∆umax (4e)

Each dynamic i is active in a polyhedral partition boundedby the so-called guard-lines:

guardXix(k) + guardUiu(k) ≤ guardCi (5)

Several other model structures were developed for thehybrid system or non-linear system at all. Much atten-tion is given to system modelling in a MLD (Mixed Inte-ger Dynamical) form [3]. In [12], the formal equivalencebetween MLD system and PWA systems is establishedand also effective algorithms were developed for trans-formation from one model structure to another [1, 28].In [13, 12], the equivalence between the following fiveclasses of hybrid systems is, under certain conditions, es-tablished: MLD systems, Linear Complementarity (LC)systems, Extended Linear Complementarity (ELC) sys-tems, PWA systems and Max-Min-Plus-Scaling (MMPS)systems. The important result of these equivalences isthat derived theoretical properties and tools can be easilytransferred from one class to another.

3.2 Identification of a hybrid systemLinearization in multiple points can be done using Taylorseries technique

fLin(x, u) = f(xi, ui) +

(∂f(x, u)

∂x

)xi,ui

(x− xi)+

+

(∂f(x, u)

∂x

)xi,ui

(u− ui).

(6)

where fLin(x, u) is tengent hyperplane to original non-linear function f(x, u) in linearization point (xi, ui), Fig.(1b).Another way to approximate non-linear function by hy-perplanes is based on solving set of optimization prob-lems in least square fashion. Consider nonlinear functionof one variable y = f(x). To find linearization parameterswe solve optimization problem:

J(ai, bi) =

P∑i=1

N∑j=1

(yij − aixij − bi)2 (7a)

subject to: aixiN + bi = ai+1x

i+11 + bi+1 (7b)

where {(xi1, yi1), (xi2, yi2), . . . , (xiN , y

iN )} denotes points of i−

th interval of linearization and P is total number of re-gions.To solve the problem (7) is quite easy unless we introducea breaking points into the optimization problem. To finda optimal breaking points of the PWA model means tofind optimal partitioning of the state space.There were introduced several techniques to find a PWAmapping in the literature. A clustering-based algorithmwas employed in traditional identification of the PWAmap. The driving idea of clustering-based procedure re-sides in local linearity of the PWA map. Then, if the localmodels around two data points are similar, it is likely thatthe data points belong to the same mode [9]. Anothertechniques are based on solution of a mixed-integer pro-gramming problem or bounded error problem [23]. A tri-angular nested partitioning was exploited in [15]. A alge-braic geometric approach was presented in [27]. The PWAmapping techniques exhibits various properties. However,a computational demand of these methods significantlygrows with a state space dimension and a number of lin-earization regions.

Information Sciences and Technologies Bulletin of the ACM Slovakia, Vol. 4, No. 1 (2012) 20-28 23

∑

mx

2x

1x

Π

Π

Π

Π

Π

∑

my

Training method

e

-

+

py

Input block 1

Input block 2

Input block m

Orthogonal neurons Product

Output neuron

1..1..1w

1..1..1w

mi nnnw ....1ˆ

mi nNNw ....1ˆ

mi NNNw ....1ˆ

Input layer Hidden layer Output layer

Figure 3: A Orthogonal Activation FunctionBased Neural Network structure

4. Problem formulationThe non-linear explicit-MPC concept is based on the hy-brid model [4]. We can say that the hybrid model is afundamental part of the whole concept. A tradeoff be-tween a accuracy and a simplicity of the hybrid modelis crucial aspect of the MPC design. If we use too com-plex model (too many linearization regions) we could failto compute a explicit control law [4, 16]. On the otherhand, the model of low accuracy can lead to poor controlperformance.Our goal is to get a multi-point piecewise linear modelof a static or a dynamic system with accuracy limitinga non-linear model. The key idea in the approximationprocess is a transformation a multi-variable non-linear al-gebraic formula into a equivalent series of a single-variablefunctions.

f(z1, z2, · · · , zn) ≈ f1(z1) + f2(z2) + · · ·+ fn(zn) (8)

This approach simplifies the complex multi-variable ap-proximation into series of single-variable identification.The optimization problem for a single term of (8) is de-fined as follows

minai,ci,ri

N∑i=1

(∫ i

ri−1

(f(z)− (aiz + ci))2dz

)(9a)

s.t.: zmin ≤ r1 ≤ . . . ≤ rN−1 ≤ zmax, (9b)

airi + ci = ai+1ri + ci+1, i = 1, . . . , N − 1. (9c)

Subsequently, several simplified optimization problems areresolved by means of existing tools.

2xπ

( )22 2 1x

π

−

( )32 4 3x x

π

−

1x

1π

2xπ

( )22 2 1x

π

−

( )32 4 3x x

π

−

2x

1π

π

π

π

π

π

π

π

π

π

π

∑

Π

Π

Π

Π

Π

Π

Π

Π

Π

ΠInput block

yInput block

Figure 4: A example of Generalized Fourier Seriesmodel

5. Orthogonal activation function based neuralnetwork

In [26] was developed an approach of PWA linearization ofmulti-variable function. The authors showed that multi-variable function f(z1, ..., zn) could be transformed to ad-ditive separable form [31] using a simple algebraic trans-formation under a certain assumption.

Assumption 5.1. The function f(z1, ..., zn) can be writ-

ten as∑n

i=1 αi

(∏qij=pi

fj(zj))

.

The assumption says that the algebraic formula has to belinearly separable function.To overcome the problem of linear separability and a re-quirement of existing algebraic formula we propose thenovel approach based on a neural network technique. In[17], a orthogonal polynomial is used as a activation func-tion for special case of neural network with one hiddenlayer - a Orthogonal Activation Function based NeuralNetwork (OAF NN). For this type of neural network a on-line and a off-line training algorithm has been defined withfast convergence properties. Actually, the Orthogonal Ac-tivation Function based neural network is a type of feed-forward neural networks which have the ability to trans-form the nonlinear input space into higher dimensional-space where linear separability is possible [8].The most important quality of the neural network tech-nique lies in the universal approximation theorem [7].

Definition 1. A standard multi-layer feed-forward neu-ral network with a single hidden layer that contains finitenumber of hidden neurons, and with arbitrary activationfunction is a universal approximator on a compact subsetof Rn.

24 Stevek, J.: Intelligent Embedded Systems

Table 2: Setup configuration, results and comparisonPWA GFS model HL CPWL model [15] HIT-PWA model [9]

Case Reg. MSE T(s) Reg. MSE T(s) Reg. MSE T(s)1. 4 1.012e-7 2.159 4 9.31e-4 - 4 1.103e-7 19.962. 6 1.659e-8 2.48 - - - 9 6.022e-8 39.933. 8 5.4e-9 4.23 16 2.23e-4 - 16 5.22e-8 70.294. 12 2.457e-9 4.72 36 5.6e-5 - - - -

The OAF neural network is a three-layer neural structurewith an input layer, an output layer and a hidden layer asshown in Fig. 3 for a multi-input-single-output (MISO)system.The hidden layer consists of neurons with orthogonal(preferably orthonormal) activation functions. The ac-tivation functions for these neurons belong to the sameclass of orthogonal functions and no two neurons have thesame order of activation function. The input and outputlayers consist of linear neurons. The weights between theinput and the hidden layer are fixed, and depend on theorthogonal activation functions. The nodes on the right ofthe orthogonal neurons implement the product (Π). Eachnode has m input signals from different input blocks. Thenodes are considered as part of the hidden layer becausethere is no weighting operation between orthogonal neu-rons and nodes (Fig. 3).The network output is defined by a linear combination ofweights

y(x, ˆw) =

N1−1∑n1=0

Nm−1∑nm=0

wn1...nmφnr...nm(x) = ΦT (x) ˆw,

(10)where x = [x1, x2, . . . , xm] is m-dimensional input vector,Ni is the neuron number of i-th input and ˆw betweenthe hidden and output layers. Functions φn1...nm(x) areorthogonal functions in m-dimensional space given by

φn1...nm((x)) =

m∏i−1

φn1(xi), (11)

where φi(x) are one-dimensional orthogonal functions im-plemented by each hidden layer neuron.

6. PWA-GFS approximationActually, the network output (10) is not the sum of onedimensional functions yet. The straightforward approachto get sum of single-variable functions can be done byremoving the product neurons which mix the inputs of thenetwork. The mixed term indicates the product neuronswhich multiply orthogonal functions from different inputs.This approach leads to a Generalized Fourier Series (GFS)model structure, Fig. (4).Next we illustrate the PWA-GFS modelling on a two-dimensional static example.

6.1 A approximation exampleThanks to the fact that EMPC can operate in nanosec-onds times it may be used in semiconductor technology[32]. A modern transistors are characterized by high-electron transport properties that designate them for highvoltage, high power and high temperature microwave ap-plications [6]. However, these devices often experienceslimitation due to trapping effects and self-heating. Thechallenge is to control working point of such devices re-specting the limitation.

In this case study an application of the modeling tech-nique for semiconductor device is shown. The current-voltage characteristics Ids(Vgd, Vgs) of a GLASMOST n-channel MOSFET transistor [24, 15] are modeled.In literature, the non-linear Current-Voltage characteris-tics of a semiconductor transistor are mostly obtained byanalytical and empirical analysis. The electrical proper-ties of the semiconductor devices depend on the produc-tion technology. The extracted models are validated bymeasurements from a experiment.In our example we consider the nonlinear Current-Voltagecharacteristics of a GLASMONT transistor derived in [24].We use just derived model, not the Current-Voltage mea-surements of a real device. Please note that this exampleonly illustrates capabilities of the modelling technique.

Nonlinear modelThe following are the equations that represents the staticbehavior of the device

(a) If Vgs ≥ V0 and Vgd ≥ V0 (linear region)

Ids =WµCox

L(Vgs − Vgd)(Vgs + Vgd + 2Vth − 2V0)

(12)

(b) If Vgs ≥ V0 and Vgd < V0 (drain saturation)

Ids =WµCox

L((Vgs − V0)(Vgs − V0 + 2Vth)

− V 2th(e(Vgd−V0/Vth) − 1))

(13)

(c) If Vgs < V0 and Vgd ≥ V0 (source saturation)

Ids =WµCox

L(V 2

th(e(Vgs−V0/Vth) − 1)

− (Vgd − V0)(Vgd − V0 + 2Vth))

(14)

(d) If Vgs < V0 and Vgd < V0 (subthreshold region)

Ids =WµCox

LV 2th

(e(Vgs−V0/Vth) − e(Vgd−V0/Vth)

)(15)

The indices gs and gd denote the gate-source and gate-drain, respectively. The following parameters values wereused:

Vth = 30mV

µ = 0.0675m2 · V −1 · s−1

Cox = 1.38e−3F ·m−2

V0 = 1V

W = 20µm

L = 2µm

From a physical point of view the model is quite simple.However, despite its simplicity the model is rather unique

Information Sciences and Technologies Bulletin of the ACM Slovakia, Vol. 4, No. 1 (2012) 20-28 25

0

2

40

1

2

3

4−0.01

−0.005

0

0.005

0.01

Vgd

(V)Vgs

(V)

I ds (

mA

)

(a) The I-V characteristic nonlinear model (6.1)

0

2

40

1

2

3

4−0.01

−0.005

0

0.005

0.01

Vgd

(V)Vgs

(V)

I ds (

mA

)

(b) The I-V characteristic GFS approximation

0

2

40

1

2

3

4−0.01

−0.005

0

0.005

0.01

Vgd

(V)Vgs

(V)

I ds (

mA

)

(c) The I-V characteristic PWA GFS model (4segments)

0

2

40

1

2

3

4−0.01

−0.005

0

0.005

0.01

Vgd

(V)Vgs

(V)

I ds (

mA

)

(d) The I-V characteristic PWA GFS model (6segments)

Figure 5: The Current-Voltage characteristics approximation results

in its unifying view on the relation between the differentmodes of operation. Each of these modes of operation fol-lows naturally from the chosen expression for the channelcharge density [24].The behavior of the model is illustrated in the Figure 5a,which shows Ids as a function of Vgs and Vgd.The four different approximations were designed, corre-sponding to four different subdivisions of the domain. Forthe approximation a chebyshev polynomials up to fourthorder were used. The final algebraic formula of the iden-tified OAF network has following form

Ioaf (Vgd, Vgs) = 0.07Vgd4 − 0.07Vgd

3 − 0.46Vgd2

− 0.43Vgd − 0.07Vgs4 + 0.06Vgs

3

+ 0.46Vgs2 + 0.43Vgs − 0.49e−15

(16)

The OAF model has no mixed product terms, therefore,it is in a Generalized Fourier Series form. Detail resultswith comparison are presented in Table 2. Table summa-rize the following data: the final number of linear regionsof the PWA model (Reg), the mean square error (MSE),

the computation time (T). The PWA GFS model resultsare compared with existing approaches [15, 9]. Thanks tothe fact that breaking points are defined for each dimen-sion independently for PWA GFS technique, the resultsin each row of the Table 2 are comparable. The resultsof HL CPWL technique is cited from [15] without com-putational time. The results of K-mean cluster techniquewere obtained from simulation using HIT toolbox. Theclustering technique failed to complete in case of manylinear regions. In Fig. 5, the nonlinear function Ids andits GFS and PWA GFS approximation are shown (eachcase corresponding to a given subdivision).

7. PWA-OAF approximationHowever, the GFS model can be used just in narrowrange of applications. For a system with a ”strong” non-linearities the GFS model would not satisfies the accuracyrequirement. Therefore we define following algorithm foridentifying a PWA model from input-output data.

26 Stevek, J.: Intelligent Embedded Systems

−1

0

1

−1

−0.5

0

0.5

1−1

−0.5

0

0.5

1

z1

z2

F(z

1,z2)

(a) Non-linear static function

−1

0

1

−1

−0.5

0

0.5

1−1

−0.5

0

0.5

1

z1

z2

F oaf(z

1,z2)

(b) OAF approximation na = 0, nb = 2

−1

0

1

−1

−0.5

0

0.5

1−1

−0.5

0

0.5

1

z1

z2

PW

A(z

1,z2)

(c) PWA approximation, 42 linearization seg-ments

−1

0

1

−1

−0.5

0

0.5

1−1

−0.5

0

0.5

1

z1

z2

PW

A(z

1,z2)

(d) PWA approximation, 36 linearization seg-ments

Figure 6: The figure shows OAF approximation for example (7) and its PWA approximation

Algorithm 7.1.

1. Normalize the I/O data into the range of orthogo-nality.

2. Train the OAF neural network with necessary num-ber of hidden neurons.

3. Prune the network. Remove the neurons with lowvalued weights (weak parameters).

4. Get the shortest algebraic formula of the pruned net-work.

5. Linearize all the terms of the algebraic formula usingprocedure [26].

The final number of linearization points will increase incomparison with the GFS approach. In return we canachieve arbitrary accuracy limiting accuracy of the OAFneural network.

We will demonstrate the procedure on next example.

ExampleConsider the two-variable non-linear static function de-picted in Figure 6a. We assume that function formulais unknown and we have got just input-output data mea-surement. Plotted data are already normalized into range[-1;1]. The training of the OAF network was performedwith 21 neurons in hidden layer. We pruned 7 neuronswhich contribution were negligible. The final formula ofthe OAF network remained as follows

FOAF = z1(−2.34z2

4 + 2.79z22 + 0.46z2 − 0.52

)+ z1

2 (−3.42z23 + 0.42z2

2 + 2.57z2 − 1.03)

− z13(1.32z2

2 + 0.62z2 − 0.66)

− 2.64z2 + 0.65z22 + 4.34z2

3 − 0.54z24 − 1.65z2

5

− 0.05 + 0.37z14

(17)

The formula (17) consist of 5 terms where each term willbe linearized independently. Last two terms are easy lin-earizable. The first three terms are linearly separable andthey are processed by procedure [26]. The final PWAsurface is depicted in Figure 6. The linearization was

Information Sciences and Technologies Bulletin of the ACM Slovakia, Vol. 4, No. 1 (2012) 20-28 27

Table 3: Example 7 benchmark resultsNo. of segments term 1. term 2. term 3. term 4. term 5. MSE Time

42 10 12 12 3 4 5.845e−3 12.679 s36 8 10 10 3 4 1.185e−2 10.527 s32 7 9 9 3 4 1.602e−2 10.22 s30 7 9 9 2 3 1.365e−2 8.4836 s29 6 9 9 2 3 1.368e−2 8.0717 s26 6 8 8 2 2 6.4995e−2 7.9196 s20 3 7 3 7 - 1.7948e−2 6.4995 s

performed with 42, 36, and 26 linearization segments.The approximation benchmark for various number of lin-earization points is presented in Tab. 3. First row con-tains total number of linearization segments. The nextfive rows contain number of linearization segments foreach term of (17). The MSE is mean square error andthe Time indicates the computation time. The table 3shows that we can achieve optional accuracy by increas-ing or decreasing of linearization segments.

7.1 Concluding remarks - PWA OAF modellingThe PWA OAF technique exhibits many good proper-ties in regard to the explicit-MPC design and competeswith existing approximation methods in a accuracy, in anumber of linearization segments and in a speed of iden-tification. The technique has two key steps. First theOAF neural network is trained by same procedure as anyother feed-forward network. Then the segmentation forthe algebraic output is chosen. The main advantages ofthe technique are as follows:

• identification from input-output measurements,

• training of the OAF network is a convex problem[33],

• training is fast with guaranteed convergence [33],

• the multi-variable approximation is transformed intoa series of a single-variable approximations,

• the technique leads to the fewer number of the bi-nary selectors in the MPC problem compared to thestandard PWA state space model,

• it is possible to achieve a arbitrary accuracy depend-ing on the number of linearization segments.

The accuracy of the final hybrid model is optimal in re-gard to the chosen segmentation.So far, the disadvantage of the technique is a heuristic seg-mentation of the algebraic formula. The notion heuristicrefers to simple decision rule: If a single-variable functionis n-th order use the n linearization segments. Yet, themixed terms of the final algebraic formula are linearizedindependently.

8. ConclusionThe benefit of methods based on the optimal control liesin inclusion of system constraints in objectives of the con-trol problem. The keynote of the optimal control ap-proach can be summarized as follows: The more accuratethe model of system the better control performance weare able to achieve. The requirement of accurate model isa fundamental restriction of the optimal control methodsfor a complex system (non-linear, hybrid).

The main result of this work is the original modellingmethod for a non-linear system from input-output databy linear Piecewise Affine model structure. The methodis inspired by neural network modelling techniques. Thetechnique is based on modified one-hidden-layer neuralnetwork with orthogonal activation functions.In terms of clearness there were considered just two -dimensional static non-linear systems in the paper. How-ever, the modelling technique can be used for a dynamicsystem as well, see [29, 30]. It is worth to mention fastlearning ability of used network during the on-line and off-line training. Thanks to the fact that the algebraic outputof the OAF network is linearly separable function, it is al-ways possible to transform multi-variable approximationproblem to problem of a series of a single variable ap-proximations. During the research, new Matlab toolboxfor PWA approximation has been developed. The toolboxsimplifies and accelerates identification of so-called PWA-OAF model using Graphical User Interface.The modelling technique has a wide applicability in au-tomotive, power electronics, computer graphics, etc..

References[1] A. Bemporad. An efficient technique for translating mixed logical

dynamical systems into piecewise affine systems. Proceedings ofthe 41st IEEE Conference on Decision and Control 2002,(December):1970–1975, 2002.

[2] A. Bemporad. Hybrid Control Toolbox - Manual, 2004.[3] A. Bemporad and M. Morari. Control of systems integrating

logic, dynamics, and constraints. Automatica, 35(3):407–427,Mar. 1999.

[4] A. Bemporad, M. Morari, V. Dua, and E. N. Pistikopoulos. Theexplicit linear quadratic regulator for constrained systems.Automatica, 38(2):3–20, 2002.

[5] F. Borrelli, A. Bemporad, M. Fodor, and D. Hrovat. AnMPC/hybrid system approach to traction control. IEEE Trans.Contr. Systems Technology, 14:541–552, 2006.

[6] M. K. Chattopadhyay and S. Tokekar. Analytical model for thetransconductance of microwave GaN HEMTs including nonlinearmacroscopic polarization and parasitic MESFET conduction.Microwave and Optical Technology Letters, 49(2):406 – 407,2007.

[7] B. C. Csáji. Approximation with Artificial Neural Networks. PhDthesis, Faculty of Sciences Eötvös Loránd University Hungary,2001.

[8] S. Dehuri. A Novel Learning Scheme for Chebyshev FunctionalLink Neural Networks. Advances in Artificial Neural Systems,2011:1–10, 2011.

[9] G. Ferrari-Trecate. Hybrid Identification Toolbox (HIT), 2005.[10] T. Geyer, G. Papafotiou, and M. Morari. Model Predictive Control

in Power Electronics: A Hybrid Systems Approach. Control,(3):5606–5611, 2005.

[11] S. Heath. Embedded Systems Design. Butterworth-Heinemann,MA, Newton, MA, USA, 1.st edition, 1997.

[12] W. Heemels, B. De Schutter, and A. Bemporad. Equivalence ofhybrid dynamical models. Automatica, 37(7):1085–1091, 2001.

[13] W. Heemels, B. De Schutter, and A. Bemporad. On theequivalence of classes of hybrid dynamical models, 2001.

28 Stevek, J.: Intelligent Embedded Systems

[14] T. A. Johansen, W. Jackson, R. Schreiber, and P. Tondel.Hardware Synthesis of Explicit Model Predictive Controllers.IEEE Transactions on Control Systems Technology,15(1):191–197, 2007.

[15] P. Julian, a. Desages, and O. Agamennoni. High-level canonicalpiecewise linear representation using a simplicial partition. IEEETransactions on Circuits and Systems I: Fundamental Theory andApplications, 46(4):463–480, Apr. 1999.

[16] M. Kvasnica. Real-Time Model Predictive Control viaMulti-Parametric Programming: Theory and Tools. VDM Verlag,Saarbruecken, 2009.

[17] C. T. Leondes. Neural Network Systems Techniques andApplications - Volume 7: Control and Dynamic Systems.Academic Press, 1997.

[18] S. Mariéthoz, U. Mäder, and M. Morari. High-speed FPGAimplementation of observers and explicit model predictivecontrollers. In IEEE IECON, Ind. Electronics Conf., Porto,Portugal, 2009.

[19] S. Mariéthoz and M. Morari. Explicit Model-Predictive Control ofa PWM Inverter With an LCL Filter. IEEE Transactions onIndustrial Electronics, 56(2):389–399, 2009.

[20] S. Mariéthoz, M. Morari, and A. G. Beccuti. Analysis and optimalcurrent control of a voltage source inverter connected to the gridthrough an LCL filter. 2008 IEEE Power Electronics SpecialistsConference, (2):2132–2138, June 2008.

[21] F. Oldewurtel, D. Gyalistras, M. Gwerder, C. Jones, A. Parisio,V. Stauch, B. Lehmann, and M. Morari. Increasing energyefficiency in building climate control using weather forecasts andModel Predictive Control. In 10th REHVA World Congress Clima,pages 9–12, 2010.

[22] S. Richter and S. Mariéthoz. High-speed online MPC based on afast gradient method applied to power converter control. AmericanControl Conference ACC 2010, pages 4737–4743, 2010.

[23] J. Roll, A. Bemporad, and L. Ljung. Identification of piecewiseaffine systems via mixed-integer programming. Automatica,40(1):37–50, Jan. 2004.

[24] M. F. Sevat. GLASMOST: A mosfet model of high numericalquality. In International Symposium on Circuits and Systems,pages 2597–2600, 1988.

[25] E. D. Sontag. Nonlinear regulation: The piecewise linearapproach. IEEE Transactions on Automatic Control,26(2):346–358, 1981.

[26] A. Szucs, M. Kvasnica, and M. Fikar. MATLAB Toolbox forAutomatic Approximation of Nonlinear Functions. InProceedings of the Process Control 2011, Štrbské pleso, HighTatras, Slovakia, 2011.

[27] R. Vidal, S. Soatto, and S. Sastry. An algebraic geometricapproach to the identification of a class of linear hybrid systems.42nd IEEE International Conference on Decision and ControlIEEE Cat No03CH37475, (December):167–172, 2003.

[28] J. L. Villa, M. Duque, A. Gauthier, andN. Rakoto-Ravalontsalama. Translating PWA systems into MLDsystems. Proceedings of the 2004 IEEE International Symposiumon Intelligent Control 2004, pages 37–42, 2004.

[29] J. Števek and S. Kozák. Improved Piecewise LinearApproximation of Nonlinear Functions in Hybrid Control. InIFAC Congress 2011, Milan, Italy, 2011.

[30] J. Števek and S. Kozák. Matlab Toolbox For PWA Approximationof Nonlinear Systems. In Proceedings of the Process Control2011, Štrbské pleso, High Tatras, Slovakia, 2011.

[31] H. P. Williams. Model Building in Mathematical Programming,volume 4 of A Wiley - Interscience Publication. John Wiley &Sons, third edit edition, 1999.

[32] F. Zanini, D. Atienza, L. Benini, and G. De Micheli.Thermal-Aware System-Level Modeling and Management forMulti-Processor Systems-on-Chip. infoscience.epfl.ch, pages2481–2484, 2011.

[33] C. Zhu, D. Shukla, and F. W. Paul. Orthogonal Functions forSystems Identification and Control. In C. T. Leondes, editor,Control and Dynamic Systems, pages 1–73. Academic Press,London, 1997.

Selected Papers by the AuthorJ. Števek, Š. Kozák. Improved Piecewise Linear Approximation of

Nonlinear Functions in Hybrid Control. In Proceedings of the18th IFAC Worls Congress 2011, 28.8.-2.9. 2011, Milan Italy,vol. 18: 14982–14987, 2011.

J. Števek, Š. Kozák. Application of Hybrid Modeling and Control forAir-Handling Unit. In International Review of AutomaticControl, Vol. 3, No. 3 ,pages 317–323, 2010.

J. Števek, Š. Kozák. New Technique of PWA Approximation ofNon-linear Systems. In Kybernetika - international journalpublished by ÚTIA, Czech Republic, 2011. - submitted/treated.

J. Števek, Š. Kozák. Hybrid Modeling and Control of ThermalSystem. In Journal of Cybernetics and Informatics, volume 10,pages 49–60. 2010.

J. Števek, Š. Kozák. Modeling and Control of Complex Systems. InProceedings of International Conference Cybernetics andInformatics, Vyšná Boca, Slovak Republic, February 10 - 13,2010.

J. Števek, Š. Kozák. Matlab Toolbox For PWA Identification ofNonlinear Systems. In Proceedings of International ConferenceProcess Control, Štrbské Pleso, Slovakia, pp. 111-118, 2011.

J. Števek, Š. Kozák Využitie fuzzy logiky v riadení procesovsystémami Xenta od firmy Schneider Electric. In iDB journal,February 2011.