IN DEGREE PROJECT ELECTRICAL ENGINEERING,SECOND CYCLE, 30 CREDITS

, STOCKHOLM SWEDEN 2016

Phase-based Extremum Seeking Control

SUYING WANG

KTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF ELECTRICAL ENGINEERING

Phase-based Extremum Seeking Control

SUYING WANG

Stockholm 2016

Automatic ControlSchool of Electrical Engineering

KTH Royal Institute of Technology

IR-EE-Dummy 2016:157

Phase-based Extremum Seeking Control

Suying Wang

October 25, 2016

Abstract

Extremum Seeking Control (ESC) is a model-free adaptive control method

to locate and track the optimal working point for nonlinear plants. However,

as shown recently, traditional ESC methods may not work well for dynamic

systems. In this thesis, we consider a novel ESC loop to locate the optimal

operating point for both static and dynamic systems. Considering that the

phase-lag of the system undergoes a large shift near a steady-state optimum

and reaches the value of ⇡/2 at the optimal operating point, the novel ESC

applies the phase-lag of the target system to track the optimum. An ex-

tended Kalman filter is used to ensure the accuracy of the phase estimation.

The structure of a phase locked loop (PLL) is employed in combination with

an integral controller to lock the phase near ⇡/2, such that the target system

will operate near the optimal working point. The controller is demonstrated

by application to optimization of the substrate conversion in a chemical re-

actor.

Keywords: extremum seeking control, phase estimation, phase locked loop,

dynamic system, online optimization.

Sammanfattning

Extremsokande reglering (ESC) ar en modellfri adaptiv reglermetod som kan

anvandas for att lokalisera den optimala arbetspunkten i olinjara processer.

Det har nyligen visats att det finns problem med traditionell ESC om det

reglerade systemet ar dynamiskt. I den har avhandlingen behandlar vi en

ny metod for extremsokande reglering som ar applicerbar for bade statiska

och dynamiska system. Metoden ar baserad pa att reglera processens ar-

betspunkt tills det lokala fasskiftet hos processen nar ⇡/2. Resultatet ar

baserat pa det faktum att fasskiftet hos processer generellt forandras kraftigt

kring optimum, och for laga frekvenser motsvarar optimum ett fasskift pa

⇡/2 radianer. Regulatorstrukturen som anvands liknar en faslast slinga

(PLL). Ett olinjart Kalmanfilter anvands for att estimera fasen och en inte-

grerande regulator anvands for att justera arbetspunkten tills fasen nar det

onskade fasskiftet. Resultaten ar illustrerade i ett exempel dar den nya regu-

latorstrukturen anvands for att optimera produktionen i en kemisk reaktor.

Acknowledgement

First of all, I would like to express my sincere gratitude to my supervisor

Olle Trollberg, for his constant encouragement and guidance. He has walked

me through all the stages of my thesis, leading me into the world of extremum

seeking control. Without his consistent and illuminating instruction, the

thesis could not have reached this final stage.

Second, I would like to express my heartfelt gratitude to my examiner

Professor Elling W. Jacobsen, for his excellent instructions and guidance on

my thesis project. Without his patient instruction, insightful criticism and

expert guidance, I would not be able to complete my thesis.

I feel grateful to all the professors and teachers at KTH, who o↵ered me

valuable courses and advice during my study.

Last but not least, I am truly grateful and thankful to Nan Qi and Diliao

Ye, for giving me lots of suggestion during my thesis period. I would also

like to thank my parents and my boyfriend for providing support. Their

encouragement and unwavering support has sustained me through frustration

and depression.

Thank you all!

Contents

1 Introduction 1

1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3 Structure of the Report . . . . . . . . . . . . . . . . . . . . . . 3

2 Background 4

2.1 Classic Extremum Seeking Control . . . . . . . . . . . . . . . 4

2.1.1 Sliding mode Extremum Seeking Control . . . . . . . . 6

2.1.2 Perturbation based Extremum Seeking Control . . . . 7

2.1.3 Phase in Perturbation based Extremum Seeking Control 11

2.2 Phase-based Extremum Seeking Control . . . . . . . . . . . . 12

3 Design of the Control Loop 18

3.1 Phase Locked Loops . . . . . . . . . . . . . . . . . . . . . . . 18

3.1.1 Adapting a PLL for ESC . . . . . . . . . . . . . . . . . 20

3.2 Phase Estimation . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.2.1 Estimation by Demodulation . . . . . . . . . . . . . . . 21

3.2.2 Updated Estimation from Variant of EPLL . . . . . . . 23

3.2.3 Estimation by Kalman Filter . . . . . . . . . . . . . . 24

3.3 Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.4 Selection of the Controller Structure . . . . . . . . . . . . . . 28

4 Controller Tuning 31

4.1 Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.2 Controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

4.3 Perturbation Signal . . . . . . . . . . . . . . . . . . . . . . . . 35

4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5 Example and Analysis 38

5.1 Performance Test . . . . . . . . . . . . . . . . . . . . . . . . . 39

5.2 Robustness Test . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

I

6 Conclusions and Further Research 47

6.1 Disscussion and Conclusion . . . . . . . . . . . . . . . . . . . 47

6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Bibliography 50

II

Chapter 1

Introduction

In process industry today, optimization is essential for most aspects of process

operation. For example, in the production process, manufacturers might

prefer to either maximize the output of a process or minimize the power

consumption; when designing a car, it is often desired to minimize the fuel

consumption. In order to reduce the cost and maximize the profits, processes

should be designed and operated both e↵ective and cost-e�cient, preferably

in combination with a low workload for the process operators. In this thesis,

we specifically focus on optimization within the control layer of a process. We

try to determine a control law to, for example, maximize process throughput

or minimize the consumption of energy or raw materials.

The optimization problem could be solved either online or o✏ine. If the

operating conditions of the plant are stable and the optimum does not vary

over time, we could do a static o✏ine optimization to find the optimal oper-

ating point and keep the process working at the optimum using a regulator.

However, o✏ine methods cannot help us locate the optimal operating point

for some systems due to disturbances. These disturbances may vary a lot,

making the location of the optimum uncertain. In order to accurately lo-

cate the optimum, we should try to do the optimization online, obtaining

a feedback based solution. When feedback is introduced in the system, the

sensitivity towards uncertainty and modelling errors would be reduced. In

addition, we might be able to track the optimal operating point over time

with the help of the feedback signals, which may improve the performance

of the controller.

Several methods could be applied to locate the optimal operating point

online. For example, we could locate the optimal operating point online using

Model Predictive Control (MPC) method [1]; self optimization control could

also be an alternative to solve the optimization problem in online model-based

situations [2]; Adaptive control is another choice for online optimization,

1

which can track the optimal operating point when the parameters of the

system are uncertain. All these methods can be used to locate the optimal

operating point when the model of a process is available. However, a model

is not always available or possible to derive. Forms of system equations, as

well as the parameters, are often unknown. In addition, the unmeasured

disturbances may vary over time, e.g., solar power plant operation depends

on the weather, dust on the panel, humidity, etc., which is hard to forecast

and varies a lot. Moreover, the time and resources available may not be

enough to develop an accurate model. Therefore, a model-free method is

required to perform process optimization.

Extremum Seeking Control (ESC), which could be regarded as a branch

of adaptive control, might be a choice for model-free optimization. ESC

is an online model-free method, which only relies on output measurements.

It mainly focuses on the gradient of the steady-state map. The gradient

will be zero when the output of the system reaches the maximum/minimum

point. Therefore, optimal operating point could be found when the gradient

reaches zero. Although traditional ESC can be used for the optimization of

online model-free systems, it has some limitations when applied to nonlinear

dynamical plant. For example, the plant has to be quasi-static and the

adaption gain should be small when the traditional perturbation ESC is

applied [3].

Trollberg and Jacobsen introduced a novel idea about model-free opti-

mization [4], which is the basic idea of this thesis. In [4], the relationship

between dynamic and static properties of plants with general nonlinear dy-

namics are investigated. It is shown that the steady-state optimal operating

point is not only reflected by a zero gradient of the equilibrium map, but

also in the local phase-lag of the system. At the optimal working point, the

phase-lag of the system will approach ±⇡/2 due to a bifurcation of the plants

zero dynamics. The novel idea is so interesting that the idea is applied to do

the optimization, i.e. phase-based extremum seeking control.

1.1 Problem Statement

This thesis tries to design a novel extremum seeking control loop based on

phase information as suggested in [4]. The method for phase estimation

should be chosen carefully to estimate the phase accurately. The structure of

the control loop should be designed to achieve the goal, i.e., locate the optimal

operating point. In addition, a guidance for parameters tuning should be

considered such that the best performance could be achieved.

2

1.2 Contribution

In this thesis report, we show that the optimization problem could be solved

by the novel method, using the structure of phase locked loops (PLLs).

We analyze the impact of the parameters in the controller and provide

a guide for controller tuning. This analysis is performed when the Kalman

filter is used to do the phase estimation, and the control part of the controller

is an integral controller.

We perform several simulations illustrating the performance of the method.

These simulations show the impact of the various parameters, which is the

same as our analysis.

1.3 Structure of the Report

The remainder of the thesis report is structured as follows. In Chapter 2,

we provide the necessary background. Here we describe the ESC problem

in more depth, and discuss how the phase may be utilized to locate the

steady-state optimum.

In Chapter 3, we go on and consider the design of the novel control loop.

The structure, the estimator as well as the controller are discussed in detail

in this part.

Chapter 4 provides a guide for controller tuning. Guidance for parameter

tuning for the estimator, the controller as well as the perturbation signal are

analysed in detail.

In Chapter 5, several simulations are performed to illustrate the perfor-

mance of the novel method.

Finally, Chapter 6 gives an overview of the thesis as well as the future

work.

3

Chapter 2

Background

This chapter provides the background of the thesis project. Section 2.1 in-

troduces the classic extremum seeking control in detail. In Section 2.2, the

working scheme of phase-based ESC is discussed.

2.1 Classic Extremum Seeking Control

Extremum seeking control is an online model-free optimization method based

on the feedback from output measurement. It is used to locate and track

the optimal operating point of a given plant, when no model information is

available. We should note that an implicit assumption in ESC is that an

optimum exists.

In addition, we only consider SISO (Single Input Single Output) systems

in this thesis. In general, ESC relies only on the feedback from output

measurements. For static plants, i.e., plants without dynamics, or memory,

the control target of ESC is the output of the system. For dynamic plants,

however, ESC tries to locate the extremum steady-state output. That is, if

we consider the state space representation of a general nonlinear plant:

x = f(x, ✓),

y = h(x),(2.1)

where f is the state equation and h is the output equation. Then the steady

states are defined when the derivative of the states is zero, i.e.,

f(xss, ✓) = 0,

yss = h(xss),(2.2)

where yss is the steady-state output and xss is the steady state.

4

Figure 2.1: Relationship between control, adaptive control and extremum

seeking control.

The idea of extremum seeking control could be traced back to 1922 [5].

Some studies were performed in Russia during the Second World War [6].

In the mid-20th century, researchers put a lot of e↵ort into ESC. For ex-

ample, Draper and Li detailed an extremum seeking control algorithm for

internal combustion engines and the performance was analyzed [7]. Obabkov

discussed multichannel ESC [8]. The first rigorous assessment of stability

of ESC was published in 2000 [3]. In this study, Krstıc and Wang proved

local stability of a near optimal solution for a general set of dynamic plants.

This study renewed the interest in the field of ESC as is evident in the large

number of publications following the break through. Nowadays, quite a few

applications are based on ESC, e.g., Anti-Lock Brake System(ABS) system,

biology system, etc. [9].

Extremum seeking control may be considered as a subfield of adaptive

control. In adaptive control, the model used by the controller is updated

online, using the information available in the measurements. A typical adap-

tive controller consists of two separate loops: one is a normal feedback loop,

the other is a parameter-adjusting loop. In extremum seeking control, the

model, which is adapted, is essentially the local gradient of the equilibrium

map. A feedback loop is then applied to drive the system to a point, where

the gradient is zero. Figure 2.1 relates extremum seeking control to adaptive

control and control in general.

There are several extremum seeking control methods, e.g., gradient-based

ESC, sliding mode ESC, perturbation-based ESC, etc. [10].

Many methods in ESC, e.g., gradient-based ESC, perturbation-based

ESC, are based on the gradient information of the performance function.

In these methods, it is assumed that the the cost function of target system

is continuous and has continuous derivatives. The optimal operating point

5

for the system is the point with zero gradient. These methods can be re-

garded as an approximation of gradient descent. Therefore, the gradient

descent method is introduced first. Gradient descent is a first-order opti-

mization method. It works as follows. Assume that we want to minimize the

performance function

y = J(✓), (2.3)

where y is the output, ✓ is the input and J is the performance function,

which is di↵erentiable. Then, y decreases fastest if it goes from J(✓0) in the

direction of the negative gradient of J at (✓0, J(✓0)). rJ(✓0) represents thegradient of J at point (✓0, J(✓0)). Assume � is positive and small enough, we

can get J(✓1).J(✓1) = J(✓0)� �rJ(✓0). (2.4)

The term �rJ(✓0) is subtracted from J(✓0), i.e., the point is moved down

toward the minimum. If we preform this repeatedly, we can get a sequence

of point J(✓0), J(✓1), · · · such that

J(✓n+1) = J(✓n)� �nrJ(✓n), n � 0, (2.5)

i.e.,

y0 � y1 � y2 � · · · . (2.6)

Therefore, the sequence may converge to the desired local minimum.

Gradient descent method can be used to find the minimum point. The

initial point can be set at any point. However, this method has some limita-

tions. The converge speed is relatively slow when the operating point is close

to the minimum. Moreover, the value of � can influence the convergence.

Smaller � leads to slower converge speed but on the other hand, the system

may diverge if a large � is employed.

Although many methods in extremum seeking control are based on the

gradient, sliding mode ESC is a notable exception. It dose not rely on any

estimation of the gradient.

2.1.1 Sliding mode Extremum Seeking Control

Sliding mode ESC is a kind of traditional ESC. In sliding mode control,

a discontinuous control law is applied to alter the dynamics of nonlinear

plants to slide along predetermined switching surfaces in state space. In

1974, Korovin and Utkin proposed a nonlinear programming method for

optimization of static plants based on sliding mode [11]. This was later

further developed by Drakunov and Ozguner, applying this method on ESC

6

[12]. The basic idea of sliding mode ESC is described as follows. Assume we

would like to maximize the output of a static system

y = J(✓), (2.7)

where J is referred as the performance function, ✓ 2 < is the control input

and y 2 < is the output. The basic idea in sliding mode based ESC is to force

the output to follow a given reference signal using a sliding mode controller.

The aim of the controller is to force the output to be ever increasing with

a specified rate k. By doing so, the output will eventually reach a local

maximum. In sliding mode ESC, we should first choose any ever-increasing

function g(t) with slope of k as reference. The error between the reference

signal g(t) and the system output y(t) is e(t) = y(t)� g(t). The input signal✓ is adjusted at a certain rate and the direction is decided by the error e(t).Then, if the controller is tuned properly, it will lock onto a switching surface

and approach the optimum at the same rate of g(t). When the working point

is near the optimum, the increase rate of y(t) cannot be sustained and the

controller will oscillate around the optimum.

2.1.2 Perturbation based Extremum Seeking Control

Perturbation based ESC is another kind of ESC, which is based on the gra-

dient information. It is developed for static systems, like y = J(✓), where Jis the performance function, y 2 < is the output and ✓ 2 < is the input. The

optimal operating point can be found when the gradient of the performance

function reaches zero. Therefore, perturbation based ESC tries to adjust the

input to make the gradient approach zero. The method works essentially as

follows.

The perturbation based ESC scheme is shown in Figure 4.3. This scheme

has five elements, namely, target system, perturbation signal, high-pass filter,

low-pass filter and the controller. This method is based on the gradient of the

performance function. Therefore, to ensure that the gradient information is

available in the output, a perturbation a sin(!t) signal is added to the input.

The relationship among controlled input, estimated optimal input and the

perturbation signal is

✓ =

ˆ✓ + a sin(!t), (2.8)

where ✓ is the controlled input,

ˆ✓ is the estimated optimal input.

Choice of perturbation signal is important in the perturbation based ESC.

On one hand, a large a is needed to get a good signal to noise ratio in order to

get a good estimation of the gradient. Additionally, a comes as a proportion-

ality constant in the gradient estimate and will a↵ect the convergence rate.

7

Figure 2.2: Structure of perturbation-based extremum seeking control.

Therefore, a larger a is preferred. But on the other hand, large fluctuations

in either the input or output is not what we desired. As the perturbation is

made larger, nonlinearities will become more important. In addition, large

fluctuations may put more wear on the equipment and it might be preferred

to minimize the variation in the output in some applications. Therefore, a

small a is preferred under this consideration. A trade o↵ should be made

in order to strike a balance between these two considerations. The value

of a should be chosen properly to get a good estimation while keeping the

fluctuations as small as possible.

In the output signal, only the variation contains the gradient informa-

tion. Therefore, a high-pass filter is applied to remove the bias of the target

output such that only variation is kept. Thus, the break o↵ frequency of

the high-pass filter should be set to a value lower than the frequency of the

perturbation signal.

The amplitude of the variation is then extracted into a constant via de-

modulation. It introduces high frequency mode as well, as shown in Equation

2.9,

A sin(!t) ⇤ sin(!t) = 0.5A(cos(0)� cos(2!t)), (2.9)

where the first term on the RHS is a constant and the second term is a high

frequency mode. Then, a low-pass filter is employed to eliminate the high fre-

quency mode, retaining the constant term. Therefore, the cut-o↵ frequency

of the filter can be set to any non-zero frequency below the perturbation

8

frequency. The remaining constant term is proportional to the amplitude of

the variation in the output and hence also the local gradient. Consequently,

the output of the low-pass filter becomes an estimate of the gradient. It is

not the exact value of the gradient, but proportional to the gradient.

An integral controller is employed to move the operating point towards

the optimum based on the gradient information from the low-pass filter. The

value of control gain may influence the whole control system. The gain es-

sentially decides the bandwidth of the controller, which indicates the speed

of the controller. The bandwidth of the controller should normally be lower

than that of the gradient estimator, since the updated gradient information

should be provided to the controller. If the speed of the controller is higher

than that of the estimator, the controller cannot get the updated gradient in-

formation. As a result, the controller may make the whole system unstable.

The bandwidth of the gradient estimator is decided by the break-o↵ fre-

quency of the low-pass filter, which is decided by the perturbation frequency.

Therefore, the value of the controller gain k should be chosen according to

the value of perturbation frequency.

When these five elements are settled, the control loop can locate the

optimal operating point automatically. Consider a static nonlinear system,

for which the performance function can be represented as

y = J(✓), (2.10)

where y 2 < is the output, ✓ 2 < is the input and J is the static plant.

Assume that the system output has a maximum value and that the controller

gain k is positive. The optimal input is represented by ✓⇤ while the estimated

optimal input could be written as

ˆ✓. The input error

˜✓ could be represented

as

˜✓ = ✓⇤ � ˆ✓. (2.11)

Since J(✓) has a maximum value, the linear part in its Taylor expansion is

zero if we evaluate at ✓ = ✓⇤. If we do the second-order Taylor expansion

near ✓⇤ and drop the higher order elements, we have

J(✓) ⇡ J(✓⇤) +J

00(✓⇤)

2

(✓ � ✓⇤)2, (2.12)

where J00(✓) is the second-order derivative of J(✓). As shown in Figure 4.3,

the input ✓ equals to the sum of

ˆ✓ and a sin(!t). Substituting ✓ in Equation

2.12 and expanding the square, we get

y ⇡ J(✓⇤) +J

00(✓⇤)

4

a2 +J

00(✓⇤)

2

˜✓2 � aJ00(✓⇤)˜✓sin(!t)� J

00(✓⇤)

4

a2 cos(2!t).

(2.13)

9

The signal y is passed through the high-pass filter, eliminating constant

terms.

y � ⌘ ⇡ J00(✓⇤)

2

˜✓2 � aJ00(✓⇤)˜✓sin(!t)� J

00(✓⇤)

4

a2 cos(2!t). (2.14)

The output of the high-pass filter is demodulated by multiplication with the

original perturbation. The demodulated signal is then passed through the

low-pass filter. Here we assume that the filter is perfect and completely

removes any components with frequency above the breako↵ frequency. The

controller input ⇠ can then be calculated as follows.

a sin(!t)(y � ⇠) ⇡ aJ00(✓⇤)

2

˜✓2 sin(!t)� a2J00(✓⇤)˜✓sin2

(!t)

� J00(✓⇤)

4

a3 cos(2!t) sin(!t),

⇠ ⇡ �a2˜✓J00(✓⇤)

2

.

(2.15)

After di↵erentiating on both sides of Equation 2.11, we could have

˙

˜✓ = � ˙

ˆ✓.Therefore, after the controller, the rate of change of the input error

˜✓ can be

represented as

˙

˜✓ = � ˙

ˆ✓ = �k ⇤ ⇠ ⇡ ka2J00(✓⇤)

2

˜✓. (2.16)

To solve the Equation 2.16, we use separation of variables.

d˜✓

dt⇡ ka2J

00(✓⇤)

2

˜✓, (2.17)

which can be rewritten as

d˜✓˜✓

⇡ ka2J00(✓⇤)

2

dt. (2.18)

Further, we have

ln˜✓ � ln˜✓0 ⇡ka2J

00(✓⇤)

2

(t� t0), (2.19)

where t0 is the initial time and

˜✓0 is the initial input error. Finally,

˜✓ is

obtained and can be given as

˜✓ ⇡ eka2J

00(✓⇤)

2 (t�t0)+ln✓0 . (2.20)

10

According to our assumption, k is positive while J(✓) has maximum value,

hence,

J00(✓) < 0, (2.21)

which means

ka2J00(✓⇤)

2

< 0. (2.22)

From the equations above, it follows that

lim

t!1˜✓(t) ⇡ lim

t!1e

ka2J00(✓⇤)

2 (t�t0)+ln✓0= 0. (2.23)

which implies that limt!1 ˆ✓(t) ⇡ ✓⇤. In other words, the input will approach

the optimal value when time goes to infinity. However, this analysis about

the optimum is valid only locally. If the start point is far from the optimum,

this analysis will not be valid any more.

2.1.3 Phase in Perturbation based Extremum Seeking

Control

The analysis above is valid only for static system. A di↵erence between static

systems and dynamic systems is that the latter will introduce a phase lag.

Perturbation based ESC works also for dynamic systems. Krstıc and Wang

applied perturbation based ESC to a dynamic system in 2000 [3]. They used

asymptotic methods that essentially brought the problem back to the static

case. However, their method require very slow estimation and control.

We will now investigate the e↵ect of the perturbation on the various signal

in the loop. Consider G(s) as a local linear approximation of the nonlinear

system at a stationary solution. For simplicity, we assume that

ˆ✓ is constant.

When the perturbation signal

ˆ✓ + a sin(!t) is set as an input of the control

loop, the frequency response of the system is given by G(i!), and |G(i!)| isthe gain of the frequency response. According to Figure 4.3, the stationary

output of the high-pass filter can be written as

y � ⌘ = a|G(i!)||HH(i!)|sin(!t+ �), (2.24)

where a is the amplitude of the input signal, |G(i!)| is the gain of the fre-

quency response, |HH(i!)| is the gain of the frequency of the high-pass filter,

� is the phase-lag of the output of the high-pass filter and ! is the frequency

of the perturbation signal.

Multiplied by the perturbation signal a sin(!t), y� ⌘ is transformed into

the signal shown below

a sin(!t)(y � ⌘) = �a2

2

(cos(2!t+ �)� cos(�))|G(i!)||HH(i!)|. (2.25)

11

To remove the high frequency components, we proceed the signal through

the low-pass filter and get the input of the controller ⇠, i.e.,

⇠ =

a2

2

cos(�)|G(i!)||HH(i!)||HL(0)|, (2.26)

where HL(0) is the amplitude of the frequency response of the low-pass filter.

The aim of the controller is then to make the signal ⇠ be zero. The value

of |HH(iw)| and |HL(0)| are known since the filters and the perturbation

frequency are selected by users. In addition, a is non-zero. Therefore, either

|G(iw)| or cos(�) has to be zero. For static systems, when ! = 0, |G(0)|is the gradient. When ⇠ reaches zero, the gradient |G(0)| reaches zero as

well. Then, the optimal operation point can be found. For dynamic systems,

however, G(iw) is unlikely to be zero since it implies that all dynamics are

gone at the optimal working point. Therefore, it is likely that it is the phase

condition which is fulfilled at the optimum. This indicates that the phase-lag

is tied to the optimal operating point and it will be further discussed in next

section.

2.2 Phase-based Extremum Seeking Control

According to [4], there is a connection between phase and optimality. It is

mentioned that there is a large phase-shift near the extremum point. The

phase-lag will reach ±⇡/2 when the system is operating at the optimum.

We are going to show how to make use of the phase-lag to find the optimal

operating point.

Assume that a nonlinear system can be represented as

x = f(x, ✓),

y = h(x),(2.27)

where x is the state, ✓ 2 < is the input and y 2 < is the output. Assume

that the steady state is parametrized by the input ✓,

x = 0, i↵x = I(✓). (2.28)

Therefore, the steady-state input-output relationship is

y = h(I(✓)). (2.29)

Assume that the steady-state input-output relationship is as shown in Figure

2.3 and G✓(s) is the transfer function of the above defined system at the

steady state corresponding to ✓. We then have

G✓(0) =dJ

d✓, (2.30)

12

Figure 2.3: Steady-state output-input relationship.

where J is the cost function.

Therefore, the gradient at point A is G✓A(0). Assume point B is the

optimal operating point, then the gradient at point B, i.e., G✓B(0), is zero.

The sign of the gradient will be changed when the operating point moves

across point B. That is, G✓A(0) is positive and G✓C (0) is negative.

Assume a finite dimensional state space representation is linearized, a

rational transfer function is in a form of

G✓(s) = Ksm + bm�1s

m�1+ · · ·+ b1s+ b0

sn + an�1sn�1+ · · ·+ a1s+ a0

, (2.31)

whereK is the gain of transfer function and a0, a1, · · · , an�1 and b0, b1, · · · , bm�1

are the coe�cients.

Then at the optimal point B, we have

G✓B(0) = 0 = Kb0a0

. (2.32)

In this equation, we have three parameters, K, a0 and b0. As the sign of

the gradient G✓(0) must switch when the input goes through the optimal

working point, at least one of the parameters among K, a0 and b0 should

13

Figure 2.4: The relationship between z(✓) and ✓.

change sign. We will analyse them one by one. If K changes sign, the value

of K will be zero at the optimal working point B. When this is the case, the

system will not have any dynamic response at the optimum. This is rare in

dynamic systems. Therefore, we don’t consider this situation. a0 is not the

parameter which changes sign, either. If a0 changes sign near the optimum,

it means that at least one pole in the transfer function should be infinity at

the optimum and change sign from +1 to �1 or the other way around. It is

impossible to move a pole from positive to negative through infinity. In other

words, it is impossible to change the sign of a0 through infinity. Therefore,

a0 should not changes sign. Upon the analysis above, only b0 changes sign

around the optimal operating point. Hence, for the rest of the analysis, we

assume only b0 changes sign.

If we rewrite Equation 2.31 as

G✓(s) = K(s+ z1)(s+ z2) · · · (s+ zm�1)(s+ zm)

(s+ p1)(s+ p2) · · · (s+ pn�1)(s+ pn), (2.33)

where z1, z2, · · · , zm represents the zeros and p1, p2, p3, · · · , pn represents the

poles. Comparing Equation 2.31 and Equation 2.33, we have b0 =

Qmi=1 zi.

Since b0 switches sign, an odd number of zeros has to switch sign. Assume

that only one zeros changes sign. Hence, the optimality condition correspond

to a zero switching sign through the origin. Assume the specific zero zm =

z(✓). An example of the relationship between z(✓) and ✓ is shown in Figure

2.4.

14

Then, we are interested in how the phase is a↵ected by such a zero-

crossing. G✓(s) could be rewrite as

G✓(s) = G0(s)(s+ z(✓)), (2.34)

where

G0(s) = k0(s+ z1)(s+ z2) · · · (s+ zm�2)(s+ zm�1)

(s+ p1)(s+ p2) · · · (s+ pn�1)(s+ pn). (2.35)

Therefore, the phase-lag can be expressed as the sum of arg(G0(i!)) and

arg(i! + z(✓)). The phase of G0(i!) will be discussed first.

According to Equation 2.35, we can obtain the phase of G0(s).

arg(G0(i!)) = arctan

!

z1+ arctan

!

z2+ · · ·+ arctan

!

zm�1

�(arctan

!

p1+ arctan

!

p2+ · · ·+ arctan

!

pn�1+ arctan

!

pn).

(2.36)

Property 1. When ! ! 0, the value of arg(G0(i!)) will approach kG0⇡,where kG0 = 0,±1,±2, · · · .Proof. As we assumed before, there is only one zero zm in G✓(s) will be zerowhen the system reaches the optimal operating point. According to Equation

2.34, G0(s) is a part of G✓(s), which does not contain z(✓). Therefore, zeros inG0(s) cannot be zero. Assume that the system is stable. Then, poles in G0(s)that cannot be zero either. Therefore, neither poles nor zeros in the G0(s)can be zero. As a result,

!ziwill approach zero, where i = 1, 2, · · · ,m�1 when

! approaches zero. Additionally,

!pj

will approach zero, where j = 1, 2, · · · , nwhen ! approaches zero as well. It is easy to find that arctan(0) = 0.

According to Equation 2.36, phase of G0(s) is the sum of the arctan results.

Hence, when ! ! 0, we have

arg(G0(i!)) ⇡ arctan(0) + arctan(0) + · · ·+ arctan(0)

� (arctan(0) + arctan(0) + · · ·+ arctan(0)) = kG0⇡,(2.37)

where kG0 = 0,±1,±2, · · · .As for the arg(i!+z(✓)) = arctan

!z(✓) , its di↵erent phase-shifts are shown

in Figure 2.5. With fixed frequency, phase-lag will change as the working

point ✓ changes. When the system reaches the optimum, the gradient of the

steady-state input-output map will be zero. Based on the discussion above,

G✓(s) will be zero, which indicates that the value of b0 will be zero as well.

In other words, the value of z(✓) should be zero, which means point (z(✓),!)will be on the imaginary axis. In this situation, the value of arg(i!+ z(✓)) is±⇡/2. When the operating point goes through the optimal operating point,

15

Figure 2.5: Di↵erent phase shifts at di↵erent frequencies.

z(✓) will go through the imaginary axis as well, which means the value of

arg(i! + z(✓)) will yield a large continuous phase shift around the optimal

operating point. When the frequency decreases, the value of the shift will

be close to ±⇡ near the optimal operating point, which will be explained in

detail the the following parts.

According to the discussion above, it can be investigated that, at the

optimal operating point, the phase-lag of the system will be

arg(G0(i!)) + arg(i! + z(✓)) = arg(G0)± ⇡/2. (2.38)

If small frequency ! is employed, the phase-lag will be close to

arg(G0(i!)) + arg(i! + z(✓)) = kG0⇡ ± ⇡/2 = ±⇡/2, (2.39)

where kG0 = 0,±1,±2, · · · . Thus, the value of arg(G0(i!)) can only influence

the sign of the phase-lag for low frequencies. Therefore, when the system

input is close to the optimal working point, the value of the phase-lag will

be close to ±⇡/2 and the phase shift near the optimum will be ±⇡ for low

frequencies. This novel optimality condition allows us to locate the optimum

by designing a controller, which drive the phase to such a specific value.

16

As discussed above, the value of arg(G0(i!)), the value of arg(i! + z(✓))and the phase-shift are all influenced by the value of !. In what follows,

we would like to show that as the frequency ! decreases, the phase-shift

increases. As shown in Figure 2.5, considering the frequencies !1 > !2 >!3 and z(✓1), z(✓2) are the value of the zero at two di↵erent near optimal

working points on either side of the optimum. The phase-shifts are di↵erent

at di↵erent frequencies, when the input ✓ changes the same value. With the

decrease of the frequency, the phase shift �� increases, i.e., ��1 < ��2 <��3. The phase changes more significantly at lower frequency. When the

frequency ! is zero, z(✓) moves on the real axis. If the working point moves

from ✓1 to ✓2, the phase will change from 0 to ⇡ accordingly, which means the

phase shift is ⇡. When it moves the other way around, the phase shift will

be �⇡ instead. Thus, when the frequency is close zero, the phase shift will

be close to ±⇡. In addition, when lower frequency is set, only the optimum

can make arg(i!+z(✓)) equal to ±⇡/2. It is possible to find the optimum by

the phase information. On the other hand, if large frequencies are employed,

the phase shift will be small. When the frequency is set to infinity, the phase

shift will be close to zero, which means a larger range of operating points

can make arg(i! + z(✓)) close to ±⇡/2. Under this situation, it is almost

impossible to find the optimal operating point any more. Therefore, it is

easier to locate the optimal operating point for lower frequencies.

This phase-based ESC method focuses on the phase of the target system.

A system come close to the optimal operating point when the phase-lag is

close to ±⇡/2. In order to locate the optimum, we should try to design

a controller to lock the phase-lag to ±⇡/2. A Phase Looked Loop(PLL)

could be used to lock the phase and frequency of the system output to a

certain value. And it is also a suitable structure for our controller. In the

next section, we will investigate how the structure of a PLL may by used for

phase-based ESC.

17

Chapter 3

Design of the Control Loop

In this chapter, we will discuss how estimation and regulation of the local

phase-lag could be utilized for extremum seeking control. A Phase locked

loop (PLL) has been used to accurately track the phase and frequency of

noisy input signals. Here we note that a similar structure is useful also for

phase-based ESC and thus we begin the chapter by reviewing the literature on

PLLs. We then go on and discuss how the elements of a PLL may be adapted

for our current purpose. Specifically, we discuss various alternative phase

estimation schemes and the impact of di↵erent loop filters, i.e., controllers.

Finally, we select a specific control structure which we analyse further in the

later chapters.

3.1 Phase Locked Loops

The technique of phase locked loops was described by Henry de Bellescize [13]

in 1932. The theory of PLL was well developed and widely used in modern

communication systems in 1970’s [14], [15], [16]. PLL was generally used to

detect and track the frequency of an incoming signal. An early application of

PLL was in analogue television where it was used to synchronize local sweep

rates with the frequency in the broadcast signal. Later, it was applied to

tune integrated circuits [17]. Today, PLL is frequently used in a variety of

applications ranging from space communications to network clocks [18].

A diagram of a basic phase locked loop is shown in Figure 3.1. It con-

tains three essential elements [18]: phase detector, loop filter, and voltage-

controlled oscillator (VCO). The phase detector compares the phase of the

input signal, i.e., reference signal, with the phase of the output signal in

order to generate the di↵erence of these two phases. The di↵erence of these

two signals is included in the output of the phase detector. The loop filter

18

Figure 3.1: A basic structure of phase locked loop.

acts as a feedback controller while the VCO behind changes the phase and

frequency of the output according to the output of the loop filter. When

the loop is locked, the frequency of the output signal is exactly the same as

the frequency of the input signal, i.e., reference signal, and the phase error

between reference signal and output signal is locked to a constant value as

well.

PLL estimates the phase error based on demodulation by multiplication.

It su↵ers from double-frequency errors introduced by the multiplication. In

order to solve this problem, Karimi and Iravani introduced an Enhanced

Phase Locked Loop (EPLL) structure [19], where the phase detection part

is re-organized. This enhanced PLL is robust with respect to both internal

settings and external noise. Furthermore, it allows the phase and amplitude

of the input signal to be estimated directly and independently. In [20], Karimi

did further improvement in EPLL, where an estimation loop is added. The

enhanced loop is able to estimate the amplitude of the input signal, which can

help the system get rid of the double-frequency errors. Patapoutian described

a PLL with the structure of Kalman filter in [21]. This system achieves rapid

acquisition and reliable tracking through replacing the constant gain with

a time-varying Kalman gain [22]. A novel PLL method for single-phase

system was proposed in [23]. Instead of the general structure, this method

generates the orthogonal voltage system, using a structure based on second

order generalized integrator. This method is easy to implement and is free

from frequency influence.

Phase locked loop focuses on the frequency error and aims at making

the output frequency equal to the input frequency. Once the frequencies

are equal, the PLL will be locked, and the phase error will remain at a

constant value. In order to locate the optimum based on the phase in the

phase-based ESC, we apply a perturbation signal to the input. Then, we

estimate the phase lag and try to regulate it to ±⇡/2 to locate the optimum.

Much of this resembles the problem addressed in PLL. But here we focus

19

on the phase error, whereas PLL often focuses on the frequency error. The

di↵erence between the two are only an integrator, since phase is the integral

of frequency. Thus, the problems are similar. In the following parts, we will

consider how the elements of a PLL may be utilized for ESC.

Among the structures mentioned above, the basic PLL could be a suitable

choice for dealing with the extremum seeking problem. Most structures dis-

cussed above are based on the basic one, e.g., the structures proposed in [20],

[19]. The majority of them enhanced one or two basic elements to improve

the performance of the PLL, while others carried out di↵erent structures,

e.g., the structure of Kalman filter, to achieve better performance. Among

all the structures, the basic structure is the simplest one. The novel ESC

focuses on locking the phase at a certain value, for which, the basic struc-

ture of PLL is enough. Therefore, it is not necessary to apply more complex

structures in the novel loop.

3.1.1 Adapting a PLL for ESC

The structure of a PLL can be modified for ESC by replacing the VCO with

the target system and adding an integrator into the loop filter. The reasons

are as follows. In the PLL structure, the VCO part changes the phase and

frequency of the output, and the loop filter acts as an feedback controller. In

the phase-based ESC, the target system changes the phase of the output for

di↵erent input signal, which is almost the same as what the VCO does. As

for the loop filter, an integrator component is essential, since ESC tries to

make the phase error, i.e., the output of the phase detector, be zero. When

the structure of PLL is used to do the phase-based ESC, the phase estimator,

i.e., the phase detection part, should be accurate and able to track a varying

phase. The reason is that all the control work is based on the phase estimated

and the phase of the output varies when the operating point changes. If the

estimator is not able to estimate and track the phase accurately, it might be

hard to show the true phase information of the target system, which may

lead to a poor performance of the whole control loop. Therefore, an accurate

estimation and tracking is essential in ESC. The controller, i.e., the loop

filter, should be free from a static error such that the real optimal operating

point can be achieved. In addition, the controller should work slower than the

estimator. If the estimator works slower than the controller, the controller

would not be able to get the updated phase information and could hardly

control the system to the optimal operating point. Moreover, a perturbation

signal should be added to the system as a part of the input signal. It is

employed to excite the plant such that the phase-lag of the target system

can be estimated.

20

Summarizing the discussion above, the ESC in the structure of PLL works

as follows:

1) The target system gives the output signal to the phase detector;

2) The phase detector part estimates and tracks the varying phase accurately,

and pass the phase information to the loop filter;

3) The loop filter adjusts the input signal of the target system with the phase

information;

4) The phase of the output will be changed since the input of the system

changes, entering the next loop.

When the phase of the output is equal to the reference phase, the control

loop will be locked, at which the optimal operating point is achieved.

3.2 Phase Estimation

An accurate estimation is the base of the control loop since the novel method

is phase based. In addition, the operation of the whole loop is based on the

estimated result. Therefore, it is important to choose a proper estimator.

In this section, we consider several di↵erent methods of phase estimator:

Estimation by demodulation, updated estimation from variant of EPLL, and

estimation by Kalman filter.

3.2.1 Estimation by Demodulation

We consider the basic estimator applied in the PLL first.

Description

The estimator in PLL is usually a multiplier as shown in Figure 3.2. In a

basic PLL, we have

Uref (t) = A1 sin(!1t+ �1),

Uo(t) = A2 sin(!2t+ �2),(3.1)

where Uref (t) is the reference signal to which we compare the phase of the

output, Uo(t) is the output signal for which we want to estimate the phase.

The output of the multiplier y(t) is

y(t) = Uref (t)⇥ Uo(t) = A1A2 sin(!1t+ �1) sin(!2t+ �2),

=

A1A2

2

[cos((!1 � !2)t+ �1 � �2)� cos((!1 + !2)t+ �1 + �2)].(3.2)

21

Figure 3.2: The standard estimator in PLL.

When the frequency of the two input signals are equal to each other, e.g.,

!1 = !2 = !, the output of the multiplier is

y(t) =A1A2

2

(cos(�1 � �2)� cos(2!t+ �1 + �2). (3.3)

High frequency term in the output of the multiplier y(t), i.e., cos(2!t+ �1 +

�2), is the double frequency error mentioned in the previous section. The

high frequency element could be attenuated by adding a low-pass filter with

break o↵ frequency lower than 2!. Assume the filter is an ideal low-pass

filter, we have

ˆ�error =A1A2

2

cos(�1 � �2). (3.4)

Therefore, the phase di↵erence between two signals can be estimated.

In the novel ESC, we aim at adjusting the phase di↵erence between the

perturbation and the system output to the value ⇡/2. We could set the

perturbation signal as the reference signal Uref (t), i.e.,

Uref (t) = a sin(!t), (3.5)

such that the structure is applied in a standard way. As the frequency of

the perturbation signal is set by the user, the frequency of the output signal

can be obtained if a suitable band-pass filter is added. The frequency of

the output signal could be adjusted to be the same as the perturbation

signal Uref (t), if the band-pass filter is designed to eliminate all the other

signals, leaving the signal with frequency ! only. Assume the output signal

is Uo(t) = A sin(!t + �). After the multiplier, the estimated phase error

22

would be

ˆ�error =aA

2

(cos(�)� cos(2!t+ �)). (3.6)

When an ideal low-pass filter is added, the high frequency term will be elim-

inated and the estimation of � could be obtained.

Properties

This demodulation method is able to estimate the phase in a simple way.

However, we might encounter problems with this method. For example,

we might get the double frequency errors by just multiplying two signals

together. However, these errors may be attenuated by adding a low-pass

filter after the multiplier. Another problem is that it is impossible to get the

phase directly. As shown in Equation 3.3, what we can deal with is only the

whole output signal, i.e.,

A1A22 cos(�1 � �2), instead of the phase information

�1��2. Both the value of amplitude

A1A22 and the value of phase �1��2 will

influence the output signal of the phase detector. Thus, when the amplitude

A1A22 is small, the value of the signal will be small as well, which may slow

down the converge speed.

3.2.2 Updated Estimation from Variant of EPLL

In [19], a new method for phase detection is proposed. We will discuss it in

detail in following parts.

Description

The structure of the phase detector is shown in Figure 3.3, where Uref is the

reference signal, A represents the estimated amplitude, Uo is the output of

VCO and e is the intermediary signal. The multiplier in conventional PLL is

replaced by three multipliers, one integration, one subtraction and a phase-

shift of 90 degrees. Instead of multiplying the input signal by the output

of VCO, a refined variant of the VCO signal is subtracted from the input

signal to produce an intermediary signal. Then the intermediary signal is

multiplied by the output of VCO. The estimated amplitude is the output of

the integration block of phase detector. The output of the loop filter is the

estimated time derivative of the total phase.

Properties

This method could provide the estimation of the amplitude and phase sepa-

rately. In addition, when the PLL is locked, the input and output angles are

23

Figure 3.3: A new estimator in PLL.

not only locked, but also equal. Moreover, this method can immune noise

and is robust with respect to externally imposed conditions. However, the

calculation of the phase information is related to the estimated amplitude.

Any change on amplitude will influence the estimation of the phase, which

may lead to a poor tracking ability of the phase.

3.2.3 Estimation by Kalman Filter

Estimation using Kalman filter is another alternative. We are going to in-

troduce how to estimate the phase-lag by Kalman filter in detail.

Description

Consider the nonlinear plant that we try to optimize, is operated with an

input signal

✓ = ✓0 + a sin(!t). (3.7)

Assume that the target system is stable, then the output in the stationary

situation is periodic, with the same base-frequency as the perturbation signal.

The output can hence be described by a Fourier series expansion

y(t) = c0 +

nX

k=1

(Ak sin(k!t) + Bk cos(k!t)), (3.8)

24

where c0 represent the constant part in y(t), Ak, Bk and k are the coe�cients.

The higher harmonics in the equation are generated by the nonlinearity and

they should be small if the amplitude of the input signal a is small.

At this point, the phase-lag is given by the fundamental components.

Therefore, we have

y(t) = A sin(!t) + B cos(!t) = C sin(!t+ �), (3.9)

where

C =

pA2

+B2,� = arctan

B

A. (3.10)

Therefore, the phase of the output could be calculated if the value of the

coe�cients A and B could be estimated.

A Kalman filter can be utilized to estimate these coe�cients. It assumes

that the model of a system is in the form

x(t) = F (t)x(t) + B(t)u(t) + w(t), w(t) ⇠ N(0, Q),

y(t) = H(t)x(t) + v(t), v(t) ⇠ N(0, R),(3.11)

where u(t) is the control input, y(t) is the system output, w(t) is the processnoise, v(t) is the measurement noise, x(t) is the state of the system, F (t)and B(t) are possibly time-varying and describe the state dynamics, H(t) isthe measurement model, Q and R are covariance matrix of process noise and

measurement noise respectively.

The predict-update model of Kalman filter is

˙x(t) = F (t)x(t) + B(t)u(t) +K(t)(y(t)�H(t)x(t)),

˙P (t) = F (t)P (t) + P (t)F (t)T �K(t)H(t)P (t) +Q,

K(t) = P (t)H(t)TR�1,

(3.12)

where K(t) is the Kalman Gain.

The initialization of Kalman filter is

x(t0) = E[x(t0)],

P (t0) = V ar[x(t0)],(3.13)

where x(t0) is the estimation of x(t) at time t0, P (t0) is the covariance matrix

of x(t) at time t0, E[·] represents the expected value and V ar[·] representsthe covariance.

In order to get an accurate estimation of the coe�cients A and B, the

model of observed output, i.e.,H(t), in the Kalman filter should be configured

as close to the true model as possible. As discussed in the beginning of this

25

section, the phase-lag is mainly given by the fundamental waveform and

harmonics contribute little on the phase-lag. Therefore, the observed model

should mainly be based on the fundamental waveform, i.e.,

y(t) = A sin(!t) + B cos(!t). (3.14)

However, if the true signal is more complex than the fundamental waveform,

the Kalman filter will try to fit this complex behaviour into just A and B,

which might lead to poor estimation. Therefore, it could be useful to have

additional terms in the Kalman filter. Then, the model of the output could

be set as

y(t) = c0 + A1 sin(!t) + B1 cos(!t) + A2 sin(2!t) + B2 cos(2!t). (3.15)

We set the state in Kalman filter as

x1 = c0,

x2 = A1,

x3 = B1,

x4 = A2,

x5 = B2.

(3.16)

Since we assume all the coe�cients are constant and fixed, we set the process

model in the Kalman filter as

F (t) =

2

66664

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

3

77775. (3.17)

B(t) = 0. (3.18)

The model of the output is set as

H(t) =⇥1 sin(!t) cos(!t) sin(2!t) cos(2!t)

⇤. (3.19)

With proper F , B, H selected, we can represent y(t) in the form assumed

by the Kalman filter. Then, the predict-update model can be written as

˙x(t) = K(t)(y(t)�H(t)x(t)),

˙P (t) = �K(t)H(t)P (t) +Q,

K(t) = P (t)H(t)TR�1,

H(t) =⇥1 sin(!t) cos(!t) sin(2!t) cos(2!t)

⇤.

(3.20)

26

All elements in Equation 3.20 are settled except the terms Q and R. We will

discuss the choice of Q and R in Chapter 4.

The amplitude C and phase of the output � can be calculated using

Equation 3.10, and we could get the amplitude and phase as

C =

qx2(t)

2+ x3(t)

2,

� = arctan

x3(t)

x2(t).

(3.21)

It is also possible to have other useful filters to filter out elements, such

as bias terms, higher order harmonics, etc., depending on how we design

the filter. A band-pass filter, with a lower cuto↵ frequency !L and a upper

cuto↵ frequency !H , could be added before the Kalman filter to allow a

lower order model for the Kalman filter. If the cuto↵ frequencies are set as

!L < ! < !H , the output of the band-pass filter, i.e., the input of the Kalman

filter, contains only the elements with basic frequency. Therefore, the band-

pass filter can modify the signal to make it suitable for the model of Kalman

filter. In addition, higher order harmonics could be eliminated by the band-

pass filter, which allows a lower order model for Kalman filter. However, if

a band-pass filter is added, it will add a bias to the phase estimation. That

is, when the output of the band-pass filter becomes the input of the Kalman

filter, the phase-lag of the output of the target system �true is not equal to

the estimated phase-lag provided by the Kalman filter any more. Instead,

�true will be the combination of the estimated phase-lag from the Kalman

filter and the phase-lag of the band-pass filter. The estimated phase-lag

�estimated is calculated from Equation 3.21. The phase-lag of the band-pass

filter �band�pass can be calculated since all coe�cients in the band-pass filter

is set by users. Therefore, the true phase-lag is

�true = �estimated � �band�pass. (3.22)

Properties

A Kalman filter can estimate the phase independent from the amplitude,

with the output of the target system only. It can possibly estimate a bias

term, as well as the phase and amplitude of higher order harmonics. Kalman

filter can not only get rid of the influence of noise but also track the variables

well. It can get an accurate estimation as well. However, the model of

Kalman filter is essential. If the model of the Kalman filter is far from the

real model, the estimation may be poor and one can hardly get an accurate

estimation. Moreover, tuning of parameters is also important for Kalman

27

filter. Parameters, e.g., Q, R, can influence the performance of the estimator

a lot. If parameters are not properly set, poor estimation may be obtained.

3.3 Controller

Once the phase-lag of the output is estimated, it should be compared with

the reference phase. The controller should try to drive the operating point

to a point, where the local phase-lag is ⇡/2 or �⇡/2. In order to make the

control target unique, we may introduce a nonlinear transformation of the

control input. We can chose a cosine function, an absolute function or any

other even function to make the control target unique, as long as the reference

is adjusted accordingly. In order to make the control loop simple, a simple

even function is preferred. In this thesis, we choose the cosine function to

make the control target unique. When the cosine function is applied, the

controller should try to control the input of the controller to be zero. Other

even functions, e.g., the absolute function, can also be an alternative.

As to the controller itself, there are many choices, e.g., proportional con-

troller, integral controller, PI controller, PID controller, etc.. In order to

achieve good accuracy, the controller should achieve a zero static error. Ad-

ditionally, the controller must be robust since the gain of the system varies

a lot. Here, the gain means the ration of the change in the local phase-lag

to the change in the input. Moreover, as mentioned in Section 3.1.1, the

controller should work slower than the phase estimator, which indicates the

bandwidth of the controller should be low compared to the phase estimator.

Therefore, a simple controller might be su�cient. An integral controller is

usually su�cient since it can always eliminate the steady-state error for the

system. However, we should note that an integral controller is not the only

choice, other controllers are also possible to do the control as well.

3.4 Selection of the Controller Structure

We have introduced di↵erent possible phase estimation schemes and con-

trollers. In this section, we will compare them and choose one combination

for the control loop.

Three scheme of phase estimation is introduced, i.e., estimation by de-

modulation, updated estimation from variant of EPLL and estimation by

Kalman Filter. In the demodulation method, the estimation result is a com-

bination of both amplitude and phase, which indicates that the value of

amplitude will influence the output of the phase estimator. In other words,

28

the value of estimated amplitude may influence the whole control loop. The

estimation method used in EPLL is able to provide the estimated phase in-

dependently. However, the calculation of the phase information is related to

the estimated amplitude, which means the value of estimated amplitude will

influence the estimation of phase. This may make it hard to track the phase

accurately. As for the Kalman filter, it can estimate both the phase and

amplitude and get the value independently, which implies that the value of

estimated amplitude will not influence the performance of the whole control

loop. In addition, it can track variables well. One important characteristics

of the estimator required is to estimate the phase accurately. The phase-lag

of the target system is the base in the novel ESC method. Hence, it is pre-

ferred to obtain the phase information without the influence of amplitude.

Comparing the three methods, the first one is the worst, since its output is

the combination of both amplitude and phase. Therefore, the second and

third methods are preferred. Another important characteristics of the esti-

mator is that the estimator should be able to track the phase. As mentioned

before, the Kalman filter is able to track the phase well, while the method

used in EPLL may have some problem in tracking. Therefore, the estimation

method using Kalman filter is preferred to be the phase estimator in the ESC

loop.

As for the controller, many structures are available. Simple controllers

might be su�cient, since bandwidth of the controller should be low compared

to the estimator. Therefore, we consider simple controllers, e.g., proportional

controller and integral controller. Both of the two controllers could achieve

zero static error if the gain of the system, i.e., the ratio of the change in the

local phase-lag to the change in the input, is a constant. However, the gain of

the system varies a lot in real situation. Then, proportional controller cannot

achieve zero static error unless the proportional gain is changed. Therefore,

proportional controller is not suitable in the ESC. On the other hand, an

integral controller can achieve zero static error. Therefore, it can be a suitable

choice for the ESC system. In the integral controller, the bandwidth is equal

to the integral gain, which gives a hint of how the integral gain should be

tuned. The tuning of the integral gain will be discussed in detail in the next

chapter. Other controllers, e.g., PI controller, PID controller, etc., are also

able to achieve zero static when the gain varies, but they might bring extra

complexity to the design. Since a simple controller can control the loop well,

there is no need to use a controller which adds more complexity to the design.

Then, the final structure of the control loop is shown in Figure 3.4. The

control loop works as follows:

Step 1. Initial value

ˆ✓0 is set first. Then, the input ✓ is put into the target

system with the perturbation signal a sin(!t).

29

Step 2. The output of the target system goes into the band-pass filter. After

the filter, all the higher order harmonics, as well as the constant term are

attenuated.

Step 3. The phase estimator, i.e., the Kalman fliter, estimates the phase of

the output �estimator with the model illustrated in Equation 3.15. The phase

of the band-pass filter �band�pass is already known, since all the parameters in

the band-pass filter are set by users. According to Equation 3.22, the phase

of the output �true can be calculated.

Step 4. The nonlinear transformation function, i.e., the cosine function,

deals with the input of the controller to make the control target unique.

Step 5. The controller, i.e., the integral controller, moves the operating

point of the target system towards the optimal operating point.

Figure 3.4: The complete structure of the controller.

30

Chapter 4

Controller Tuning

There are several parameters that can be adjusted in the novel ESC loop: (1)

Parameters in Kalman filter: the model of Kalman filter, measurement noises

covariance R and process noises covariance Q; (2) Prameters in controller:

the controller gain k; (3)Parameters in perturbation signal: the amplitude of

the signal and the frequency of the signal. The values of these parameters

may significantly influence the performance and stability of the controller.

Therefore, it is important to investigate the e↵ects of these parameters and

provide insight into how to select them.

4.1 Kalman Filter

Kalman filter is utilized to estimate the phase of the output and the estimated

result is fed to the controller to control the target system. The parameters

in the Kalman filter will influence the accuracy of the estimation and speed

of the estimation, which significantly impacts the overall performance of the

method.

We will discuss the model of Kalman filter first. A proper model of the

input signal of the Kalman filter is essential for an accurate estimation. If

the real signal is much more complex than the model, the estimator will try

to put all the information to the limited model, which may lead to inaccurate

estimation. As we assumed in section 3.2.3, the output y is periodic with the

same base-frequency as the perturbation signal. The higher harmonics in the

equation are generated by the nonlinearity and they should be small if the

amplitude of the input signal a is small. Therefore, the phase-lag is mainly

given by the components with frequency ! and we can write the input signal

of the Kalman filter as

y(t) = C sin(!t+ �), (4.1)

31

where

C =

pA2

+B2, � = arctan

B

A. (4.2)

However, this model is valid only in the stationary point and that it is not

able to capture transient behaviour in the system. Since the phase-lag is

mainly given by the fundamental wave, a band-pass filter can be employed

to allow a lower order model in Kalman filter.

With the band-pass filter, we can modify the input to the estimator to

make the signal more suitable for the model of Kalman filter. As discussed

in section 3.2.3, we choose

y(t) = A1 sin(!t) + B1 cos(!t) + A2 sin(2!t) + B2 cos(2!t) + c0 (4.3)

as the model of the input signal. The parameters of the band-pass filter

should be chosen depending on the perturbation frequency utilized.

Consider a band-pass filter that can eliminate signals with frequency lower

than !low and higher than !high. An ideal band-pass filter has a complete

flat passband and signals with frequencies within the passband will be kept

completely. All the signals with frequencies outside the passband will be

completely attenuated. However, there are no ideal band-pass filters in prac-

tice. There is a region outside the passband where frequency are attenuated,

but not rejected, which is called the filter roll-o↵. Therefore, we should take

roll-o↵ into consideration when choosing the cut-o↵ frequency of the band-

pass filter. In order to keep the intended information and achieve better

estimation on Kalman filter, we require that: (1) !low should be lower than

or equal to the frequency of the perturbation signal !; (2) !high should be

higher than or equal to the frequency ! and lower than the frequency 2!,i.e., ! !high < 2!. Since the phase-lag is mainly given by the terms with

basic-frequency, we should leave the components with frequency !. There-

fore, we could choose !low = !high = !, such that the signal with frequency !would be kept due to the roll-o↵. Therefore, the model of the Kalman filter

can be suitable for di↵erent systems, since high order harmonics as well as

the bias are attenuated. In addition, the bias introduced by the band-pass

filter could be easily calculated and removed since all filter parameters as

well as the frequency are user determined.

With the model set, the matrices F (t), B(t), H(t) are settled as well. The

covariance matrices Q, R and the initial conditions have to be determined

by the user. We try to figure out how to set these values. By substituting

K(t) = P (t)H(t)TR�1, (4.4)

into

˙P (t), we can get

˙P (t) = F (t)P (t) + P (t)F (t)T � P (t)H(t)TR�1H(t)P (t) +Q. (4.5)

32

Since all the states in Kalman filter are assumed as constant, we have

F (t) =

2

66664

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

3

77775. (4.6)

Therefore, Equation 4.5 can be simplified as

˙P (t) = �P (t)H(t)TR�1H(t)P (t) +Q. (4.7)

P (t) will probably oscillate since H(t) contains sinusoids. From Equation

4.7, we can find that the increase in Q and R will result in the increase

in the value of

˙P (t). If P (t) oscillates, a larger

˙P (t) might imply a larger

amplitude. This in turn would mean that the covariance of the estimated

error will oscillate with a larger amplitude. Then the Kalman filter may be

unstable and cannot achieve an accurate estimation. Therefore, the value of

Q and R should not be large in order to make the estimator stable. As the

value of P (t) may oscillate, it may be easier to analyse the estimator with the

average value of P (t). According to Equation 4.4, the value of K(t) would be

influenced by the value of P (t), H(t) as well as the value of R. H(t) is a part

of the model of the Kalman filter and P (t) is the error covariance matrix,

which is decided by the estimated state. Therefore, for K(t), R is the only

element that can be adjusted. From Equation 4.4, we can find that when Rdecreases, the value of K(t) will increase and vice versa. Since the value of

F (t) and B(t) are set to zero in the model of Kalman filter, we have

˙x(t) = K(t)(y(t)�H(t)x(t)). (4.8)

We can find that K(t), the Kalman gain, will influence the converge rate

of the estimated value x(t). With larger K(t), x(t) may converge faster.

Since R is the covariance matrix of measurement noise, with smaller R, the

measurements are trusted more. If the model is su�ciently accurate, this

may lead to a faster convergence.

In the Kalman filter, the value of

RQcan influence the performance a lot.

If R is small compare to Q, the measurement is trusted. In addition, the

process noise is assumed large and the predict will be trusted less. When the

value of R is much smaller than Q, the filter may not trust the prediction

any more and the estimation will be mainly based on the observed value. If

R is large compare to Q, the measurement noise is assumed to be large and

the measurement would be less trusted. In addition, the prediction would be

assumed correct and the output of the Kalman filter would mainly be based

on the prediction.

33

Figure 4.1: Two di↵erent situations of the phase-shift.

4.2 Controller

The integral controller has the transfer function G(s) = ks. The sign and the

value of k would influence the performance of the controller.

We will investigate how the sign of the control gain is related to the system

being controlled first. It is discussed in the section 2.2 that z(✓) will pass

the imaginary axis when the system goes through the optimal working point

✓⇤. However, we cannot predict what direction z(✓) will go, when it crosses

the imaginary axis. According to the characteristics of di↵erent systems, the

phase-shift can be divided into two cases, as shown in Figure 4.1. We cannot

be certain in which direction the phase-lag will change when we move the

operating point. If the sign of control gain k is not chosen to match the

target system, we will get an unstable closed loop and the operating point

will never move to the optimal working point. Therefore, we could perform

a quick initial experiment to determine the sign of k.Apart from the sign of k, its value is also important to the controller.

The gain decides the bandwidth of the controller. The bandwidth of the

controller should be chosen according to the bandwidth of the Kalman filter.

As mentioned in Chapter 3, output of the Kalman filter is fed to the controller

as the input signal. If the estimator works slower than the controller, the

controller would not be able to get the updated information provided by

the estimator, resulting in a poor performance of the whole control loop.

Therefore, in order to get the updated phase information to perform a better

control, a proper k should be set to make the controller work slower than the

34

estimator. On the other hand, the value of k will influence the stability of

the system. If a large k is chosen, the output of the controller may oscillate

more, which may make the system unstable. If the value of k is small, the

converge rate will be small, which may slow down the converge speed of the

whole control loop. Therefore, there is a trado↵ between the speed and the

robustness.

4.3 Perturbation Signal

The perturbation signal has two parameters, one is perturbation frequency

and the other is amplitude.

As discussed in Chapter 3, the frequency of the input will influence the

phase-lag of the output of the target system. According to section 2.2, the

phase-lag of the system is

� = arg(G0(i!)) + arg(i! + z(✓)) = g0 + arctan

!

z(✓)+ kG0⇡, (4.9)

where

kG0 = 0,±1,±2, · · · . (4.10)

With lower frequency, the value of g0 will be close to zero. Then the phase-

lag of the system is mainly based on the value of arctan

!z(✓) . As introduced

in section 2.2, z(✓) will be zero when the operating point reaches the op-

timum working point. Therefore, at the optimum, the value of arctan(

!z(✓))

will be close to ±⇡/2. Hence, the phase-lag will be close to ⇡/2+kG0⇡, kG0 =

0,±1,±2, · · · at the optimal operating point for low frequency. If the per-

turbation frequency is large, the value of g0 cannot be ignored. As for the

arg(i!+z(✓)), with larger !, there will be a wider range of inputs ✓ that can

make arg(i!+z(✓)) close to ±⇡/2. As a result, the phase-lag at the optimum

will be g0 +±⇡/2 + kG0⇡ instead of ⇡/2 + kG0⇡, where kG0 = 0,±1,±2, · · · .Hence, it is hard to locate the real optimal operating point with the reference

phase ±⇡/2 when the perturbation frequency is large. In order to locate the

optimal operating point with more accuracy, a lower perturbation frequency

should be set.

The perturbation frequency will influence the converge speed of the sys-

tem as well. With higher perturbation frequency, the bandwidth of the es-

timator can be larger, which allows a faster controller. Hence, the converge

rate of the whole system can be larger than the system with lower pertur-

bation frequency. Therefore, a higher frequency can be used first to locate

a working point ✓⇤1 around the optimum. The start point can then be set at

35

Figure 4.2: Di↵erent phase error at di↵erent frequency.

the optimal working point u⇤1 and the frequency can be decreased. By re-

ducing the perturbation frequency step by step, we can achieve the optimal

operating point with less error and at a higher speed.

However, lower frequency may have a higher converge rate in some special

situations. When the frequency is the only di↵erence, the lower frequency

system may converge faster than the higher frequency system. It might be

the phase error that result in this situation. The phase error between the

true phase and the reference phase can be larger for lower frequency at the

same working point, as shown in Figure 4.2. As the integral controller gain

are the same, the converge speed will be faster for the system with larger

phase error. Therefore, a system with lower frequency can converge faster

than a higher frequency system.

As for the amplitude, it should be small, but still be larger than the

amplitude of the noises. The perturbation signal is employed here to excite

the plant. It is added to the output of the controller. The controller is

designed to shift the output of the controller to the optimal working point,

while its output might be small. Therefore, if the perturbation signal has

large amplitude, it will exert the significant fluctuation on the value of the

36

input signal ✓. This may lead to a large fluctuation on the output signal,

which is not preferred in some applications. Hence, the amplitude should be

small. However, in a real system, noise always exists. Thus, the amplitude ashould be large in order to distinguish the perturbation signal from the noise.

In other words, the amplitude should be large enough to obtain a large signal

to noise ratio.

4.4 Summary

According to the discussion above, some conclusions can be made.

1) With larger Q and R, the value of P (t) might oscillate with large ampli-

tude. The value of

RQ

will influence the performance of the estimator. If Ris smaller than Q, measurement is trusted more and vice verse. If the model

of is su�ciently accurate, smaller R may lead to a faster convergence.

2) The sign of the control gain k should be determined by a small experiment.

Moreover, the value of K should be chosen carefully to make the controller

work both fast and stable.

3) The amplitude of the perturbation signal should be small but large enough

to achieve large signal to noise ratio. As for the frequency, higher frequency

allows larger bandwidth of the controller, which enables the system to con-

verge faster. To achieve an accurate result, a lower frequency should be

chosen.

37

Chapter 5

Example and Analysis

In this chapter, an example is given to show how the control loop works. The

following system comes from [4].

Example: Isothermal CSTR: Consider an isothermal perfectly mixed tank

reactor with two consecutive reactions A ! B, 2B ! C, with standard mass

action kinetics

V cA = F (cAf � cA)� V k1cA, (5.1)

V cB = �FcB + V k1cA � V k2c2B, (5.2)

where cA and cB are concentrations of A and B, respectively, cAf is the con-

centration of input flow. The parameters are V = 1.0, cAf = 1.0,k1 = 2.0and k2 = 0.1.

The steady-state input-output relationship is shown in Figure 5.1. The

optimal working point is around ✓⇤ = 0.375. We will now assume that the

model is unknown and apply the ESC algorithm designed in Chapter 3 in

order to locate the optimum.

In the example, some parameters are set first:

The measurement noises covariance matrix is set as R(t) = 0.1. A low-

pass with break-o↵ frequency !l = 1.3! and a high-pass filter with break-o↵

frequency !h = 0.8! are employed to combine a band-pass filter, where ! is

the input frequency. The process noises covariance matrix is set as

Q(t) =

2

66664

1 0 0 0 0

0 1 0 0 0

0 0 1 0 0

0 0 0 1 0

0 0 0 0 1

3

77775. (5.3)

38

Figure 5.1: The steady-state input-output realtionship.

With these parameters settled, we could run the control loop and analyze

the performance.

5.1 Performance Test

Perturbation signals with di↵erent frequencies are tested in this example.

The result is shown clearly in Figure 5.2. The comparison result is shown in

Table 5.1. It is obvious that e1 < e2 < e3 < e4. We can draw the conclusion

that with lower frequency, the final control result will be closer to the exact

optimal working point for this system, the same as discussed in Section 4.3.

In addition, it can be inferred from the figure that as the frequency increases,

the speed of converge decreases as discussed in the previous chapter.

Frequency ! Output of the controller

ˆ✓ Steady-state error e = ✓ � ✓⇤

0.02 0.3789 e1 = 0.0039

0.1 0.3792 e2 = 0.0042

0.2 0.3588 e3 = 0.0162

0.3 0.3138 e4 = 0.0612

Table 5.1: Steady-state error of the output of the controller at di↵erent

frequencies.

39

Figure 5.2: Control result with frequency ! = 0.3, ! = 0.2, ! = 0.1 and

! = 0.02, with k = 0.0001.

Figure 5.3 shows that the phase error is larger for lower frequency at the

same working point as we discussed in Chapter 4, which may lead to a faster

convergence for lower frequency when other parameters are the same.

In Chapter 4, we suggested that an approach to achieving both fast con-

vergence and accuracy would be to initially set the perturbation frequency

high with a correspondingly aggressive tuning. And then, we successively

detune the frequency as well as the controller, in order to eventually achieve

accuracy. As shown in Figure 5.4, the blue line shows the result that the sys-

tem starts with frequency ! = 0.1, controller gain k = 0.001 before t = 1500.

When t reaches 1500, the system will change the frequency to ! = 0.02 and

the controller gain will be k = 0.0001. The red line shows the result with

frequency ! = 0.02 and controller gain k = 0.0001 throughout the entire sim-

ulation. It is shown clearly that the system converges faster in first situation

if other parameters are adjusted properly.

Interestingly, we find that the value of k cannot be increased much when

! = 0.02 as is illustrated in Figure 5.5. Comparing the curve in Figure

5.5, the only di↵erence between these two curves is the value of controller

gain k. With certain perturbation frequency, large value of k may introduce

instability to the system, as discussed in the Controller Tuning part.

We have also compared the novel ESC method with one kind of classic

40

Figure 5.3: Phase-lag with frequency ! = 0.3, ! = 0.2, ! = 0.1 and ! = 0.02.

Figure 5.4: Changing of the operating point when ! = 0.02 VS. changing of

the operating point when perturbation frequency is switched from ! = 0.1to ! = 0.02.

41

Figure 5.5: Control result with frequency ! = 0.02 control constant k = 0.001VS. the result with frequency ! = 0.02 and control constant k = 0.01

ESC. In the classic ESC, the control gain k is set to 100 to obtain a large

convergence rate; the cuto↵ frequency of the low-pass filter is set to !l = 0.3!;the cuto↵ frequency of the high-pass filter is set to !h = 0.6!. The result

is shown in Figure 5.6. It implies that in this particular case the system

controlled by classic ESC converges slower than the one controlled by the

phase-based ESC method. The steady-state error in the classic ESC is 0.0159

and the error in the phase-based ESC is 0.0042, which indicates that the

phase-based method achieves a result with less steady-state error than the

classic one.

5.2 Robustness Test

In order to test the robustness of the controller, we perform two di↵erent

tests below.

First is a disturbance test, where we want to show if the operating point

can come back to the optimum when a sudden change is added to the input.

At first, the system input is ✓(t) = a sin(!t) + ˆ✓, where

ˆ✓ is the output

of the controller. When the time reaches 7000, a disturbance d = 0.1 is

added to the input. The e↵ect of the disturbance is illustrated in Figure

5.7 and Figure 5.8. Obviously, the e↵ect on the output is small and the

42

Figure 5.6: The di↵erent control results by di↵erent ESC methods with fre-

quency ! = 0.1

controller successfully adjusts the input in order to compensate for the added

disturbance. Therefore, we can draw the conclusion that the novel extremum

seeking controller has good robustness for disturbances in this situation.

Then comes the test where the first order inertial element is added to the

system. The output in s domain becomes Y (s) = xbki

Ts+1 in the test case,

where xb represents cB. We set T = 1, ki = 0.1, and the result is shown in

Figure 5.9. We can see from the result that the controlled output converges

to a certain value. With the same frequency and control gain, the converge

rate is almost the same as the system without the inertial element. The

steady-state error under this situation, i.e., e = 0.0228, is a little bit larger

than the error without the first order inertial element. This result indicates

that the control loop can deal with inertial element well and the operating

point can be moved to a steady value, which is close to the optimal operating

point.

5.3 Summary

These simulations show that the phase-based ESC loop is able to find the

optimal operating point when proper parameters are chosen. With larger

perturbation frequency, the other parameters can be set larger to make the

43

Figure 5.7: The result of the controller output when disturbance d = 0.1added at time 7000 with frequency ! = 0.1.

Figure 5.8: Output of the target system when disturbance d = 0.1 added at

time 7000 with frequency ! = 0.1.

44

Figure 5.9: Input signal of the target system when the first order inertial

element is added with frequency ! = 0.1, control gain k = 0.001 VS. the

input signal with the ordinary system with frequency ! = 0.1, control gaink = 0.001

45

system converge faster. However, system with lower frequency converges

faster if only the value of frequency changes. As to the accuracy, system

with lower frequency achieves less steady-state error. This system is robust

for disturbances and is capable to move the operating point back to the

optimum in a short time. In addition, when inertial elements is added to the

system, the control loop is able to control the target system working near

the optimal operating point as well.

46

Chapter 6

Conclusions and Further

Research

6.1 Disscussion and Conclusion

A phase-based extremum seeking control loop is designed in this thesis report.

The structure of PLL is employed to lock the control loop when the phase-

lag is ±⇡/2. Kalman filter is employed to ensure an accurate estimation. To

achieve a zero steady-state error, an integral controller is utilized.

The impact of the parameters in the control loop are analyzed, as well

as a guide for controller tuning. Simulations shown in Chapter 5 imply that

this model free method can move the operating point of the target system

to the optimum. In addition, this control loop is robust for both disturbance

and inertial terms.

6.2 Future Work

The main purpose of this thesis work is to locate the optimal operating

point of a dynamic system. Discussions in the previous chapters show that

the goal has been achieved with the phase-based method. However, there

are still some tasks left.

In this thesis we have focused on local behaviour of the loop. A global

analysis is left for future work. The tuning and structure of the controller

and its parameters have significant impact on the performance of the loop. In

this thesis, we performed an initial analysis and provided some simple tuning

guides. However, further research into this topic is necessary to provide useful

guides for all parameters.

47

Bibliography

[1] P. E. F. Camacho and A. P. C. Bordons, Model Predictive Controllers.Springer London, 2007.

[2] S. Skogestad, “Near-optimal operation by self-optimizing control: From

process control to marathon running and business systems,” Computers& Chemical Engineering, vol. 29, no. 1, pp. 127–137, 2004.

[3] M. Krstic, “Performance improvement and limitations in extremum

seeking control,” Systems & Control Letters, vol. 39, no. 5, pp. 313–326, 2000.

[4] O. Trollberg and E. W. Jacobsen, “On bifurcations of the zero dynamics-

connecting steady-state optimality to process dynamics,” IFAC AD-CHEM 2015, IFAC-PapersOnLine, vol. 48, no. 8, pp. 170–175, 2015.

[5] M. Leblanc, “Sur lelectrification des chemins de fer au moyen de courants

alternatifs de frequence elevee,” Revue generale de lelectricite, vol. 12,no. 8, pp. 275–277, 1922.

[6] V. Kazakevich, On extremum seeking. PhD thesis, PhD Thesis, Moscow

High Technical University, 1944.

[7] C. S. Draper and Y. T. Li, Principles of optimalizing control systemsand an application to the internal combustion engine. American Society

of Mechanical Engineers, 1951.

[8] V. Obabkov, “Theory of multichannel extremal control systems with

sinusoidal probe signals,” Automation and Remote Control, vol. 28,

pp. 48–54, 1967.

[9] J. WANG and Z. M. HE, “System construction and application analysis

of extremum seeking control,” Journal of Kunming University of Scienceand Technology (Science and Technology), vol. 5, pp. 56–59, 2010.

48

[10] C. Zhang and R. Ordonez, Extremum-seeking control and applications:a numerical optimization-based approach. Springer Science & Business

Media, 2011.

[11] S. Korovin and V. Utkin, “Using sliding modes in static optimization

and nonlinear programming,” Automatica, vol. 10, no. 5, pp. 525–532,1974.

[12] S. Drakunov and

¨

U.

¨

Ozguner, “Optimization of nonlinear system output

via sliding mode approach,” in Proceedings of the IEEE InternationalWorkshop on Variable Structure and Lyapunov Control of UncertainDynamical Systems, She�eld, UK, pp. 61–62, 1992.

[13] H. de Bellescize, La reception synchrone. E. Chiron, 1932.

[14] W. C. Lindsey and M. K. Simon, Telecommunication systems engineer-ing. Courier Corporation, 1973.

[15] A. Blanchard, “Phase-locked loops: application to coherent receiver de-

sign,” New York, Wiley-Interscience, 1976, vol. 1, 1976.

[16] A. J. Viterbi, Principles of coherent communication. McGraw-Hill, 1966.

[17] A. Grebene and H. Camenzind, “Phase locking as a new approach for

tuned integrated circuits,” in Solid-State Circuits Conference. Digest ofTechnical Papers. 1969 IEEE Internationa, vol. 12, pp. 100–101, IEEE,1969.

[18] F. M. Gardner, Phaselock techniques. John Wiley & Sons, 2005.

[19] M. Karimi-Ghartemani and M. Iravani, “A new phase-locked loop (PLL)

system,” in Circuits and Systems, 2001. MWSCAS 2001. Proceedings ofthe 44th IEEE 2001 Midwest Symposium on, vol. 1, pp. 421–424, IEEE,2001.

[20] M. Karimi-Ghartema, Enhanced phase-locked loop structures for powerand energy applications. John Wiley & Sons, 2014.

[21] A. Patapoutian, “On phase-locked loops and Kalman filters,” Commu-nications, IEEE Transactions on, vol. 47, no. 5, pp. 670–672, 1999.

[22] P. F. Driessen, “DPLL bit synchronizer with rapid acquisition using

adaptive Kalman filtering techniques,” IEEE Transactions on Commu-nications, vol. 42, no. 9, pp. 2673–2675, 1994.

49

[23] M. Ciobotaru, R. Teodorescu, and F. Blaabjerg, “A new single-phase

pll structure based on second order generalized integrator,” in PowerElectronics Specialists Conference, 2006. PESC’06. 37th IEEE, pp. 1–6,IEEE, 2006.

50

TRITA TRITA-EE 2016:157ISSN 1653-5146

www.kth.se