Introduction to optimization Problems

1

Presented by:

Eng. Mohamed Youssef Selim

I. Introduction

� Introduction to

• An optimization problem seeks to find the largest(the smallest) value of a quantity (such asmaximum revenue or minimum surface area)

22 of 48

maximum revenue or minimum surface area)given certain limits to a problem .

� An optimization problem can usually beexpressed as “find the maximum (or minimum)value of some quantity Q under a certain set ofgiven conditions”.

I. Introduction

A schematic view of modeling/optimization process

Real-world problem

Mathematical model

assumptions, abstraction, data, simplifications

33 of 48

Solution to model

Solution toreal-world problem

optimization algorithm

interpretation

makes sense? change the model, assumptions?

I. Introduction

� Mathematical models in Optimization

• The general form of an optimization model:

min or max f(x 1,…,xn) (objective function)

44 of 48

subject to g i(x1,…,xn) ≥ 0 (functional constraints)

x1,…,xn ∈ S (set constraints)

• x1,…,xn are called decision variables

• In words,

the goal is to find x 1,…,xn that� satisfy the constraints;� achieve min (max) objective function value.

I. Introduction

� Types of Optimization Models

Stochastic(probabilistic information on data)

Deterministic(data are certain)

55 of 48

Discrete, Integer(S = Zn)

Continuous(S = Rn)

Linear(f and g are linear)

Nonlinear(f and g are nonlinear)

I. Introduction

� Examples of Discrete Optimization Models:Traveling Salesman Problem (TSP)

� There are n cities. The salesman

• starts his tour from City 1,

66 of 48

• starts his tour from City 1,

• visits each of the cities exactly once,

• and returns to City 1.

• For each pair of cities i,j there is a cost c ij associated with traveling from City i to City j .

� Goal: Find a minimum -cost tour.

I. Introduction

� Examples of Discrete Optimization Models:Shortest Path Problem

� In a network, we have distances on arcs ; source nod e s and sink node t .

a d

1 1 1

3

4

77 of 48

� Goal: Find a shortest path from the source to the sink.

s

b e

tc

1 1

1

1

2

2

25

4

7

2

4

I. Introduction

� Problems that can be modeled and solved bydiscrete optimization techniques

• Scheduling Problems (production, airline, etc.)

• Network Design Problems

88 of 48

• Network Design Problems

• Facility Location Problems

• Inventory management

• Transportation Problems

I. Introduction

� Problems that can be modeled and solved bydiscrete optimization techniques

• Minimum spanning tree problem

• Shortest path problem

99 of 48

• Shortest path problem

• Maximum flow problem

• Min-cost flow problem

• Assignment Problem

I. Introduction

� Solution Methods for Discrete OptimizationProblems

• Integer Programming

• Network Algorithms

1010 of 48

• Network Algorithms

• Dynamic Programming

• Approximation Algorithms

I. Introduction

� Global vs. local optimization

1111 of 48

I. Introduction

� Global optimization• Finding, or even verifying, global minimum is diffi cult, in general

• Most optimization methods are designed to find local minimum, which may or may not be global minimum

1212 of 48

• If global minimum is desired, one can try several w idely separated starting points and see if all produce same result

• For some problems, such as linear programming, glob al optimization is more tractable

I. Introduction

� Optimization Problem Types – Real Variables

• Linear Program (LP)� (P) Easy, fast to solve, convex

1313 of 48

• Non-Linear Program (NLP)� (P) Convex problems easy to solve� Non-convex problems harder, not guaranteed to find global optimum

I. Introduction

� Optimization Problem Type – Integer/Mixed Variables

• Integer Programs (IP) : � (NP-hard) computational complexity

• Mixed Integer Linear Program (MILP)

1414 of 48

� Our problem of interest, also generally (NP-hard)

� However, many problems can be solved surprisingly quickly!

• MINLP, MILQP etc.� New tools included in CPLEX 9.0!

II. Convex Optimization

CONVEX OPTIMIZATION

1515 of 48

(CVX OPTIMIZATION)


� Introduction to

1616 of 48


� Introduction to

1717 of 48


� Introduction to

1818 of 48


� Introduction to

1919 of 48


� Introduction to

2020 of 48


2121 of 48


2222 of 48


� Introduction to

2323 of 48


� Introduction to

2424 of 48


� Introduction to

2525 of 48


� Introduction to

2626 of 48


� Introduction to

2727 of 48


� Introduction to

2828 of 48


Different Optimization Programs

Linear Program

2929 of 48

Linear Program

Quadratic Program

QCQP

Robust Linear Program


� Introduction to

3030 of 48


� Introduction to

3131 of 48


� Introduction to

3232 of 48


� Introduction to

3333 of 48


� Introduction to

3434 of 48


� Introduction to

3535 of 48


Duality

3636 of 48

Lagrange dual problem

For solving non convex problems

KKT Conditions


� Introduction to

3737 of 48


� Introduction to

3838 of 48


� Introduction to

3939 of 48


� Introduction to

4040 of 48


� Introduction to

4141 of 48


� Introduction to

4242 of 48

III. COLLABORATIVE RESOURCE ALLOCATION (CRA) ALGORITHM

� CRA Algorithm

�� The problem is a constrained non-convex optimization problem, so

finding optimal solution is NP hard . Thus, an algorithm is proposed to

solve this problem sub -optimally .

4343 of 48

solve this problem sub -optimally .

III. COLLABORATIVE RESOURCE ALLOCATION (CRA) ALGORITHM

� Resource Allocation

�� ToTo findfind optimaloptimal powerpower allocationallocation,, wewe considerconsider thethe LagrangianLagrangian ofof thethe optimizationoptimization

problemproblem ((11)) dualizeddualized withwith respectrespect toto aa totaltotal powerpower constraintconstraint..

4444 of 48

� where λ = (λ1, ..., λ

M)T isis aa nonnon negativenegative LagrangianLagrangian multipliermultiplier..


CVX software package on MATLAB

Example for using CVX package

4545 of 48

Example for using CVX package


� References

• Boyd Lectures

http://stanford.edu/class/ee364a/

4646 of 48

• Coursera

• Other resources from the internet

http://stanford.edu/class/ee364a/

ا� �د � رب ا��ن

Thank You

4747 of 48

ا� �د � رب ا��ن

I. Backup Slide

� Complexity Analysis

• (P) – Deterministic Polynomial time algorithm

• (NP) – Non-deterministic Polynomial time algorithm,

4848 of 48

• (NP) – Non-deterministic Polynomial time algorithm, � Feasibility can be determined in polynomial time

• (NP-complete) – NP and at least as hard as any known NP problem

• (NP-hard) – not provably NP and at least as hard as any NP problem,

� Optimization over an NP-complete feasibility problem

Introduction to optimization Problems

Engineering

Transcript of Introduction to optimization Problems