OR-1 2011 1

Backgrounds-Convexity

Def: line segment joining two points is the collection of points

nRxx 21,10,)1( 21 xxx

)0,,1,( 21212

21

1 xxx

) called ,0,1, ,(Generally 11 ncombinatioconvexixx mi ii

mi

ii

1x

2x

1x

2x

21212 )1()( xxxxx

)( 21 xx

OR-1 2011 2

Def: is called convex set iff whenever

nRC Cxx 21 )1( .10,, 21 andCxCx

Convex sets Nonconvex set

OR-1 2011 3

Def: The convex hull of a set S is the set of all points that are convex combinations of points in S, i.e.

conv(S)={x: x = i = 1k i xi, k 1, x1,…, xkS, 1, ..., k 0, i = 1

k i = 1}

Picture: 1x + 2y + 3z, i 0, i = 13 i = 1

1x + 2y + 3z = (1+ 2){ 1 /(1+ 2)x + 2 /(1+ 2)y} + 3z

(assuming 1+ 2 0)

x

y

z

OR-1 2011 4

Proposition: Let be a convex set and for , define

Then is a convex set.

Pf) If k = 0, kC is convex. Suppose

For any x, y kC,

Hence the property of convexity of a set is preserved under scalar multiplication.

Consider other operations that preserve convexity.

nRC } |{ CxsomeforkxyRykC n

Rk

.0k' ,'such that ',' kyykxxCyx

kC'y'xk,C'y'x

'y'xk'ky'kxyx

))1((hence))1((But

))1(()1()1(Then

kC

OR-1 2011 5

Convex functionDef: Function is called a convex function if for all x1 and

x2, f satisfies

Also called strictly convex function if

RRf n :

,10 ),()1()())1(( 2121 xfxfxxf

,10 ),()1()())1(( 2121 xfxfxxf

1x 2x21 )1( xx

))1(( 21 xxf

)()1()( 21 xfxf

)(xf

x

111 ))(,( nRxfx ))(,( 22 xfx

Meaning: The line segment joining (x1, f(x1)) and (x2, f(x2)) is above or on the locus of points of function values.

OR-1 2011 6

OR-1 2011 7

Def: f: Rn R. Define epigraph of f as epi(f) = { (x, ) Rn+1 : f(x) } Equivalent definition: f: Rn R is a convex function if and only if epi(f) is

a convex set.

Def: f is a concave function if –f is a convex function.

Def: xC is an extreme point of a convex set C if x cannot be expressed as y + (1-)z, 0 < < 1 for distinct y, z C ( x y, z )

(equivalently, x does not lie on any line segment that joins two other points in the set C.)

: extreme points

OR-1 2011 8

Review-Linear Algebra 2052 4321 xxxx

30545 4321 xxxx

20263 4321 xxxx

2613

5451

1512

A

4

3

2

1

x

x

x

x

x

20

30

20

b

notation vectormatrix,in bAx

inner product of two column vectors x, y Rn :

x’y = i = 1n xiyi

If x’y = 0, x, y 0, then x, y are said to be orthogonal. In 3-D, the angle between the two vectors is 90 degrees.

( Vectors are column vectors unless specified otherwise. But, our text does not differentiate it.)

OR-1 2011 9

Submatrices multiplication

2221

1211

34333231

24232221

14131211

AA

Aa

aaaa

aaaa

aaaa

A

2

1

4

3

2

1

B

b

b

b

b

b

B

222121

212111

BAbA

BAbaAB

OR-1 2011 10

submatrices multiplications which will be frequently used.

'

'

'

2

1

21

21

22221

11211

m

n

mnmm

n

n

a

a

a

AAA

aaa

aaa

aaa

A

bxAxAxAxA

x

x

x

x

AAAbAx jnj jnnn

12211

4

3

2

1

21

nn AyAyAyAAAyAy ''''' 2121

mi iimm

m

m 'ay'ay'ay'ay

'a

'a

'a

yyyA'y 122112

1

21

OR-1 2011 11

Def: is said to be linearly dependent if

, not all equal to 0, such that

( i.e., there exists a vector in which can be expressed as a linear combination of the other vectors. )

Def: linearly independent if not linearly dependent.

In other words,

(i.e., none of the vectors in can be expressed as a linear combination of the remaining vectors.)

Def: Rank of a set of vectors : maximum number of linearly independent vectors in the set.

Def: Basis for a set of vectors : collection of linearly independent vectors from the set such that every vector in the set can be expressed as a linear combination of them. (maximal linearly independent subset, minimal generator of the set)

},,,{ 21 mxxx mccc ,,, 21 02

21

1 mmxcxcxc

},,,{ 21 mxxx

},,,{ 21 mxxx

},,,{ 21 mxxx mi i

ii icxc1 allfor 0 implies 0

OR-1 2011 12

Thm) r linearly independent vectors form a basis if and only if the set has rank r.

Def: row rank of a matrix : rank of its set of row vectors

column rank of a matrix : rank of its set of column vectors

Thm) for a matrix A, row rank = column rank

Def : nonsingular matrix : rank = number of rows = number of columns. Otherwise, called singular

Thm) If A is nonsingular, then unique inverse exists.

)( 11 AAIAA

OR-1 2011 13

Simutaneous Linear Equations

Thm: Ax = b has at least one solution iff rank(A) = rank( [A, b] )

Pf) ) rank( [A, b] ) rank(A). Suppose rank( [A, b] ) > rank(A).

Then b is lin. ind. of the column vectors of A, i,e., b can’t be expressed as a linear combination of columns of A. Hence Ax = b does not have a solution.

) There exists a basis in columns of A which generates b. So Ax = b has a solution.

Suppose A: mn, rank(A) = rank [A, b] = r.

Then Ax = b has a unique solution if r = n.

Pf) Let y, z be any two solutions of Ax = b. Then Ay = Az = b, or Ay – Az = A(y-z) = 0. A(y-z) = j=1

nAj(yj – zj) = 0.

Since column vectors of A are linearly independent, we have yj

– zj = 0 for all j. Hence y = z.

(Note that m may be greater than n.)

OR-1 2011 14

OR-1 2011 15

Operations that do not change the solution set of the linear equations

(Elementary row operations)Change the position of the equationsMultiply a nonzero scalar k to both sides of an equationMultiply a scalar k to an equation and add it to another equation

Hence X = Y. Solution sets are same.

The operations can be performed only on the coefficient matrix [A, b], for Ax = b.

}',,','|{ 2211 mm bxabxabxaxX

}',,'),()''(|{ 222121 mm bxabxakbbxkaaxY

)( implies and

)( implies that Show**

**

XYXxYx

YXYxXx

OR-1 2011 16

Solving systems of linear equations (Gauss-Jordan Elimination, 변수의 치환 ) (will be used in the simplex method to solve LP problems)

2252

103

104

32

21

321

xx

xx

xxx

2252

2042

104

32

32

321

xx

xx

xxx

2252

102

104

32

32

321

xx

xx

xxx

2

102

206

3

32

31

x

xx

xx

2

6

8

3

2

1

x

x

x

OR-1 2011 17

Infinitely many solutions

2252

103

104

432

421

4321

xxx

xxx

xxxx

2252

20242

104

432

432

4321

xxx

xxx

xxxx

2252

102

104

432

432

4321

xxx

xxx

xxxx

2

102

2026

43

432

431

xx

xxx

xxx

2

63

88

43

42

41

xx

xx

xx

43

42

41

2

36

88

xx

xx

xx

txtxtx

ttx

2 ,36 ,88

get and arbitrary for Assign

321

4

OR-1 2011 18

43

42

41

2

36

88

xx

xx

xx

method)simplex in the used be (will

0 2, 6, 8, Here

solution. basic a called is variablesdependent for the solving and

0 toariablesindepent v thesettingby obtainedsolution thely,Particular

variables.dependent are ,, and t variableindependen is

4321

3214

xxxx

xxxx

OR-1 2011 19

2

102

2026

43

432

431

xx

xxx

xxx

2

63

88

43

42

41

xx

xx

xx

2

123

248

43

32

31

xx

xx

xx

434241 2 ,36 ,88 xxxxxx 343231 2 ,312 ,824 xxxxxx

different istion representabut solutions, ofset same thehave systemsBoth

0,2,6,8.). 4321 xxxxge

OR-1 2011 20

Elementary row operations are equivalent to premultiplying a nonsingular square matrix to both sides of the equations

2252

103

104

32

21

321

xx

xx

xxx

22

10

10

520

031

411

3

2

1

x

x

x

2252

2042

104

32

32

321

xx

xx

xxx

22

10

10

1

11

1

520

031

411

1

11

1

3

2

1

x

x

x

22

20

10

520

420

411

3

2

1

x

x

x

Ax = b

OR-1 2011 21

2252

102

104

32

32

321

xx

xx

xxx

2252

2042

104

32

32

321

xx

xx

xxx

22

10

10

1

11

1

520

031

411

1

11

1

3

2

1

x

x

x

22

10

10

1

11

1

1

2/1

1

520

031

411

1

11

1

1

2/1

1

3

2

1

x

x

x

OR-1 2011 22

22

10

10

1

11

1

1

2/1

1

520

031

411

1

11

1

1

2/1

1

3

2

1

x

x

x

22

10

10

1

2/12/1

1

520

031

411

1

2/12/1

1

3

2

1

x

x

x

22

20

10

1

2/1

1

520

420

411

1

2/1

1

3

2

1

x

x

x

OR-1 2011 23

So if we multiply all elementary row operation matrices, we get the matrix having the information about the elementary row operations we performed

2252

103

104

32

21

321

xx

xx

xxx

2

6

8

3

2

1

x

x

x

22

10

10

111

25.25.2

65.65.7

520

031

411

111

25.25.2

65.65.7

3

2

1

x

x

x

2

6

8

100

010

001

3

2

1

x

x

x

111

25.25.2

65.65.71A

OR-1 2011 24

Finding inverse of a nonsingular matrix A.

Perform elementary row operations (premultiply elementary row operation matrices) to make [A : I ] to [ I : B ]

Let the product of the elementary row operations matrices be C.

Then C [ A : I ] = [ CA : C ] = [ I : B]

Hence CA = I C = A-1 and B = A-1.

OR-1 2011 25

bBAxB 11

nn ABABABAAABAB 12

11

121

11

)(:ar)(nonsingul ,: where Suppose mnmNmmBNBA

NBINBBBNBBAB 11111Then