LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic...
Transcript of LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic...
![Page 1: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/1.jpg)
LECTURE 10: CONSTRAINED
OPTIMIZATION – LAGRANGIAN
DUAL PROBLEM
1. Lagrangian dual problem
2. Duality gap
3. Saddle point solution
![Page 2: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/2.jpg)
Lagrangian dual problem
![Page 3: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/3.jpg)
Property 1 – weak duality
![Page 4: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/4.jpg)
Weak duality theorem
![Page 5: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/5.jpg)
Property 2 – concavity and subgradient
![Page 6: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/6.jpg)
Property 3 – duality gap• Duality gap may exist
![Page 7: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/7.jpg)
Example of duality gap
![Page 8: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/8.jpg)
Property 4 – strong duality
![Page 9: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/9.jpg)
Geometric interpretation of LD
![Page 10: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/10.jpg)
Picture of duality gap
![Page 11: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/11.jpg)
Full Lagrangian dual
•
![Page 12: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/12.jpg)
Partial Lagrangian dual (1)•
![Page 13: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/13.jpg)
Partial Lagrangian dual (2)
•
![Page 14: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/14.jpg)
Lagrangian dual of LP
![Page 15: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/15.jpg)
Lagrangian dual of QP
![Page 16: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/16.jpg)
Lagrangian dual of QP
![Page 17: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/17.jpg)
Saddle point solution
• Definition
![Page 18: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/18.jpg)
Saddle point and duality gap
• Basic idea : The existence of a saddle point solution to the
Lagrangian function is a necessary and sufficient
condition for the absence of a duality gap!
![Page 19: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/19.jpg)
Proof
![Page 20: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/20.jpg)
Saddle point theorem
![Page 21: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/21.jpg)
Saddle point and KKT conditions
![Page 22: LECTURE 10: CONSTRAINED OPTIMIZATION LAGRANGIAN DUAL … · Saddle point and duality gap •Basic idea : The existence of a saddle point solution to the Lagrangian function is a necessary](https://reader030.fdocuments.in/reader030/viewer/2022040800/5e35ab43bdf855326c3415fd/html5/thumbnails/22.jpg)
Proof