Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h....

56
ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) EE210A: Adaptation and Learning Professor Ali H. Sayed

Transcript of Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h....

Page 1: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED)

EE210A: Adaptation and Learning Professor Ali H. Sayed

Page 2: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 2

LECTURE #10

MEAN-SQUARE PERFORMANCE Sections in order: 15.1-15.4, 16.1-16.6, 19.3

Page 3: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 3

GRADIENT NOISE

Page 4: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 4

PERFORMANCE MEASURE

Page 5: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 5

PERFORMANCE MEASURE

Page 6: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 6

PERFORMANCE MEASURE

Page 7: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 7

STOCHASTIC EQUATIONS

Page 8: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 8

STOCHASTIC EQUATIONS

Page 9: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 9

EXCESS MEAN-SQUARE ERROR (EMSE)

Page 10: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 10

EXCESS MEAN-SQUARE ERROR (EMSE)

Page 11: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 11

STATIONARY DATA

Page 12: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 12

STATIONARY DATA

Page 13: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 13

STATIONARY DATA

Page 14: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 14

INDEPENDENCE RELATIONS

Page 15: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 15

INDEPENDENCE RELATIONS

Page 16: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 16

INDEPENDENCE RELATIONS

Page 17: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 17

EMSE

Page 18: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 18

EMSE

Page 19: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 19

ENERGY CONSERVATION

Page 20: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 20

ENERGY CONSERVATION

Page 21: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 21

ERROR RELATIONS

Page 22: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 22

PERFORMANCE METRICS

Page 23: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 23

ALGEBRAIC DERIVATION

Page 24: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 24

ALGEBRAIC DERIVATION

Page 25: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 25

ALGEBRAIC DERIVATION

Page 26: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 26

ENERGY CONSERVATION RELATION

Page 27: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 27

VARIANCE RELATION

Page 28: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 28

STEADY-STATE OPERATION

Page 29: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 29

STEADY-STATE PERFORMANCE

Page 30: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 30

STEADY-STATE PERFORMANCE

Page 31: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 31

VARIANCE RELATION

Page 32: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 32

MEAN-SQUARE ANALYSIS

Page 33: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 33

MEAN-SQUARE ANALYSIS

Page 34: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 34

APPLICATION TO LMS FILTER

Page 35: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 35

VARIANCE RELATION

Page 36: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 36

SMALL STEP-SIZES

Page 37: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 37

SEPARATION PRINCIPLE

Page 38: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 38

SEPARATION PRINCIPLE

Page 39: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 39

SIMULATION

Page 40: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 40

SEPARATION PRINCIPLE

Page 41: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 41

WHITE GAUSSIAN INPUT

Page 42: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 42

STEADY-STATE ASSUMPTION

Page 43: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 43

INDEPENDENCE ASSUMPTIONS

Page 44: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 44

INDEPENDENCE ASSUMPTIONS

Page 45: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 45

MEAN-SQUARE PERFORMANCE

Page 46: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 46

MEAN-SQUARE PERFORMANCE

Page 47: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 47

MEAN-SQUARE PERFORMANCE

Page 48: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 48

COMPLEX DATA

Page 49: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 49

REAL DATA

Page 50: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 50

RESULTS

Page 51: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 51

SIMULATION RESULTS

Page 52: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 52

SIMULATION RESULTS

Page 53: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 53

SIMULATION RESULTS

Page 54: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 54

SIMULATION RESULTS

Page 55: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 55

SIMULATION RESULTS

Page 56: Professor Ali H. Sayed · electrical engineering department ee210a: adaptation and learning (a. h. sayed) 2 . lecture #10 . mean-square performance sections in order: 15.1-15.4, 16.1-

ELECTRICAL ENGINEERING DEPARTMENT EE210A: ADAPTATION AND LEARNING (A. H. SAYED) 56

OTHER FILTERS