Field-weighted Factorization Machines for Click-Through Rate...
Transcript of Field-weighted Factorization Machines for Click-Through Rate...
![Page 1: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/1.jpg)
Field-weighted Factorization Machines for Click-Through Rate Prediction in Display Advertising
Author : Junwei Pan, Jian Xu, Alfonso Lobos Ruiz, Wenliang Zhao, Shengjun Pan, Yu Sun, Quan Lu Source : WWW’ 18 Advisor : Jia-Ling Koh Speaker : Chia-Yi Huang Date : 2018/05/22
![Page 2: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/2.jpg)
Outline
▸ Introduction ▸ Method
▸ Experiment
▸ Conclusion
!2
![Page 3: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/3.jpg)
▸Goal • Display ad CTR prediction use Field-
weighted Factorization Machines
Introduction
!3
![Page 4: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/4.jpg)
▸Goal • Display ad CTR prediction use Field-
weighted Factorization Machines
Introduction
!4
![Page 5: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/5.jpg)
▸ Field & Feature
Introduction
!5
![Page 6: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/6.jpg)
▸ Feature interactions are prevalent and need to be specifically modeled.
▸ Features from one field often interact differently with features from different other fields.
▸ Potentially high model complexity needs to be taken care of.
Challenge
!6
![Page 7: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/7.jpg)
▸ Factorization Machine(FM, 因子分解機)
▸ Field-aware Factorization Machine(FFM)
▸ Field-weighted Factorization Machine(FwFM)
Background
!7
![Page 8: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/8.jpg)
Outline
▸ Introduction
▸Method ▸ Experiment
▸ Conclusion
!8
![Page 9: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/9.jpg)
!9
Evolution▸ Logistic Regression model
▸ Degree-2 Polynomial model
![Page 10: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/10.jpg)
!10
Evolution▸ Factorization Machine
▸ Field-aware Factorization Machine(FFM)
![Page 11: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/11.jpg)
!11
Evolution▸ Field-weighted Factorization
Machine(FwFM)
![Page 12: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/12.jpg)
!12
Mutual Information
![Page 13: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/13.jpg)
Outline
▸ Introduction
▸ Method
▸ Experiment ▸ Conclusion
!13
![Page 14: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/14.jpg)
▸Data sets
!14
Experiment
![Page 15: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/15.jpg)
!15
Experiment▸ Comparison of FwFMs with Existing
Models.
![Page 16: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/16.jpg)
!16
Experiment▸ Comparison of FwFMs and FFMs
using the same number of parameters.
![Page 17: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/17.jpg)
▸ L2 Regularization
!17
Experiment
![Page 18: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/18.jpg)
▸ Learning Rate
!18
Experiment
![Page 19: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/19.jpg)
▸ Embedding Vector Dimension
!19
Experiment
![Page 20: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/20.jpg)
▸ Learned field interaction strengths
• For • For • For
!20
Experiment
![Page 21: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/21.jpg)
!21
Experiment
P1356
![Page 22: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/22.jpg)
!22
Experiment
![Page 23: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/23.jpg)
Outline
▸ Introduction
▸ Method
▸ Experiment
▸ Conclusion
!23
![Page 24: Field-weighted Factorization Machines for Click-Through Rate …184pc128.csie.ntnu.edu.tw/presentation/18-05-22/Field-weighted... · Learning Rate !18 Experiment Embedding Vector](https://reader035.fdocuments.in/reader035/viewer/2022071013/5fcbf70c244c2049b826316a/html5/thumbnails/24.jpg)
Conclusion▸ FwFMs are competitive to FFMs with
significantly less parameters. ▸ FwFMs can indeed learn different
feature interaction strengths from different field pairs.
!24