Adaptive Filtering of Tweets with Machine Learning by Neri Van Otten
-
Upload
pydata -
Category
Technology
-
view
109 -
download
2
description
Transcript of Adaptive Filtering of Tweets with Machine Learning by Neri Van Otten
![Page 1: Adaptive Filtering of Tweets with Machine Learning by Neri Van Otten](https://reader033.fdocuments.in/reader033/viewer/2022061223/54c6c8d34a795938448b45a6/html5/thumbnails/1.jpg)
ADAPTIVE FILTERING OF TWEETS WITH MACHINE LEARNINGNeri Van Otten
Data Scientist, Conversocial
![Page 2: Adaptive Filtering of Tweets with Machine Learning by Neri Van Otten](https://reader033.fdocuments.in/reader033/viewer/2022061223/54c6c8d34a795938448b45a6/html5/thumbnails/2.jpg)
About Conversocial• Social customer service platform
![Page 3: Adaptive Filtering of Tweets with Machine Learning by Neri Van Otten](https://reader033.fdocuments.in/reader033/viewer/2022061223/54c6c8d34a795938448b45a6/html5/thumbnails/3.jpg)
Intelligent Prioritization
![Page 4: Adaptive Filtering of Tweets with Machine Learning by Neri Van Otten](https://reader033.fdocuments.in/reader033/viewer/2022061223/54c6c8d34a795938448b45a6/html5/thumbnails/4.jpg)
Continual Improvement• Incorporate feedback from customers• Constant monitoring performance
![Page 5: Adaptive Filtering of Tweets with Machine Learning by Neri Van Otten](https://reader033.fdocuments.in/reader033/viewer/2022061223/54c6c8d34a795938448b45a6/html5/thumbnails/5.jpg)
How to Process Text• Create features• Tf-idf for bag of words• N-grams• Other features e.g. length of text, contains url, …• Convert to matrix of 0/1• Singular value decomposition (SVD)• Machine learning models
![Page 6: Adaptive Filtering of Tweets with Machine Learning by Neri Van Otten](https://reader033.fdocuments.in/reader033/viewer/2022061223/54c6c8d34a795938448b45a6/html5/thumbnails/6.jpg)
Parameter Optimization• Separate server• Runs on backup copy of database
![Page 7: Adaptive Filtering of Tweets with Machine Learning by Neri Van Otten](https://reader033.fdocuments.in/reader033/viewer/2022061223/54c6c8d34a795938448b45a6/html5/thumbnails/7.jpg)
Training Models• Daily training• Queued and trained when servers are under utilized• Lower priority as the system still has working models• Store to S3
![Page 8: Adaptive Filtering of Tweets with Machine Learning by Neri Van Otten](https://reader033.fdocuments.in/reader033/viewer/2022061223/54c6c8d34a795938448b45a6/html5/thumbnails/8.jpg)
Models in Production• Servers specific for prioritization• Chef to configure new servers• Servers download models from S3• Cache as many models in memory as possible• Evict older models• Use the client specific model to classify message
![Page 9: Adaptive Filtering of Tweets with Machine Learning by Neri Van Otten](https://reader033.fdocuments.in/reader033/viewer/2022061223/54c6c8d34a795938448b45a6/html5/thumbnails/9.jpg)
Questions