Recommendation engine
-
Upload
vignesh-prajapati -
Category
Technology
-
view
1.559 -
download
0
description
Transcript of Recommendation engine
![Page 1: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/1.jpg)
Recommendation Engine
![Page 2: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/2.jpg)
OutlinesIntroductionObjectivesScopeProblem with existing systemPurpose of new systemProposed architectureTechnologies to be usedModules of systemIntegration of technologiesImplementation Issues to be solvedApplicationFuture Enhancement
![Page 3: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/3.jpg)
ObjectivesInformation Filtering System
Recommendation engine recommends - User based - Item based - Slop based
Run On Cloud Environment
![Page 4: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/4.jpg)
IntroductionEngine - Gives Suggestion Based on
movies,songs,videos,websites,books,images and also social elements.
Applicable for E-business.
Useful for both Customers and online Retailers
Recommendation engine is being used at Amazon, Youtube, Facebook,Twitter
![Page 5: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/5.jpg)
ScopeOur system will only provide Recommendation
service only.
Recommendation will be genrated based on user’s historical activity like purchase pattern as well as rating and like.
Recommendation will be either stored on database ,file or directly retrieved to retailers web application.
![Page 6: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/6.jpg)
Problems with existing System
Take more Time to generate recommendations
No real time recommendation for large data
![Page 7: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/7.jpg)
Purpose of new System Less time for generating recommendations
Applicable for Bigdata
Recommendations be several algorithms User based Item based Slop based Association rule mining
Evaluation of recommendation
![Page 8: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/8.jpg)
Recommendations-TypeUser Based Recommendation
![Page 9: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/9.jpg)
Recommendations-TypeItem Based Recommendation
![Page 10: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/10.jpg)
Proposed System Architecture
![Page 11: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/11.jpg)
Technologies to be usedHadoop
Mahout
Graphlab
Google prediction
Google Storage
Google App engine
![Page 12: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/12.jpg)
Modules of SystemUser Module
Admin Module
Recommendation Module
File management Module
Search Module
![Page 13: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/13.jpg)
Integration of TechnologiesMahout based Recommendation
Graph based Recommendation
Google prediction Based Recommendation
![Page 14: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/14.jpg)
Technology: HADOOPHadoop is a top-level Apache project being built
and used by a global community of contributors.Hadoop project develops open-source software
for reliable, scalable, distributed computing.It enables applications to work with thousands of
nodes and peta bytes of data.Hadoop also support Map/Reduce Algorithm. It provides HDFS file system that stores data
on the compute nodes.
![Page 15: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/15.jpg)
Hadoop
![Page 16: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/16.jpg)
Graphlab It is New Parallel Framework for Machine
Learning Algorithm .Now a day ,Designing and implementing
efficient and correct parallel machine learning (ML) algorithms can be very challenging.
Designed specifically for ML needsAutomatic data synchronization.Map phase like – Update Function .Reduce phase like – Sync Operation .
![Page 17: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/17.jpg)
17
Data GraphShared Data Table
Scheduling
Update Functions and Scopes
GraphLabModel
![Page 18: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/18.jpg)
CPU 1 CPU 2 CPU 3 CPU 4
MapReduce – Map Phase
18
Embarrassingly Parallel independent computation
12.9
42.3
21.3
25.8
No Communication needed
![Page 19: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/19.jpg)
CPU 1 CPU 2 CPU 3 CPU 4
MapReduce – Map Phase
19
Embarrassingly Parallel independent computation
12.9
42.3
21.3
25.8
24.1
84.3
18.4
84.4
No Communication needed
![Page 20: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/20.jpg)
CPU 1 CPU 2
MapReduce – Reduce Phase
20
12.9
42.3
21.3
25.8
24.1
84.3
18.4
84.4
17.5
67.5
14.9
34.3
2226.
26
1726.
31
Fold/Aggregation
![Page 21: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/21.jpg)
Graphlab in RecommendationGraphlab provide better way in
recommendation engine.Its just first load fits simple dataset file. In graphlab we can also implement various
algortihm like k-means clustering ,fuzzy logic, pagerank and etc.
Its first translated dataset into Matrix form.And then according to different algorithm it
generated recommendated output.
![Page 22: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/22.jpg)
Google Prediction ServiceGoogle cloud service used for Building smart
Application.Having Machine learning Algorithms.Related to Artificial Intelligence.
![Page 23: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/23.jpg)
Google Prediction Service
Google Prediction API : Set of Methods for Data Analysis.Libraries support multiple languages.
Google App Engine :Enable Application to Cloud environment
Application serverGoogle Cloud Storage :
Enable Data to store on Google Cloud database.
![Page 24: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/24.jpg)
Google Prediction Service
![Page 25: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/25.jpg)
Technology : MAHOUT • Apache Mahout is open source project by the
Apache Software Foundation (ASF).• The primary goal of Mahout is creating
scalable machine-learning algorithms.• Several Map-Reduce in Mahout enabled
clustering implementations, including k-Means, fuzzy k-Means, Canopy, Dirichlet, and Mean-Shift.
• Mahout have fix datasets which generally take as data input.
• Amzon EC2 are working with Hadoop and Mahout.
![Page 26: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/26.jpg)
Implementation Issues to solvedLack of knowledge about hadoop,mahout,hiveMemory issueOperating system supportLoad BalancingConfiguration Data normalizationDeveloping Clustering algorithmConfiguring mahout with hadoop
![Page 27: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/27.jpg)
Application of recommendationYahoo!FacebookTwitterBaidueBayLinkedInNew York TimesRackspaceeHarmonyPowerset
Recommendation Engine
![Page 28: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/28.jpg)
Future enhancementIntegration with Web Application like Jsp , Servlet
Integration with Database like Hive, Hbase, Mongodb, Couch db
Cloud based recommendation Service
Integration of Mahout , Graphlab and Google prediction based recommendation services.
Mobile application integration
![Page 29: Recommendation engine](https://reader036.fdocuments.in/reader036/viewer/2022062303/554f0c6db4c90580698b53d6/html5/thumbnails/29.jpg)
Thank You