Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013
Reco4J Project Intelligent RecommendaDons for
Your Business
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 1
Recommender Systems • A system that can recommend or present items to the user based on the user’s interests and interacDons
• One of the best ways to provide a personalized customer experience
• Built by exploiDng collecDve intelligence to perform predicDons
• Examples: Amazon, YouTube, NeUlix, Yahoo, Tripadvisor, Last.fm, IMDb
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 2
The Example: NeUlix • The world largest online movie rental services, 33 million members in 40 countries
• 60% of members selecDng movies based on recommendaDons (September 2008)
• NeUlix Prize: US$ 1,000,000 was given to the BellKor's PragmaDc Chaos team which bested NeUlix's own algorithm for predicDng raDngs by 10.06% (September 2009)
• 75% of the content watched on the service comes from its recommendaDon engine (April 2012)
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 3
Why Recommender Systems • Standard uses:
– Increase the number of items sold – Sell more diverse items – Increase the user saDsfacDon – Increase user fidelity – Beger understand what the user wants
• Advanced uses: – Create ad hoc campaigns (per geographic area, per type of users) – OpDmize products distribuDon over a wide area for large retail chains
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 4
Problem • There are no available sohware products for state-‐of-‐the-‐art recommender systems
• A high-‐end recommender engine can be built only through expensive custom projects
• Large scale user/item datasets require a big data approach
• There is no "best soluDon" • There is no "one soluDon fits all” • The NeUlix winner composed 104 different algorithms
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 5
SoluDon: Reco4J
A graph-‐based recommender engine
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 6
Reco4J Main Goals • Implement the state-‐of-‐the-‐art in the recommendaDon on top of a graph model
• Provide sohware / cloud services / consultancy
• Contribute to the RecSys research field
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 7
Reco4J Features • Composable models/algorithms • Persistent models (mulD model supported) • Updatable models • Commercial and research-‐oriented algorithms • Independent from source knowledge datasets • Cluster and cloud-‐ready • Context-‐aware recommendaDons • MulDtenant • Social recommendaDons
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 8
Reco4J Under the Hood • J is for Java • CollaboraDve filtering algorithms – Neighborhood-‐based methods – Latent factor models
• Neo4J graph database: – Data source repository – Persistent model repository
• Hadoop cluster/MapReduce • Apache Mahout
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 9
Algorithms Roadmap • CollaboraDve filtering
– Memory based (Neighborhood) • User/Item based
– Several distance algorithms (Cosine, Euclidean, Tanimoto, etc.) • Graph based
– Path Based Similarity (Shortest Path, Number of Paths) – Random Walk Similarity (Item Rank, Average first-‐passage/commute Dme)
– Model based (Latent factor) • StochasDc gradient descendant • AlternaDng least square
• Social recommendaDon – Trust based approach – ProbabilisDc approach
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 10
Algorithms Roadmap (2) • Crosswise Features – Context awareness – Composability – Real-‐Dme – ParallelizaDon
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 11
Context-‐Aware RecommendaDon “The ability to reach out and touch customers anywhere means that companies must deliver not just compe;;ve products but also unique, real-‐;me customer experiences shaped by customer context”
C. K. Prahalad
• Incorporate contextual informa5on in the recommendaDon process • Modeling contextual InformaDon
– From: User x Item -‐> RaDng – To: User x Item x Context -‐> RaDng
• Hierarchical structure • Three approaches
– Contextual pre-‐filtering – Contextual post-‐filtering – Contextual modeling
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 12
Reco4J Context Model
users items rate(c1, …, cn)
………….
contextualized ra;ngs
Context Tuple DAG
usera-‐mp1 itemc-‐split1
…….. usera-‐mp2
itemc-‐split2
itemd-‐splitk
userb-‐mpk
…….. ……..
user micro profiles item spliEng
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 13
Advantage of graph database • NoSQL database to handle BigData issue • Extensibility • No aggregate-‐oriented database • Minimal informaDon needed • Natural way for represenDng connecDons:
– User -‐ to -‐ item – Item -‐ to -‐ item – User -‐ to -‐ User
• Graph Based/Social Algorithms • Graph ParDDoning (sharding) • Performance
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 14
Example: Find Neighbors
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 15
Why Neo4J? • Java based • Embeddable/Extensible • NaDve graph storage with naDve graph processing engine
• Open Source, with commercial version • Property Graph • ACID support • Scalability/HA • Comprehensive query/traversal opDons
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 16
RecommendaDon Model
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 17
Persistence Model
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 18
Persistence Model
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 19
Persistence Model
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 20
Reco4J + Hadoop • Queue Based Process • Operates both on cluster and cloud • Each process downloads data from
Neo4J/Reco4J before or during computaDon
• Stores data into Reco4J Model
• Scaling augmenDng the number of: • Neo4J Nodes (only one master) • Hadoop Nodes
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 21
Reco4J in the Cloud • Recommenda)on as a service (RaaS) • Reco4J cloud infrastructure offers: – Pay as you need – Pay as you grow – Support for burst – Periodical analysis at lower costs – Test/evaluate several algorithms on a reduced dataset – Compose algorithms dynamically
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 22
Consultancy Goals
Analysis
Data Source
ExploraDon
Process DefiniDon
Import Data
Test/EvaluaDon
Deploy
Alessandro Negro – Luigi Giuri Reco4J Project @ Paris Meetup -‐ May 2013 Page 23
Thank you
Alessandro Negro Linkedin: hgp://it.linkedin.com/in/alessandronegro/ Email: [email protected] Luigi Giuri Linkedin: it.linkedin.com/in/luigigiuri Email: [email protected] Reco4J Site: hgp://www.reco4j.org Twiger: @reco4j GitHub: hgps://github.com/reco4j
Top Related