Delivering Search for Today's Local, Social, and Mobile Applications
-
Upload
amazon-web-services -
Category
Technology
-
view
685 -
download
0
description
Transcript of Delivering Search for Today's Local, Social, and Mobile Applications
![Page 1: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/1.jpg)
Jon Handler
Delivering Search for Today's Local, Social, and Mobile
Applications
CloudSearch Solution Architect
![Page 2: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/2.jpg)
Agenda
• Search for social and mobile applications
• Amazon CloudSearch
• Customer stories and panel
– Tim Ramsey, CTO/Co-founder for Ziplist / Conde Nast
– Pat Binkley, VP Engineering for Zumobi
– Ken Chung, CTO for Viddy
![Page 3: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/3.jpg)
Time Spent Online (source: go-gulf.com)
![Page 4: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/4.jpg)
Social Activities
• People are talking!
• Word of mouth is the primary factor behind
20% to 50% of all purchasing decisions
(Jonah Berger, "Contagious")
• Search brings relevance to the process of
pulling in friends
![Page 5: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/5.jpg)
The Rise of Mobile Search (Source: BIA / Kelsey)
• In 2011, Search was 75% desktop,
25% mobile
• In 2013, they project 60% / 40%
• By 2016, 57% mobile, 43% desktop
![Page 6: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/6.jpg)
In the Mobile Space, Local Is Growing
![Page 7: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/7.jpg)
The Path to Search
• Search is central to user experience
• Options for building a search experience: BYO, open source, legacy enterprise
• Challenges
![Page 8: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/8.jpg)
Amazon CloudSearch: Simple Search Simply
• Pay for infrastructure as you need it
• Lower total cost of operation
• No need to guess at capacity
• Increase innovation – low risk, inexpensive, simple experimentation
• Does the undifferentiated heavy lifting
• Available in 5 regions: go global in minutes
![Page 9: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/9.jpg)
Amazon CloudSearch Overview
DNS / Load Balancing AWS Query
Search API Console Config
API
Command
Line Interface Console Doc
Svc API
Command
Line Interface Console
SEARCH SERVICE Search Documents
DOCUMENT SERVICE Add Documents
Update Documents
Delete Documents
Create Domains
Configure Domains
Delete Domains
CONFIG SERVICE
Search Domain
![Page 10: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/10.jpg)
CloudSearch in Context
![Page 11: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/11.jpg)
Automatic Scaling
SEARCH INSTANCE Index Partition n
Copy 1
SEARCH INSTANCE Index Partition 2
Copy 2
SEARCH INSTANCE Index Partition n
Copy 2
SEARCH INSTANCE Index Partition 2
Copy n
SEARCH INSTANCE
DATA Document Quantity and Size
TRAFFIC Search Request Volume and Complexity
Index Partition n Copy n
SEARCH INSTANCE Index Partition 1
Copy 1
SEARCH INSTANCE Index Partition 2
Copy 1
SEARCH INSTANCE Index Partition 1
Copy 2
SEARCH INSTANCE Index Partition 1
Copy n
![Page 12: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/12.jpg)
Pricing
• Get started for just $2.40/day; $75/month
• AWS Calculator http://calculator.s3.amazonaws.com/calc5.html
Get started now – 30 days free
![Page 13: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/13.jpg)
Customer Stories
![Page 14: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/14.jpg)
Searching Recipes and Recipe Boxes on ZipList
Tim Ramsey
Chief Architect and Co-Founder,
Ziplist Inc.
![Page 15: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/15.jpg)
ZipList.com - High-Level Description and Context
• Shopping List
• Recipes and Recipe Box
o 1.6M Recipes in Global Index
o 25M Recipes in Boxes
• 3.5M Users
• iPhone and Android Mobile Apps –
315K Downloads
• Partners
o 300+ White Label Partners
(SkinnyTaste.ZipList.com)
o 1,500+ Recipe Partners
(DarcyDiva.com)
• Hardware/Software Platform
o Amazon EC2
o Amazon RDS
o Amazon CloudSearch
o Ruby on Rails
![Page 16: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/16.jpg)
ZipList Recipes
Indexed from Around the Web
• Structured (AllRecipes.com) or
Unstructured (Saveur.com)
• Recipes collected with and without
website cooperation
• Pushes to API or JIT with the Recipe
Clipper
* Attributes used in recipe box search/filtering
Attributes Collected and Indexed
• Title*
• Description
• Photo* (yes/no)
• Publisher* (Martha Stewart, Serious
Eats, etc.)
• Ingredients (text and number*)
• Instructions
• Publisher Tags*
• Course* (main, dessert, breakfast, etc.)
• Season (Thanksgiving, summer,
birthday, etc.)
• Etc.
![Page 17: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/17.jpg)
Searching ZipList Recipes
• First try – Ferret o Ability to index on any attribute without DB changes
o Zero latency between insert and available in index
o Runtime load and stability
• Next try – Sphinx o Incremental indexing load and time
o Indexing DB load
o Search daemon load and latency
• Current – CloudSearch o SDF allows for synthetic fields (like ferret)
o Fast query results
o Indexing latency issues
![Page 18: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/18.jpg)
ZipList Platform
ZipList Recipe Indexing
WWW Recipes
ZL REcipes
User A Recipe Box
User B Recipe Box
Recipes CloudSearch
SDF Documents
Save Recipe
Save Recipe
Recipe Box Cloudsearch
SDF Documents
Recipes Clipped or Pushed
![Page 19: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/19.jpg)
Recipe Box Indexing Latency
• Desire to unify the UX for Global Recipe Search and Recipe Box Search
o Filtering
o Facet Counts
o Pagination
• Scenario
o User finds a recipe on Epicurious.com and adds it to their box
o They then go immediately to ZipList.com to view their Recipe Box and filter by
Epicurious publisher
o The time it takes for the recipe to appear is LTOT = LSDF + LCSI
o Even 10s is unacceptable
![Page 20: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/20.jpg)
ZipList Platform
Handling Indexing Latency
ZL Recipes
Recipe Box
Recipe Box Cloudsearch
SDF Documents
Save Recipe
Query Module
RB Cache
Search Query
Search Results
![Page 21: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/21.jpg)
Summary and Conclusions
• With the latency issues resolved, CloudSearch works very well for our
searching needs
• Searching, filtering, and faceting provided by CloudSearch fit in well with our
application
• We have a consistent interface / capabilities between Global Recipe Search
and Recipe Box Search
• Positions ZipList well for moving to NoSQL solutions
![Page 22: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/22.jpg)
Zumobi
Pat Binkley, VP Engineering
![Page 23: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/23.jpg)
Zumobi is the leading mobile media company that partners with top media brands to publish premium applications
and provide integrated native advertising experiences on smartphones and connected devices.
The Zumobi Brand Integration (ZBi) native advertising platform offers an SDK to drive a wide array of features and formats enabling seamlessly integrated brand experiences on mobile.
Zumobi’s Long History of Mobile Innovation
![Page 24: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/24.jpg)
CloudSearch Example: THE WEEK App
• Why? Parity with Desktop experience
– Most news apps are not yet integrated with search
capabilities, but desktop is. Mobile is quickly catching up
with desktop functionalities
• Problem? Volume of data on device
– Zumobi apps store content locally for performance and
offline experience
– Too much data to store on phone to provide meaningful
search results
• Solution? AWS CloudSearch
– Store documents in the cloud and query
![Page 25: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/25.jpg)
New Capabilities
• Keyword Searching
• Relevant Results Display
• Article Viewing
• Social sharing
– Facebook & Twitter
• Save
– Save your favorite articles to review later
![Page 26: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/26.jpg)
Architecture
Zumobi
Platform
(Rails App)
CloudSearch DOCUMENT
SERVICE RSS
Optional: >2K items
Document Request
Response
CloudSearch SEARCH
SERVICE Step 1: Search for Item/List Returned
Step 2: Fetch optional > 2K items
App Content SQLite/S3
CDN
** Implementation Time: - 5 days
![Page 27: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/27.jpg)
Impact
• Feature parity with THEWEEK.com website
• Enhanced App Performance
– Increased Usage
• Amplified User Engagement
• More Pageviews
• New Users
• Data Collection
– Find out what your audience is searching for in order to provide more relevant
content
Impact
![Page 28: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/28.jpg)
Zumobi App Network Future Search Integrations…
![Page 29: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/29.jpg)
Learn More – Check Out
• Zumobi.com
• Zumobi.com/AdLab
![Page 30: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/30.jpg)
Viddy
Ken Chung, CTO
![Page 31: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/31.jpg)
Viddy
Viddy is a simple way to capture,
create, and share short, 30-second
social videos with friends.
![Page 32: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/32.jpg)
![Page 33: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/33.jpg)
Viddy – Search Use Cases
• Users want to discover content around them
• Users want to find their friends and celebrities
• Users want to discover content with certain #hashtags and keywords
• Users want to discover more relevant content by language, country, and more
context
![Page 34: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/34.jpg)
Viddy – Unique Problem? No
• Problems not unique to Viddy
• With rise of social, mobile, and geo-aware mobile apps, search is basically
everyone’s challenge
• Why re-invent the wheel?
• Let the platform handle the headaches and let us focus on innovations!
![Page 35: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/35.jpg)
Viddy – B.C. aka “Before CloudSearch”
• Custom managed Lucene + Indexers + Search API Servers
• A full dedicated backend development effort for search product (out of two
backend engineers)
• Lack of tooling for optimizing queries and improving search relevancy by
non-engineers
• Lack of monitoring and reporting around search
![Page 36: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/36.jpg)
… and then 300K users to 40 million users in less than 6 months
![Page 37: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/37.jpg)
Viddy – Hello, CloudSearch!
• Migrated entirely from Lucene to CloudSearch in 3 days
• Simple CloudSearch REST API for Indexing and Querying
• 2–5 hrs/wk of maintenance effort for search product
• Interns improving search result through CloudSearch console
• Better monitoring and report
![Page 38: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/38.jpg)
Viddy – Aftermath
• Operation cost: $5–6K/mo to $1600/mo
• More development resources for innovation
• 5x increase in search usage
• Faster incremental index: avg. delay from 15 mins to 30 secs
![Page 39: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/39.jpg)
Customer Panel
![Page 40: Delivering Search for Today's Local, Social, and Mobile Applications](https://reader033.fdocuments.in/reader033/viewer/2022060108/554e5934b4c905532d8b49e6/html5/thumbnails/40.jpg)
Thank You!
• Learn more at http://aws.amazon.com/cloudsearch
• Please fill out your evaluation forms