Moving from fast to solr on atg
-
Upload
lucenerevolution -
Category
Education
-
view
718 -
download
0
description
Transcript of Moving from fast to solr on atg
![Page 1: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/1.jpg)
Moving from FAST to Solr on ATG Ricardo Merizalde Software Development Manager Backcountry.com
![Page 2: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/2.jpg)
Who We Are
§ Online retailer of high-‐end outdoor gear § Founded in 1996 by former Olympic skier § 7 Ecommerce sites § Internet Retailer Top 100
“We use the gear we sell”
![Page 3: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/3.jpg)
More Background
§ Legacy Perl/Postgres ecommerce plaEorm § Using FAST Impulse/ESP since 2008 § Implemented Oracle/ATG Commerce in 2012 § 1.5MM daily search engine hits during Q4 2012
![Page 4: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/4.jpg)
Why change search engines?
§ Product End of Life § Index latency § Hardware footprint § License limitaUons § Support effort is high § Not a good fit for conUnuous delivery
![Page 5: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/5.jpg)
Goals
§ Low index latency § High availability § Easy to scale § ConUnuous delivery § Improve user experience § Improve merchandising tools
![Page 6: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/6.jpg)
OpenCommerceSearch
![Page 7: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/7.jpg)
What’s OpenCommerceSearch?
§ Oracle ATG/Solr IntegraUon § Search Module: Business Tools § Framework to ease development
![Page 8: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/8.jpg)
Integration Overview
Each environment has its own collections
![Page 9: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/9.jpg)
Search Module § Manage search assets
• Facets • Redirects • Synonym Lists & Synonyms • Boosts & Blocks • Ranking Rules
§ Rule engine uses Solr • Has its own index (very small) • Extra search call to get rules (fast &
cacheable)
![Page 10: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/10.jpg)
Search Module
![Page 11: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/11.jpg)
Search Module
![Page 12: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/12.jpg)
Search Module
§ Rule’s context • Target Page: Search, Category or All Pages • Query: shirts, [the north face] • MulU-‐site aware • Catalog aware • Category aware
§ Rule’s life Ume • Start & End Date
![Page 13: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/13.jpg)
Search Module
§ Rule’s custom properUes
![Page 14: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/14.jpg)
Search Module
§ Ranking rules • Boost/demote chunks of products instead of
individual products • Less manual boosUng • Ideal state is lidle/no manual boosUng. Use
mulUple data points to automate ranking (clicks, revenue, etc.)
• Uses boost funcUons
![Page 15: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/15.jpg)
Search Module
![Page 16: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/16.jpg)
Search Module
![Page 17: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/17.jpg)
Search Module
![Page 18: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/18.jpg)
Search Module
§ Rule based categories • PromoUon landing pages • Keyword landing pages • Price range, discount off, gender, hand pick
products, etc.
![Page 19: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/19.jpg)
Search Module
![Page 20: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/20.jpg)
Search Module § Asset MigraUon
• Redirects: imported as exact matches • Synonyms: manual cleanup • Boosts & Blocks: not imported • Facets: not imported
§ Query Synonyms • FAST is a black box • Analysis Tool • Synonym-‐Expanding EDisMax Parser wriden
by Nolan Lawson
![Page 21: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/21.jpg)
Who We Are
Index Latency
![Page 22: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/22.jpg)
Index Latency § Feed limited to XML feed/exlt tool § Slow process with mulUple points of
failure 1. Generate XML files 2. scp XML files to FAST servers 3. exlt cronjob loads feed into SQL schema 4. JDBC Connector checking for changes 5. Document Processing Pipeline 6. Index documents
![Page 23: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/23.jpg)
Index Latency § Index documents using API calls § Different SLAs for change types (price,
inventory, content, taxonomy) § Auto soj & hard commits § Products & Skus are not versioned
![Page 24: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/24.jpg)
Index Latency
![Page 25: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/25.jpg)
Who We Are
ConUnuous Delivery
![Page 26: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/26.jpg)
Continuous Delivery § Hard to do integraUon tesUng § Automated tests were flaky
• Data dependencies • No applicaUons states
§ Leverage EmbeddedSolrServer • RAM index • Custom JUnit Test Runner for IntegraUon
tests • Solr XML for data • TestNG & Selenium for Automated tests
![Page 27: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/27.jpg)
Continuous Delivery § IntegraUon Test
• Shared core with bootstrapped data • Clone core & index test data • Test code & server run within same process
![Page 28: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/28.jpg)
Continuous Delivery § Automated Test
• Cores live within applicaUon server • Test data sent over HTTP
![Page 29: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/29.jpg)
Continuous Delivery § Deployments
![Page 30: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/30.jpg)
Continuous Delivery § Rundeck jobs
![Page 31: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/31.jpg)
User Experience
![Page 32: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/32.jpg)
User Experience § Search results not consistent with
product detail page § De-‐normalize data & index skus § Group by product id
![Page 33: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/33.jpg)
User Experience
![Page 34: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/34.jpg)
User Experience
![Page 35: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/35.jpg)
User Experience § Orange jacket not available in XXL
![Page 36: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/36.jpg)
User Experience
![Page 37: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/37.jpg)
User Experience § Orange jacket available in XXL
![Page 38: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/38.jpg)
User Experience § Grouping is awesome but is CPU
intensive § Monitor query response Umes § Pay adenUon to parUal match search (use
mm parameter) § Use the facets you really need
![Page 39: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/39.jpg)
User Experience
QuesUons
![Page 40: Moving from fast to solr on atg](https://reader034.fdocuments.in/reader034/viewer/2022052602/559bd83c1a28ab49158b45c2/html5/thumbnails/40.jpg)
CONTACT Ricardo Merizalde [email protected] https://github.com/rmerizalde/opencommercesearch http://ricardomerizalde.blogspot.com/