The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015
Transcript of The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015
![Page 1: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/1.jpg)
SANDcamp, San Diego, CA - February 27, 2015
The Great ConsolidationEntertainment Weekly migration case study
![Page 2: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/2.jpg)
Who are we?
• Four Kitchens Web Chefs
• Jon Peck - drupal.org/u/fluxsauce
• Matt Grill - drupal.org/u/drpal
• Patrick Coffey - drupal.org/u/pcoffey
![Page 3: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/3.jpg)
Project overviewWhat did we do & who did it?
![Page 4: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/4.jpg)
Entertainment Weekly
• Magazine that covers film, TV, music, Broadway theater, books, and pop culture
• 40th largest in the US
• ew.com
• 13.7 million consumers / week
![Page 5: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/5.jpg)
Migration scope
• WordPress
• 10 blogs
• 51,616 terms
• 101,581 posts
• 90,435 images
• Vignette V6
• 117,326 articles
• 201,986 images
• 97,611 terms
• 4,924 galleries
![Page 6: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/6.jpg)
Development Teams
• Four Kitchens - 3 developers to 6
• Time Inc. - 1 developer to 4
• Zoom, HipChat for regular communication
• Video really helps
• As does GitHub selfies :-)
![Page 7: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/7.jpg)
Timeline
• First commit: April 29th, 2014
• Launch: January 29th, 2015
• 17 sprints
![Page 8: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/8.jpg)
Delegation of responsibilities
• Four Kitchens
• Project Management
• Data structures
• Migration
• Implement design, advertising
• Time Inc.
• Product Owner
• Design
• Workflow
• Infrastructure, build process
![Page 9: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/9.jpg)
ProcessStandards and documentation
![Page 10: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/10.jpg)
Definition of Readiness and Completion• Stakeholders
• Context necessary for development
• Developers
• Code reviews, PO acceptance, demo, docs
• Evolving framework
![Page 11: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/11.jpg)
Code and style standards
• Algorithmically enforced
• PHP_CodeSniffer (rules from coder)
• jshint - syntax
• jscs - code style
• EditorConfig settings
![Page 12: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/12.jpg)
Peer code reviews
• Functional - does it…
• Fulfill the intent of the story?
• Use best practices?
• Avoid technical debt?
![Page 13: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/13.jpg)
Development Environments
• Documented project-specific process
• Superset of existing instructions
• Greatly reduced on-boarding time
• Provided plentiful feedback to DCMS
• Beta tested new VMs
![Page 14: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/14.jpg)
Branching standard
• Reduces clutter
• Improves communication, navigation
• Best practices
• sprint-xx (kept for one sprint)
• SEGEWCMSM-YY (deleted after merge)
• Tags added when deleting sprint branch
![Page 15: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/15.jpg)
Front-endTheming, performance and advertising
![Page 16: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/16.jpg)
THEMING
![Page 17: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/17.jpg)
https://www.drupal.org/project/aurora
Aurora, mobile first theming
• Used Aurora as base theme
• HTML5, Sass & Compass
• Mobile theme is sub-theme of Aurora
• Desktop is a sub-theme of Mobile
![Page 18: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/18.jpg)
Gulp task management
• Compiles Sass
• Runs JavaScript checkers
• jshint
• jscs
![Page 19: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/19.jpg)
JAVASCRIPT
![Page 20: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/20.jpg)
JavaScript global scoping
• Created global JS object for all EW functions
• Best practice
• Centralized point of entry
• Easy access
![Page 21: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/21.jpg)
Atomic JavaScript functionality• Break related functions into individual files
• Only load what’s needed
• No monolithic structure
• Improves front-end performance
![Page 22: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/22.jpg)
PREPROCESSING
![Page 23: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/23.jpg)
Balancing performance with business logic• Node template data built in pre-processing
• Does not use views, panels
• Clean & reusable
• Intuitive structure
• Helper and abstractions
![Page 24: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/24.jpg)
BEACONS
![Page 25: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/25.jpg)
Global metadata object
• Useful for integrations requiring metadata
• Avoids repetition
• Facilitates data collection by crawlers
![Page 26: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/26.jpg)
Beacon performance
• Delay loading of beacons (non-blocking)
• Use defer for injecting JS tags
![Page 27: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/27.jpg)
ADVERTISING
![Page 28: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/28.jpg)
Collaborating with DCMS
• TGX improvements
• Single-request mode
• More native GPT functionality
• Ad Manager
• Patches, recommendations
![Page 29: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/29.jpg)
Custom ad rendering to maximise performance• No logic in templates
• Data attributes contain values
• Avoids inline JS
• Improves rendering performance
• Ad renders in footer (after page load)
![Page 30: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/30.jpg)
Editorial UI & Workflow
![Page 31: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/31.jpg)
Packages and Channels
• Automatic dynamic content
• Editors define rules for selection
• Optional manual curation
![Page 32: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/32.jpg)
Custom content types
• Used existing DCMS features to begin
• Rounds of PO, editor feedback
• Usability key to adoption
![Page 33: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/33.jpg)
Collaboration with DCMS
• Provided feedback on tools
• Search interfaces
• Transient messages
• Crop tool
• Auto save
![Page 34: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/34.jpg)
Dynamic Entity References
• Lots of aggregate views in EW
• Editorial control of views, but with overrides
• Custom module that “fills in” empty parts of a view based on context
• View lists 15, editor defines 5, DER populates remaining
![Page 35: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/35.jpg)
Dynamic ER challenges
• Modular, extensible, reusable
• Accurate dynamically curated nodes
• High performance
• Editoral expectations for caching
• State Machine integration
![Page 36: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/36.jpg)
Content migrationsWordPress, Vignette to Drupal
![Page 37: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/37.jpg)
WORDPRESS
![Page 38: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/38.jpg)
Challenges
• 10 different blogs
• Different structures
• Multiple taxonomies
• Invalid markup
• Custom shortcodes, filters
• WordPress Migrate module not enough
![Page 39: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/39.jpg)
Solutions
• Interviewed other teams, shared code
• Pre-processing
• Strip comments
• Combine into one monolithic file
• Separates authors, tags, images
![Page 40: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/40.jpg)
Shortcodes and Filters
• Custom markup
• Created custom short code system
• Most will not be migrated
• Pre-processing solution
• Render as HTML
• Used WordPress, existing code
![Page 41: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/41.jpg)
WordPress Migrate
• Can’t use directly…
• Can extend!
• Helps normalize WXR structure
• Focus on custom logic
![Page 42: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/42.jpg)
Mapping terms on import
• Separates monolithic taxonomy
• Specific vocabularies
• New content types (people, creative works)
• CSV Spreadsheet used for defining rules
• Curated by Editors, PO
![Page 43: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/43.jpg)
VIGNETTE
![Page 44: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/44.jpg)
Exporting from Vignette
• Coordinated with TI team
• Defined XML structure
• Gave feedback on export UI
• Resolve edge cases
![Page 45: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/45.jpg)
Importing
• XML content is sanitized and validated
• Transforms overloaded taxonomies to content
• People, Creative Works
• Articles, Packages, Galleries
![Page 46: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/46.jpg)
MIGRATION PERFORMANCE
![Page 47: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/47.jpg)
Improving migration performance• Validate all XML before ingestion
• Eliminate all PHP errors, warnings
• Avoid redundant migrations
• Separate in pre-import
• Reduce overall number of migrations
• Implement cache_counts
![Page 48: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/48.jpg)
Reducing migration overhead
• Disabled solr, pathauto, metatag, others during migration
• Project requirement was 2.5; you should use 2.7
• Ran migrations in parallel
• Ensure HW resources are sufficient
![Page 49: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/49.jpg)
Migration redirects
• Minimize Drupal performing redirects
• Redirect farm (rules, 1:1 redirects)
• Exported if source, target known
• DNS changes
• Redirects from migrated servers
![Page 50: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/50.jpg)
Performance and caching
![Page 51: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/51.jpg)
Frontend performance
• CSS
• Sass and Compass optimized
• Targeted - only load what’s needed
• JavaScript
• Linted, strict standards
• In the footer (non-blocking)
• Global objects (less is more)
![Page 52: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/52.jpg)
Frontend caching strategy
• Editorial 1 minute publish to live
• Ensure cacheable headers are set
• Shorter TTL on Akamai than Varnish
• Ex: 5 minutes on Akamai, 10 on Varnish
• Send purges on publishing
![Page 53: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/53.jpg)
Backend performance
• Eliminate PHP errors
• Cache, EXPLAIN custom queries
• Minimize unnecessary overhead
• Module count
• Benchmarking
![Page 54: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/54.jpg)
Performance testing
• Load test production with migrated data
• blitz.io
• New Relic for introspection
• Exercised different content types, exceeded TTLs
• Front-end testing
• WebPageTest.org
![Page 55: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/55.jpg)
Questions?
![Page 56: The Great Consolidation - Entertainment Weekly Migration Case Study - SANDcamp 2015](https://reader030.fdocuments.in/reader030/viewer/2022020208/55a64bc11a28abe7028b4658/html5/thumbnails/56.jpg)
Thank you!