Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf ·...
Transcript of Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf ·...
![Page 1: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/1.jpg)
Joomla, 40 Million Page Views, and YouMitch PirtleNYPHP, January 2009
![Page 2: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/2.jpg)
Summary Overview
MTV brought Mitch Pirtle in to recommend a platform for a very high-traffic website - he chose an open source platform and then led development to a successful
release. Problems solved: lowering costs, fostering a can-do philosophy, promotion of the free software approach of
sharing with others, unique development approaches learned from working on large FOSS projects
![Page 3: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/3.jpg)
Obligatory ‘Me’ Slide
American, married Swiss
Lived in US and EU
Currently in NYC
Web-only since 1995
VP at KickApps
High traffic sites
Pragmatic, not dogmatic
Joomla! co-founder
Former Mambo core
Booster of many FOSS
Co-founded NYCJUG
Too many projects to list
![Page 4: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/4.jpg)
Let’s Get This Thing Started
![Page 5: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/5.jpg)
Audience Breakdown
![Page 6: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/6.jpg)
TechnologyAre you a technologies person?
![Page 7: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/7.jpg)
BusinessAre you a business person?
![Page 8: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/8.jpg)
Not Technology or BusinessAre you neither?
![Page 9: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/9.jpg)
Drum Roll, Please
![Page 11: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/11.jpg)
The Challenge
www.quizilla.com
Acquired in 2006
Peaks of 40,000,000 unique pageviews in a week
Mix of two codebases (perl & php)
Difficult to maintain
Difficult to augment/evolve
![Page 12: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/12.jpg)
The Objectives: New World
High performance
Reliable, stable
Scale to extremes
Easy to maintain
Easy to extend
![Page 13: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/13.jpg)
The Shopping List
Open Source
Framework
MVC
Strong Community
Active Development
Common Language
Available Resources
Modular Architecture
Flexible, Customizable
![Page 14: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/14.jpg)
Winner: Joomla! Frameworkhttp://docs.joomla.org/Category:Framework
![Page 15: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/15.jpg)
Ok, we could have used CakePHP, Symfony, Zend Framework, Django, Ruby on Rails, Merb....
However PHP was desired, and I was available.
Psssssst! Hey buddy!
![Page 16: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/16.jpg)
Why Joomla! Framework?
Meets all objectives
PHP, MySQL
FOSS
Flexible
Framework
MVC...
![Page 17: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/17.jpg)
Result: Success
Only needed 3 webservers
Only needed 2 database servers
Becoming standard base platform for future development
![Page 18: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/18.jpg)
Impact: Development
Atomic MVC - many developers, few conflicts
Source code - Tight reliance on revision control systems
Issue Tracker - Daily SCRUM-style planning meet
Change management - Build process with controls
![Page 19: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/19.jpg)
Atomic MVC?
Ok, I made that up. Sounds cool though.
![Page 20: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/20.jpg)
What MVC Looks Like
![Page 21: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/21.jpg)
This is Atomic MVC
![Page 22: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/22.jpg)
Plain MVC
![Page 23: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/23.jpg)
Atomic MVC
![Page 24: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/24.jpg)
Impact: Philosophy
Anything developed that wasn’t a proprietary feature or function was contributed back to the project
All code was packaged for ease of reuse on other projects within the enterprise
![Page 25: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/25.jpg)
So What Did We Learn?
![Page 26: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/26.jpg)
One Size Fits AllOk, almost.
![Page 27: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/27.jpg)
Modular FrameworkNot fully decoupled from CMS (yet)
![Page 28: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/28.jpg)
Database ClassStill very specific to one instance - no tiers or pools
![Page 29: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/29.jpg)
So How Does It End?No biggie. We didn’t fall ALL the way down.
![Page 30: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/30.jpg)
One Size Fits All: Classes!
Joomla is a stack framework
No need for language support
No need for dynamic navigation
No need for third party API support
JDate? A date class? Are you freaking kidding me?
![Page 31: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/31.jpg)
Modular Framework: Extend
Extended existing classes for specific needs
Introduced helper classes to simplify complex tasks
![Page 32: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/32.jpg)
Taming JDatabase
We cheated. And we’re proud of it.
Extended jConf with a constructor for poor-man’s query pool
Extended JDatabase mysqli driver to have one connection for reads, one for writes
![Page 33: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/33.jpg)
JSession Ate My Brain
Changes too numerous to mention :-)
Fixes to storage driver
Fixes to cookie mechanism
Auto-remember
Auto-resume
![Page 34: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/34.jpg)
Input, Output
Hacked the output buffers so modules could share information with components
![Page 35: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/35.jpg)
Special Note on Caching
JCache needs help. Serious help.
Used memcached driver
Chose our own key namespace strategy
Switched to a write-through cache approach
Dropped output caching entirely
Added per-item expiration TTL
![Page 36: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/36.jpg)
Current Status
Codebase is now part of the standard offering for MTV websites that would like to take advantage of the platform
![Page 37: Joomla, 40 Million Page Views, and Younyphp.org/...MTV-Quizilla-High-Traffic-Scaling-PHP.pdf · platform and then led development to a successful release. Problems solved: lowering](https://reader034.fdocuments.in/reader034/viewer/2022042320/5f0a48f37e708231d42ae873/html5/thumbnails/37.jpg)
Questions?