SymfonyCamp - Lessons Learned
-
Upload
dustin-whittle -
Category
Technology
-
view
2.709 -
download
2
Transcript of SymfonyCamp - Lessons Learned
![Page 1: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/1.jpg)
LessonslearnedatYahoo!
DustinWhittleYahoo!
Acasestudyonusingopensourcetoolstobuildaplatformforenterprisewebapplicationsusingsymfony.
![Page 2: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/2.jpg)
Overview
‣ HowdidYahoo!buildYahoo!Bookmarks,Yahoo!Widgets,Yahoo!Answers,andDeliciouswiththesymfonyproject?
‣ Whatisascalablearchitecture?
‣ Howtoworkwithopensourceframeworksintheenterprise?
‣ Whatworkedandwhatdidnotwhenbuildingalargeweb2.0productfromopensourcetools?
‣ Q&A?
‣ symfony‐project.com|developer.yahoo.com
![Page 3: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/3.jpg)
WhoamI?‣ Workingwithsymfonysinceopensource
‣ symfonyCoreTeamMember
‣ ResponsibleforthedevelopmentandsupportofsymfonyatYahoo!
‣ WorkedwithY!Answers,Delicious,Y!Widgets,Y!Bookmarks,Yahoo!ApplicationPlatform
‣ Consultant‣ Commercialsymfonysupport+training(USA)
![Page 4: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/4.jpg)
AlookattheopensourceYahoo!‣ Yahoo!isbuiltfromopensourcetoolsandcommunities
‣ FreeBSD,Linux,Apache,PHP,MySQL
‣ Wecontributebackandeveryonewins
‣ Weshareourtools,experience,anddatathroughopenAPIsandwebservices
‣ Yahoo!UserInterfaceLibraries+EdgeCaching‣ UserInterfaceDesignPatterns‣ ExceptionalPerformanceRules+YSlow
‣ R3I18n+Template+TranslationManagement
![Page 5: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/5.jpg)
AlookatYahoo!Answers‣ http://answers.yahoo.com
‣ Yahoo!AnswersisthelargestcollectionofhumanknowledgeontheWebwithmorethan135millionusersand515millionanswersworldwide(Yahoo!InternalData,March2008).
‣ Yahoo!Answersisthe2ndrankededucation&referencesiteontheweb(comScore)
‣ Availablein26marketsand12languages
![Page 6: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/6.jpg)
Yahoo!Answersatthebeginning‣ StartedasasmalldevelopmentteamonPHP4fromaforkof
Yahoo!TaiwanKnowledge+
‣ LaunchedDecember2005byDecember2006therewere60millionusersand65millionanswers
‣ Thecodebaseeventuallybecamedifficulttomaintainanditeratenewfeatures
‣ Largedistributeddevelopmentteams(US/UK)
![Page 7: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/7.jpg)
Whyafrontendpresentationplatform?‣ Rasmussays“frameworksarenotwellsuitedforY!”
‣ Buildapplicationstorequirements
‣ Doexactlywhatyouneed:nomore,noless
‣ Understandthatframeworksaddalotofoverhead
‣ Despitechoosingopensourceorbuildingyourown‣ Everyoneusesaframework
‣ Ifyouuseopensource,useonlythepiecesyouneed
![Page 8: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/8.jpg)
Thechoicetouseaframework‣ symfonywasinvestigatedforuseonYahoo!Bookmarksasa
testforthelaterre‐architectureofYahoo!Answers
‣ InvestigationwasperformedinFebruary2006
‣ symfony,CakePHP,Prado,ZendFramework,Struts,Tapestry,Spring,Seaside,RubyonRails,Django
‣ PHPhasmostsupportinternally,sothechoicewasreallybetweensymfony,CakePHP,Prado,php.MVC,andZendFramework
![Page 9: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/9.jpg)
Whatdidweneedfromafrontendplatform?‣ Fitexistingenvironment(BSD/PHP5/Apache)forphased
migration
‣ DevelopmentCycle–Howeasytodevelop,test,anddeploy?
‣ Cleanseparationbetweendata,logic,anddisplay(MVC)
‣ Independentmodellayertofitserviceorientedarchitecture
‣ Extensibleandpluggable‣ Internationalizationandlocalizationsupport‣ Detaileddocumentationandactivecommunityofsupport
‣ Opensourceandabilitytocontributeback
![Page 10: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/10.jpg)
Howweusesymfony‣ FrontendProductionSites‣ Yahoo!Answers‣ Delicious‣ Yahoo!Bookmarks
‣ Yahoo!Widgets
‣ Yahoo!ApplicationPlatform
‣ InternalTools‣ CorporateReporting+MiscManagement
‣ WebServices
![Page 11: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/11.jpg)
Thechoicetousesymfony‣ Configurability/Flexibility‣ Featureswedonotwantareeasilydisabled‣ Useoffactoriesforeasycustomization
‣ Documentation/SupportCommunity
‣ TheDefinitiveGuidetosymfony(freeonline)
‣ Excellenttutorialsandexampleapplications‐Askeet
‣ Activecommunitywithwiki,mailinglists,forums,ircchannel
![Page 12: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/12.jpg)
Thechoicetoadoptsymfony?‣ Philosophy
‣ Full‐stackframeworkforbuildingcomplexwebapplications
‣ Adoptbestideasfromanywhere,usingexistingcodeifavailable(Mojavi,Prado,Rails,Django)
‣ Design
‣ CleanseparationbetweenModel,View,andController
‣ Controllerusingmodulesandactions
‣ ViewsusingtemplatesinstraightPHPwithhelpers
‣ Easytoreuseviewmodulestocomposeapage
‣ Layouts,Components,Partials,Slots
![Page 13: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/13.jpg)
symfonyinfeatures‣ MVCDesign+Project/ApplicationStructure
‣ CascadingConfigurationSystem(YAML‐>PHP)
‣ Environments–Dev,Prod,Test,QA,Staging
‣ Flexibleviewlayer–layouts,partials,components
‣ TaskSystem–Taskssystemforprojectmanagement
‣ Internationalization+LocalizationSupport
‣ Form/Widget/ValidationSystem
‣ Security–CSRF+XSSProtection+UserManagement
‣ Caching–APC,Memcache,File,Database
‣ TestingFramework–LIMEUnit+FunctionalTesting
![Page 14: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/14.jpg)
symfonyMVCframework
![Page 15: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/15.jpg)
Acompletefrontendplatform‣ Acommunitymaintainedapproachtobuildingweb
applicationsfromframeworks
‣ PHPFramework
‣ JavaScriptFramework(YUI)
‣ CSSFramework(YUI)
‣ UIDesignPatterns+BestPractices‣ DevelopmentTools(logger,profiler,debugger,docs)
‣ Unit+FunctionalTestingFrameworks(LIME/YUITest)
‣ DeploymentTools(packagedeploymentsystem)
![Page 16: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/16.jpg)
Users
LoadBalancers
Frontend
PHPAPC,PEAR,PECL,CustomExtensions
FreeBSD4.x/6.x,Linux2.6.x
symfony/YUIApacheCustomModules
Backend
MySQL/Oracle WebServices AdAPI UserAPI
Y!HighLevel
![Page 17: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/17.jpg)
Whatdoesitmeantoscale?‣ Asystemwhoseperformanceimprovesafteraddinghardware,
proportionallytothecapacityadded,issaidtobeascalablesystem.
‣ HighAvailability+Scalability+Performance‣ Biggerdataset,moretraffic,maintainable
‣ Notaboutperformance
‣ PHPisslow,butitisnotyourbottleneck
‣ Languagesdonotscale,architecturesdo.
‣ Planningtogrowandplanningtofail
‣ CapacityPlanning‣ BusinessContinuityPlanning
![Page 18: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/18.jpg)
Scaling–Planning‣ Planninghardwarepurchasesandhostingoptionstohaveas
muchasyouneedwithoutbreakingyourwallet
‣ Partitioninganddistributingdatabasestosupportlargedatasetsandsimultaneoustransactions
‣ Monitoringyourapplicationstofindandclearbottlenecks
‣ ProvidingservicesAPIsandusingservicesfromotherproviderstoincreaseyoursite'sreachandcapabilities
‣ ThinkMinimal,Plantogrow,Plantofail.
![Page 19: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/19.jpg)
Scaling–ThebasicsinPHP‣ PHPisrarelythebottleneck‣ “Mostperformancecomesnotfromthelanguage,butfromapplicationdesign”‐
Rasmus‣ ShareNothingArchitecture‣ Independent,self‐sufficient,nosinglepointofcontention‣ Nolocalstorage=NoPHPSessions‣ Useadatabase(worksfordistributed)‣ Useasmallsignedcookie(ideal)‣ Importantdataindatabase‣ Individualexpirationonsessionobjects‣ Smalldataitems
‣ Useadistributedcache‣ Memcache
‣ Forgetaboutsmallefficiencies‐‐Prematureoptimizationistherootofallevil.
![Page 20: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/20.jpg)
PropelorDoctrineor???‣ NoORMforlargeprojects
‣ PropelorDoctrineforlargeprojects‣ Propelforinternalprojects(bestsupported)‣ ExploringDoctrineforthefuture
‣ ServiceOrientedArchitecture‣ Platformsasservices(reusabletoall)
‣ Noheavylifting,pushdownthestack‣ ThinController/FatModel(wheremodel==services)
‣ Java/C++/Erlang+JSON/XML
![Page 21: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/21.jpg)
ScalingDatabases–Thebasics‣ Master/SlaveReplication
‣ Firststeps‣ Helpswithreads,writesarestillbottleneck
‣ Partitioning‣ Segmentingdata
‣ Sharding(horizontalpartitioning)‣ Segmentingdataontodifferentphysicalmachines
‣ Makeproblemssmaller,easiertogrow
![Page 22: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/22.jpg)
ImprovinglatencywithCaching‣ AlwaysusePHPopcodecache(APC,Xcache,etc)‣ Useforroutingandi18ncache
‣ Memcache(distributedcache)‣ Useforviewcache‣ Distributedinvalidationcanbeapain‣ sfViewCacheManagermakesthiseasy!‣ Beintelligentaboutcache_keys(uri,user,state)
‣ Thereisafinelinetocaching‣ Atwhatpointdoyouspendmoretimemanagingthe
cache,thanreadingfromit?
![Page 23: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/23.jpg)
TweakingPerformance‣ Don’tusefeaturesyoudonotneed‣ settings.yml/factories.yml
‣ Usecore_compile(aggregateclasses)‣ Removedebugstatements(sfOptimizerPlugin)‣ Donotuse.htaccess(movetorealapacheconfig)‣ Setaminimalincludepath‣ Increaserealpath_cache_size+realpath_cache_ttl‣ Useapc.stat=0‣ Use@routeName‣ Donotusecomponentsinloop
![Page 24: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/24.jpg)
WhatdoesYahoo!change?‣ Minorchangestofitourenvironment(bsd/php/apache)
‣ Mostofourchangesareeasilyimplementedviafactories
‣ User,Request,Response,I18ncontroller‣ DroppedtheORMandpusheddownthestack(SOA)
‣ AddedaparallelAPIDispatcher(ysfAPIClientPlugin)‣ Addeddimensionstoconfigurations(ysfDimensionsPlugin)
‣ IntegrateR3translation/templatemanagement(ysfR3Plugin)
‣ R3‐http://developer.yahoo.com/r3/
‣ IntegratesupportforY!UserInterfacelibraries(ysfYUIPlugin)‣ Createdabuildanddeploymentsolution(ysfBuildPlugin)
![Page 25: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/25.jpg)
Localizingtheexperiencewithdimensions‣ CascadingConfigurationbasedonYAML
‣ Framework‐>Project‐>Application‐>Module
‣ Extendingthecascadetobebasedondimensions
‣ Dimensionscanbeanything(andcanbechainedtogether)
‣ DataCenter+Environmentforcustomizingconfigurations
‣ Cultureforlocalizinguserinterface+data‣ Themeforcustomizinglookandfeel
‣ Userinfo(isuseroncorporateintranet?)‣ Caching
![Page 26: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/26.jpg)
Extendingtheview+i18nsystemwithR3+intl‣ Translation+TemplateManagement
‣ Integratedwithsymfonyi18n+viewlayers
‣ <r3:trans>hello</r3:trans>vs__(‘hello’)‣ DictionaryManagement
‣ SQLite/XLIFFBackend‣ r3:include,r3:var,r3:cphp
‣ Addedtasks‣ Importing/ExportingXLIFFdictionary
‣ Generatingtranslationsforprojects‣ UseINTLextension+graphme_*function(replacesmbstring)
![Page 27: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/27.jpg)
YahooUserInterfacelibraries‣ YUILogger/Debugger‣ EventSystem(unobtrusivejavascript)
‣ Helpers‣ AJAX(link_to_*,*_to_remote)
‣ Forms
‣ Widgets(RichTextEditor+Slider+Calendar)
![Page 28: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/28.jpg)
Abuildanddeploymentsystem‣ Aggregateandminifystylesheetsandjavascripts
‣ Rewritetemplates,css,jsforCDN(Akamai,S3,…)
‣ Generatetranslationsforconfigurations+templates
‣ Generateconfigurationcache‣ Aggregatecoreclasses+removedebugstatements
‣ Runlint,unit,functionaltests‣ Packageapplications‣ ReplacewithPEARor.debor.rpm
‣ Deploymentviapackages
‣ Manageenvironment/systemdependencies
![Page 29: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/29.jpg)
SecurityRules‐http://developer.yahoo.com/security/
‣ XSS‣ FilterInput‣ Usefilterextension‣ EscapeOutput‣ symfonyoutputescapingisslow,butsafe
‣ Forperformance,escapemanually
‣ CSRF‣ Alwaysvalidateformstousers
![Page 30: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/30.jpg)
YSlowRules‐http://developer.yahoo.com/performance/1.MakeFewerHTTPRequests
2.UseaContentDeliveryNetwork
3.AddanExpiresoraCache‐ControlHeader
4.GzipComponents
5.PutStylesheetsattheTop
6.PutScriptsattheBottom
7.AvoidCSSExpressions
8.MakeJavaScriptandCSSExternal
9.ReduceDNSLookups
10.MinifyJavaScriptandCSS
11.AvoidRedirects
12.RemoveDuplicateScripts
13.ConfigureETags
14.MakeAjaxCacheable
15.FlushtheBufferEarly
16.UseGETforAJAXRequests
17.Post‐loadComponents
18.PreloadComponents
19.ReducetheNumberofDOMElements
20.SplitComponentsAcrossDomains
21.MinimizetheNumberofiframes
22.No404s
23.ReduceCookieSize
24.UseCookie‐freeDomainsforComponents
![Page 31: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/31.jpg)
ProfilingandDebugging‣ Xdebug/Kcachegrind/Valgrind
zend_extension=/usr/local/.../xdebug.so
xdebug.profiler_enable=1
xdebug.profiler_aggregate=On
xdebug.profiler_output_dir=/tmp
kcachegrindcachegrind.out.aggregate.*
‣ PEARincluedextension=inclued.so
inclued.enabled=1
inclued.dumpdir=/tmp
php~/pecl/inclued/gengraph.php‐iinclude
![Page 32: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/32.jpg)
Doityourselfforcheap‣ Opensourcesoftware=Free
‣ Apache
‣ PHP
‣ MySQL‣ Memcache/Perlbal/MogileFS/Squid
‣ symfony/Propel/Doctrine/Swift
‣ Nagios
‣ AmazonSharedInfrastructure=Cheap
‣ EC2CloudComputing
‣ S3DistributedStorage‣ SimpleDB
![Page 33: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/33.jpg)
Booksworthreading
![Page 34: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/34.jpg)
Questions?
![Page 35: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/35.jpg)
Directandindirecthelpfrom...
•CalHenderson,Flickr•SaraGolemon,Yahoo!•RasmusLerdorf,Yahoo!•SteveSouders,Google
![Page 36: SymfonyCamp - Lessons Learned](https://reader033.fdocuments.in/reader033/viewer/2022042821/55d732bcbb61eb692b8b4658/html5/thumbnails/36.jpg)
ThanksforlisteningandenjoytherestofsymfonyCamp2008!
WanttomovetoCalifornia?Yahooishiring!