>>>>>>>>>>>>>>>>>>>>>

CREATING THE FUTURE OF BIG DATA THROUGH "THE APACHE WAY” WHY THIS MATTERS TO THE COMMUNITY

Dr. Justin R. Erenkrantz, Bloomberg LP justin@erenkrantz.com / @jerenkrantz

WHY SHOULD I PAY ATTENTION?

»  MentortoApacheGeodeandHAWQ»  Commi5ertoApacheHTTPServer,APR,Subversion,Serf»  FormerPresidentandDirectorofTheApacheSoBware

FoundaDon»  Ph.D.fromUniversityofCalifornia,Irvine»  DissertaDon:"ComputaDonalREST:ANewModelfor

Decentralized,Internet-ScaleApplicaDons”»  HeadofComputeArchitectureatBloombergLP»  ~50billionDcksDAILYflowthroughoursystems

2

TECH @ BLOOMBERG: OPEN SOURCE

3

»  ThecoreofourBloombergProfessionalpla\ormhasevolvedawayfromproprietarycode

»  FoundaDonsofournext-generaDoninfrastructure-OpenStack,Ceph,Hadoop,Spark,Solr,Chromium,Chef-areallopen-source

»  Nolongercanvendorstellusthattheywon’tfixacriDcalbug»  Placesalotofpressureonourpartnerstocollaborateopenly

»  Givingbacktothecommunity-h"ps://github.com/bloomberg/

»  Allowsustoinnovateatthehigherlevels–helpingourcustomersmakesenseofthefirehoseofinformaDonthatisavailabletothem

TECH @ BLOOMBERG: OPEN CAN BE HARDWARE TOO!

4

HISTORY LESSON…

5

» StartedasApacheGroupwith8membersinFeb1995resumingworkonNCSAh5pd» UIUCplacedtheservercodeinpublicdomain» MostoftheUIUCteamleBtojoinNetscape» WebmastersleBinthelurchandjoinedtogether

» TheApacheSoBwareFoundaDonincorporatedin1999» Today,thereareover350communiDesaffiliatedwithApacheperformingover16,000codecommits/month

Why?

PHILOSOPHY OF THE APACHE SOFTWARE FOUNDATION

6

»  Letthecontributorsdowhattheydobest:contribute.FoundaDonexiststodotherest.

» DoesnotpayforcontribuDons» Manyaresponsoredbyathird-party» StaffASFhasarefocusedoninfrastructure/PR/etc

» Doesnotpick“winners”or“losers”»  “CompeDDon”betweenASFprojectsperfectlyacceptableaslongastherearehealthycommuniDes…thinkGeodeandIgnite(!)

ANTI-PHILOSOPHY

7

»  “TheApacheWay”isnot…» DumpingyourcodeonGitHub» Single-sponsorcontribuDons» RunningaBenevolentDictatorship(BDFL)

» TheApacheSoBwareFoundaDonmaynotbebestforallprojects...that’sperfectlyOK.

»  IfyouwishtobepartofApache,youneedtoadheretosocialconstructsandnorms

» Technicaldecisionsareuptothecommunitytodecide

ROLE OF APACHE INCUBATOR

8

» Eachproject(TLP)isrunrelaDvelyautonomously» ProjectkarmadoesnotautomaDcallycarryover

»  IfIcancommittoGeode,itdoesn’tmeanIcancommittoIgnite!(But,Icouldlikelyearniteasily!)

»  Incubatorwasformedin2003aswewerestrugglingtoscalethefoundaDonandrepeatthemodel.Itworked.

»  Ifapodlingdoesnothaveahealthycommunity,it’llnevergraduate.That’sOK.IfthepodlingdoesbecomeaTLP,butlaterlosesitscommunity,it’llendupintheArc.That’sOK,too.

TRANSPARENCY & MERITOCRACY

9

»  Roy’s Mantra: "If it's not on the list, it didn't happen.” »  Apache in the age of GitHub, JIRA, ReviewBoard, etc. »  Is the mailing list doomed?

»  Generation gap may mean email isn’t preferred »  Tools are always secondary to process »  Transparency is the aim: allows others to have a voice »  The tools and process are never about prohibiting face-to-

face contact - but, ensuring that there is equal access for participation and permitting asynchronous decision making

»  Making decisions in a synchronous echo chamber (Slack, IRC, etc.) is not conducive to transparency

MAKING DECISIONS

10

»  Voting is the way contributors are (and feel) empowered »  “Binding” votes from recognized contributors (PMC)

»  Vote on code, ideas, and, most importantly, releases »  Minimum acceptable quorum: 3 voters »  Minimum acceptable time frame: 72 hours »  The power of the dreaded “-1” (veto)

»  Code can be vetoed, but not releases »  Veto should be cast as a last resort; used to foster

discussion

GROWING COMMUNITY

11

» ContribuDonscancomefromanywhere » Reliesuponcorecontributorsbeingopentoideas

» Yet,thereoBenisasetofagreeduponprinciples» GoingtoGeodecommunityandsaythatyoushouldremoveallconsistencycodeisanon-starter

» Thisisthepowerofthemythical"TheApacheWay”» Meritocracy:accessbasedondemonstratedskills

» MichaelYoung'sTheRiseoftheMeritocracy(1958)–negaDveconnotaDonsacrossanenDresociety

GROWING COMMUNITY

12

» AsadownstreamconsumerofApacheprojects,willtherebesomeonewhoismaintainingthecodebase?CanIhelpvolunteertomaintainit?

» Acodebasebyitselfisinert» Codeisneverperfect,butahealthyandinclusivecommunitywillbeimprovingthecodeconstantlybaseduponfeedbackandothers

»  “CommunityoverCode”

ROLES IN INCUBATOR

13

» Thinkofapodlingasbeingprovidedasetoftrainingwheelsastheylearntherulesoftheroad.

» RequiredquarterlyreporDngisoneofthefewmechanismsthattheBoardimposestoallprojectstoensurethatthecommunityishealthy.»  Ifnoonesubmitsthereport,noonemaybehome!

» MentorsarearoundtoanswerquesDons,shareknowledge,andbestpracDces.Mentorsarenottheretocontributecode–though,oBenwecould;but,thatroleisdisDnct.

NORMS OF THE COMMUNITY

14

» Overtheyears,mostdisputesIhaveseencomedowntonormsthatwerenotagreeduponordocumented

» ForminganexplicitconsensusonreleaseversioningandcompaDbilityrulesup-frontissoincrediblyhelpful.» Projectsalwayshaveatensionbetween“newfeatures”andcompaDbility.Decidewherethecommunitywantstobeearlyon.

» TheGeodewikisecDonisgreat.Keepitup!

EXPECTATIONS FOR CONTRIBUTORS

15

» ExplicitlycommunicaDngtocontributorswhoarenotyetinPMCwhattheexpectaDonsareforreceivingcommitaccess(vote)toaprojectisextremelyhelpful.

»  It’spainfultoseecontributorswhodonotfeelempoweredbythecommunity.It’sahugeredflag.

» Eachprojectcanandshouldsetitsownbar.» MygutfeelingnowistoerronthesideofinclusivenessandgivecommitrightsearlierthanIdid.It’sallunderversioncontrolanyway.Worstcase,revokethatperson’sbit.

GRADUATION

16

» WhenwillApacheGeodegraduatefromIncubator?»  “Whenit'sready”istheonlyhonestanswer.» Geodecommunityneedstodemonstratethatitcangovernitselfandbeinclusiveandtransparent

»  Itdoesn’thavetobeperfect–nocommunityis.» ThisiswheretheBoardcanbeextremelyhelpful.

»  IamextremelyhappytoseetheprogressthatGeodehasmadesofarandwishittheverybestonitspath.

Join the Apache Geode Community!

•  Check out: http://geode.incubator.apache.org

•  Subscribe: user-subscribe@geode.incubator.apache.org

•  Download: http://geode.incubator.apache.org/releases/

18

THANKS!

Dr. Justin R. Erenkrantz, Bloomberg LP justin@erenkrantz.com / @jerenkrantz