Software Engineering in the Open Source Age

The Secrets of the Strong DeveloperLife in the commercial trenchesDave CowdenDirector of DevelopmentCoLinx, LLCAbout Dave..

Where I work:Industrial E-commerce and Logistics services providerA non-profit co-op owned by large industrial companiesFounded in 2001 with 40 employees, 850 todayAnnual revenues 80MMostly Java, some pythonSmall dev team 11 developers

Statistics: Ptplace.com2.5M transactions/yr and 20M hits per monthAbout 50,000 users750k lines of code 86 storefronts serving users in nearly time zones17 languages

Statistics: Logistics1M sqft of warehouse space, plus 500k sqft equivalent storage in super-sweet automated storage systems24x7x364 operation1.2 M pounds of freight shipped dailyShipments to most of the US next day via truckReleases monthly ( releases must occur during a 20 minute shift change )

What is Dev Strength?The size of problem you can solve

Your value to the organization

3 secrets to being a strong developerBe a superhero

Study Logistics

Write Less Code

Study LogisticsAmateurs study tactics, professional soldiers study logistics.-- General Omar Bradley, WWII

Logistics in the software biz?Version control/Release ManagementContinuous integrationTestingInfrastructureDependency managementProject Management

Strong developers are experts at the activities that deliver high quality software on a sustainable basis.

Could you set up a VCS system for your team?Keep track of all changes, who made them, when, and why.Coordinate concurrent changes by different developersDevelop multiple versions of a product at the same time, while also allowing combining those changes at a later timeRelease different versions of the same product at different times

Can you build and deploy your product with one button press?Automate all of the work that is not writing code!Do you automatically and actively manage dependencies?

Track dependencies, and deps of deps, all the way down the chainSometimes there can be conflictsDo you automate your tests?Unit tests (testng /junit ) ( part of your CI builds right? )Integration tests (Selenium)Load Tests ( apache ab )

Do you know how to deploy to the cloud?Deploy a new server with tomcat, java, etc on amazon EC2 in 5 minutesLeverage virtualization

Strong developers can:Are you agile (with a little a)?Strong shops use a combination of classical and agile development to create an efficient delivery processCoLinx uses Scrum in particular:

With strong logistics, the bazaar kills the cathedralVery large corporations and teams historically had the upper hand on tooling,Small teams had the flexibility, and made do with lesssTodays OSS tips the scale decidedly in favor of small, agile teams with strong logistics

The bazaar, CoLinx $80M/yr Monthly releases10 developersThe cathedral, CoLinx Owners $5B/yr Annual releasesHundreds of developersWith strong Logistics, developers can do amazing things!Create a source code repository, with continuous integration builds, release management tools, etc.Create a bug tracking site, a public blog, a web presence, and email accounts for communicationCreate an internet scale application,Deploy the application on geo-distributed servers around the worldCreate a mobile application that is immediately available in the market to tens of millions of usersDerive revenue without having any significant invested capital.Bring an idea to market many times faster than very large companies could.With milk money, a developer can single handedly:Write Less CodeThere are no points for writing code that is:Not necessaryAlready availableStrong developers are 40% integrator, 20% author, 20% researcher, 20% sysadmin/dba/network

Example: Ptplace.com1M CoLinx LOCs represents about 15% of the product144 open source libraries are the other 85%CoLinx delivers new releases to production monthly, about 5x faster than the IT team of our owners( You need strong logistics to pull it off! )

Use Open Source Software (OSS)Higher QualityLower cost ( both upfront and total ) ( Free as in beer )Freedom from:annoying license management and budgetingincompetent support helpdesks who waste your timeVendor lock-inEnhances agility. Corporate annual budgets do not make annual releases easyEncourages an optimistic, can-do attitude

Use a best of breed approach when architectingNothing is good at everything. Do it all products are the dark side.Do not be afraid to integrate multiple products. Applies for closed and open source productsThe strategy will lead to less code in the long run

Passionately Evangelize code re-use in every way possibleAssume that someone else has written what you are writingLearn and teach places to find toolkits and libraries. It often viewed as non-work but nothing could be further from the truth!Prefer already written, tested code to yours. Replace yourself when you canStrive for a mix of young and experienced devs working togetherFight for ability to download and still third party libraries freely. Learn what licenses work for you/your company.Install automated copy-paste code detectorsCultivate a culture of get-r-done, not build-my-fiefdom

Superheros are all about contextSuperheros apply a superpower in a context of mortals. Superhero developers realize this as well:Knowing how to make computers do what you want is your super-power.A non-computer business domain is your context of mortals.Programmer @Google => superman on Krypton.Programmer with business mojo: => superhero!

Part 1: Making computers do what you want.Strong developers extend their work to include non-software too. Devops is a commonly used term

These folks are Generalists you can build an entire company around themNetworksDatabasesUnix/Windows System AdminFirewalls/RoutersQA/TestingSecurityPart 2: Become an expert in your business.In a complex business: there are usually lots of developersand lots of business people And sometimes even lots of business analystsThere is lots of competition for superhero status. Those who generalize and adapt will become superheros.In a small business:There is no dev staff or business analysts at allSuperheros are rare ( why is that? More later )If you can architect tech solutions and get it 80% right 80% of the time, you are a superhero in either environment!

Growing superheros: 3R+CFuture jobs are either:Telling computers what to doBeing told by a computer what to doProgramming should be taught in schools at a level of importance as the 3-RsEducation still views programming as a vocation.Programming teaches critical thinkingProgramming knowledge makes you a superhero when combined with another discipline.Learning to code is not a non-refundable ticket to krypton or geekworld!

Questions?It is the most exciting time ever to be a software developerThanks for the opportunity!Questions?