A CTO's Guide to Scaling Organizations

A CTO’s Guide to Scaling Organizations

Randy Shoup @randyshoup

linkedin.com/in/randyshoup

OrganizationalScalability

• Scalability Principles

• Team Size and Scope

• Team Autonomy

• Global and Distributed Teams

• Concluding Thoughts

Universal Scalability Law

System throughput is limited by• Contention

o Queueing on a shared resource, O(N)

• Coherenceo Coordination and communication between all nodes, O(N2)

http://www.perfdynamics.com/Manifesto/USLscalability.html



Universal Scalability Law

• Implicationso Find ways to remove contention points o Find ways to reduce or eliminate coordination overheado Increased N more contention, more coherence

• Multicore processor designo Fast to stay within a coreo Expensive to synchronize across cores

• Distributed system designo Sharding o Eventual Consistency

“What if we designed our organizations like we design our systems?”




• Team Autonomy



Small “Service” Teams

• Amazon “2 Pizza” Teamso No team should be larger than can be fed by 2 large pizzaso Typically 3-5 peopleo Mix of junior and senior people

• Team == Component | Serviceo Clear, well-defined area of responsibilityo Single service or set of related serviceso Minimal, well-defined “interface”

• Applying the Universal Scalability Lawo Reduce N within teamso Well-defined responsibilities reduce synchronization / coordination points

between teams

End-to-EndOwnership

• Teams own their roadmap

• No separate maintenance or sustaining engineering team

• Engineers own service from design to deployment to retirement

TeamAnti-Patterns

• Skill-based teamso Based around tiers or technologies (e.g., front-end team, application

team, DBA team, Ops team)o (-) Every project crosses many team boundarieso (-) No end-to-end ownership of the systemo (-) No end-to-end ownership of the customer experience

• Project-based teamso Form ad-hoc team for a particular project, then disbando (-) No long-term ownership of code, product, serviceo (-) Encourages short-term approach instead of sustainable technical

debt

TeamAnti-Patterns

• Large teamso (-) Teams larger than 6-8 should be splito (-) Communication and coordination overhead makes it increasingly

difficult to sustain velocity




• Team Autonomy



Autonomy and Accountability

• Give teams autonomy• Freedom to choose technology, methodology, working environment• Responsibility for the results of those choices

• Make teams self-sufficient• Team has inside it all skill sets needed to do the job• Depend on other teams for supporting services

• Hold team accountable for *results*• Give a team a goal, not a solution• Let team own the best way to achieve the goal

Autonomy and Accountability

• Clear “contract” provided to other teams• Functionality: agreed-upon scope of responsibility• Service levels and performance

DecisionmakingAnti-Patterns

• Single authorityo Decisions made or approved by single person (CTO?)o (-) Single bottleneck / contention pointo (-) Single point of failureo (-) Unsustainable for decisionmakero (-) Discourages autonomy, ownership, growth

• Unanimity / Consensuso Decisions made or approved by “everyone”o (-) Constant need for coordination / coherenceo (-) Increasingly ineffective / counterproductive as organization growso (-) Discourages autonomy, ownership, growth




• Team Autonomy



Effective Global Teams

• Local Ownershipo Well-defined area of responsibilityo Clean interface with the rest of the organization

• Individual teams are co-locatedo High-bandwidth communication within a teamo Minimal coordination across teams

Global TeamAnti-Patterns

• Anti-Pattern: Split Teams Over Geographieso (-) Constant need for coordination over time zoneso (-) Local conversations become disruptive rather than helpfulo (-) No local pride of ownership

• Anti-Pattern: Remote Team as Job Shopo (-) Constant need for management and task assignmento (-) Resentment between first-tier and second-tier siteso (-) No local pride of ownershipo Ex. eBay remote offices vs. Google remote offices

DistributedTeams

• Fully distributed *OR* fully co-locatedo Distributed teams rely on virtual proximity (chat, hangouts, IRC)o Co-located teams rely on physical proximity (co-working)

• Anti-Pattern: “Mostly” co-locatedo (-) Co-located majority ends up determining communication methodso (-) Remote individuals left out, less able to contribute, less productive




• Team Autonomy



One Idea,Many Facets

Small Teams

MicroservicesDevOps

Agile Practices

A CTO's Guide to Scaling Organizations

Software

Transcript of A CTO's Guide to Scaling Organizations