Microsoft Developer Division’s Transformation to DevOps

Continuous Value Delivery

Continuous Delivery?

Visual Studio& TFS

Update 2

Visual Studio& TFS

Update 3

Visual Studio& TFS

Update N

Visual Studio Online

One Code Base with Multiple Delivery Streams

Let‘s talk about transforming our Organization…..

DevOps Lifecycle

Looking back....

Before - Quality

Code Test & Stabilize Code Test & Stabilize

Beta

Code

Complete

RTM

Planning

Before - Debt

Adopting Agile

Roles

Organization

Teams

Cadence

Plan

Practices

Guiding principles

Alignment

Autonomy

Organization

Program Management Development QA

Organization - New

Program Management Engineering

Cross discipline

8-12 people

Self managing

Autonomous backlog

Intact for 12-18 months

Physical team rooms

Teams

Version Control

Team Rooms

Vision

18 month vision

Planning Horizons

Sprint

3-week sprints

Planning Chat

3 sprints

Season

6 month season

Internal Sprint Cadence

6 weeks

10 – 12 weeks

3 weeks

3 week Sprints

Sprint Planning

Backlog Grooming

Deployment!

Sprint Planning

Done!

Week 1 Week 2 Week 3

Week 1 Week 2 Week 3

Sprint Mails

Week 1 Week 2 Week 3

Week 1 Week 2 Week 3Week 2 Week 3

Sprint 57Sprint 56 Sprint 58

The sprint plan What we accomplished

Sprint mails

Plan Accomplished

Vision

18 month vision

Planning Horizons

Sprint

3-week sprints

Planning Chat

3 sprints

Season

6 month season

Continuous Planning & Learning

SpringFallSpring Fall

6 month plan

Experience

Reviews

18 month vision

Scenario

Review

Done!Done!

What about the details?

Experience

User Story

Task

Our Backlog

DevOps Lifecycle

Deployment automation

• Online operation – service stays up

• Fully automated

• Multiple versions must coexist peacefully

• Staged, incremental rollout

• Exposure control

• Feature flags and experiments

Feature Flags

Live Site Culture

DevOps Lifecycle

Live

Site

Culture

Our telemetry system collects rich data for both real-time and latent analysis

We needed 24x7 monitoring system that ran reliably all the timeParticularly when the service was down or misbehaving

Activity Logs

TracesKPI

MetricsJob

HistoryCustomer

IntelligenceSynthetic

Perf Counters

Customer IntelligenceBusiness IntelligenceOperational Intelligence

DependentServices

Perf Counters

ActivityLogging

TracesKPI

MetricsJob

HistoryCustomer

IntelligenceSynthetic

Customer IntelligenceBusiness IntelligenceOperational Intelligence

DependentServices

Ping Mesh

ActivityLogging

TracesKPI

MetricsJob

HistoryCustomer

IntelligenceSynthetic

Perf Counters

Customer IntelligenceBusiness IntelligenceOperational Intelligence

We follow a core principle: Fail Fast, Detect Fast and Recover Fast

Alerting is key to fast detection

Alerting – making it right is not easy

We set alerting goalsEvery alert must be actionable and represent a real issue with the system.

Alerts should create a sense of urgency –false alerts dilutes that

But poor alerting makes DevOps unhappyRedundant alerts for same the issue

Needed to set right thresholds and tune often

Stateless alerts contributed to further noise

Health model in action

• 3 errors for memory

and performance

• All 3 related to same

code defect

• APM component mapped to feature team

• Auto-dialer engaged Global DRI

Eliminated alert noise

~928 alerts per week to

~22 and reduced DRI

escalations by ~56%

DevOps Lifecycle

Hypothesis-driven development

HYPOTHESIS

EXPERIMENT (MVP)

LEARNING

MEASURE

We believe developers want a simplified project creation experience so they can quickly leverage VSO and its rich development platform in the cloud.

HYPOTHESIS

Have a simplified project creation experience for web and IDE featuring smart defaults and contextual information.

EXPERIMENT

% of accounts creating a project the same day they created an account

LEARNING

Sample - Experiment

Old project creation experience

Too many actions and distractions

preventing customers from taking

the next natural step of creating a

project.

WEB

IDE

Experience

Focused project creation

experience and also

integrated in IDE flow

Learnings

First project creation

jumps from 15% to 50%First account creation from IDE

jumped from 3% to 20%

Another experiment running in parallel

negated results

Why?

• React to market/competitive changes quickly

• Maximize investments in what customers want

• Empower the Organization to be a great workplace

• Tight Customer Feedback loop

Christian Binder

ALM Architect Microsoft

cbinder@microsoft.com

Twitter - @xcbinder

http://blogs.msdn.com/cbinder