Reaching for the cloud3 ways that the cloud will change the way you develop software

Atlassian NirajBhawnani

What is the cloud?

• Remote servers run applications and store your data

• Accessible over the internet

• Users need not care where the application is running or

where their files are hosted

• Examples of cloud companies:

What is the cloud?

Atlassian & the cloud

• We build software tools for issue tracking, collaboration,

continuous integration, and source control

Who is Atlassian?

• Founded in 2002 by 2

Sydney-siders

• ~600 employees in:

Who is Atlassian?

Who is Atlassian?

Atlassian’s Vietnamese Team, November 2013

• Atlassian started selling software for customers to install

on their own servers

• Customers would keep asking if we had a hosted

solution

• In 2011, Atlassian OnDemand was born

• 41,150 VMs

• 5,640 CPU cores

• 58 TB RAM

• 368 TB Disk

Why is everything moving to the cloud?

• Great for business. One time payments become regular

payments

Why move to the cloud?

Confluence BTF vs. OnDemand

100 user license = $4000

Assuming an upgrade every 2 years:

BTF = $4000

100 user license = $300/month

OnDemand = $300 × 2 × 12= $7200

• Cheaper for customers – no staff or hardware required

Why move to the cloud?

Confluence BTF vs. OnDemand

100 user license = $4000

Staff/hardware = $70000/yr

Cost every 2 years:

BTF = $70000 × 2 + $4000= $144000

100 user license = $300/month

No staff/hardware = $0/yr

OnDemand = $300 × 2 × 12= $7200

• Always accessible via the internet

• Full control of the application server environment

Why move to the cloud?

Delivering software to the cloud

Traditional delivery pipeline

Production

Merge code QA Verification Staging deployment

& verification

Build Release

Manual Task

Automatic Task

Continuous Delivery Pipeline

Production

Merge code Continuous Integration

(Build & Test)

Manual Task

Automatic Task

How will the cloud change the way you

build software?

1. Incremental delivery

• Waterfall model does not work for cloud software

• Deliver features in small increments instead of “big bang

releases”

• Good for customers: they get features faster

• Good for you: less investment before validation

Incremental Delivery

• Start with a Minimum Viable Product (MVP)

• Simplest version of the feature that still adds value for users

• Example: Facebook Photos

• MVP allows users to just upload and view photos

Incremental Delivery

• Then deliver incrementally in order of importance

• V1/MVP: Upload and view standard definition photos

• V2: Face tagging

• V3: Video support

• V4: High definition photos & full screen mode

• V5: Automatic face detection

Incremental Delivery

What to do when the MVP is too big?

• Atlassian calls them “dark features”

• Wrap the feature in some condition and provide a way to

switch the condition on and off

• Also provide a way for tests to turn them on/off

• Eliminates the need for long lived code branches

Feature Switches

• Also allows us to deploy features to a subset of users

• Can be used to test more risky features without upsetting

too many users

Feature Switches

All users

Experiment

Group

Control

Group

2. Automation

• Since there might not be a manual verification step, we

must write a lot of automated tests to verify the system:

• Unit tests

• Integration tests

• Smoke tests

Automated Testing

• Automated tests need to run for every commit on every

branch

• Branches can be merged automatically or manually

• Automatic deployment when the main branch passes

• Tools: Atlassian Bamboo, Hudson, Cruise Control

Automated Testing

Automated Testing

• Check in code

• Build runs:

• Unit tests

• Integration tests

• Merge code

• Used to test individual units of source code in isolation

• Can run very very fast

• Might miss some bugs

• E.g. JUnit, Mockito, QUnit, etc.

Unit Tests

• Tests the whole product by functionality

• Runs slowly

• Selenium tests to drive the UI

• Hamcrest/JUnit to do assertions

Integration Tests

• Similar to integration tests

• Runs on one or more staging servers with real data that

has been upgraded

• Verifies that the data is intact and that basic functionality

still works

• Also uses Selenium

Smoke Tests

• Test code sends commands to Selenium Server

• Selenium Server starts a browser

• Selenium Server sends commands to the browser

• Type in this form

• Click the submit button

• Is this element visible?

Selenium

Test codeSelenium

Server

Web

Browser

• Since you need to write a lot of tests, you need to reuse

as much code as possible

• Write “page objects” that represent each page and “page

components” that represent shared components (e.g.

header and navigation)

Selenium

• Example code (Selenium with page objects):

Selenium

@Testpublic void testQuickSearchWithResults(){

viewPage = product.login(User.TEST, ViewPage.class, testData.page);QuickSearch quickSearch = viewPage.getHeader().getQuickSearch();SearchResultPage resultsPage = quickSearch.doSiteSearch(searchWithResults);Poller.waitUntilTrue(resultsPage.hasMatchingResults());

}

• Deployments happen very frequently:

• : 25/day

• : 1/day

• : 1/week

• Updates need to be fast and automatic

• Usually a collection of custom software and scripts

Automated Deployment

3. Analytics

• Collect usage data from all instances and users in a

centralised place

• This can be a very large amount of data, you need to

use tools that can handle it:

• Graphite

• Amazon Redshift

Analytics

• Query the data to answer important questions:

• What features are people using?

• What features are people not using?

• How are the features being used?

• What kind of users are using what kind of features?

• Use the data to make important business decisions

Analytics

• You should come up with the list of questions before you

ship the feature. Why?

Analytics

• You should come up with the list of questions before you

ship the feature. Why?

• To answer some kinds of questions you may need to write

additional code.

Analytics

• Leverages the existing events system

• Support for client-side events

• Reporting via Graphite or Apache Hive

• Google Analytics

• Greenzone

Atlassian Analytics

Atlassian Analytics

Product

instance

Product

instance

Product

instance

Product

instance

Product

instance

…

Product

instance

Analytics

Server

Analytics

Server

UDP

Graphite

HiveAmazon

Redshift

Application Server

Atlassian Analytics

Web Browser

Google Analytics

Analytics Plugin

Application

code

Analytics

Server

Database

Server

Greenzone

Database

Hive, etc.

Java Event

Batched

AJAX

Daily SQL

Queries

UDP

HAMS

Sales information

• Hardware costs (particularly RAM) is the largest expense

• RAM usage can be reduced by:

• Multitenancy

• Microservices architecture

• Optimising code

Other Considerations

• Deployment should not cause downtime

• Upgrade a standby server

• Switch proxy to new server

• Store sessions in a central location e.g. database

• Users experience no downtime

Other Considerations

• Things don’t always go as planned

• Always try to roll forward, never backward

• Have a procedure to deploy hotfixes in case of emergency

• Only revert to older versions as a last resort

Other Considerations

What should I do next?

• Consider moving your application to the cloud

• Deliver features incrementally

• Write a lot of automated tests

• Make deployment easy and automatic

• Use collected data to make business decisions

What should I do next?

• Learn more about Atlassian software

• Hear talks like this one about industry best practices

• Network with other professionals

• Free beer!

• https://aug.atlassian.com/

Thank You!

Questions?