July 2016 | Behind the Scenes

Agenda.

● Welcome & introduction● From Code to Node● Short Break● A look to the future of deployment● Tour of our office● Drinks● Fin

July 2016 | Behind the Scenes Flynn Bundy <@bundyfx> & Matthew Hodgkins <@matthodge>

“I was trying to sleep last night but all I could think about was how to make our pipelines better.”

Matt

PLAN.

CODE.[sensu]Get(){...}

[void]Set(){...}

[boolean]Test(){...}

static [void] XMLConversion (){...}

static [void] InstallService (){...}

static [void] ConfigureAgentTransport([Transport]$Transport){...}

CODE.node $AllNodes.Where{$_.Role -eq "our_app"}.role{...}

node $AllNodes.Where{$_.Role -eq "our_app_1"}.role{...}

node $AllNodes.Where{$_.Role -eq "our_app_2"}.role{...}

node $AllNodes.Where{$_.Role -eq "our_app_3"}.role{...}

node $AllNodes.Where{$_.Role -eq "our_app_4"}.role{...}

BUILD.

BUILD.

TEST. Context Download Agent Method [+] Should download the Sensu client 7.58s [+] Should be the full file 101ms Context Install Agent Method [+] Should Install the Sensu client 38.49s Context ConfigureAgentClient Method [+] Should be correctly formatted into JSON 1.77s [+] Should contain the Correct Information in the JSON | Subscriptions 20ms [+] Should contain the Correct Information in the JSON | Name 19ms [+] Should contain the Correct Information in the JSON | Private IPv4 Address 15ms Context ConfigureAgentTransport Method [+] Should be formatted correctly into JSON 154ms [+] Should contain the Correct Information in the JSON | Name 16ms [+] Should contain the Correct Information in the JSON | Reconnect on Error 13ms Context InstallService Method [+] Should deploy and install the Sensu client service 256ms Context Conversion XML Method [+] Should correctly create an valid XML configuration file 119ms Context RemoveAgent Method [+] It should remove the agent 239.16sDescribing DSC Based methods Context Testing DSC based methods [+] Test() Method should return a Boolean Type 283ms [+] Get() Method should return a Sensu Type 31ms

TEST.

Move fast and break things. Unless you are breaking stuff, you are not moving fast enough.

Mark Zuckerberg

DEPLOY.

DEPLOY.

WHAT IS DSC?.

WHAT IS DSC?.

WHAT IS DSC?.

OPERATE.

PILLARS OF MONITORING.

WHY MONITORING IS HARD.

▪Automation is an after thought▪Forgetting to add new services▪Checks are hard to write▪Don’t want to do annoying things

ENTER SENSU.

IIS

/purchases

Sensu Agent

Sensu Server

RabbitMQ

STEP 1 - DEPLOY AGENT WITH DSC.Sensu Install{ Ensure = "Present" DownloadURL = 'https://sensu.global.net/msi/sensu-0.24.0-1.msi' RabbitMQServer = 'coolblue-rabbitmq.internal.fake' Transport = 'rabbitmq' vHost = 'sensu' User = 'sensu' Port = '5672' Password = 'some_password' #Encrypted of course Subscription = 'windows_base'}

IIS

/purchases

Sensu Agent

Sensu Server

RabbitMQ Deploy Agent

STEP 2 - CREATE SOME CHECKS.{ "checks": { "disk_usage": { "command": "check-windows-disk.rb.bat -w 85 -c 95", "interval": 600, "subscribers": [ "windows" ] }, "windows_update": { "command": "powershell.exe -file C:/coolblue_checks/win_last_updated.ps1 -w 30 -c 60", "interval": 86400, "subscribers": [ "windows" ] } }}

SENSU CHECK DEFINITIONS.

IIS

/purchases

Sensu Agent

Sensu Server

RabbitMQ

Add Check Definitions

STEP 3 - UPDATE THE CONFIG.Sensu Install{ Ensure = "Present" DownloadURL = 'https://sensu.global.net/msi/sensu-0.24.0-1.msi' RabbitMQServer = 'coolblue-rabbitmq.internal.fake' Transport = 'rabbitmq' vHost = 'sensu' User = 'sensu' Port = '5672' Password = 'some_password' #Encrypted of course Subscription = 'windows_base','my_iis_app'}

STEP 4 - ADD APP SPECIFIC CHECKS.{ "checks": { "iis_purchases": { "command": "check-http.rb http://localhost/purchases", "interval": 60, "playbook": "https://github.com/devTeam/readme.md", "subscribers": [ "my_iis_app" ] } }}

WHY MONITORING IS HARD.

▪Automation is an after thought▪Forgetting to add new services▪Checks are hard to write▪Don’t want to do annoying things

THE SENSU SOCKET INPUT.

IIS

/purchases

Sensu Agent

Sensu Server

RabbitMQ

3030TCP/UDP

My App

JUST SEND IN JSON.

{ "name": "ravendb_connectivity", "output": "Could not connect to RavenDB!", "status": 2, "team": "TeamName", "application": "My App", "playbook": "https://github.com/devTeam/readme.md"}

USING A DEAD-MAN SWITCH.

{ "name": "daily_sql_backup", "output": "Backup of SQL databases performed", "status": 0, "team": "OpsTeamName", "application": "sql_backups", "playbook": "https://github.com/opsTeam/readme.md", "ttl": 86400}

MONITORING CAN BE EASY.

▪Automation is an after thought▪Forgetting to add new services▪Checks are hard to write▪Don’t want to do annoying things

FIN.

Matthew Hodgkins

@matthodge

hodgkins.io

Flynn Bundy

@bundyfx

flynnbundy.com

BREAK TIME.

July 2016 | Behind the Scenes Flynn Bundy <@bundyfx> & Chris Thorp <@thorpgeek>

Challenges.

● OS Base Memory Usage

● Large HDD Usage

● Build Times for Servers

● Further increase the development speed of Coolblue

SAD (Server And a Desktop).

Install web server feature

Server Core.

WE

Nano Server.

Nano Server Features.

Results.

Server Core Nano Server

Build Time: 6 min ~40 Seconds

Start Time: 25 Seconds 2 Seconds

Restart Time: 15 Seconds 1 Seconds

Hard Drive Space used:

8 GB 600 MB

Basic Memory Usage 250 MB 160 MB

Nano Server Thoughts.

● No 32 Bit Support

● MSI is gone

● Not all server core features available

● No Full .Net Framework Support

Challenges.

● OS base Memory Usage

● Large HDD Usage

● Build and Deployment Times for Servers

● Further increase the development speed of Coolblue

Summing up challenges.

● OS Base Memory Usage

● Large HDD Usage

● Build Deployment Times for Servers

● Further increase the development speed of Coolblue

FIN.

Chris Thorp

@thorpgeek

Flynn Bundy

@bundyfx

flynnbundy.com

Events: coolblue.nl/behindthescenes DevBlog: devblog.coolblue.nl