NATHAN BURRELL • SENIOR DEVELOPER • ATLASSIAN

Connect + AWS + Docker = Bitbucket Pipelines

A short time ago in an office far far away…

Connect Microservices on AWS Docker

Agenda

Connect Descriptor

iFrames and Webhooks

Security

Connect

Connect Descriptor Describes in JSON how the integration will integrate with the Atlassian cloud product.

Permissions

iFrames

Webhooks

Scopes control what your integration can do with JWT tokens generated using the shared secret provided on installation.

{

…

“scopes”: [

“account”,

“repository:admin”,

“pipeline”,

“pipeline:variable”

],

…

}

Permissions

iFrames

Webhooks

Modules are used to specify where you will inject UI fragments into the parent application.

{

…

“modules”: {

“webPanel”: [ {

“key”: “pipelines-overview”,

“name”: { “value”: “Pipelines Overview” },

“url”: “pipelinesOverview.html?accountUuid={repo_owner_uuid} ”,

“location”:  “org.bitbucket.repository.overview.informationPanel”,

“condition”: [ { …} ]

}]

},

…

}

Permissions

iFrames

Webhooks

Webhooks are used to specify URLs to be called back to when certain events occur.

{

…

“lifecycle”: {

“installed”: “/api/events/connect/installed”,

“uninstalled”, “api/events/connect/uninstalled”

},

…

“webhooks” : [ {

“event”: “repo:push”,

“url”: “/api/events/repository/push

}]

}

iFrames and Webhooks iFrames are your UI injection points into an Atlassian cloud product. Webhooks allow you to interact with a product when certain events occur.

AP.require(‘request')!…!

AP.js!

AP.js!

Window.postMessage()!

Bitbucket Pipelines

Use the arrows, lines and symbols included here. Do not change the styles. Delete this instructional text.

repo:push Services

Security

Using a JWT token you can implement authentication and authorization.

Creating

Authenticating

OAuth

Each and every request be it for an iFrame or webhook contains a JWT token either as a query parameter or in a header parameter.

Request

Bitbucket Pipelines

JWT

Creating

Authenticating

OAuth

Each and every request be it for an iFrame or webhook contains a JWT token either as a query parameter or in a header parameter.

Request

Pipelines

JWT

Bitbucket

Authenticating

Creating

OAuth

Supporting text should be kept short and to the point; Limit text to a maximum of 2 lines.

access_token Bitbucket Pipelines

JWT

oauth

Transparent Integrations

Connect

Anatomy of Microservice

How we use AWS

Microservices on AWS

Anatomy of a Microservice The architecture of a typical pipelines microservice and how it interacts with connect.

Use the arrows, lines and symbols included here. Do not change the styles. Delete this instructional text.

Microservice

DynamoDB ElastiCache

EC2 ELB

Dropwizard

SQS SNS

Hystrix RxJava Java

Microservice

HTTP

Endpoints

Bitbucket

Browser SNS

Event

Handlers

Webhooks

iFrames

Events

Events Services

Models

Daos

Microservices

SQS

Requests Clients

DynamoDB Requests

How we use AWS How we configure our service stacks running on AWS and deploy updates to our services.

Cloudformation

Code deploy

Monitoring

Use cloudformation to specify how to configure your entire service stack.

AWSTemplateFormatVersion: “2010-09-09”

Resources:

serviceTable:

Type: “AWS::DynamoDB:Table”

Properties: …

serviceELB:

Type: “AWS:ElasticLoadBalancing::LoadBalancer”

Properties: …

serviceInstance:

Type: AWS::EC2::Instance

Properties: …

aws cloudformation create-stack …

Cloudformation

Code deploy

Monitoring

Use compose to manage your entire local environment.

Version: 0.0

os: linux

files:

- source: app/service.jar

destination: /opt/app/service.jar

hooks:

ApplicationStop:

- location: scripts/shutdown.sh

ApplicationStart:

- location: scripts/startup.sh

application.tar.gz

- app

- service.jar

- AppSpec.yml

- scripts

- shutdown.sh

- startup.sh

AppSpec.yml

S3

Cloudformation

Code deploy

Monitoring

Use datadog to provide you with realtime monitoring.

docker

datadog-agent

EC2

crawler

Battle Tested Easy to Use Cloud scale effortlessly

Microservices on AWS

Runtime Environment

Container Management

Security Tips

Docker

Docker Using docker for runtime environments locally and in the cloud.

Local development

Using compose

Building containers

Use containers to play with multiple versions of tools.

Use container to spin up dependencies.

Dev machine

docker

Working directory

postgres container

-v $(pwd):/opt/code –w /opt/code openjdk:8-jdk javac … -v $(pwd):/opt/code –w /opt/code openjdk:9-jdk javac …

Java container

-d --name postgres postgres:9.6 … docker run ... --link postgres openjdk:8-jdk …

Local development

Using compose

Building containers

Use compose to manage your entire local environment.

version: ‘2’

services:

java:

image: openjdk:8-jdk

links:

- postgres

volumes:

- .:/opt/code

postgres:

image: postgres:9.6

docker-compose up -d down

Local development

Using compose

Building containers

Using compose and dockerfiles to produce your own images.

version: ‘2’

services:

service:

image: service:dev

build:

context: .

dockerfile: DOCKERFILE

docker tag service:dev service:production &&

docker push service:production

FROM openjdk:8-jdk

ADD [ “*.jar”, “/opt/app/service.jar”]

ENTRYPOINT [ “java”, “-jar” “service.jar”]

DOCKERFILE

docker-compose.yml

docker-compose build

Container Management Using kubernetes to manage and run thousands of containers

Pods

Namespaces

Management

Pods are the base concept of kubernetes. They describe a collection of related containers, volumes and their settings.

Node

Web Server File synchronizer

Shared directory

Shared network

Pod

Pods

Namespaces

Management

Namespaces in kubernetes are where you collect groups of related resources together.

A B

Pods

Namespaces

Management

Kubectl is your new best friend.

kubectl create get delete namespace pod

Security Tips Using docker and kubernetes in a secure way in production.

Sibling Containers

User namespace

Networking

Sibling containers are a much safer alternative to privilliged mode and docker in docker.

Node

docker.sock Container with docker

Sibling container

Sibling containers

User namespace

Networking

Enable user namespace remapping.

Using: --userns-remap=default

Node

docker daemon

root build-user

root dockremap

Sibling containers

User namespace

Networking

Disable intercontainer communication and turn on ip table rules.

Using: --icc=false --ip-tables=true

Docker Bridge

Node

iptable rules

Docker

Trust It Use It

Docker Microservices on AWS Connect

Your entry point to an Atlassian cloud product

Speed up time to market and free up

time to focus on your business logic

A runtime environment for your services and

more ;)

Review

Connect + AWS + Docker = Bitbucket Pipelines

NATHAN BURRELL • SENIOR DEVELOPER • ATLASSIAN