7+1 myths of the New OS

Alexis Richardson, Weave

Weave Network “SDN for hipsters”

The main points

1. In minutes, Weave solves Docker multi host problem

1. Inconceivably easy to use. And: dynamic, multi-cloud,

multi-hop, easy to wire up to rest of world...

1. Application developers don’t have to be experts in

networking or distributed systems, the SDN does it for

you...

Use cases and examples

http://weaveblog.com/

https://github.com/zettio/weave

The interesting part..

Weave Network is 100% decentralised

Standard APIs - IP, DNS, DHCP…

“Enables, then gets out of your way” -- supports a much

wider set of use cases...

Can we solve more

problems this way?

1994

SUN Microsystems invents a lingua franca for

the new world of distributed computing

→ JAVA

Promises portable web based applications by

providing a runtime container. Is designed

by distributed computing people...

7+1 myths of distributed computing

“Fallacies” - according to SUN Microsystems when Java was born

1. The network is reliable.

2. Latency is zero.

3. Bandwidth is infinite.

4. The network is secure.

5. Topology doesn't change.

6. There is one administrator.

7. Transport cost is zero.

8. The network is homogeneous. (added by Gosling, 1997)

http://en.wikipedia.org/wiki/Fallacies_of_distributed_computing

Where did Java go to?

Java turned into J2EE - tried to hide

complexity in a ‘platform’

→ the app server disaster

One size fits all “platforms” supporting

alleged enterprise requirements like

transactions

Let’s NOT DO THIS AGAIN

Ubiquitous and cheap

physical compute infra

Are we in post-fallacy world?

1. A homogeneous cloud with zero administrators?

2. Unbounded, cheap bandwidth albeit not zero cost?

3. Decouple applications from physical topology?

No we are NOT

1. A homogeneous cloud with zero administrators?

2. Unbounded, cheap bandwidth albeit not zero cost?

3. Decouple applications from physical topology?

// TODO: get latency to zero and system

integrity to 100% (CAP/Zooko)

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER

DOCKER DOCKER DOCKER

A profound change just occurred

Docker delivers a new alignment between dev

and ops: a container is what you ship, when

you ship an application

Google - 2B containers / week

If we are all like Google → 10T / year?

The new Docker

Scalable distributed application platform,

or, if you like: the New OS.

read.about.it@

http://weaveblog.com/2014/11/13/life-and-

docker-networking/

The new Weave

OK, so people want a New OS...

Let’s learn from Java & the Fallacies & the J2EE mistake.

1. Update the fallacies to “7+1 myths of the New OS”.

2. This is partly for fun :-)

3. But it is also to motivate future work!

MYTH 1

The application lifecycle is independent of the OS

But the reality is...

Dev and Ops align on a single build, the container

Neil Ellis:

“After Docker many of us are starting to look at things in

a different way. Our apps are now the same as machines

thanks to Docker. This means our applications can act like

they once did, as the sole piece of software running on a

machine.”

This leads to using containers for microservices, and in

the future, to microkernels & short lived microcontainers

MYTH 2

Distributed systems, and hence new apps, need a panoply of

unfamiliar tools and skills

But the reality is...

Because containers align app with machine lifecycle, we

can take advantage of decades of best practice

Neil again: “... this has the interesting side effect of

making older and low level technologies directly relevant

to application development and operations.

Forget your UDDI, you can go back to using DNS (optionally

with SRV records) instead. Forget application servers,

just load balance your containers.

It is as if we have turned back the clock 30 years, in a

good way.”

MYTH 3

Apps must be fundamentally rewritten and all operational

tooling replaced

But the reality is...

Because “we have turned back the clock 30 years, in a good

way”, every single API and tool can *already work*

We can envisage a composable runtime system, like Unix,

but for the era of containers and cloud scale apps

This can consist of a set of critical micro-services, like

networks (IP), service discovery (DNS), naming (DHCP).. in

which every service is completely obvious and intuitive

This makes it much easier to migrate and refactor apps.

MYTH 4

Platforms are better at designing your app than you are

But the reality is...

Mostly, you already know how to write apps and manage them

In a Unix like world of composable micro services that

have obvious APIs, you can choose what to build and you

already have the skills to do it

Opinionated platforms are extremely useful when aligned

with special cases (eg mobile) but can never be general

purpose. E.g not every app is a 12-factor web app.

A general purpose system must be unopinionated: it must

first enable, and then get out of your way.

MYTH 5

You really need an all-inclusive system

But the reality is...

You want adaptability & to plug in your own services

If you have to “learn” a platform before building an

application, you MAY be doing the wrong thing

All inclusive platforms tend to get in the way of choice

and application design. Enterprises also *require*

pluggability within the model and not as a ‘bolt on’.

Examples of good models: Amazon’s IaaS+services model; and

Netflix OSS which should be called “Netflix OS” ;-)

MYTH 6

Dev/Test and Production need different tooling

But the reality is...

Good tools scale from dev/test to prod

The ideal system should ‘scale from laptop to cloud’ with

a single set of APIs and tooling.

You need a system that is scale and location invariant, in

order to be truly portable. You need dynamic topologies

eg. using a Weave Network :-)

Note to Docker - can haz downloads <800MB plz

MYTH 7

Co-ordination is a platform concern, and not an

application level concern

But the reality is...

The internet is completely decentralised and your systems

should be too

Example: Consensus. YAGNI a lot of the time.

When you do need it, you want the semantics exposed up to

the app and not buried inside the platform (Java app

servers made this mistake with transactions)

MYTH N+1

Docker will solve all known problems of computing

Recap: New myths for New OS

7+1 new myths

1. The application lifecycle is independent of the OS

2. Distributed systems require a panoply of new skills and tools.

3. Apps must be fundamentally rewritten, and tooling replaced.

4. Platforms are better at designing your app than you are.

5. You really need an all inclusive system.

6. Dev/test and production need different tooling.

7. Coordination is a platform concern not an app level concern.

8. Docker will solve all known problems of computing

The new Weave: help with the faff

Download Docker and Weave and build an app in the way that

you already know how to do, but: using containers.

Weave will NOT get in your way: apps do not need

fundamental rewrites and ops teams do not need to retool.

Like Unix, a runtime system that is obvious, unopinionated

and composable. But: for 2014 concerns.

Mix in your own (micro)services and orchestration to taste!

Many challenges, here are a few

Logging without tears

Decentralised access control

Distributed LB and proxying