Papers We Love:Jails and Zones

CTO

bryan@joyent.com

Bryan Cantrill

@bcantrill

Papers we love: Jails and Zones

• Discussing two important papers that form the foundation of thinking about OS-based virtualization and containers:

• Jails: Confining the Omnipotent Root by Poul-Henning Kamp and Robert Watson, presented at SANE 2000

• Solaris Zones: Operating System Support for Consolidating Commercial Workloads by Dan Price and Andy Tucker, presented at LISA 2004

• As much as possible, want to let these papers speak for themselves — and provoke discussion!

Jails: Problem statement

Jails: Prior work

Jails aside: chroot(2)

Jails: Proposed solution

Jails: Advantages

Jails: jail(2)

Jails: Confining the filesystem

Jails: Confining the network

Jails: Implementation

Jails: Network management complexities

Jails: Filesystem management complexities

Jails: User management complexities

Jails: Unintended consequences

Jails: Networking limitations

Jails: Resource management limitations

Jails: Management limitations

Jails: Epilogue

• Jails became easier to manage with jls/jps/ezjail/iocage

• Jails were allowed to have multiple IPv4 addresses

• Some jail-based resource management was added, including CPU binding and

• System V IPC was virtualized, but remains out-of-tree

• VIMAGE added exclusive IP stacks to jails, but it remains a build-time option and “is considered experimental”

Zones: Problem statement

Zones: Problem statement detail

Zones: Proposed solution

Zones: Block diagram

Zones: Design principles

Zones: Design principles, cont.

Zones: State model

Zones: Configuration

Zones: Installation

Zones: Application environment

Zones: Virtual platform

Zones: Console

Zones: Process model

Zones: Process model, cont.

Zones: IPC

Zones: System V IPC

Zones: Networking

Zones: Filesystem

Zones: Resource management

Zones: Observability and debugging

Zones: Security experience

Zones: Workloads

Zones: Epilogue

• Crossbow added virtual NICs and exclusive IP stacks — and anti-spoof allowed exclusive IP stacks to be deployed safely

• Resource management became much more complete, adding memory capping, CPU capping, I/O throttling

• ZFS revolutionized zone installation/configuration

• With introduction of IPS packaging, Solaris got rid of so-called “sparse root” zones...

• ...and Joyent added sparse root zones back to SmartOS (thanks to no IPS and no global zone package management)

Zones: Epilogue, cont.

• Sun added notion of branded zones in 2006, including a nascent Linux brand (LX) — and then ripped LX out in 2010

• LX brand revived by Joyent in 2014 in SmartOS and completed (first deployed into production in early 2015)

• Overlay network support added to SmartOS by Joyent, allowing software-defined VXLAN-based networks in non-global zones

Jails and Zones: Conclusions

• Each of these technologies has served to inspire the other: zones was explicitly inspired by jails — and the jails networking work has been explicitly inspired by Crossbow

• These two papers are important because they capture not just the what, but the why of their respective works

• These technologies were both ahead of their time; it’s invaluable now to be able to understand their motivations!

• In the words of the late, great Jim Gray: You need to write more!