Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby...
Transcript of Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby...
![Page 1: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/1.jpg)
1
Solving the Package Problem? Or Making Solving the Package Problem? Or Making
it Infinitely Worse?it Infinitely Worse?
Joe BrockmeierJoe [email protected]@redhat.comTwitter: @jzbTwitter: @jzbhttp://dissociatedpress.net/http://dissociatedpress.net/
http://projectatomic.io
![Page 2: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/2.jpg)
2
Who's this guy?Who's this guy?
![Page 3: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/3.jpg)
3
Solving the Package ProblemSolving the Package Problem(Or Making it Infinitely Worse?)(Or Making it Infinitely Worse?)
● The Packaging Problem We FaceThe Packaging Problem We Face
● Solution: Software CollectionsSolution: Software Collections
● Solution: rpm-ostreeSolution: rpm-ostree
● Solution: Linux ContainersSolution: Linux Containers
● Potential PitfallsPotential Pitfalls● QuestionsQuestions
![Page 4: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/4.jpg)
4
In the Beginning...In the Beginning...
![Page 5: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/5.jpg)
5
Distributions as the Distributions as the Center of the UniverseCenter of the Universe
![Page 6: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/6.jpg)
6
Developers do not want to be Developers do not want to be limited to system versions of limited to system versions of
softwaresoftware
![Page 7: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/7.jpg)
7
Developers want easier ways to Developers want easier ways to deploy complex software from deploy complex software from
desktop to serverdesktop to server
![Page 8: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/8.jpg)
8
Automate ALL THE THINGSAutomate ALL THE THINGS
![Page 9: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/9.jpg)
9
Software Collections, rpm-ostree, and Software Collections, rpm-ostree, and Docker (oh my)Docker (oh my)
![Page 10: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/10.jpg)
SCALE 12x
Let's talk about Software CollectionsLet's talk about Software Collections
![Page 11: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/11.jpg)
11
Do not require changes to RPMDo not require changes to RPM
![Page 12: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/12.jpg)
12
Software Collections areSoftware Collections are not not just a just a different version packaged for your OSdifferent version packaged for your OS
![Page 13: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/13.jpg)
13
Do not overwrite system filesDo not overwrite system files
![Page 14: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/14.jpg)
14
Example: PHP 5.4
On CentOS 6.xOn CentOS 6.x
PHP 5.4 package isPHP 5.4 package is php54
This pulls in:This pulls in:
php54phpcli.x86_64
php54phpcommon.x86_64
php54phppear.noarch
php54phpprocess.x86_64
php54phpxml.x86_64
php54runtime.x86_64
Lives in:Lives in: /opt/rh/php54/root
![Page 15: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/15.jpg)
15
Avoid conflicts with system filesAvoid conflicts with system files
![Page 16: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/16.jpg)
16
Require Require minorminor changes to your changes to your existing spec filesexisting spec files
![Page 17: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/17.jpg)
17
Do not conflict with updates on your Do not conflict with updates on your systemsystem
![Page 18: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/18.jpg)
18
Nifty: Nifty: CanCan depend on other SCLs depend on other SCLs
![Page 19: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/19.jpg)
SCALE 12x
Let's talk about using SCLsLet's talk about using SCLs
![Page 20: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/20.jpg)
20
Getting StartedGetting Started
● Assuming using a SLC with CentOS● yum install centosreleaseSCL
● yum install php54 (or whatever...)
● scl enable php54 “application option”
● Your application now uses PHP 5.4 ... the rest of the system ignores it.
● Python & Django with SCL (by Langdon White):● http://red.ht/scldjango
● Find packages for CentOS here: http://mirror.metrocast.net/centos/6.5/SCL/x86_64/
![Page 21: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/21.jpg)
21
Packaging SCLs
● Grab the necessary packages (CentOS or Grab the necessary packages (CentOS or Fedora or RHEL 6.5):Fedora or RHEL 6.5): yum install sclutils sclutilsbuild
● Instructions on converting an existing Instructions on converting an existing package:package: http://bit.ly/sclspecfile
● For Conversion: For Conversion: spec2sclspec2scl
● General instructions on packaging SCLs:General instructions on packaging SCLs: http://bit.ly/pkgingscls
![Page 22: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/22.jpg)
SCALE 12x
SoftwareCollections.orgSoftwareCollections.org
![Page 23: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/23.jpg)
23
Software Collections CurrentlySoftware Collections Currently
● See: https://www.softwarecollections.org/en/
● CentOS SCL SIG: http://wiki.centos.org/SpecialInterestGroup/SCLo
● Git repo: https://git.centos.org/project/?p=sig-sclo
● Upstream mailing list: https://www.redhat.com/mailman/listinfo/sclorg
●
![Page 24: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/24.jpg)
24
What is SoftwareCollections.org?
● Upstream community for development of SCLs.
● Build and hosting services for collections.
● Resources (documentation, forums, mailing lists) for developers/packagers.
● An index of packaged software for users of CentOS, Fedora, RHEL, and other RPM-based distributions.
![Page 25: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/25.jpg)
25
The Lifecycle of Collections
● SCLs can be used to provide newer software support on older releases, or (going forward) to provide legacy support on newer releases:
● Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7 (when released) using SCL.
● OpenShift leverages SCLs for its cartridges using RHEL supported and non-supported components.
● SCLs can be used inside Docker containers to simplify container deployment.
![Page 26: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/26.jpg)
SCALE 12x
rpm-ostreerpm-ostree
![Page 27: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/27.jpg)
27
The Problem with Packages
● RPM (and dpkg) are designed to go one way: forward
● Upgrades are difficult to “roll back” in the event something goes wrong
● Switching between two distinct OSes / versions is more or less impossible
![Page 28: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/28.jpg)
28
What is rpm-ostree?
● Derived from ostree● Initially conceived of as a way to parallel install
multiple UNIX-like OSes (e.g., Fedora Rawhide and Fedora 20)
● “git for operating system binaries”● Creates an installable tree from RPMs● Not a package manager, but does take on some
of the role from package managers
![Page 29: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/29.jpg)
29
What rpm-ostree Enables
● Install one or more operating system trees to a system
● Gives “atomic” updates ● An update is, essentially, one unit – it succeeds
or fails● An update can be rolled back
● Allows switching between “trees”
● Provides tools for creating tree composes
![Page 30: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/30.jpg)
30
Current Limitations
● Currently, an rpm-ostree “tree” is an immutable system
● Doesn't allow for adding packages to a system w/out rebuilding the tree
● Build tools are still being developed, but moving quickly
![Page 31: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/31.jpg)
SCALE 12x
So, anybody heard of this Docker So, anybody heard of this Docker thing?thing?
![Page 32: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/32.jpg)
32
The Problem with Packages
● Deploying complex services / applications is difficult with packages
● Packages aren't as portable as we'd like● Application is developed on CentOS 6, but production is
using CentOS 7? ● Packaging guidelines can be ... difficult
● Packages don't provide any solution for running containerized applications...
![Page 33: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/33.jpg)
33
Docker: It's Like Deluxe Super Awesome Packaging
● Docker is application-centric
● Docker containers are portable
● Supports versioning for an entire container
● Components can be re-used
● Allows for supplying ready-to-run services rather than half-configured packages
● Buzzword compliant
![Page 34: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/34.jpg)
34
Docker to the Rescue?
● Docker containers: relatively easy to work with
● Provide far more “services” than package systems● Application isolation● Image format, sharing, API
● Allows “layering” of applications● One group provides base image● Another group provides base image +
framework/service (e.g., Apache)● Another group provides base image + framework +
finished application ready to deploy
![Page 35: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/35.jpg)
SCALE 12x
PitfallsPitfalls
![Page 36: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/36.jpg)
36
Docker isn't Mature
● “Containers Don't Contain”
● Signing, etc. are still in their infancy
● Packaging apps in containers is still not well-understood
● Deploying apps in containers is still not well-understood
![Page 37: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/37.jpg)
37
Additional Problems
● Auditing software is difficult (or impossible) in containers
● Updates to containers – who tracks? How to automate?
● Host/Container mis-matches
● What else?
![Page 38: Solving the Package Problem? Or Making it Infinitely Worse?€¦ · Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7](https://reader036.fdocuments.in/reader036/viewer/2022071010/5fc7f17511255b73b936ee9f/html5/thumbnails/38.jpg)
Links and Pointers● Website: projectatomic.io● Github: github.com/projectatomic● Facebook.com/projectatomic● Twitter: @projectatomic● Mailing Lists:
http://www.projectatomic.io/community/