RPM experiments

42
RPM Building in 2016 RPM Building in 2016 RPM Building in 2016 RPM Building in 2016 RPM Building in 2016 RPM Building in 2016 RPM Building in 2016 RPM Building in 2016 RPM Building in 2016 RPM Building in 2016 RPM Building in 2016 RPM Building in 2016 RPM Building in 2016 RPM Building in 2016 RPM Building in 2016 RPM Building in 2016 RPM Building in 2016 Julien Pivotto Julien Pivotto Julien Pivotto Julien Pivotto Julien Pivotto Julien Pivotto Julien Pivotto Julien Pivotto Julien Pivotto Julien Pivotto Julien Pivotto Julien Pivotto Julien Pivotto Julien Pivotto Julien Pivotto Julien Pivotto Julien Pivotto Inuits TTT Inuits TTT Inuits TTT Inuits TTT Inuits TTT Inuits TTT Inuits TTT Inuits TTT Inuits TTT Inuits TTT Inuits TTT Inuits TTT Inuits TTT Inuits TTT Inuits TTT Inuits TTT Inuits TTT February 26th, 2016 February 26th, 2016 February 26th, 2016 February 26th, 2016 February 26th, 2016 February 26th, 2016 February 26th, 2016 February 26th, 2016 February 26th, 2016 February 26th, 2016 February 26th, 2016 February 26th, 2016 February 26th, 2016 February 26th, 2016 February 26th, 2016 February 26th, 2016 February 26th, 2016

Transcript of RPM experiments

RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016RPM Building in 2016

Julien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien PivottoJulien Pivotto

Inuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTInuits TTTFebruary 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016February 26th, 2016

whoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoamiwhoami

• JJJJJJJJJJJJJJJJJulien Pivotto• rrrrrrrrrrrrrrrrroidelapluie• SSSSSSSSSSSSSSSSSysadmin• FFFFFFFFFFFFFFFFFLOSS since 2004• IIIIIIIIIIIIIIIIInuits since 2011• CCCCCCCCCCCCCCCCCentOS CfgMgmtSig

Native packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packagesNative packages

• RRRRRRRRRRRRRRRRReally? It is still needed?• PPPPPPPPPPPPPPPPPypi/Rubygems/npm/pear…• OOOOOOOOOOOOOOOOOmnibus/FPM/…

No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .No need for native packages. . .

It is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easyIt is so easy

gem install fpm

ERROR: Could not find a valid gem `fpm'(>= 0), here is why:

Unable to download data from https://rubygems.org/ − Errno::ECONNREFUSED:Connection refused − connect(2) for "api.rubygems.org" port 443 (https://api.rubygems.org/latest_specs.4.8.gz)

I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)I have it installed (4 times :-/)

• /////////////////usr/bin/fpm• ./vendor/bin/fpm• /////////////////usr/local/bin/fpm• ~~~~~~~~~~~~~~~~~/.ruby/2.4/gems/rubygems/fpm/bin/fpm

Now I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prodNow I can deploy to prod

• DDDDDDDDDDDDDDDDDev: version 1.1• UUUUUUUUUUUUUUUUUAT (deployed next day): 1.2• PPPPPPPPPPPPPPPPProd (a week after): 2.0. broken.

Open questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questionsOpen questions

• WWWWWWWWWWWWWWWWWhat is installed?• WWWWWWWWWWWWWWWWWhere does the file come from?• CCCCCCCCCCCCCCCCCompilation at install time?

RPMRPMRPMRPMRPMRPMRPMRPMRPMRPMRPMRPMRPMRPMRPMRPMRPM

How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011How I built RPM in 2011

• hhhhhhhhhhhhhhhhhttps://github.com/roidelapluie/vagrant-build-mapnik

• /////////////////usr/bin/wget -O /tmp/mapnik.tar.bz2https://github.com/mapnik-2.0.0.tar.bz2

• /////////////////usr/bin/screen -d -m/usr/local/bin/build_mapnik.sh

• AAAAAAAAAAAAAAAAAll of that in puppet manifests

I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…I was so young…

After vagrant provision , wait until thecompilation is done.

You can see the compilations process:vagrant sshsudo screen −r

The packages will be located in the "rpms"folder.

Building in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VMBuilding in the VM

cd /opt/mapnik−2.0.0python scons/scons.py configure PREFIX=${

target} PYTHON_PREFIX=${target}python scons/scons.pypython scons/scons.py install

FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…FPM all the things…

fpm −s dir −n mapnik −v 2.0.0 −−iteration "${start_date}" \

−C "${target}" −t rpm −−prefix /usr −−urlhttp://mapnik.org/ \

−−description "Mapnik is a Free Toolkit fordeveloping mapping applications." \

−−exclude include

Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…Even the devel package…

fpm −s dir −n mapnik−devel −v 2.0.0 −−iteration "${start_date}" \

−C "${target}/include" −t rpm −−prefix /usr/include −−url http://mapnik.org/ \

−−description "The mapnik−devel packagecontains header files for developingprograms using the Mapnik library." \

−−depends "mapnik = 2.0.0"

How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014How I built RPM in 2014

• hhhhhhhhhhhhhhhhhttps://github.com/roidelapluie/collectd-rpm

• SSSSSSSSSSSSSSSSSpec file in collectd upstream• wwwwwwwwwwwwwwwwwget sources• iiiiiiiiiiiiiiiiinstall build deps• rrrrrrrrrrrrrrrrrpmbuild -bb

install build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build depsinstall build deps

egrep "^BuildRequires" collectd−5.5.0/contrib/redhat/collectd.spec | cut −d `:' −f 2 | tr−d ` \t' | sed `s/,/\n/' | uniq | tr −d `\n' | tr `,' ` '|xargs yum install −y

And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?And in 2016?

PackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglovePackaginglove

• hhhhhhhhhhhhhhhhhttp://packaginglove.github.io/packaging-book/

• CCCCCCCCCCCCCCCCCurrently limited to RPM• AAAAAAAAAAAAAAAAA pragmatic approach• DDDDDDDDDDDDDDDDDeb coming• PPPPPPPPPPPPPPPPPlz contribute

Pick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better toolsPick better tools

• dddddddddddddddddocker instead of vagrant• yyyyyyyyyyyyyyyyyum/rpm helpers commands• rrrrrrrrrrrrrrrrreuse/rebuild fedora packages• ttttttttttttttttthe Cloudr©: copr/travis

RPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpersRPM/yum helpers

• yyyyyyyyyyyyyyyyyum-builddeps install build dependencies• ssssssssssssssssspectool download the sources

Reuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packagesReuse fedora packages

• ccccccccccccccccclone spec file• fffffffffffffffffix if needed

dockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdockerdocker

• dddddddddddddddddocker runs on travis• FFFFFFFFFFFFFFFFFaster than vagrant• uuuuuuuuuuuuuuuuuseful run options: --rm -v -w -t -i --env

Docker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" image

FROM centos:7MAINTAINER Julien Pivotto <roidelapluie@inuits

.eu>RUN yum install −y /usr/bin/wgetRUN yum install −y /usr/bin/spectoolRUN yum install −y /usr/bin/rpmbuildRUN yum install −y /usr/bin/yum−builddepRUN yum install −y /usr/bin/makeRUN yum install −y epel−release

Docker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" imageDocker "el7-build" image

docker build −t el7−build .

Docker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker runDocker run

docker run −e XUID="$(id −u)" −−rm −ti −v $PWD:/work −w /work el7−build ./el7−build.sh

• ------------------rm: do not fill my disk• -----------------t: allocate a tty• -----------------i: interactive• -----------------v $PWD:/work mount current dir on host to/work in container

• -----------------w /work: use /work as working dir

el7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.shel7-build.sh

useradd −u ${XUID:−1000} rpmmkdir −p /home/rpm/rpmbuild/SOURCESspectool −C /home/rpm/rpmbuild/SOURCES −g test

.specyum−builddep −y facter.specchown −R rpm: /home/rpmsu − rpm −c "rpmbuild −bb /work/facter.spec"cp −a /home/rpm/rpmbuild/SRPMS/*.src.rpm .cp −a /home/rpm/rpmbuild/RPMS/*/*.rpm .

Cloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud toolsCloud tools

travis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secretstravis secrets

• TTTTTTTTTTTTTTTTTravis "secrets" = encrypted variables• UUUUUUUUUUUUUUUUUse for password, credentials• VVVVVVVVVVVVVVVVVariables or files• TTTTTTTTTTTTTTTTThe travis gem• PPPPPPPPPPPPPPPPPublish to github releases

travis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gemtravis gem

travis setup releases

travis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymltravis.ymlsudo: requiredservices:

− dockerscript:

− makedeploy:

provider: releasesapi_key:

secure: Some encrypted stuffskip_cleanup: truefile_glob: truefile: python−*.rpmon:

tags: true

coprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcoprcopr

hhhhhhhhhhhhhhhhhttps://copr.fedorainfracloud.org/coprs/roidelapluie/• fffffffffffffffffedora project• IIIIIIIIIIIIIIIIInfra to build RPM• WWWWWWWWWWWWWWWWWeb interface• pppppppppppppppppublic builds• llllllllllllllllless limitations than travis

copr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and travis

travis encrypt−file copr−config −−addecho copr−config > .gitignore

copr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and traviscopr and travis

sudo: requiredservices:− dockerscript:− makebefore_install:− openssl aes−256−cbc −K

$encrypted_2ec4d63b6867_key −iv$encrypted_2ec4d63b6867_iv

−in copr−config.enc −out copr−config −d

MakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefileMakefile

el7:docker run −e XUID="$(shell id −u)" −−rm −v

$(shell pwd):/work −w /work −ti fedora:23./dockerscript.sh

dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)dockerscript.sh (1/2)

#!/bin/bashset −x;set −ednf install −y /usr/bin/spectool /usr/bin/

rpmbuild /usr/bin/copr−cli /usr/bin/python3−config

useradd −u ${XUID:−1000} bobmkdir −p /home/bob/rpmbuild/SOURCESfind . −maxdepth 1 −type f −exec cp −v '{}' /

home/bob/rpmbuild/SOURCES ';'spectool −C /home/bob/rpmbuild/SOURCES −g /

work/boost.specchown −R bob: /home/bob

dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)dockerscript.sh (2/2)

su − bob −c "rpmbuild −bs /work/boost.spec"copr−cli −−debug −−config ./copr−config build

boost /home/bob/rpmbuild/SRPMS/*.src.rpm

• rrrrrrrrrrrrrrrrrpmbuild -bs: build only SRPM• cccccccccccccccccopr-cli --debug is to avoid the 10 minuteslimitation in travis-ci

ConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusion

ConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusionConclusion

• fffffffffffffffffedora: source of pkgs• tttttttttttttttttravis/copr: Public builds• dddddddddddddddddocker: reproducible locally