Groovy grails20110120Présentation sur Groovy / Grails par Olivier Gourment
Groovy DevOps for GGX 2015
-
Upload
andrey-adamovich -
Category
Technology
-
view
2.747 -
download
5
Transcript of Groovy DevOps for GGX 2015
Andrey AdamovichBio: Developer, coach, speaker, author
Company: Aestas/IT (http://aestasit.com)
Email: [email protected]
Linkedin: http://www.linkedin.com/in/andreyadamovich
Lanyrd: http://lanyrd.com/profile/andreyadamovich
GitHub: https://github.com/aadamovich
SO: http://stackoverflow.com/users/162792/andreyadamovich
Twitter: @codingandrey, @aestasit
••••••••
03
What's this presentation about?DevOps
Intrastructure Provisioning
Continuous Integration
Continuous Delivery
••••
04
TechnologiesGroovy http://groovy.codehaus.org
Gradle http://gradle.org
Jenkins http://jenkinsci.org
Puppet http://puppetlabs.com
AWS http://aws.amazon.com
•••••
05
Ant + Gradleant.taskdef(
name: 'scp',
classname: 'o.a.t.a.t.o.ssh.Scp',
classpath: configurations.secureShell.asPath)
ant.taskdef(
name: 'sshexec',
classname: 'o.a.t.a.t.o.ssh.SSHExec',
classpath: configurations.secureShell.asPath)
01.
02.
03.
04.
05.06.
07.
08.
09.
08
Simple callant.sshexec(
host: host,
username: user,
password: password,
command: command,
trust: 'true',
failonerror: failOnError)
01.
02.
03.
04.
05.
06.
07.
09
Next step: wrapper functiondef ssh(String command,
Properties props,
boolean failOnError = false,
String suCommandQuoteChar = "'",
String outputProperty = null)
...
01.
02.
03.
04.
05.
06.
07.
10
Next step: wrapper functiondef scp(String file,
String remoteDir,
Properties props)
...
01.
02.
03.
04.
05.
11
Task example Itask installFonts <<
forAllServers props ‐>
ssh('yes | yum install *font*', props)
01.
02.
03.
04.
05.
12
Task example IItask uninstallNginx <<
forAllServers props ‐>
ssh('/etc/init.d/nginx stop', props)
ssh('yes | yum remove nginx', props, true)
ssh('rm ‐rf /etc/yum.repos.d/nginx.repo', props)
ssh('rm ‐rf /var/log/nginx', props)
ssh('rm ‐rf /etc/nginx /var/nginx', props)
01.
02.
03.
04.
05.
06.
07.
08.
09. 13
DrawbacksNew connection each time
Excplicit repeating parameters
Complex scripts are hard to maintain
Tasks are not idempotent
••••
14
Sshoogr featuresGroovybased SSH DSL for:
Remote command execution
File uploading/downloading
Tunneling
•••
16
Sshoogr usage (import)@Grab(
group='com.aestasit.infrastructure.sshoogr',
module='sshoogr',
version='0.9.16')
import static com.aestasit.ssh.DefaultSsh.*
01.
02.
03.
04.
05.
17
Sshoogr usage (defaults)defaultUser = 'root'
defaultKeyFile = new File('secret.pem')
execOptions
verbose = true
showCommand = true
01.
02.
03.
04.
05.
06.
18
Sshoogr usage (connection)remoteSession
url = 'user2:654321@localhost:2222'
exec 'rm ‐rf /tmp/*'
exec 'touch /var/lock/my.pid'
remoteFile('/var/my.conf').text = "enabled=true"
01.
02.
03.
04.
05.
06.
19
Sshoogr usage (multiline content)remoteFile('/etc/yum.repos.d/puppet.repo').text = '''
[puppet]
name=Puppet Labs Packages
baseurl=http://yum.puppetlabs.com/el/
enabled=0
gpgcheck=0
'''
01.
02.
03.
04.
05.
06.
07.
20
Sshoogr usage (file copying)remoteSession
scp
from localDir "$buildDir/application"
into remoteDir '/var/bea/domain/application'
01.
02.
03.
04.
05.
06.
21
Sshoogr usage (command result)def result = exec(command: '/usr/bin/mycmd',
failOnError: false, showOutput: false)
if (result.exitStatus == 1)
result.output.eachLine line ‐>
if (line.contains('WARNING'))
throw new RuntimeException("Warning!!!")
01.
02.
03.
04.
05.
06.
07.
08.
09. 22
Sshoogr usage (shortcuts)if (ok('/usr/bin/mycmd'))
...
if (fail('/usr/bin/othercmd'))
...
01.
02.
03.
04.
05.
06.
23
Sshoogr usage (tunnels)tunnel('1.2.3.4', 8080) int localPort ‐>
def url = "http://localhost:$localPort/flushCache"
def result = new URL(url).text
if (result == 'OK')
println "Cache is flushed!"
else
throw new RuntimeException(result)
01.
02.
03.
04.
05.
06.
07.
08.
09. 24
Sshoogr usage (prefix/suffix)prefix('sudo ')
exec 'rm ‐rf /var/log/abc.log'
exec 'service abc restart'
suffix(' >> output.log')
exec 'yum ‐y install nginx'
exec 'yum ‐y install mc'
exec 'yum ‐y install links'
01.
02.
03.
04.
05.
06.
07.
08.
09. 25
Still problemsComplex scripts are still not easy to maintain
Scripts are usually not idempotent••
26
Why Puppet?More mature than competition
Large community
Readable DSL
Good acceptance from DEVs and OPs
No need to learn Ruby ;)
•••••
28
Upload modulestask uploadModules <<
remoteSession
exec 'rm ‐rf /tmp/repo.zip'
scp
from localFile "$buildDir/repo.zip"
into remoteDir "/root"
...
01.
02.
03.
04.
05.
06.
07.
08.
45
Upload modules ...
exec 'rm ‐rf /etc/puppet/modules'
exec 'unzip /tmp/repo.zip ‐d /etc/puppet/modules'
01.
02.
03.
04.
05.
46
Apply manifeststask puppetApply(dependsOn: uploadModules) <<
remoteSession
scp
from localFile "$buildDir/setup.pp"
into remoteDir "/tmp"
exec 'puppet apply /tmp/setup.pp'
01.
02.
03.
04.
05.
06.
07.
08.
09. 47
What we solved?Separated infrastructure state description and operations tasks
Scripts became more maintainable and idempotent••
48
In the meanwhile...We started developing complex/generic Puppet modules
Modules need proper testing
...on different platforms
•••
49
Do you test, right?How to test this stuff?
How to reuse a JUnit approach to testing?
We wanted things to be SIMPLE!
•••
50
PUnitSimple testing tool for verifying remote server state
Uses Sshoogr and JUnit
Reuse reporting features of JUnit
As simple as ...
••••
52
PUnit example (derby)class DerbyInstallTest
extends BasePuppetIntegrationTest
@Before
void installDerby()
apply("include derby")
...
01.
02.
03.
04.
05.
06.
07.
08.
53
PUnit example (derby)@Test
void ensureDerbyRunning()
command('service derby status > derbystatus.log')
assertTrue fileText("/root/derbystatus.log")
.contains('Derby')
assertTrue fileText("/root/derbystatus.log")
.contains('is running.')
01.
02.
03.
04.
05.
06.
07.
08.
54
PUnit example (derby)@Test
void ensureCanConnect()
Thread.sleep(10000)
uploadScript()
command('/opt/derby/db‐derby‐10.9.1.0‐bin/bin/ij ' +
'testDataScript.sql > derbytest.log')
...
01.
02.
03.
04.
05.
06.
07.
55
ScalabilityHow do we test on different OS?
How do we run parallel tests on multiple architectures?
How do we avoid selling our houses?
•••
59
Gramazon example Itask startInstance(type: StartInstance)
keyName 'cloud‐do'
securityGroup 'cloud‐do'
instanceName 'gramazon/cloud‐do'
stateFileName 'cloud‐do.json'
ami 'ami‐6f07e418'
instanceType 't1.micro'
waitForStart true
01.
02.
03.
04.
05.
06.
07.
08.
09. 64
Gramazon example IItask terminateInstance(type: TerminateInstance)
stateFileName 'cloud‐do.json'
01.
02.
03.
65
The flowStart instance(s)
Upload manifests
Run tests
Generate report
Terminate instance(s)
1.
2.
3.
4.
5.
66
ConclusionsGroovy/Gradle is an ultimate automation glue!
Reuse your existing Java knowledge
...to build a bridge between DEVs and OPs
Reuse development best practices for OPs
Infrastructure configuration is outsourcable now!
Automate!
••••••
75
Source codeSshoogr: https://github.com/aestasit/sshoogr
Sshoogr Gradle: https://github.com/aestasit/sshoogrgradle
Groowin: https://github.com/aestasit/groowin
Groowin Gradle: https://github.com/aestasit/groowingradle
PUnit: https://github.com/aestasit/puppetunit
Gramazon: https://github.com/aestasit/gramazon
Imgr: https://github.com/aestasit/imgr
•••••••
76