Zen and-the-art-of-build-script-maintenance-skillsmatter

ZENand the art of

Build Script Maintenance

An Inquiry into Build Automation Quality

John Ferguson SmartWakaleo Consulting

Web: http://www.wakaleo.comEmail: [email protected]: wakaleo

http://www.wakaleo.com/blog


mailto:[email protected]


Who is this guy, anyway?John Ferguson SmartConsultant, Trainer, Mentor, Author, Speaker

Specialities:

Java, Groovy/Grails

Software Development Lifecycle

Agile stuff (CI, TDD, BDD,...)

Open Source

AgendaWhat are we discussing today?What makes a good build script?

Smelly build scripts

Choosing your tools

Maven tips

Ant tips

IntroductionQuality build scripts - why botherMaintenance costs

Learning curve

Turn-over

Portability

Automation

Build quality - quality buildsWhat makes a good build script?Gold Standard

Portable

Reproducible

Standard

Maintainable

Build quality - quality buildsGold StandardReference build process

Reference binaries

Build quality - quality buildsPortableRuns anywhere

Runs on any OS

No local dependencies

Environment-specific configurations

Specially-installed software or databases

...

Build quality - quality buildsReproducible“Play it again, Sam”

Build quality - quality buildsStandardKnowing what to expect

Build quality - quality buildsMaintainableThink of the next dude

Smelly buildsSo what makes a poor build script?

1) The hard coded build

2) The OS-specific build

3) The IDE-only build

4) The Magic Machine build

5) The Oral Tradition build

6) The Nested build

7) The Messy Dependencies build

8) The DYI build

9) The untrustworthy build

10) The slow build

Smelly buildsThe hard coded build

C:/bea/weblogic-9.1/..

Paths

http://testserver.acme.com:7001

URLs

<svn username="scott" password="tiger"...>Passwords

<property name="dir.jboss" value="${env.JBOSS_HOME}"/>

Environment variables

http://testserver:7001























































































<target name="checkstyle">

<delete dir="./reports" quiet="true" /> <mkdir dir="./reports" /> <checkstyle config="./docs/sun_checks.xml"> <formatter type="xml" tofile="./reports/checkstyle.xml"/> <fileset dir="./src" includes="**/*.java"/> </checkstyle>

<style in="./reports/checkstyle.xml" out="./reports/checkstyle.html" style="checkstyle.xsl"/>

</target>

Hard-coded directories










































































































<target name="checkstyle">

<delete dir="./reports" quiet="true" /> <mkdir dir="./reports" /> <checkstyle config="./docs/sun_checks.xml"> <formatter type="xml" tofile="./reports/checkstyle.xml"/> <fileset dir="./src" includes="**/*.java"/> </checkstyle>

<style in="./reports/checkstyle.xml" out="./reports/checkstyle.html" style="checkstyle.xsl"/>

</target>

<property name=”reports.checkstyle.dir” value=”${basedir}/reports”/>

<target name="checkstyle"> <delete dir="${reports.checkstyle.dir}" quiet="true" /> <mkdir dir="${reports.checkstyle.dir}" /> <checkstyle config="./docs/sun_checks.xml"> <formatter type="xml" tofile="${reports.checkstyle.dir}/checkstyle.xml"/> <fileset dir="./src" includes="**/*.java"/> </checkstyle>

<style in="${reports.checkstyle.dir}/checkstyle.xml" out="${reports.checkstyle.dir}/checkstyle.html" style="checkstyle.xsl"/>

</target>

Project-relative directory

DRY




























































































































<target name="war" > <war destfile="c:/tomcat/jakarta-tomcat-5.0.19/webapps/app.war" webxml="${src}/app.xml" basedir="${bin}" /></target>

Hard-coded directories



































<property name="wardir" location="c:/tomcat/jakarta-tomcat-5.0.19/webapps"/>

<target name="war" > <war destfile="${wardir}" webxml="${src}/app.xml" basedir="${bin}" /></target>


Still hard-coded












































<svn username="scott" password="tiger"> <checkout url="http://subversion.acme.com/myapp/trunk" destPath="${subproject.dir}" /></svn>

Hard-coded username/password

http://subversion.acme.com/someproject/trunk











<property environment="env"/><property name="dir.jboss" value="${env.JBOSS_HOME}"/>

Environment variable

Smelly buildsThe OS-specific build

<exec command="grep \"@\" ${build.dir} | wc -l" outputproperty="token.count"/>

Smelly buildsThe OS-specific build

...CALL PAUSE.CMD...

build.cmd

...:: Check for a non-existent IP address:: Note: this causes a small extra delay!IF NOT DEFINED NonExist SET NonExist=10.255.255.254PING %NonExist% -n 1 -w 100 2>NUL | FIND "TTL=" >NUL...

pause.cmd

Smelly buildsThe IDE-only build

Smelly buildsThe Magic Machine build

Directories

App servers

Databases

Configuration files


Installed software or tools

Smelly buildsThe Magic Machine build

Directories

App servers

Databases

Configuration files


Installed software or tools

<proprerty weblogic.dir="/u01/app/bea/weblogic-9.1"/>

Smelly buildsThe Oral Tradition build

Smelly buildsThe Nested Build

#! /bin/shANT_HOME=/u01/app/tools/ant-1.7.1...$ANT_HOME/ant $1project/tools/ant.sh

Smelly buildsThe Nested Build

<target name="build-subproject"> <svn username="scott" password="tiger"> <checkout url="http://subversion.acme.com/someproject/trunk" destPath="${subproject.dir}" /> </svn> <ant dir="${subproject.dir}" target="build-all" /></target>

build.xml









































The Messy Dependencies buildJAR files in the SCM

Unversioned JAR files

Unclear dependencies

Smelly builds

The DYI build“Not invented here”

DYI dependencies

DYI deployments

DYI Maven releases

...

Smelly builds

Smelly buildsThe untrustworthy build<junit fork="yes" haltonfailure="false" dir="${basedir}"> <classpath refid="test.class.path" /> <classpath refid="project.class.path"/> <formatter type="plain" usefile="true" /> <formatter type="xml" usefile="true" /> <batchtest fork="yes" todir="${logs.junit.dir}"> <fileset dir="${test.unit.dir}"> <patternset refid="test.sources.pattern"/> </fileset> </batchtest></junit>



















































































Smelly buildsThe slow build

Choosing your toolsFlexibility verses ConventionWhat’s better: flexibility or standards?

It depends what you’re doing...

Choosing your toolsStandards and Conventions

No standards

Make up your own standards

Support standards

Encourage/enforce standards

Standards and ConventionsAd-hoc scripting

Eas

y to

rea

dH

ard

to

rea

d

23

Choosing your toolsFlexibility and expressiveness

Eas

y to

rea

dH

ard

to

rea

d

Easy to do whatever you wantMakes you stick to conventions

3

Encourage/enforce standards

Do whatever you want

2

Choosing your toolsFlexibility verses ConventionBuild Scripting Rule 1

“A build script will tend to reflect the personality of it’s developer”

Choosing your toolsFlexibility verses ConventionBuild Scripting Rule 2

“The more flexible a build script, the more likely it is to become unmaintainable”

Choosing your toolsFlexibility verses ConventionFlexibility is great for some jobs:

Ad-hoc tasks

Some deployment tasks

“Out-of-the-box” stuff

Choosing your toolsFlexibility verses ConventionBut too much flexibility is hard to maintain

Ant tipsBetter Ant scriptsConsistent conventions

Declare your dependencies

Make it readable

Tidy up your mess

Avoid long scripts

Ant tipsBe consistent

Standardize target names

Document your public targets

Ant tipsDeclare your dependencies

Use an Enterprise Repository Manager

Several tool choices:

Maven Ant Tasks

Ivy

Ant tipsUsing the Maven Ant Tasks

Declare dependencies

Deploy to a Maven Enterprise Repository

<artifact:dependencies pathId="dependency.classpath"> <dependency groupId="junit" artifactId="junit" version="3.8.2" scope="test"/> <dependency groupId="javax.servlet" artifactId="servlet-api" version="2.4" scope="provided"/></artifact:dependencies>

Ant tipsMake it readable

Write a build script like your source code...

Avoid long targets

Avoid long build scripts

Use descriptive target names

Ant tipsTidy up your mess

Always define a ‘clean’ target.

Ant tipsMove to Maven ;-)

Maven tipsBetter Maven scriptsSimple

Portable

Reproducible

Clean

Automated

Maven tipsKeep it simpleUse modules

Use an organization-level POM

Maven tipsKeep it portableNo hard-coding

Define sensible defaults for properties and profiles

Avoid resource filtering for production code

Maven tipsKeep it reproducibleAvoid external snapshots

Specify plugin versions

Use consistent environments

Maven tipsConsistent environmentsEnforcing a minimum Maven version

<?xml version="1.0"?><project...> <modelVersion>4.0.0</modelVersion> <groupId>com.ciwithhudson.gameoflife</groupId> <artifactId>gameoflife</artifactId> <version>0.0.1-SNAPSHOT</version> <name>gameoflife</name> <prerequisites> <maven>2.2.1</maven> </prerequisites>

Minimum Maven version

Maven tipsConsistent environmentsUse the same version of Maven

Use a “standard” Maven installation across the organization

Use a global settings.xml file

Store a copy in SCM

Enforce a minimum Maven version in your projects

Maven tipsEnforcing consistency with the enforcer pluginMaven version

JDK version

Snapshots

Plugin versions

OS

...

Maven tipsEnforce the Maven version

<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-enforcer-plugin</artifactId> <version>1.0-beta-1</version> <executions> <execution> <id>enforce-maven-version</id> <goals> <goal>enforce</goal> </goals> <configuration> <rules> <requireMavenVersion> <version>2.2.1</version> </requireMavenVersion> </rules> </configuration> </execution> </executions> </plugin>

Minimum Maven version

Maven tipsEnforce the JDK versionAll developers should be using the same JDKs

Incompatible bytecode

Different XML parsers

Different Maven behaviour

Enforce the JDK version

<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-enforcer-plugin</artifactId> <version>1.0-beta-1</version> <executions> <execution> <id>enforce-jdk-version</id> <goals> <goal>enforce</goal> </goals> <configuration> <rules> <requireJavaVersion> <version>[1.5.0,1.6.0)</version> </requireJavaVersion> </rules> </configuration> </execution> </executions> </plugin>

Maven tips

Authorized JDK versions

Maven tipsSpecify your plugin versionsUndeclared version numbers are bad

Inconsistent builds across different machines

Non-repeatable builds

Plugin changes can break the build

Don’t use SNAPSHOT plugins either

Specify your plugin versions

<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-enforcer-plugin</artifactId> <version>1.0-beta-1</version> <executions> <execution> <id>enforce-versions</id> <goals> <goal>enforce</goal> </goals> <configuration> <rules> <requirePluginVersions/> </rules> </configuration> </execution> </executions> </plugin>

Maven tips

Plugin versions must be defined

Maven tipsKeep it cleanKeep tabs on your dependencies:

What dependencies are you actually using?

What dependencies do you really need?

Maven tipsDependency list What dependencies are you actually using?$ mvn dependency:list[INFO] Scanning for projects...[INFO] Searching repository for plugin with prefix: 'dependency'.[INFO] ------------------------------------------------------------------------[INFO] Building babble-core[INFO] task-segment: [dependency:list][INFO] ------------------------------------------------------------------------[INFO] [dependency:list][INFO] [INFO] The following files have been resolved:[INFO] antlr:antlr:jar:2.7.6:compile...[INFO] commons-collections:commons-collections:jar:2.1.1:compile[INFO] commons-logging:commons-logging:jar:1.0.4:compile[INFO] dom4j:dom4j:jar:1.6.1:compile[INFO] javax.persistence:persistence-api:jar:1.0:compile[INFO] javax.transaction:jta:jar:1.0.1B:compile[INFO] junit:junit:jar:4.5:test[INFO] net.sf.ehcache:ehcache:jar:1.2:compile[INFO] org.hamcrest:hamcrest-all:jar:1.1:compile[INFO] org.hibernate:hibernate:jar:3.2.0.ga:compile[INFO] org.hibernate:hibernate-annotations:jar:3.2.0.ga:compile[INFO] [INFO] ------------------------------------------------------------------------[INFO] BUILD SUCCESSFUL[INFO] ------------------------------------------------------------------------

mvn dependency:list

Maven tipsDependency tree Where do they come from?$ mvn dependency:tree[INFO] Scanning for projects...[INFO] Searching repository for plugin with prefix: 'dependency'.[INFO] ------------------------------------------------------------------------[INFO] Building babble-core[INFO] task-segment: [dependency:tree][INFO] ------------------------------------------------------------------------[INFO] [dependency:tree][INFO] com.sonatype.training:babble-core:jar:1.0-SNAPSHOT[INFO] +- org.hibernate:hibernate:jar:3.2.0.ga:compile[INFO] | +- net.sf.ehcache:ehcache:jar:1.2:compile[INFO] | +- javax.transaction:jta:jar:1.0.1B:compile[INFO] | +- commons-logging:commons-logging:jar:1.0.4:compile[INFO] | +- asm:asm-attrs:jar:1.5.3:compile[INFO] | +- dom4j:dom4j:jar:1.6.1:compile[INFO] | +- antlr:antlr:jar:2.7.6:compile[INFO] | +- cglib:cglib:jar:2.1_3:compile[INFO] | +- asm:asm:jar:1.5.3:compile[INFO] | \- commons-collections:commons-collections:jar:2.1.1:compile[INFO] +- org.hibernate:hibernate-annotations:jar:3.2.0.ga:compile[INFO] | \- javax.persistence:persistence-api:jar:1.0:compile[INFO] +- junit:junit:jar:4.5:test[INFO] \- org.hamcrest:hamcrest-all:jar:1.1:compile[INFO] ------------------------------------------------------------------------[INFO] BUILD SUCCESSFUL[INFO] ------------------------------------------------------------------------

mvn dependency:tree

Maven tipsDependencies in EclipseEclipse has an equivalent screen

Maven tipsDependency analyse What dependencies do you really need?$ mvn dependency:analyze[INFO] Scanning for projects...[INFO] Searching repository for plugin with prefix: 'dependency'.[INFO] ------------------------------------------------------------------------[INFO] Building babble-core[INFO] task-segment: [dependency:analyze][INFO] ------------------------------------------------------------------------[INFO] Preparing dependency:analyze...[INFO] [dependency:analyze][WARNING] Used undeclared dependencies found:[WARNING] javax.persistence:persistence-api:jar:1.0:compile[WARNING] Unused declared dependencies found:[WARNING] org.hibernate:hibernate-annotations:jar:3.2.0.ga:compile[WARNING] org.hibernate:hibernate:jar:3.2.0.ga:compile[INFO] ------------------------------------------------------------------------[INFO] BUILD SUCCESSFUL[INFO] ------------------------------------------------------------------------

mvn dependency:analyse

Used but not declared

Declared but not used

Maven tipsExcluding dependenciesWhat if you don’t want a dependency? <dependencies> <dependency> <groupId>org.springframework</groupId> <artifactId>spring</artifactId> <version>2.5.5</version> <exclusions> <exclusion> <groupId>javax.jms</groupId> <artifactId>jms<artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.apache.geronimo.specs</groupId> <artifactId>geronimo-jms_1.1_spec</artifact> <version>1.1</version> </dependency> <dependencies>

Don’t include JMS
























































































































Maven tipsStandardizing versionsUse dependencyManagement for consistency

<dependencyManagement> <dependencies> <dependency> ! <groupId>mysql</groupId> ! <artifactId>mysql-connector-java</artifactId> ! <version>5.1.6</version> </dependency> <dependency> ! <groupId>postgres</groupId> ! <artifactId>postgres</artifactId> ! <version>7.3.2</version> </dependency> </dependencies></dependencyManagement>

<dependencies> <dependency> !<groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> </dependency></dependencies>

Parent pom

Child pom































































































































Maven tipsKeep it automatedPlan your release strategy

Use a Repository Manager

Automatic snapshot deployments

Automated releases

Maven tipsMaven best practices for CI buildsUse batch mode (-B)

Always check or snapshot updates (-U)

Use a repository per project

Print test failures to stdout (-Dsurefire.useFile=false)

Maven tipsKnow when to script itGroovy or Ant scripting is easy in Maven

Call external scripts when appropriate

Maven tipsKnow when to script itIt’s pretty easy in Maven 2...<project> <build> <plugins> <plugin> <groupId>org.codehaus.groovy.maven</groupId> <artifactId>gmaven-plugin</artifactId> <version>1.0-rc-5</version> <executions> <execution> <phase>compile</phase> <goals> <goal>execute</goal> </goals> <configuration> <source> println "Hi there I’m compiling ${project.name}" </source> </configuration> </execution> </executions> </plugin> </plugins> </build> ...

























































































































Maven tipsKnow when to script itIt’s even easier in Maven 3...project { build { $execute(id: 'compilation-script', phase: 'compile') { println "Hi there I’m compiling ${project.name}" } $execute(id: 'validation-script', phase: 'validate') { println "Hi there I’m validating ${project.name}" } ... }}







































































Want to learn more?

Java Power Tools BootcampLondon - October 13-17 2010

Thank You

John Ferguson SmartWakaleo Consulting

Web: http://www.wakaleo.comEmail: [email protected]: wakaleo





Zen and-the-art-of-build-script-maintenance-skillsmatter

Technology

Transcript of Zen and-the-art-of-build-script-maintenance-skillsmatter