ContinuumA World Class Continuous Integration

Tool Stack

April 18, 2023

2

What is Continuous Integration?

• Continuous Integration is generally defined as:

– a software development practice where members of a team integrate their work frequently

– usually each person integrates at least daily - leading to multiple integrations per day.

– each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.

• What does this really mean?

– Instead of going two months, with multiple developer branches and having to do a painful, error prone merge process at the end, changes are merged every day, by the person who wrote the code.

– Breaks to the build, tests, or divergence in design philosophy are spotted quickly and early

3

What is Test Driven Development

• Test Driven Development is generally defined as:

– a development technique where unit tests are written before the actual functional code

– all function units can be exercised and tested in an repeatable manner

• What does this really mean?

– At any time, any piece of code, software module or entire product can be verified by running the unit test(s) associated with it

– It provides early feedback to the developer that• The code they have written is correct• The code they have changed is still correct• They have not unintentionally broken another part of the system

– Any break of a unit test immediately highlights a problem

4

CI and TDD create better quality software

• Employing CI and TDD together ensures that

– You will always know when someone has broken the build

– You will always know when someone has broken a unit test

– You will always know when someone has broken the functional

logic of your code/module/application

– A developer’s code base never diverges too far from the rest of

the team

• Identifying problems as early as possible in the development

process leads to higher quality software products that are cheaper

to develop

ContinuumBeyond Continuous Integration:

Continuous Project Insight

6

What is Continuous Project Insight?

• We live in a networked, hyperlinked world• Our development tools should do too

• Continuous Project Insight is:– A step further than purely integrating code and tests– All tools and the information they provide is interlinked– Knowledge and information is easy to share and navigate

through– Project reports and documentation is always current and

accurate– Developers and management can connect to the heartbeat of a

project with minimal effort

7

What is Continuum?

• Continuum is:

– a world class CI and TDD tool stack

– created to initially support the NextGen project

– assembled from cheap or Open Source components

– primarily focused on (but not limited to) Java development

• It supports:

– Automated builds, unit tests, code coverage, project documentation, etc

– Integrated SCM, Knowledge capture, Issue tracking, Peer Code review

– Access is via• All major web browsers• From within developer tools

– Notification via email, RSS, IM client

8

Why Continuum?

• The Goals for Continuum were:

– Create the best possible CI/TDD tool stack possible

– Be a demonstrable experiment of what CI/TDD can and should

be

– Enable and encourage developers to adopt CI/TDD practices

– Enable managers and project leaders to easily view project

status

– Provide a mechanism for tying all aspects of the development

process into a coherent, interlinked whole

9

Continuum Tool Stack

• The CI/TDD tools used in Continuum are:

– Subversion – Source code repository

– Fisheye – Web based source repository viewer

– Pulse – Continuous Integration build server

– Jira - Issue tracking software

– Confluence Wiki – Knowledge sharing/collaboration

– Crucible – Web base peer code review

– OpenLDAP – user credential management

– Clover – code coverage reports and metrics

– Ant – build system

– testNG – Unit test framework

– yDoc – Javadoc + UML documentation

10

Continuum Tool Relationships

11

Subversion• Source Code Control System• Like CVS, is a Concurrent Versioning System• The bedrock of all the CI stack

– All tools support/integrate with it • Encourages continuous integration in developers

– I.e. developers have local copy and check in frequently– Discourages multi-branch explosion

• No performance issues – everyone has a local copy• Developers can work in disconnected manner

• Works over the VPN – accessed via port 80 • It’s free and open source• All tools to support using it are free• Integrates into the explorer shell (TortoiseSVN)• The manual and books are free• Support for it on all OS platforms• Support for it in all major IDEs

12

Fisheye

• Web based Subversion repository viewer• Supports

– Browsing of project files via the web– Multiple repositories– Related checkins are grouped into changesets– Viewing diffs of file versions– Extensive search capabilities– Displays annotated views of source files (i.e. who changed what)– Historical display of source churn– Graphing of line counts (project, dir, file)– Graphical display of a file’s branch/tag relationships– Linking to issue tracking systems– RSS feeds for change notifications at any level of repository

13

Pulse

• Continuous Integration Build server• Manages continuous integration builds• Triggers any time a developer checks in code• Executes automated

– builds, unit tests, documentation generation, code coverage– and any other artifact we need automated

• Supports RSS, email and Jabber (IM) notification• Links to issue tracking and SCCS viewers• Supports distributed agent builds• Supports scheduling• Has remote API (xml-rpc)• Can build with Ant, Maven, Make or any arbitrary binary

14

Jira

• Issue tracking system• Multi project support (i.e. different repositories)• User configurable dashboard • Fine grained permissions model• Customizable workflow models• Sub-tasks• Time tracking• Configurable reports• Links to Confluence, Subversion and Fisheye• Notification via email, RSS feeds• Export to xml, excel• Remote API• Plugin extension architecture• …

15

Confluence

• The Wiki– Used for knowledge capture, sharing and collaboration– New pages– Threaded comments– News items– Attachments

• WYSIWYG editing of content (as easy as using Word)• Maintains revision history (just like a SCCS)• Fine grained permissions on per space or page• Notification via Email and RSS feeds• Powerful search• Extensive set of user contributed plugins

– Managed from Plugin Manager within Confluence• Dynamically pulls content from Jira• Links to Fisheye (or any system that pushes RSS)• Remote API • …

16

Crucible

• Web based tool for peer code review• Easy to manage review process through web interface• Dynamically pulls changeset information from Fisheye• Workflow notification via Email and RSS• Review information captured in-situ via web browser• Ability to add inline, threaded comments at the review, file, and line

level. • Full auditing capability with historical review access • Annotated view of repository artifacts with inline review comments• Summary reporting of review statistics and metrics • Current running version 0.7. Extensive set of features still to be

added in the future.

17

Clover

• Code coverage analysis tool• Measures effective coverage of unit tests• Fast, accurate, configurable, detailed coverage reporting of Method,

Statement, and Branch coverage • Accurately measures per statement coverage, rather than per-line

coverage • Creates historical trending graphs of code coverage• Full IDE integration• Reports in HTML, PDF, XML or a Swing GUI

18

Coding tools

• Intellij– Preferred IDE– Huge selection of plugins– Most powerful support for power coding– Supports (via plugins) Subversion, Clover, Jira, Ant, TestNG

• Ant– Used for building everything– Lowest common denominator– Industry standard– IDE Support

• testNG– Unit testing framework– Created to solve many deficiencies in JUnit– IDE support

• yDoc– Javadoc extension/replacement– Automatically creates UML diagrams from source at build time– Customizable colors

Implementing Continuum

20

Physical Implementation*

• Hardware:– Single 2U machine– Dual CPU, 4GB Ram– Suse Enterprise Linux 10– Raid 1 disk configuration

• Software (not including tools)– Apache2 web server serves as gateway– AJP or mod_proxy used to pass through to other servers– OpenLDAP – Mysql database– Custom user management web application

• Account creation, verified from corp ActiveDirectory server

• Backups– Nightly backups of all important data– Backed up locally, and to two offsite locations– Rolling 14 day backups

21

Total Cost

• Manpower

– 3 man weeks

• Hardware

– Scavenged box – effectively free

• SoftwareJira $4800

Confluence $4000 (but no cost to this project)

Pulse $1300

Fisheye $1500

Clover $2500

yDoc $1490

Crucible $0 (not a 1.0 GCA product yet)

Total $11590