How to grow a sustainable software development process (for scientific software)

C. Titus Brown6/8/15

Questions and suggestions and skepticism welcome!

The khmer project

• ~13k lines of C++, ~7k lines of Python• About 8 years old• ~5 regular developers, ~60 contributors total• Formal dev/contrib process (khmer.rtfd.org)

• We took a circuitous path… see

“Walking the talk”http://dx.doi.org/10.6084/m9.figshare.791567

Example

(Creating a new pull request)

A simple three step process.

1. Use version control and create branches for each feature set.

2. Make up a checklist to fill out at each merge.

3. Adhere to the checklist :)

Version control

With distributed VCS like git or mercurial, there is no good reason not to use version control

even for small personal projects.

Version control will save your bacon.

A checklist

What goes on the checklist?!

Ideas:• Two-person merge rule;

• “I ran the tests, and they succeeded”• “Someone else ran the tests, and they

succeeded”• “A computer ran the tests automatically, and

they succeeded.”

What goes on the checklist?!

Ideas:• Code formatting guidelines;• Code coverage guidelines;• Spellcheck;• Changes described in ChangeLog;• Commit messages make sense;• Specific feature checks (“command line scripts

should be documented”; “CPython objects should have XXX”);

Important thing about these rules:

Target things that are causing you pain.

• There are an infinite number of edges to sand down on any reasonably involved software project!

• Focus on the painful edges.

(Running the tests.)(Continuous Integration)

Testing

I don’t personally do Test Driven Development, although I’m happy to work with people who do.

“Test-Enhanced Development” –

1. Write really simple tests that cover most of the new code, along with any tricky bits that you were worried about when writing the code (~TDD-lite).

2. Then do SDD.

“Stupidity Driven Development”

• Wait until something breaks (user report, bug found, whatever).

• Write a test that triggers that bug.• Then, and only then, fix the bug.

That exact bug will never reappear guise in the codebase :)

Code coverage

Use code coverage as a guide to writing new tests.

If a line if code isn’t executed by your tests, then it’s not tested by your tests.

(Just because it’s successfully executed by your tests doesn’t mean it’s correct!)

Random advice

Automate your tests

• Anything that isn’t automated won’t be run regularly, if at all.

Keep many tests lightweight

• Only tests that run quickly will be run regularly.

• E.g. Save obscure cross-platform tests for releases.

Set up continuous integration, maybe?

• Very useful for cross-platform testing, integration testing…

• …but people should be running the tests before they commit, and certainly before they push.

Have a release process, if you do releases.

…yet another checklist.

Releases are usually done infrequently, so remove need for good memory.

Never rewrite both your tests and your code at the same time.

Otherwise you’ll not only invent exciting new bugs, but you’ll reimplement boring old ones!