Continuous Integration with Jenkins and Launchpad

This post will describe how we are using Jenkins and Launchpad together to achieve Continuous integration.

Right now we are running our tests right after the code is (manually) reviewed and just before pushing them to trunk. We refer to this part of the process as autolanding and there are tools (tarmac) which can automate the process. However the manual process of code review can take quite some and our jenkins jobs are not triggered until the review is ready.

We would like to run tests much sooner, though. Ideally the workflow would be as follow:

1. A new Merge Proposal in launchpad is created
2. Jenkins job is triggered and starts the defined testing on defined configurations (e.g. tests on precise-i386, precise-amd64, quantal-i386, quantal-amd64, etc)
3. Once all the tests finished the merge proposal is updated with the test result and a link to the jenkins job

In our case we are using libvirt and VMs to achieve a clean and isolated test environment. Therefore we are doing some post-test clean-up actions (reverting the VM to a predefined snapshot).

Our jenkins setup looks roughly like this.

In this case there are jenkins jobs:

1. unity-ci – job which gets triggered as soon as a new merge proposal exists for lp:unity and does the testing
2. generic_update-mp – parametrized job which will take test result, merge proposal and few other parameters and updates the merge proposal in launchpad with the result.

Flowchart for the whole process:

The setup for autolanding is very similar:

The only difference is that

1. unity-autolanding job is triggered when a merge proposal is approved
2. generic-land job will also merge the branch from merge proposal to trunk and does the actual push to trunk

The flowchart is slightly more complicated as there are more possible failure points (e.g. the testing might pass but the autolanding will fail because we require non-empty commit message and the commit message was not specified):

Right now we have a prototype in Python using a ScriptTrigger plugin for jenkins which is implementing all of this. I will describe in the next post how use it in practice together with some examples.