//

Generated Jenkins Jobs and automatic Branch Merging for Feature Branches

7.4.2015 | 7 minutes of reading time

In my current project we are using feature branches to keep the master-branch clean and stable. Development, peer code reviews and also the first pre-integration acceptance tests by the product owner team takes place in the feature branches.

Many projects are using a set of jenkins-jobs to execute builds, run tests and provide code-metrics. Most of these projects are only running these jobs for the master-branch.

Our goal was to reduce the integration risk at the end of a feature branch`s lifecycle. The same Jenkins-Jobs as used for the master-branch should be used for all feature branches. Developers for example should fix their test-problems and ui-test-failures as long as they are working in the branch. As a nice side-effect, the product owner team will have an overview with all branches and their current state. They will not start any acceptance test, until the feature-branch-jobs are green.

Bamboo for example has a feature called plan branches, but there is nothing comparable in Jenkins. If you search the community, a lot of solutions for Jenkins are based on job cloning and mostly depend on a set of jenkins-plugins. There is a nice blog post by zeroturnaround including a thesis which describes a possible solution, but we missed an easy branch-overview for example.

So we implemented another solution based on the Job-DSL-Plugin , which fits our needs:

  • Branch-Dashboard with a clear status for each branch
  • Automatic creation/deletion of jobs, no manual work
  • version-controlled job-definitions
  • reduced fix-time in master-branch after integration
  • reduced effort for master-to-feature-branch-merges (see tipps & tricks: automatic merges)

What we did

  1. Express your jobs in DSL-Scripts
    We always wanted to have our job configurations in our SCM. Changes should be comprehensible and version controlled. So we used the Job-DSL-Plugin and started to express our Job-Definitions in DSL-Scripts.
    1// basic example-job, which checks out sources from mercurial,
    2// runs a maven build and sends mails
    3job {
    4    name('build.job')
    5    logRotator(-1,3)
    6    scm {
    7    hg('http://mercurial.example.com/project123/','default') 
    8    }
    9    triggers { scm('* 7-20 * * 1-5') }
    10    steps {
    11    maven {
    12        rootPOM('parent/pom.xml')
    13        goals('clean install -T 1C')
    14        property('skipITs','true')
    15        property('maven.test.failure.ignore','true')
    16    }
    17    }
    18    publishers {
    19    mailer('devs@example.com',true,false)
    20    archiveJunit('**/target/surefire-reports/*.xml')
    21    }
    22}
    23
  2. Set up the seed job
    The seed-job will create the jobs based on your DSL-Scripts. You can use this Tutorial , which describes the creation of the seed-job. But instead of using a provided script, we used the “Look on Filesystem”-option with a regular expression to use our scripts from our SCM.

    Build-Step-Definition for the seed-job


    The seed-job is triggered by SCM-changes in our master branch and additionally one time per day. That’s needed if nobody is working in the master branch for example, but a new branch is added.
  3. Get a new-line-separated file containing your branches
    To create jobs for all branches, we need to know, which branches exist. We are using a small Shell-Script to create a new-line-separated text-file containing the branch names. It runs as an additional build-step in the seed-job. We are using Mercurial, and Mercurial is running on the same server. So we can just change to the mercurial-repo and ask mercurial.
    1WORKING_DIR=$PWD
    2cd /opt/mercurial/hg-repo
    3hg branches | column -t | awk '{printf "%s\n",$1}' | sort > ${WORKING_DIR}/branches.txt
    4cd $WORKING_DIR
    5

    Shell-Step in seed-job


    All our scripts are also stored in our SCM, thats why the script has it’s own file. You could also write the commands directly in the shell step box.

  4. Set up your jobs to be generated for every branch
    Now we have a basic build job and a file containing all branches. Now we extend the build-job-DSL to run for every branch.
    1// read the text-file containing the branch list
    2def branches = []
    3readFileFromWorkspace("seed-job","branches.txt").eachLine { branches << it }
    4 
    5// for every branch...
    6branches.each {
    7    def branchName = it
    8 
    9    job {
    10        // use the branchName-variable for the job-name and SCM-configuration
    11        name("branch.${branchName}.build.job")
    12        // ...
    13        scm {
    14        hg('http://mercurial.example.com/project123/',"${branchName}") 
    15        }
    16        // ...
    17    }
    18}
    19

    The job-console should look like this:

    1Processing DSL script build.groovy
    2Adding items:
    3    GeneratedJob{name='branch.featureFoo.build.job'}
    4    GeneratedJob{name='branch.featureBar.build.job'}
    5

That’s it. Now a build-job for every branch is created automatically. The Jobs are also deleted automatically, after the branch is merged back to the master branch. There is no manually work needed.

Some tipps & tricks, if you like to implement something like this:

  • Use a repository cache
    We are using Mercurial and every job would create an own repository copy for each branch. But you can set up the Mercurial-Jenkins-Plugin to “Use Repository Caches” and “Use Repository Sharing”. So make sure to enable this in the global Jenkins settings.
  • Create views
    If you have many branches and/or create multiple jobs for each branch, it might be a good idea to create some views to sort your jobs. The Job-DSL-Plugin can also generate views.
    1// example-view containing all jobs starting with "branch"
    2view(type: ListView) {
    3    name 'Builds per Branch'
    4    jobs { regex("branch.*") }
    5    columns {
    6    status()
    7    weather()
    8    name()
    9    lastSuccess()
    10    lastFailure()
    11    lastDuration()
    12    buildButton()
    13    }
    14}
    15
  • Maybe use choice parameters
    We have a job, which is used by our product owner team to deploy the branch they want to test on one of the test servers. We don’t generate a deploy job for each branch, instead we are using a job parameter for the branch name. We can easily reuse our text-file containing the branch-names to provide a Select-Box.
    1def branches = []
    2readFileFromWorkspace("seed-job","branches.txt").eachLine { branches << it }
    3 
    4job {
    5    name('deploy.test')
    6    parameters {
    7        // create a parameter "BRANCH", containing all entries from "branches"
    8        // default-value is the first entry from "branches"
    9        choiceParam('BRANCH',branches,'Which branch do you want to deploy?')
    10    }
    11    scm {
    12        hg('http://mercurial.example.com/project123/',"$BRANCH") 
    13    }
    14    //...
    15}
    16
  • Think about automatic merges
    After we implemented the first jobs, we thought about automatic merges from our master branch to the feature branches. In our project, the Interfaces to some needed subsystem are changed quite often and the project-code to access this subsystems is always fixed in the master branch. So the product owner team was always frustrated, when they decided to start a test of a feature branch which was currently not working against the changed interfaces. We spent a lot of time with no-brainer-merges into feature-branches. Based on the techniques described above, we started to implement an automatic merge. Now every change in our master branch will get merged into all feature-branches if there is no merge-conflict. The merge is done by a small shell script. The script is called by a job, which is generated by the Job-DSL-Plugin for every branch. If there is a merge-conflict, a developer has to do the merge manually. The goal of every project should be, that there are not so many open feature branches. But if this goal is not reachable for some reasons, you might think about automatic merges.
    1# Job-DSL Automerge
    2 
    3def branches = []
    4// we use another file here, to filter some branches which should not get automerged
    5readFileFromWorkspace("seed-job","branchesAutomerge.txt").eachLine { branches << it }
    6 
    7branches.each {
    8    def branchName = it
    9 
    10    job {
    11    name("branch.${branchName}.automerge")
    12    triggers { cron('H 5 * * 1-5') }
    13    wrappers {
    14        environmentVariables {
    15        env('BRANCH', "${branchName}")
    16        }
    17 
    18        // we are using a single repository for the automerge-jobs. So we have to be sure, that only one job is using the repository
    19        exclusionResources('AUTOMERGE_REPO')
    20    }
    21    steps {
    22        criticalBlock {
    23        shell(readFileFromWorkspace('parent/jenkinsJobs/scripts/automerge.sh'))
    24        }
    25    }
    26    }
    27}
    28
    1# automerge.sh
    2 
    3# Jenkins uses "-e" parameter, but we want to handle the exit-code manually
    4set +e
    5 
    6WORKING_DIR=$PWD
    7cd /var/lib/jenkins/repoAutomerge
    8 
    9# reset local changes
    10hg update -C .
    11# get changes from repository
    12hg pull
    13# update to branch
    14hg update -C ${BRANCH}
    15 
    16# try the merge
    17hg merge develop --config "ui.merge=internal:merge"
    18mergereturn=$?
    19 
    20case $mergereturn in
    21    0) 
    22        echo '##################################'
    23        echo '#####   Merge successfully   #####'
    24        echo '##################################'
    25 
    26        # commit and push
    27        hg commit -m 'Automerge' -u 'AutoMerger'
    28        hg push
    29 
    30        rc=0
    31        ;;
    32    1) 
    33        echo '####################################################'
    34        echo '#####   Merge-Conflict, manual merge needed!   #####'
    35        echo '####################################################'
    36        rc=1
    37        ;;
    38    255) 
    39        echo '############################################'
    40        echo '#####   No Changes (Return-Code 255)   #####'
    41        echo '############################################'
    42        rc=0
    43        ;;
    44    *) 
    45        echo '###############################################'
    46        echo "#####   Merge-Returncode : $mergereturn   #####"
    47        echo '###############################################'
    48        rc=1
    49        ;;
    50esac
    51 
    52# reset local changes
    53hg update -C .
    54 
    55exit $rc
    56

share post

Likes

0

//

More articles in this subject area\n

Discover exciting further topics and let the codecentric world inspire you.

//

Gemeinsam bessere Projekte umsetzen

Wir helfen Deinem Unternehmen

Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.

Hilf uns, noch besser zu werden.

Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.