Overview

Tutorial: Jenkins Plugin Development

7 Comments

Some time ago I thought about developing a plugin for Jenkins. Inspired by the Performance plugin, I wanted to develop a plugin for AppDynamics, making use of the REST interface it provides. It would enable to execute performance tests in e.g. an acceptance environment, and fetch certain performance measurements via the AD REST interface by supplying the start and end time of the test. My adventure however would be more of a challenge than I anticipated beforehand.

I started off trying to modify the Jenkins Performance plugin, this would be a troublesome exercise. The Performance plugin is based on reading files generated by JMeter. But because we can use a REST interface, less parsing and saving of files is necessary.

screen-capture-appdynamics-overall-responsetime

Marcel Birkner already did a great post about a Jenkins plugin for Nexus, which I’m not going to repeat. The Jenkins Wiki is also a good starting point. There are also some other tutorials you can find on the Internet, I found them somewhat limited however, so I’ll try to go more in-depth and hopefully add another valueable tutorial.

The sources can be found on GitHub: AppDynamics Jenkins Plugin

Project Setup

The following will be a summary, for more detail see Marcel’s tutorial.

First create the Maven project:

$ mvn -cpu hpi:create

To easily test the plugin, I wrote the following bash script (in case you’re using Linux or Mac), named run-fast.sh:

#! /bin/sh
rm -rf work/plugins
mvn -Dmaven.test.skip=true -DskipTests=true clean hpi:run

I wanted to easily reload the plugin while developing, without restarting Jenkins every time. Unfortunately I haven’t found a way that the Jetty container / Jenkins will correctly reload the updated class files for the plugin. If you find a way (other than JRebel), please let me know. In the mean time, after code changes are made, exit the Jenkins run script (^c) and restart the script again.

Necessary Plugin Objects

For your plugin to hook into the Jenkins build system, certain classes need to be extended so that they are discovered automatically. I will first give an overview of the main classes our plugin is using and will later describe them in more detail.

  • AppDynamicsResultsPublisher  – extends –  Recorder
    The Recorder is a specific BuildStep intended to run after the build completed, collects statistics from the build and marks it as unstable / failed.
  • AppDynamicsBuildAction  – implements –  Action and StaplerProxy
    Actions are exposed and create an additional URL subspace. They can appear e.g. in the left-hand menu of a build and these objects are persisted to disk. By adding the StaplerProxy interface we can point to a different ModelObject which is our BuildActionResultsDisplay.
  • BuildActionResultsDisplay  – implements –  ModelObject
    A ModelObject is some object referenced by an URL. It can be referenced e.g. by Jelly pages which we’ll describe later.
  • AppDynamicsProjectAction  – implements –  Action
    Another Action but this time on project level instead of individual build level.

screen-capture-appdynamics-buildresult
To expose the above objects on the Jenkins web interface, corresponding Jelly files are necessary. There are various types and to map them to certain classes (for retrieving the actual data or graphs) the paths need to match. For example to display the outcome of a specific build, we have a Jelly file on the following path:

/nl/codecentric/jenkins/appd/BuildActionResultsDisplay/index.jelly

This file maps to the BuildActionResultsDisplay class (remember, proxied by AppDynamicsBuildAction) and can show its data.
Other types of Jelly files are (which can live besides each other in the same directory):

  • config.jelly – for configuration, mainly on Publisher (our Recorder) level to configure the plugin
  • floatingBox.jelly – can show a floating graph on a build or project page
  • summary.jelly – can show a summary on a build or project page

Now let’s go into more detail…

Publisher

The Publisher object or Recorder is the base of our plugin. It needs a BuildStepDescriptor to provide certain information to Jenkins, but this descriptor also makes it possible to provide defaults for certain configuration fields and even to validate data. Below is a snippet of the descriptor.

  public static class DescriptorImpl extends BuildStepDescriptor {
 
    @Override
    public String getDisplayName() {
      return PUBLISHER_DISPLAYNAME.toString();
    }
 
    public String getDefaultUsername() {
      return DEFAULT_USERNAME;
    }
 
    public ListBoxModel doFillThresholdMetricItems() {
      ListBoxModel model = new ListBoxModel();
 
      for (String value : AppDynamicsDataCollector.getAvailableMetricPaths()) {
        model.add(value);
      }
 
      return model;
    }
 
    public FormValidation doTestAppDynamicsConnection(@QueryParameter("appdynamicsRestUri") final String appdynamicsRestUri,
                                                      @QueryParameter("username") final String username,
                                                      @QueryParameter("password") final String password,
                                                      @QueryParameter("applicationName") final String applicationName) {
      FormValidation validationResult;
      RestConnection connection = new RestConnection(appdynamicsRestUri, username, password, applicationName);
 
      if (connection.validateConnection()) {
        validationResult = FormValidation.ok("Connection successful");
      } else {
        validationResult = FormValidation.warning("Connection with AppDynamics RESTful interface could not be established");
      }
 
      return validationResult;
    }

The methods that return a FormValidation object provide a nice way of validating input, or in this case verifying that the connection to the AppDynamics REST interface is valid. It is coupled to our Jelly file in the following way (snippet from config.jelly):

  <f:entry title="${%appdynamics.rest.username.title}" description="${%appdynamics.rest.username.description}">
    <f:textbox field="username" default="${descriptor.defaultUsername}" />
  </f:entry>
 
  <f:validateButton
      title="${%appdynamics.connection.test.title}" progress="${%appdynamics.connection.test.progress}"
      method="testAppDynamicsConnection" with="appdynamicsRestUri,username,password,applicationName" />

screen-capture-appdynamics-configuration

The “validateButton” gives a separate button that will invoke the doTestAppDynamicsConnection method of our AppDynamicsResultsPublisher class. For validation of fields, it is sufficient to add a doCheckxxx method where xxx maps to the correct parameter name and where you add an @QueryParam annotation to the method input parameter.The descriptor needs to be bound to the corresponding class as follows:

  @Extension
  public static final DescriptorImpl DESCRIPTOR = new DescriptorImpl();
 
  @Override
  public BuildStepDescriptor<Publisher> getDescriptor() {
    return DESCRIPTOR;
  }

Finally, the perform method will be executed whenever a build is started. From here, also the AppDynamicsBuildAction must be instantiated, shown in the following snippet:

  @Override
  public boolean perform(AbstractBuild<?, ?> build, Launcher launcher, BuildListener listener)
      throws InterruptedException, IOException {
    PrintStream logger = listener.getLogger();
 
    AppDynamicsDataCollector dataCollector = new AppDynamicsDataCollector(connection, build,
        minimumMeasureTimeInMinutes);
    AppDynamicsReport report = dataCollector.createReportFromMeasurements();
 
    AppDynamicsBuildAction buildAction = new AppDynamicsBuildAction(build, report);
    build.addAction(buildAction);
 
    ...
 
    Result result;
    if (performanceFailedThreshold >= 0
        && performanceAsPercentageOfAverage - performanceFailedThreshold < 0) {
       build.setResult(Result.FAILURE);
     } else if (performanceUnstableThreshold >= 0
        && performanceAsPercentageOfAverage - performanceUnstableThreshold < 0) {
      result = Result.UNSTABLE;
      if (result.isWorseThan(build.getResult())) {
        build.setResult(result);
      }
    }

Everything written to the logger will show up in the build console output.

Build Action

The most important thing to know about the AppDynamicsBuildAction is to provide a weak reference to our BuildActionResultsDisplay object and return it as target so it can be shown on the build page:

  private transient WeakReference buildActionResultsDisplay;
 
  public BuildActionResultsDisplay getTarget() {
    return getBuildActionResultsDisplay();
  }
 
  public BuildActionResultsDisplay getBuildActionResultsDisplay() {
    BuildActionResultsDisplay buildDisplay = null;
    WeakReference wr = this.buildActionResultsDisplay;
    if (wr != null) {
      buildDisplay = wr.get();
      if (buildDisplay != null)
        return buildDisplay;
    }
 
    try {
      buildDisplay = new BuildActionResultsDisplay(this, StreamTaskListener.fromStdout());
    } catch (IOException e) {
      logger.log(Level.SEVERE, "Error creating new BuildActionResultsDisplay()", e);
    }
    this.buildActionResultsDisplay = new WeakReference(buildDisplay);
    return buildDisplay;
  }
 
  public void setBuildActionResultsDisplay(WeakReference buildActionResultsDisplay) {
    this.buildActionResultsDisplay = buildActionResultsDisplay;
  }

Our BuildActionResultsDisplay will actually expose the data for a certain build. It is fed with a AppDynamics report containing all information for / from the build. The class will parse the data and generate e.g. a graph as shown in the code and corresponding Jelly:

  public void doSummarizerGraph(final StaplerRequest request,
                                final StaplerResponse response) throws IOException {
    final String metricKey = request.getParameter("metricDataKey");
    final MetricData metricData = this.currentReport.getMetricByKey(metricKey);
 
    final Graph graph = new GraphImpl(metricKey, metricData.getFrequency()) {
    ....
    };
 
    graph.doPng(request, response);
  }
       <j:set var="report" value="${it.getAppDynamicsReport()}"/>
      <j:forEach var="metricData" items="${report.metricsList}">
              
      </j:forEach>

Project Action

Finally the AppDynamicsProjectAction is quite similar to the previous objects. One thing that is interesting though is how to get a list of all previous reports, as we would like to show some overall statistics. The following code shows how the AbstractProject can be used to fetch a list of builds and from each build grab the stored AppDynamicsReport object:

  private List getExistingReportsList() {
    final List adReportList = new ArrayList();
 
    if (null == this.project) {
      return adReportList;
    }
 
    final List<? extends AbstractBuild<?, ?>> builds = project.getBuilds();
    for (AbstractBuild<?, ?> currentBuild : builds) {
      final AppDynamicsBuildAction performanceBuildAction = currentBuild.getAction(AppDynamicsBuildAction.class);
      if (performanceBuildAction == null) {
        continue;
      }
      final AppDynamicsReport report = performanceBuildAction.getBuildActionResultsDisplay().getAppDynamicsReport();
      if (report == null) {
        continue;
      }
 
      adReportList.add(report);
    }
 
    return adReportList;
  }

 

Conclusion

What seemed like a quite daunting task eventually (after lots of struggeling) turned out to be quite easy. I hope by writing this blog post that some ideas and principles behind Jenkins become a bit more clear. And without adding too much other stuff, you have gotten a nice overview of which basic classes are necessary for a Jenkins plugin.

Now go and write your own plugin!!

Kommentare

  • NITIN KUMAR SHARMA

    25. June 2014 von NITIN KUMAR SHARMA

    hi Miel,

    i trying to use “appdynamics-dashboard-1.0.1.hpi” in my Jenkin job to fetch the performance metrics after executing my Jmeter test cases.

    i can see in the Jenkins console log that the connection with Appdynamic controller was successful but later while fetching the result “null pointer exception was thrown”…

    Kindly let me know if the below log makes any sense to you:
    ===================================
    Connection successful, continue to fetch measurements from AppDynamics Controller …
    ERROR: Publisher nl.codecentric.jenkins.appd.AppDynamicsResultsPublisher aborted due to exception
    java.lang.NullPointerException
    at nl.codecentric.jenkins.appd.AppDynamicsReport.addMetrics(AppDynamicsReport.java:32)
    at nl.codecentric.jenkins.appd.AppDynamicsDataCollector.createReportFromMeasurements(AppDynamicsDataCollector.java:58)
    at nl.codecentric.jenkins.appd.AppDynamicsResultsPublisher.perform(AppDynamicsResultsPublisher.java:219)
    at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
    at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:772)
    at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:736)
    at hudson.model.Build$BuildExecution.post2(Build.java:183)
    at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:685)
    at hudson.model.Run.execute(Run.java:1757)
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
    at hudson.model.ResourceController.execute(ResourceController.java:88)
    at hudson.model.Executor.run(Executor.java:234)

    • Miel Donkers

      26. June 2014 von Miel Donkers

      Dear NITIN KUMAR SHARMA,

      Thank you for leaving a comment. However I think bugs don’t belong here as part of this blog post. That’s why I created a bug-report in the Github repository, where the source-code lives. You can find it here; https://github.com/jenkinsci/appdynamics-plugin/issues/1

      I will pick it up from there.

      Best regards,
      Miel

  • ashish singh shah

    16. February 2015 von ashish singh shah

    Hi Miel,
    It was nice to read your blog and understand the same from code.
    I too have a similar kind of requirement like AppDynamics plugin. But in my case i need to trigger some performance test using rest call after the build and once that test is complete i need to get the result back and show it.
    My doubt is after triggering the test how can i get the results back since the test may run for half an hour and how to make jenkins wait for that much time.

    • Miel Donkers

      16. February 2015 von Miel Donkers

      Dear Ashish Singh Shah,

      Currently the AppDynamics plugin does not support the behaviour you are describing. To make this possible you would need to fork the plugin and modify it so you can provide the duration afterwards. Right now the duration for which to take the measurements is taken from the build-time. So you would just need to change the plugin to provide the duration in some other way, e.g. via a property.
      I think you can somewhat mimic the behaviour by specifying the minimal duration for monitoring to be long enough and trigger the plugin immediately after your performance test. Then it should get the data you need.

      Unfortunately this is not functionality I foresee for the AppDynamics plugin, and so you would need to build it yourself.

      Best regards,
      Miel

  • ashish singh shah

    23. February 2015 von ashish singh shah

    Got your point, thanks Miel

    • ashish singh shah

      24. February 2015 von ashish singh shah

      Hi Miel,
      Need your urgent help. I am facing one issue. I am not able to see the locally built plugin in “add build step” and “add post build action” drop downs menu of jenkins job. I tried with your appdyanmics[source code from git] and tried with sample say hello builder[code generated using “mvn hpi:create”]. i packaged them in .hpi files and installed them in jenkins and restarted it. i can see these plugins under manage plugins—> Installed section with version as “sanpshot private dated” but when i try to add them as build step or post build action these plugins are not available in the dropdown. then i downloaded .hpi form jenkins wiki after installing them i was able to see them under dropdown menu. So why i am not able to use locally built plugins?

      I am packaging them using maven command “mvn package” and from the target folder using .hpi file. Is there something wrong i am doing while packaging them to .hpi file.

      And can help in how to debug the plugins.

      • Miel Donkers

        25. February 2015 von Miel Donkers

        Hi Ashish,

        I cannot tell why deploying the hpi file under Jenkins won’t work. Instead, have you tried the running/debugging option directly from Maven as also mentioned in the blog post? You can run using this command:

        mvn -Dmaven.test.skip=true -DskipTests=true clean hpi:run

        Via this way, you should see in the list of post-build actions the AppDynamics Performance Publisher. If it doesn’t show up, I suggest carefully going through the Jenkins log output to see why the plugin might not have been initialized.

        Good luck,
        Miel

Comment

Your email address will not be published. Required fields are marked *