Overview

Part 2 of Agile testing of JIRA plugins: Wired Tests

No Comments

In the last post we – i.e. Thomas Strecker and I – looked at the setup of a JIRA plugin project and how to add and run unit tests. In this post we take a look at “wired tests” which are another test type.

Wired Test

Atlassian defines the term “wired test” as a test which is executed inside a running host application e.g. JIRA. Technically the test classes are bundled in a separate test plugin which is deployed into the running host application. A special test runner (AtlassianPluginsTestRunner) is required which contacts the plugin and triggers the test executing from the VM where JUnit is executed. The test results are sent back to the source VM. This approach has several pros and cons. First the advantages:

  • The tests are run in a way very similar to the production system.
  • You can inject dependencies through the constructor of the test class.
  • You will get a proper stack trace in case of an error.

But there are also disadvantages:

  • Starting a JIRA instance is slow. This means for example that your integration test on the CI/CD server can easily take more than 10 minutes.
  • The runner will execute all methods of your test class even if you want to run only one method.
  • The wired tests will not contribute to the code coverage of the tests out of the box. We will cover the topic of coverage in a later post.
  • You are restricted in the choice of dependencies. For example you have to use the provided JUnit version used (4.10). Even if you declare another JUnit version such dependencies will not be included in the test plugin.
  • Usually the host application instance is shared between tests. This means that tests have to pay extra attention to avoid changing global state like static fields but also to cope with the data from other tests or (during development where a long running JIRA instance is used) from previous test runs.
  • There is no way to change the variant of a dependency (for example replace the email sender with an implementation which does not send emails but records them) of the implementation code. So wired tests are more system tests than integration tests.
  • Because the test code is added to the OSGi container as a separate plugin the code can only test the exported methods of the plugin under test. This means that the class under test must implement an interface and the test code has to use this interface. In addition the plugin under test has to export all interfacing classes required for the test. This also includes value classes. Since for calls between classes in the same plugin such an interface declaration or defining an export is not required this means extra effort only for the wired test.

Please also note:

  • A running JIRA instance is required. If executed in the integration test phase of Maven such an instance is started and destroyed by Maven. Alternatively the developer can start such an application instance with “atlas-run”. A normal JIRA instance can not be used since special debug plugins are required.
  • When the default setup is used, all wired tests must reside inside the it.* package in order to be properly discovered and executed by the Atlassian maven plugins.

Our opinion

The list of disadvantages is long but understandable for such a test type. The more important question is why is there a need for such a test type. In our humble opinion the reason for the existence of this test type is the limited ability to interact in a test with the base JIRA components without starting the whole platform. To still be able to test the business code without taking the disadvantages from above we recommend to create a facade which encapsulates the JIRA methods used and test the business logic with a mocked facade to avoid using wired tests. However there are still cases where a wired test is the best choice. In the above example it would be the test of the facade.

Sample wired test

In order to execute a wired test, it is necessary to have a plugin descriptor for the test plugin (src/test/resources/atlassian-plugin.xml). When using the Atlassian commands to add components or resources, the corresponding test plugin descriptor is created and updated automatically, otherwise you have to do it manually.

A sample plugin descriptor would look like this (Foo is the interface of the component under test):

<atlassian-plugin key="${project.groupId}.${project.artifactId}-tests" 
    name="${project.name}"plugins-version="2">
  <plugin-info>
    <description>${project.description}</description>
    <version>${project.version}</version>
    <vendor name="${project.organization.name}" 
        url="${project.organization.url}"/>
  </plugin-info>
 
  <!-- from the product container -->
  <component-import key="applicationProperties" 
      interface="com.atlassian.sal.api.ApplicationProperties"/>
 
  <!-- from the plugin under test -->
  <component-import key="foo-component" interface="com.example.Foo"/>
</atlassian-plugin>

The test itself is then similar to a Unit test, with the exception that its constructor gets all relevant components as parameters and the special test runner is used:

@RunWith(AtlassianPluginsTestRunner.class)
 
public class FooWiredTest {
  private Foo component;
 
  public FooWiredTest(Foo component) {
    this.component = component;
  }
 
  @Test
  public void test_foo_saves() {
    component.setValue("myTestValue");
    assertEquals("myTestValue", component.getValue());
  }
}

Running wired tests

You can run wired tests as part of the build process by using the command “atlas-integration-test”. It is also possible to use the standard maven commands which trigger the integration tests, e.g. “mvn integration-test” or “mvn verify”.

It is also possible to run wired tests against a running instance by using the command “atlas-remote-test”, which needs additional parameters, however, for the server to test against. If you are running the test instance with the default setup, the command should be “atlas-remote-test –server localhost –http-port 2990 –context-path /jira”.

In order to run a wired test from the IDE you have to first launch the test instance (e.g. “atlas-run” or “mvn jira:run”). This will install the current version of the plugin and the test plugin into the test instance. In addition, it is necessary for running the test to set the system property “baseurl” like for example via “-Dbaseurl=http://localhost:2990/jira” in the test execution.

When running tests from the IDE it is, however, necessary to keep in mind that the actual test code is executed inside the test instance – not the JVM running in the IDE. Therefore, all changes to the test code or the plugin must be made available to the test instance first, before re-running the tests. This can be achieved by using the command “atlas-install-plugin”, which updates the main plugin, or “mvn package jira:install jira:test-install”, which installs both the plugin and the test plugin if the tests have been modified.

Summary

While the idea of running a test inside the host application is nice in theory it is not really convincing in practice. The main drawbacks are that the tests are not as isolated as they should be and that updating the plugin under test and the test plugin is not as fast or easy as it should be.
In a later post we will take a look at FastDev (how it can help but also has drawbacks) which is designed to reduce the time needed for the turn-around. However frontend tests are the topic of the next post.

Comment

Your email address will not be published. Required fields are marked *