Robot Framework Acceptance Tests: Develop With Eclipse, Run With Maven

The Robot Framework is a very versatile tool and never confronted us with unsolvable problems so far. In some regards it still feels a little rough and needs some polishing, to make development and maintenance of automated tests more efficient. One of those areas is the integration into the well known development environment. There is the Robot IDE, but this only covers a fraction of my requirements. The following will show, how easily the Robot Framework and your own Java Keywords can be integrated in Maven and Eclipse.

Robot is started with a script, depending on your operating system this is jybot.sh or jybot.bat (when you don’t use Java Keywords, pybot is enough). With many command line parameters you can control, which tests shall be executed. Before doing so, you want to compile your own keywords and include them into the classpath. Since development is often enough happening on windows, but the continuous integration server is running on Linux, you have two non-trivial scripts to maintain to invoke the robot tests. This can be done much easier:

1. Maven

Instead of maintaining a plethora of start scripts manually, you can execute jybot directly from within maven. The following pom.xml shows the details:

• The maven-dependency-plugin copies all dependant libraries into a folder, which is later included into the classpath. Now you can easily use any third party library.
• With the exec-maven-plugin you can invoke ‘jybot’ in the integration-test lifecycle phase. This requires that jybot is in your PATH of your environment. Luckily, the .sh or .bat can be omitted.
• The classpath is configured to
  

  ${project.build.directory}${file.separator}classes${path.separator}
  ${project.build.directory}${file.separator}dependency${file.separator}*

  This ensures that it will include

  1. The compiled classes of your own Java keywords
  2. All jar libraries that the pom.xml lists as dependency. (The wildcard notation is only available with Java 6!)
• Afterwards, the jybot script is parametrized:
  • -C off turns off the colored output
  • –variable … sets some variables. Developerfriedly defaults are set in the pom.xml’s properties section. Is the environment variable ‘env’ set to ‘ci’, the ci profile will be activated, which changes those variables to make use of the real test system.
  • –variable RESOURCES:${project.basedir}/src/main/robot/resource Setting a RESOURCES variable makes it possible to refer to resources from your testcases simply with ${RESOURCES}/keyword-file.txt.
  • –exclude inprogress Excludes test cases, that are currently in progress. A developer can work on a test case until it is fully working, and can commit parts of his work, without having the test case executed by default. Of course, once the testcase is fully automated and functional, that tag must be deleted..
  • –include author-* In my opinion, all tests should me tagged with an author. This includes all tests that are existing, but also offers an easy way for the developer to limit the test execution to a set of test, that he is currently working on. This could be further improved with maven properties and profiles. There’s currently a bug in Jython that expands any parameter to matching filenames. (Jython Issue 1567) – so there should be no file that matches author-*!
  • ${ROBOT_TESTS} This tells robot which tests to execute. By default this refers to all tests in ${project.basedir}/src/main/robot/suite. By using tags and folders you have two independant ways to structure and manage your test cases and select a set for execution. This should be enough flexibility for everybody.

On the long term, it would be nicer to have a full-blown maven plugin for the robot framework, in order to save the awkward script parameterization. Until this is done, the presented solution should be sufficient.

<project>
...
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-dependency-plugin</artifactId>
        <version>2.1</version>
        <executions>
          <execution>
            <id>get dependencies</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>copy-dependencies</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <version>1.1.1</version>
        <executions>
          <execution>
            <id>robot</id>
            <phase>integration-test</phase>
            <goals>
              <goal>exec</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <executable>jybot</executable>
          <environmentVariables>
            <CLASSPATH>${project.build.directory}${file.separator}classes${path.separator}${project.build.directory}${file.separator}dependency${file.separator}*</CLASSPATH>
          </environmentVariables>
          <arguments>
            <argument>-C</argument>
            <argument>off</argument>
            <argument>--variable</argument>
            <argument>SYSTEM_UNDER_TEST:${system-under-test}</argument>
            <argument>--variable</argument>
            <argument>DB_PORT:${db.port}</argument>
            <argument>--variable</argument>
            <argument>RESOURCES:${project.basedir}/src/main/robot/resource</argument>
            <argument>--outputdir</argument>
            <argument>${project.build.directory}/robot</argument>
            <argument>--output</argument>
            <argument>${project.groupId}-${project.artifactId}-${project.version}-output.xml</argument>
            <argument>--log</argument>
            <argument>${project.groupId}-${project.artifactId}-${project.version}-log.html</argument>
            <argument>--report</argument>
            <argument>${project.groupId}-${project.artifactId}-${project.version}-report.html</argument>
            <argument>--exclude</argument>
            <argument>inprogress</argument>
            <argument>--include</argument>
            <argument>author-*</argument>
            <argument>${ROBOT_TESTS}</argument>
          </arguments>
        </configuration>
      </plugin>
    </plugins>
  </build>
  <profiles>
    <profile>
      <id>ci-common</id>
      <activation>
        <property>
          <name>env</name>
          <value>ci</value>
        </property>
      </activation>
      <properties>
        <db.port>1234</db.port>
        <system-under-test>testservername</system-under-test>
      </properties>
      </build>
    </profile>
  </profiles>
  <properties>
    <ROBOT_TESTS>${project.basedir}/src/main/robot/suite</ROBOT_TESTS>
    <db.port>4711</db.port>
    <system-under-test>localhost</system-under-test>
  </properties>
</project>

2. Project Structure

The above quoted pom.xml results in Eclipse in the depicted project structure below – either with the m2eclipse-Plugin or by running the eclipse:eclipse goal of the maven-eclipse-plugin.

3. Starting Robot

Executing the automated tests is now just a matter of executing the appropriate maven lifecycle phase:

4. Selenium Server

If you test Web UIs with robot, it is nice to integrate also the selenium server as external tool into Eclipse:

5. Full Workflow

By means of a concrete Example, a short screen cast should give you an impression, how to develop test cases in eclipse and how to execute them. Since details are hardly seen in the embedded video, there also a fullscreen version of it.