Overview

The Listener Interface of the Robot Framework

No Comments

Having an eye on the Robot’s nuts & bolts…

Sooner or later there is always this single question – the question regarding debugging Robot. Whether during an introduction or in a training, suddenly it’s always “How do I debug this?”

Well, you might say that one should not (need to) debug any automated test suite at all as your very detailed tests will always tell you where the problem is. But that is an illusion. We all know the tests that fail just because an unexpected modal pop up prevents any focus switch or because a required UI element appears too late. That’s the point when you just want to peek what’s going on there.

And this is possible with the Robot Framework, too. The magic words are Listener Interface.

The Robot Framework provides an interface to integrate listener implemented as classes or modules. Listeners needs to be registered during the start up of the framework and messages will be sent to all listener covering important events. Listeners are called synchronously and block the execution of the test suite.

Listeners can be implemented in Java or Python. The Robot Userguide includes an overview of all events respectively all valid hooks. Lucky enough you just have to implement the methods / functions you are actually interested in.

Implemented listeners need to be in the Robot Framework’s path during runtime, either by putting them in the module search path or by using paths (relative, absolute) upon registration.

To register a listener you have to use the --listener cómmand line option when starting the Robot Framework. This is also the place to put arguments as we will see later.

A very basic listener

Let’s start with an example: We would like the test run to be interupted right before a new test case is started. The test case execution shall continue after clicking a button in a dialog.

First our very complex and realistic test suite:

File name: SimpleTest.txt

*** Test Case ***
Simple Test Case
  [Documentation]  A simple test case as listener demo
  ${test_var}=  Set Variable  OK
  Should Be Equal  ${test_var}  OK

Now we need a listener that opens a dialog and waits for a button click when a new test case is about to be executed. Therefore we create a Python module that implements the start_test function. start_test has two parameters – the name of the test case as a string and the test case’s attributes as a dictionary. What attributes are passed over in the dictionary is documented in the userguide.

File name: PleaseWait.py

import tkMessageBox
from Tkinter import Tk
 
 
ROBOT_LISTENER_API_VERSION = 2
 
def start_test(name, attributes):
    Tk().withdraw() # Remove root window
    tkMessageBox.showinfo("Please click 'OK'...",
                          "Test Case Name: " + name)

Now we can start Robot with the listener included and have a look at the result.

pybot --listener PleaseWait SimpleTest.txt

As the result the following dialog should be shown:

Listener with arguments

This approach falls short when you have more than one test case in your test suite. Nobody wants to click “Ok” 20 times just to get to the desired test case. In this situation listeners configured with arguments come in handy.

When using listeners with arguments it is necessary to switch from module to class when implementing in Python. Arguments will be passed to the class’s constructor. Thus the listener class must implement an __init__ method that accepts all expected parameters in the correct order. In our situation we just need a single parameter, the test case name.

Please note: As arguments are optional the parameters of the __init__ method should always have sensible default values.

File name: PleaseWaitWithArg.py

import tkMessageBox
from Tkinter import Tk
 
 
class PleaseWaitWithArg():
    ROBOT_LISTENER_API_VERSION = 2
 
    def __init__(self, test_name = ''):
        self.test_name = test_name
 
    def start_test(self, name, attributes):
        if (name == self.test_name) or ('' == self.test_name):
            Tk().withdraw() # Remove root window
            tkMessageBox.showinfo("Please click 'OK'...",
                                  "About to start test '%s'" % name)

This listener waits before the given test case is started (respectively any test case if the argument is omitted).

To check the functionality we have to add at least one more test case to our test suite…

File name: TwoTests.txt

*** Test Cases ***
Simple Test Case
  [Documentation]  A simple test case as listener demo
  ${test_var}=  Set Variable  OK
  Should Be Equal  ${test_var}  OK
 
Additional Test
  [Documentation]  Don't be evil...
  Log  Don't panic!

Arguments are passed to the listener upon registration. They are separated from the listener’s name by colons. Each additional argument is appended to the other by an additional colon.

pybot --listener PleaseWaitWithArg:"Simple Test Case" TwoTests.txt

If everything works as expected the test execution is interupted before the first test case but not before the second test case.

Please note: double quotes are used just to mask the spaces in the test case name. Generally there is no need to enclose arguments in quotes.

You should leave out the argument and see what happens when using the listener without it!

Showing variables via listeners

But listeners can do more. It is possible to use other test libraries within a listener. By using Robot’s “BuiltIn” library you can access a variable’s current value.

File name: ShowVariable.py

import tkMessageBox
from Tkinter import Tk
from robot.libraries.BuiltIn import BuiltIn
 
 
ROBOT_LISTENER_API_VERSION = 2
 
def end_test(name, attributes):
    Tk().withdraw() # Remove root window
    tkMessageBox.showinfo("Please click 'OK'...",
                          "test_var = '%s'" % BuiltIn().get_variables()['${test_var}'])

When executing this…

pybot --listener ShowVariable SimpleTest.txt

… the following result should be shown:

And how to continue from here?

These examples should show you what the listeners of the Robot Framework can do. For sure someone thought just four paragraphs ago “Hey, it should be pretty easy to build a real debugger with those!” Before everbody starts hacking on the console and creates a new repository on GitHub: Someone already did so – have a look here. I will write a dedicated blog entry about how to use this remote debugger for Robot later.

 

Remarks on “API version”

May be you already noticed the variable ROBOT_LISTENER_API_VERSION in my examples. This variable and it’s value define what version of the Listener API will actually be used when calling the listener. Release 2.1 of the Robot Framework introduced heavy changes in the API. To ensure compatibility of existing listeners with the new release the variable ROBOT_LISTENER_API_VERSION was created. When it exists and equals 2 the new API version is used. When the variable is missing the old API version will be used as the default behaviour. There is no reason to use the old API version together with new code (you would have to search the repository for an old documentation first…) but the default behaviour might puzzle you – especially if Python is not your native language or when “writing” some code by adopting Copy & Paste…

Forgetting the API version or putting it in the wrong position (outside the class definition) will result in the listener being called with the old API version creating quite surprising error messages. Thus when stumbling over messages like

[ ERROR ] Calling listener method 'start_test' of listener 'PleaseWaitWithArg' failed: TypeError: start_test() takes exactly 3 arguments (4 given)

just check whether ROBOT_LISTENER_API_VERSION exists, has the right value of 2 and is well placed. In this particular case I moved the variable in front of class PleaseWaitWithArg(): and therefore outside the class definition – a typical “Copy & Paste” error.

 

Remarks on “Java”

Right a the beginning I wrote that listeners can be implemented in Java, too. But how does that look like exactly and what do you have to keep in mind?

First the source code of all three listeners implemented in Java…

File name: PleaseWaitJ.java

import java.util.Map;
import javax.swing.JOptionPane;
 
public class PleaseWaitJ {
    public static final int ROBOT_LISTENER_API_VERSION = 2;
 
    public void startTest(String name, Map attributes) {
        JOptionPane.showMessageDialog(null, "Test name: " + name);
    }
}

File name: PleaseWaitWithArgJ.java

import java.util.Map;
import javax.swing.JOptionPane;
 
public class PleaseWaitWithArgJ {
    public static final int ROBOT_LISTENER_API_VERSION = 2;
 
    private String testName;
 
    // Empty constructor in case argument is omitted...
    public PleaseWaitWithArgJ() {
        this.testName = "";
    }
 
    // Constructor taking the expected argument...
    public PleaseWaitWithArgJ(String testName) {
        this.testName = testName;
    }
 
    public void startTest(String name, Map attributes) {
        if (name.equals(testName) || ("".equals(testName)))
            JOptionPane.showMessageDialog(null, "Test name: " + name);
    }
}

File name: ShowVariableJ.java

import java.util.Map;
import javax.swing.JOptionPane;
 
import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;
import javax.script.ScriptException;
 
public class ShowVariableJ {
    public static final int ROBOT_LISTENER_API_VERSION = 2;
 
    public void endTest(String name, Map attributes) throws ScriptException {
        ScriptEngine engine = new ScriptEngineManager().getEngineByName("python");
        engine.eval("from robot.libraries.BuiltIn import BuiltIn");
        engine.eval("v = BuiltIn().get_variables()['${test_var}']");
        Object testVar = engine.get("v");
        JOptionPane.showMessageDialog(null, "test_var = " + testVar);
    }
}

First you have to say: It really works! But compared to the functional scope of these examples that handling of Java based listeners is rather tricky.

The following points are worth to be mentioned:

  • For sure Robot must be started with Jython (jybot) in these cases.
  • File extensions are ignored during registration of listeners. Thus when mixing Python and Java listeners you might use module / class names twice without noticing. Robot will load the Python code.
  • In contrast to Python code you must compile the Java code before starting Robot.
  • Arguments are optional! Thus you should implement constructors with “missing” arguments as well (see second example).
  • The access to Python based libraries is rather awkward (see third example).

Comment

Your email address will not be published. Required fields are marked *