The Listener Interface of the Robot Framework

Having an eye on the Robot’s nuts & bolts…

Sooner or later there is always this single question – the question regarding debugging Robot. Whether during an introduction or in a training, suddenly it’s always “How do I debug this?”

Well, you might say that one should not (need to) debug any automated test suite at all as your very detailed tests will always tell you where the problem is. But that is an illusion. We all know the tests that fail just because an unexpected modal pop up prevents any focus switch or because a required UI element appears too late. That’s the point when you just want to peek what’s going on there.

And this is possible with the Robot Framework, too. The magic words are Listener Interface.

The Robot Framework provides an interface to integrate listener implemented as classes or modules. Listeners needs to be registered during the start up of the framework and messages will be sent to all listener covering important events. Listeners are called synchronously and block the execution of the test suite.

Listeners can be implemented in Java or Python. The Robot Userguide includes an overview of all events respectively all valid hooks. Lucky enough you just have to implement the methods / functions you are actually interested in.

Implemented listeners need to be in the Robot Framework’s path during runtime, either by putting them in the module search path or by using paths (relative, absolute) upon registration.

To register a listener you have to use the --listener cómmand line option when starting the Robot Framework. This is also the place to put arguments as we will see later.

A very basic listener

Let’s start with an example: We would like the test run to be interupted right before a new test case is started. The test case execution shall continue after clicking a button in a dialog.

First our very complex and realistic test suite:

File name: SimpleTest.txt

1*** Test Case ***
2Simple Test Case
3  [Documentation]  A simple test case as listener demo
4  ${test_var}=  Set Variable  OK
5  Should Be Equal  ${test_var}  OK

Now we need a listener that opens a dialog and waits for a button click when a new test case is about to be executed. Therefore we create a Python module that implements the start_test function. start_test has two parameters – the name of the test case as a string and the test case’s attributes as a dictionary. What attributes are passed over in the dictionary is documented in the userguide.

File name: PleaseWait.py

1import tkMessageBox
2from Tkinter import Tk
3 
4 
5ROBOT_LISTENER_API_VERSION = 2
6 
7def start_test(name, attributes):
8    Tk().withdraw() # Remove root window
9    tkMessageBox.showinfo("Please click 'OK'...",
10                          "Test Case Name: " + name)

Now we can start Robot with the listener included and have a look at the result.

1pybot --listener PleaseWait SimpleTest.txt

As the result the following dialog should be shown:

Listener with arguments

This approach falls short when you have more than one test case in your test suite. Nobody wants to click “Ok” 20 times just to get to the desired test case. In this situation listeners configured with arguments come in handy.

When using listeners with arguments it is necessary to switch from module to class when implementing in Python. Arguments will be passed to the class’s constructor. Thus the listener class must implement an __init__ method that accepts all expected parameters in the correct order. In our situation we just need a single parameter, the test case name.

Please note: As arguments are optional the parameters of the __init__ method should always have sensible default values.

File name: PleaseWaitWithArg.py

1import tkMessageBox
2from Tkinter import Tk
3 
4 
5class PleaseWaitWithArg():
6    ROBOT_LISTENER_API_VERSION = 2
7 
8    def __init__(self, test_name = ''):
9        self.test_name = test_name
10 
11    def start_test(self, name, attributes):
12        if (name == self.test_name) or ('' == self.test_name):
13            Tk().withdraw() # Remove root window
14            tkMessageBox.showinfo("Please click 'OK'...",
15                                  "About to start test '%s'" % name)

This listener waits before the given test case is started (respectively any test case if the argument is omitted).

To check the functionality we have to add at least one more test case to our test suite…

File name: TwoTests.txt

1*** Test Cases ***
2Simple Test Case
3  [Documentation]  A simple test case as listener demo
4  ${test_var}=  Set Variable  OK
5  Should Be Equal  ${test_var}  OK
6 
7Additional Test
8  [Documentation]  Don't be evil...
9  Log  Don't panic!

Arguments are passed to the listener upon registration. They are separated from the listener’s name by colons. Each additional argument is appended to the other by an additional colon.

1pybot --listener PleaseWaitWithArg:"Simple Test Case" TwoTests.txt

If everything works as expected the test execution is interupted before the first test case but not before the second test case.

Please note: double quotes are used just to mask the spaces in the test case name. Generally there is no need to enclose arguments in quotes.

You should leave out the argument and see what happens when using the listener without it!

Showing variables via listeners

But listeners can do more. It is possible to use other test libraries within a listener. By using Robot’s “BuiltIn” library you can access a variable’s current value.

File name: ShowVariable.py

1import tkMessageBox
2from Tkinter import Tk
3from robot.libraries.BuiltIn import BuiltIn
4 
5 
6ROBOT_LISTENER_API_VERSION = 2
7 
8def end_test(name, attributes):
9    Tk().withdraw() # Remove root window
10    tkMessageBox.showinfo("Please click 'OK'...",
11                          "test_var = '%s'" % BuiltIn().get_variables()['${test_var}'])

When executing this…

1pybot --listener ShowVariable SimpleTest.txt

… the following result should be shown:

And how to continue from here?

These examples should show you what the listeners of the Robot Framework can do. For sure someone thought just four paragraphs ago “Hey, it should be pretty easy to build a real debugger with those!” Before everbody starts hacking on the console and creates a new repository on GitHub: Someone already did so – have a look here . I will write a dedicated blog entry about how to use this remote debugger for Robot later.

1import java.util.Map;
2import javax.swing.JOptionPane;
3 
4public class PleaseWaitJ {
5    public static final int ROBOT_LISTENER_API_VERSION = 2;
6 
7    public void startTest(String name, Map attributes) {
8        JOptionPane.showMessageDialog(null, "Test name: " + name);
9    }
10}

1import java.util.Map;
2import javax.swing.JOptionPane;
3 
4public class PleaseWaitWithArgJ {
5    public static final int ROBOT_LISTENER_API_VERSION = 2;
6 
7    private String testName;
8 
9    // Empty constructor in case argument is omitted...
10    public PleaseWaitWithArgJ() {
11        this.testName = "";
12    }
13 
14    // Constructor taking the expected argument...
15    public PleaseWaitWithArgJ(String testName) {
16        this.testName = testName;
17    }
18 
19    public void startTest(String name, Map attributes) {
20        if (name.equals(testName) || ("".equals(testName)))
21            JOptionPane.showMessageDialog(null, "Test name: " + name);
22    }
23}

1import java.util.Map;
2import javax.swing.JOptionPane;
3 
4import javax.script.ScriptEngine;
5import javax.script.ScriptEngineManager;
6import javax.script.ScriptException;
7 
8public class ShowVariableJ {
9    public static final int ROBOT_LISTENER_API_VERSION = 2;
10 
11    public void endTest(String name, Map attributes) throws ScriptException {
12        ScriptEngine engine = new ScriptEngineManager().getEngineByName("python");
13        engine.eval("from robot.libraries.BuiltIn import BuiltIn");
14        engine.eval("v = BuiltIn().get_variables()['${test_var}']");
15        Object testVar = engine.get("v");
16        JOptionPane.showMessageDialog(null, "test_var = " + testVar);
17    }
18}