Robot Framework Tutorial – A complete example

18 Comments

Robot Framework Tutorial

Part I: Robot Framework Tutorial – Overview
Part II: Robot Framework Tutorial – A complete example
Part III: Robot Framework IDE
Part IV: How to Structure a Scalable And Maintainable Acceptance Test Suite
Part V: Robot Framework Tutorial – Writing Keyword Libraries in Java
Part VI: Robot Framework Tutorial – Loops, Conditional Execution and more
Part VII: Robot Framework – Testing Windows Applications
Appendix A: Robot Framework – Compact Sheet


The biggest problem when starting a new project with the Robot Framework is probably the pure amount of possible options it is offering:

  • Using Python, Jython, or Java installation?
  • Which input format should be used to implement the tests (HTML, Text, BDD)?
  • This has direct impact on the question whether or not the Robot IDE (RIDE) can be used?
  • How to enable running the same tests locally and on some CI environment?
  • How to exceute the tests at all (scripting, ANT, Maven)?

So what would be the best way to go? Well, I have seen quite some different Robot Framework setups and I sense a certain urge to overly complicate things. Of course it is cool to start the Robot Tests from Eclipse using Maven (we have blogged on this). And writing tests in BDD-style is certainly more agile than using the HTML-format (funny thing, isn’t it, we have blogged on this as well :-)).

But there is one thing that is really cool: Simplicity! And this does not only mean it is simple once everything is setup and works, but it is also simple to understand a problem when things are not working. This is especially important in teams with different technical background.

So let’s start setting up a simple and bulletproof Robot Framework project …

Some initial Thoughts

But B-4 (any Trekkies around :)) starting to setup a new Robot Framework project it must always be analysed which Test-Libraries are needed for testing the System under Test (SuT). A few examples:

  • Testing a web application? Then one will probably include the SeleniumLibrary or Selenium2Library.
  • There is a database involved? Then one of the Database Libraries (Python or Java) should be considered.
  • Need to test SSH/SFTP? Then SSHLibrary is for sure worth a try.

The list could of course be continued for all the available Test Libraries, up to the point where no ready-made Test Library is available. Then you would need to plan writing your own one. (But this is something to cover in an article of its own.)

But why is this important? The Test Libraries used might have direct impact on whether the Python or Jython/Java installation of the Robot Framework must be used (compare Robot Framework Tutorial – Overview). But again, only because some required Test Library is implemented in Java it could still be possible to use the pure Python installation if the Remote Library Interface is supported by that library. Thus these things should really be well thought of before starting.

Tip: The complete code example for this article can be found from GitHub.

Luckily I have done this thinking already for a hypothetical SuT :-). Let’s imagine that this is a web application that makes use of some database (quite typical use case from my experience). To keep things as simple as possible the tested web application will be Google and the database will simply be a local MySQL database instance. Of course there is then no real relation between Google and this local database, but that does not matter for demonstrating the tests. What is more important is to show how to mix different Test Libraries (in this example SeleniumLibrary and DatabaseLibrary) and make use of the Remote Test Library feature of the DatabaseLibrary at the same time. I cannot really tell why, but I tend to use the Java version from the available database libraries.

“Testing Architecture”

The following figure gives an overview on the “Testing Architecture” (sounds cool, doesn’t it?):

Overview on the "Testing Architecture"

Selenium always requires a running Selenium Server. That could be run on the same machine where the Robot Framework is running, but it can also be any other Server that can be reached via TCP/IP. The same holds true for the Database Library Server. DatabaseLibrary could be used without RPC, but as then we would be forced to use Jython let’s use it as a server as well. Of course locally all these servers will be running on the same machine (namely your PC). In some official testing environment that is typically setup for a project it is often the case that the Robot Framework is running on some Linux box together with the CI-server. Then the Selenium Server is running on some Windows-Server, as you would like to test with browser versions that are close to those used by the end users. The DatabaseLibrary Server could then again run on the CI-Server. That does not hurt.

Test Implementation & Organisation

Finally we are coming to the implementation of the tests. We will not go to all the details in the implementation here, as those can be seen from the sample project. But again let’s more look at some general considerations that should be made when implementing tests using the Robot Framework.

As mentioned in the beginning here a decision is needed which format (HTML, Text, BDD) to use and whether or not RIDE should be used. The usage of RIDE is having direct impact on the choice of the used format. Of course one should also consider the – probably different – technical knowledge of the team members and – one step further – of a maybe different team supporting a potential maintenance phase.

Tip: If there is already testdata available in a tool like Excel this data can be easily Copy & Pasted to your tests when using RIDE.

If I am in charge to make a choice – and in this article I am :) – it is very simple: Use the HTML-format and use RIDE. Here is my reasoning for this:

  • RIDE has made big progress since its beginning and its auto-completion for keywords and guidence in implementing Test Suites and Resource Files is extremely helpful.
  • Using RIDE there is no need to consider writing tests in BDD-style anyway. But honestly there are some syntactical elements required here that I simply do not like. Furthermore I believe it is harder to write/maintain for less technical oriented team members, because there is not yet such a thing like a machine that can understand human language. That is an illusion. I also think that other Testing Frameworks that have started with BDD (and often only supporting BDD) are having an advantage if BDD is the one and most important requirement.
  • The HTML-format is simply great. You can visualize the tests nicely in a Browser and I would say it is quite easy to understand by just looking at it. Especially less technical oriented team members feel at home with the table-style test descriptions as those are looking to some extend familiar from using tools like Excel.
  • There is also a disadvantage in using HTML-format, as merging changes in Source Control Systems can be problematic. But this is a problem that can be tackled.

Of course one of the most important aspects for implementing the tests is to be able to execute the same tests locally and on the official test environment (CI environment). Luckily this can be achieved quite easily with the Robot Framework, as it is possible to pass in parameters to the tests that can then be used in the corresponding keywords. Some typical examples:

  • The starting URL of the web-application to be tested.
  • IP-address and port of the Selenium-Server.
  • JDBC connection string to connect to the database using the Database Library.

These parameters can be collected in variable files. These variable files can then be passed to the Robot Framework in the command line. As we will anyway have different startup-scripts locally and on the CI-environment this can be easily steered. The parameters from those files are then used in the test implementation as variables.

This leads us to some central part of organizing the tests: The directory structure.

Project Directory Structure

Defining a common directory structure – that might even be re-used in different projects – is of course helpful as it is then much easier to work in different projects. The directory structure depicted above is working very well for me since quite some time already (with minor tweak every now and then).

Tip: All path information should be always given relative. For example references from Testsuites to Resource Files or a Testsuite given as a parameter to a robot start script.

First of all we bundle everything below one top-level directory (robot). Then we distinguish between the implementation and the execution of the tests. On the implementation side all Testsuites go to the testsuites-directory and all Resource Files go to the, well I guess you guessed it already. Of course in some really big projects some additional sub-directories might be created to further structure the tests. What is really important that – when referencing any other files (e. g. Resource Files from Testsuites) – all path information is given only relative. This way it does not matter at all on which operating system and in which directory the top-level robot-directory is located. This is especially important as all these files are under some Source Control System and every team member might check out files to different locations.

The execution-branch is used to handle the different target environments. Typically this means a local environment on the development machines of the team and some official CI environment. If more environments would be required additional branches can be implemented here. Quite obviously the scripts-directory is holding all scripts required for execution (robot itself, Selenium Server, Database Library Remote Server) and the settings-directory is holding the specific variable files. It should be noted that especially the scripts are only written once in the beginning of the project and are very rarely changed afterwards. The same is true for the settings as those should only cover differences in the execution environments.

Finally the is the lib-folder. Of course this one can be used to check-in own Test Libraries, but it might also make sense to store ready-made libraries here to ensure they are kept stable. It heavily depends on the project whether or not this folder is required.

Test Execution

When it comes to executing the tests I simply stick to shell-scripts. Easy to understand, working without problems since ages (do I have to feel old saying this?) and easy to use in any CI environment. Of course we will probably have to write two different versions of all those startup-scripts as it is often the case that locally the tests must be started on a Windows PC, but on the official CI-environment some bash– or csh-script is required. But as mentioned beforehand this is a “do once and forget” job and the scripts are not really that complicated.

For the beginning we need three scripts:

  1. One to start the robot tests as such.
  2. One to start the Selenium Server.
  3. One to start the Database Library Remote Server.

Again we could do some fancy stuff here having one script starting everything, but why should we do that. We anyway have to start the latter two scripts only once and afterwards only the tests are re-executed. Let’s better spend that time for implementing the tests. So basically that’s it for the execution.

Putting the pieces together

First of all we need to install the Robot Framework and required Libraries on the development machines. We assume this is Windows. Installation on Unix should be checked from the corresponding tool pages, but should also not be too complicated.

Tip: Robot Framework is not supporting 3.x versions of Python and wxPython (required for RIDE) is having problems with 2.7x version of Python, thus stick to Python 2.6x.

For this example the following versions of the required tools have been used:

Just install the first five artefacts in the given order using the installers (I am using defaults for everything all the time). Then add the following part to your PATH variable: “C:Python26;C:Python26Scripts”. Now you should be able to execute Robot Framework using “pybot –version” and to start RIDE from “C:Python26Scriptsride.py”.

Robot Framework & RIDE after installation

Downloading the Database Library JAR was more like an exercise as this one is bundled (just copied) to the lib-folder of the sample project anyway :).

The Selenium Library is always bundled with a corresponding Selenium Server JAR. To be able to use this one independent from Robot (well, Python) installed to different folders by different team members you have to define a new environment variable RF_HOME pointing to your Python installation directory (default would be “C:Python26″). This variable is used in the local scripts starting the selenium server.

To make use of a local MySQL database the same database settings are used as in the DBLibrary Sample Project. Thus in order for this example to work a local MySQL database must be installed and a testing schema and a corresponding user must be created as follows:

C:xamppmysqlbin>mysql -u root -p
mysql> create database databaselibrarydemo;
mysql> create user ‘dblib’@’localhost’ identified by ‘dblib';
mysql> grant all privileges on databaselibrarydemo.* to ‘dblib';

Of course this will look slightly different on different operating systems. Basically this is all that is required to start executing the sample tests.

All other required files are either already installed together with the Robot Framework (like for example the Selenium Server JAR) or are coming from the project that needs to be checked out from GitHub.

In the directory robot/execution/local/scripts are all the scripts needed to execute the tests. The test implementation can be found from robot/implementation/testsuites. The testcases can be best viewed/edited in RIDE by opening the implementation directory directly. Then all files can be viewed and edited directly.

To start the tests first the Selenium-Server and the DBLibrary-Server must be started. The the Sample Testsuite can be started. Windows batch scripts to start everything can be found from robotexecutionlocalscripts. If everything works out as planned (hopefully it does) all scripts should work without any modifications, as all path information is given relative to the project directory structure. There are no server-side files given, but those can be easily adapted from the local files.

Conclusion & Impressions

We have seen that the Robot Framework is offering a lot of functionality and possibilities to do – maybe even the same – things in different ways. Thus it really makes sense to start with some basic analysis of the required test functionality before starting to hack some tests.

Editing Testsuites and Resource Files in RIDE

Using RIDE makes implementing test functionality easier, especially for less technical oriented team members. Keeping things simple will help in the long run maintaining the project (this is not only true for Robot Framework tests ;-)).

Selenium Server up and running

By the way, I think it was not explicitly noticed in this article yet, but the reporting and logging capabilities of the Robot Framework are really great and are helping a lot in troubleshooting problems in the test implementation.

Robot Framework Logfile with embedded screenshot

The Robot Framework is offering a lot of predefined testing functionality in various test libraries. This really helps a lot in starting to write tests once it is clarified which libraries are best to be used. Doing some pre-testing (prototyping) to check different test libraries will help a lot to decide which test libraries should be used productively.

Database Library Server running

Especially when doing project work in a lot of different projects (in opposite to testing one in-house product for example) a common project structure and common tooling will help a lot. When testing a product it makes of course much sense to create own keywords in some common resource file to encapsulate the testing functionality and making changes easier (centralized).

Hopefully this gives you some ideas how to start working with the Robot Framework and how to structure test functionality. Of course there are a lot of possibilities still to enhance this sample project. But hopefully it is a good starting point nonetheless.

Please leave comments and your own experiences in the comments section and I have the feeling there are more Robot Framework related articles to come on this channel :-).

Author

Thomas Jaspers

Share on FacebookGoogle+Share on LinkedInTweet about this on TwitterShare on RedditDigg thisShare on StumbleUpon

More content about Agile Testing

Kommentare

  • So far I have no experience in Selenium and Roborframework but I found this article really nice and helpful. Thanks.

  • Hi Leute,

    i have tried to install all the packets as descriped here, but I got an error starting start_Sample_TestSuite.bat:

    These is, what I have installed:

    python-2.6.6.amd64.msi
    jython-installer-2.5.3.jar
    robotframework-2.7.4.win-amd64.exe
    wxPython2.8-win64-unicode-2.8.12.1-py26.exe
    robotframework-ride-0.51.win-amd64.exe
    dblibrary-2.0-server.jar
    robotframework-seleniumlibrary-2.9.1.win-amd64.exe

    and MySQL 5.5 Server

    OS is Windows 7, 64 Bit

    Here is the error log from std out:
    18:39:55.077 ERROR – Failed to start new browser session, shutdown browser and clear all session data
    java.lang.RuntimeException: Firefox refused shutdown while preparing a profile
    at org.openqa.selenium.server.browserlaunchers.FirefoxChromeLauncher.waitForFullProfileToBeCreated(FirefoxChrome
    Launcher.java:367)
    at org.openqa.selenium.server.browserlaunchers.FirefoxChromeLauncher.populateCustomProfileDirectory(FirefoxChrom
    eLauncher.java:120)
    at org.openqa.selenium.server.browserlaunchers.FirefoxChromeLauncher.launch(FirefoxChromeLauncher.java:90)
    at org.openqa.selenium.server.browserlaunchers.FirefoxChromeLauncher.launchRemoteSession(FirefoxChromeLauncher.j
    ava:412)
    at org.openqa.selenium.server.browserlaunchers.FirefoxLauncher.launchRemoteSession(FirefoxLauncher.java:114)
    at org.openqa.selenium.server.BrowserSessionFactory.createNewRemoteSession(BrowserSessionFactory.java:381)
    at org.openqa.selenium.server.BrowserSessionFactory.getNewBrowserSession(BrowserSessionFactory.java:125)
    at org.openqa.selenium.server.BrowserSessionFactory.getNewBrowserSession(BrowserSessionFactory.java:86)
    at org.openqa.selenium.server.SeleniumDriverResourceHandler.getNewBrowserSession(SeleniumDriverResourceHandler.j
    ava:811)
    at org.openqa.selenium.server.SeleniumDriverResourceHandler.doCommand(SeleniumDriverResourceHandler.java:437)
    at org.openqa.selenium.server.SeleniumDriverResourceHandler.handleCommandRequest(SeleniumDriverResourceHandler.j
    ava:407)

    I have tried this work around:
    https://3lncr.wordpress.com/2010/03/04/selenium-rc-0-1-1-and-firefox-3-6-profile-issue/

    changing in “install.rdf” the value of em:maxVersion to 99.6 because Firefox is very fast counting the version number and now it is 16 😉

    Is there any solution for this problem?

    Best regards and thank you very much
    Uwe

    PS: The RIDE is also not very stable…

  • Hi Uwe,

    unfortunately I cannot directly check things out currently, but the reason could be very well be a too old version of the Selenium-Server.

    Did you use the latest one coming with the Robot Framework Selenium package for your tests?

    With RIDE it depends a bit, what are your concrete problems with it? Sometimes it might be wise not to use a new release right away ;).

    Best Regards
    – Thomas

  • OK – I have got it by myself

    After installing these packets:

    python-2.6.6.amd64.msi
    jython-installer-2.5.3.jar
    robotframework-2.7.4.win-amd64.exe
    wxPython2.8-win64-unicode-2.8.12.1-py26.exe
    robotframework-ride-0.51.win-amd64.exe
    dblibrary-2.0-server.jar

    and of course MySQL 5.5 Server

    I installed these here

    easy_install rtomac-robotframework-selenium2library-e8d7493
    easy_install decorator
    easy_install selenium
    easy_install Selenium2Library

    Now it works :-)

  • April 25, 2013 von Ranjana

    i have started executing test cases ,i have three .txt files commom ,google and testcase in same folder C:\myDemo .but while executing test case getting error like
    C:\Python27>pybot C:\mydata\RFramework\Demo\myFirstTest.txt
    [ ERROR ] Error in file ‘C:\mydata\RFramework\Demo\myFirstTest.txt’ in table ‘Se
    tting': Non-existing setting ‘Resource common.txt’.
    [ ERROR ] Error in file ‘C:\mydata\RFramework\Demo\myFirstTest.txt’ in table ‘Se
    tting': Non-existing setting ‘Resource google-search.txt’.
    ==============================================================================
    myFirstTest :: This is our first test case.
    ==============================================================================
    Go To Google Page[Documentation] Go to google page and serch somet… | FAIL |
    Test case contains no keywords

    Can u help me..why my resource file not recognising

  • It is quite hard without seeing the complete setup ;), but when you include the Resource-Files you either have to use a complete path or a path relative to the location of your Testsuite File. You could also tr executing pybot from the “C:\mydata\RFramework\Demo\” directory, just in case.

    Another potential problem – which might be even more likely – when using the TXT-format of Robot Framework is that you have to add additional spaces between “commands” and parameters.

    Non-existing setting ‘Resource google-search.txt’.

    The above error might indicate that you have to add additional spaces after the “Resource” statement.

    • April 26, 2013 von Ranjana

      Yes I do agree that without seeing the set up..I have tried all options of changing the spaces from 2 to 4 :) but no luck.gud to have ur email id in this case…

  • May 8, 2013 von Sri

    HI ,
    I am facing error while starting the test scripts with ride tool..
    WARN ] Keyword ‘Capture Page Screenshot’ could not be run on failure: No browser is open. Please help

  • Hi Sri,

    it has been some time since I did this example. Is the Selenium-Server running? Otherwise it would be good to have more details from the Robot Framework Log-File (not the Report) to see what might be going wrong there.

    Cheers
    – Thomas

  • Hi,

    Is there a way to stop the screen shot capturing for every [WARN] message..
    I know there is a setup called run_on_failure=nothing which would not capture screen shots on any failures .In my case I have a keyword which Opens browser if not already open

    ex: ${status} ${msg} Run KeyWord and ignore error Get browser
    run keyword if ‘${status}’==’FAIL’ open browser @args

    In the above example selenium first checks for browser
    window and if it doesn’t find any window already open then it generates a warning
    msg ( captures screen shot at this point)

    How could I stop capturing screen shots on this particular scenarios

  • Hi Sai,

    for these kind of questions is probable better to use the Robot Framework forum:
    https://groups.google.com/forum/#!forum/robotframework-users

    It is very active and I guess you will get an answer there quite quickly :).

    Best Regards
    – Thomas

  • Hi

    Could you also please explain how to get the coverage data for the robot test cases using jacoco coverage engine.

    Thanks
    Greg

  • How would code coverage help you? I can’t see what it will tell you that would be useful.

    Cheers
    Mark

  • May 15, 2014 von joechang

    hi everybody
    I Just beginning to use it
    I have one question
    I set Variable inside Url Parameter
    but this Parameter Will add Plus
    *** Variables ***
    ${SN} this is test String

    *** Test Cases ***

    Open Browser https://www.google.com?SN=${SN} browser=gc

    Browser URL
    https://www.google.com?SN=+this is test String

    surplus ‘+’
    How can usage in Correct
    anybody can teach me ?
    thank everybody

  • Thanks for this very clear article. It makes starting to use the framework really easy. Would you have any other articles comparing the RObot Framework to other comparable, open source tools?

  • Thanks Ronald, that is good to hear :).
    I am almost always planning such articles, but I have little time currently to write them up, as this of course requires some decent knowledge of the tool to compare with as well :-).

  • August 8, 2014 von vishnu

    here is what i get when i execute the “pybot –version” on the windows cmd.
    I have set the PATH variable to C:\Python26;C:\Python26\Scripts
    I still get this error::

    Microsoft Windows [Version 6.2.9200]
    (c) 2012 Microsoft Corporation. All rights reserved.

    C:\Users\VishnuSaiReddy>python –version
    Python 2.6.6

    C:\Users\VishnuSaiReddy>pybot –version
    ‘pybot’ is not recognized as an internal or external command,
    operable program or batch file.

    Have installed a 64 bit python version and

    wxPython3.0-win64-py26 64-bit Python 2.6

  • October 8, 2014 von Frank

    I could not get RobotFramework IDE running under Windows7 64 bits with
    python-2.7.8
    jython 2.5.3 (on java1.7.0_60)
    wxPython2.8-win64-unicode-2.8.12.1-py27
    robotframework-2.8.6.win32
    robotframework-ride-1.2.3.win32

    PATH was updated to include python, jthon, jybot, and scripts…etc.
    The IDE icon was created, but nothing was shown.
    Anyone with similar problem?

Comment

Your email address will not be published. Required fields are marked *