Robot Framework – Testing Windows Applications


Robot Framework Tutorial

Part I: Robot Framework Tutorial – Overview
Part II: Robot Framework – A complete example
Part III: Robot Framework IDE
Part IV: How to Structure a Scalable And Maintainable Acceptance Test Suite
Part V: Robot Framework Tutorial – Writing Keyword Libraries in Java
Part VI: Robot Framework Tutorial – Loops, Conditional Execution and more
Part VII: Robot Framework – Testing Windows Applications
Appendix A: Robot Framework – Compact Sheet

The new Robot Framework Tutorial 2016 series


So far when talking about test automation using the Robot Framework – or other tools – this was mainly related to testing web applications. Mostly in addition with testing the created database content and potentially directly testing some (Java) services. This is not really surprising as the web world is huge and is often dominating the application landscape of companies.

But of course there are also (still) a lot of Windows applications in the field. And thus it is only natural that those should be included in the test automation cycle. The keyword-driven approach of the Robot Framework offers an excellent possibility to seaminglessly integrate those tests with – for example – the tests required for web applications. The following figure depicts this scenario.


Installation Stack

Testing Windows applications requires a somewhat specific installation stack. Of course the machine running the tests must be running on Windows :-). My laptop is running a “Windows 7 64 Bit Edition”.

The answer to testing native Windows applications is AutoIt and luckily there exists a corresponding Robot Framework library. According to this thread in the Robot Framework support forum AutoIt requires a 32 Bit installation of Python, Robot Framework and related libraries. This leads to the following installation stack:


Robot Framework Core:

Robot Framework Libraries:

After installing all of the above packages there is also already an example installed for using the AutoIt library. This can be found from C:\RobotFramework\Extensions\AutoItLibrary\tests.

Unfortunelty the installation – and later on running – the tests is having a major problem with UAC that is also adressed in the before mentioned forum discussion. I used the brute force approach here and disabled UAC completely while evaluating the AutoIt library. As typically the tests will later on run on dedicated machines this is hopefully not a problem. But it should be kept in mind.

Ok, let’s proceed to the AutoIt example.

AutoIt Example

The example installed with AutoIt is really sophisticated and gives a good impression on how to implement these kind of tests. And what could be tested under Windows naturally? Of course the calculator.

Running the example will probably work out of the box on any Windows installation that is running in English language. As on my laptop there is an installation of Windows 7 using German language I had the opportunity to immedietly dig into some of the concepts of testing with AutoIt :-).

First of all a lot of names must be changed as the “Calculator” is called “Rechner” in german language and also all of the menu items and so on are different. So I went through all the example code – using try & error – replacing the relevant strings:


Beside this there was a need to adjust the maps defined in CalculatorGUIMap.py that can be found from the sample directory. There are maps defined – for different version of the calculator – to define how to navigate to certain menu entries using the ALT-Shortcut shortcuts. In addition names of Window elements (mostly buttons in this case) are defined there similar to ID-values used in Selenium to identify elements.

Here is the adjusted menumap. Ok, I was lazy and did not change all of the entries, but only those I needed to run a subset of the tests.

# MENUMAP_61 is the mapping for Calculator version 6.1 (Win7 version)
MENUMAP_61 = {"Ansicht Standard"             : "AS",
              "Ansicht Wissenschaftlich"     : "AW",
              "View Decimal"                 : "VD",
              "View Hex"                     : "VP",
              "Ansicht Zifferngruppierung"   : "AI",
              "Bearbeiten Kopieren"          : "BK",
              "Bearbeiten Einfuegen"         : "BE",
              "Exit"                         : "{F4}"

And an example on how to define elements that can be “clicked”:

# GUIMAP_61 is the mapping for Calculator version 6.1 (Win7 version)
GUIMAP_61 = {"0" : "Button6",
             "1" : "Button5",
             "5" : "Button10",
             "8" : "Button9",
             "9" : "Button14",
             "/" : "Button20",
             "*" : "Button21",
             "-" : "Button22",
             "+" : "Button23",
             "=" : "Button28",

             "C"     : "Button13",
             "Clear" : "Button13",

Of course I wanted to know how the names of the elements can be detected. AutoIt brings the corresponding tool direclty with its installation. Simply start the Au3Info.exe and then start inspecting the elements of the application to be tested. The following screenshot shows for example inspecting the Button “5” to be “Button10”:


Let’s have a closer look at one of the defined tests. It does not make sense to show all of the keywords here, but I try to explain what they are doing. Probably it makes a lot of sense to install the sample and take a closer look in addition to reading this blog post.

*** Test Case ***
Integer Addition
    [Documentation]    Get "The Answer" by addition.
    Click Buttons    4 1 + 1 =
    Win Wait    Rechner    42
    ${Ans} =    Get Answer
    Should Be Equal As Numbers    ${Ans}    42

The “Click Buttons” keyword is using the above mentioned guimap to find out which elements must be addressed. Those are then selected using appropriate keywords from the AutoIt library. The result is copied to the windoes clipboard using the “Get Answer” keyword. This way it is then possible to compare this with the expected result.


The biggest problem in getting the tests to run was the UAC-related problems during installation. This might be a problem for some companies in installing and using AutoIt library.

Chances are good that most native Windows applications can be tested this way, but of course this remains to be seen individually for every case. But as some kind of evaluation phase is a very usual thing to do – before starting any (bigger) test automation project – it should be possible to find out relatively quickly if AutoIt works for an application or not.

The integration into the Robot Framework ecosystem is very good. Everything works as expected and again here one of the big advantages of the Robot Framework – the concept of keyword-driven testing – can shine. This way testing with AutoIt can seeminglessly be integrated into testing any other technologies like Web- or database-tests for example.

Happy testing 🙂

Thomas Jaspers

Long-term experience in agile software projects using
Java enterprise technologies. Interested in test automation tools and concepts.

Share on FacebookGoogle+Share on LinkedInTweet about this on TwitterShare on RedditDigg thisShare on StumbleUpon


  • Alex

    Hi great post… Like it very much… I give this definitely a spin as soon as I can… Just did something similar with Sikuli.org… I wrote a litte python framework and used a variant of gherkin to achieve something similar…
    Thanks for sharing… Alex

  • Thomas Jaspers

    Thx 🙂 … happy you liked it and Sikuli sounds interesting, I hope I find some time to take a closer look 🙂

  • Serhat Bolsu

    24. February 2014 von Serhat Bolsu

    Thanks for another great solution.

    Right now, I am dealing with Mobile Native test automation. I am lookign forward for your post on this topic with newest tools and integration with Robot framework.

  • David Luu

    Good post. Just wanted to point out, beyond AutoItLibrary, one could test Windows UI by also writing an RF library around Microsoft’s UI Automation interface, and/or the open source White framework extension for MS’s UIA. Unfortunately, no one’s yet created such a library.

  • greg

    Has ANYONE really got AutoitLibrary to work in the real world?
    Beyond the calculator example?
    Everyone talks about the library, and everyone uses the calculator example, but I have yet to see anyone show proof it works beyond that in a real world, 3rd Party, windows app.

    I find that once installed only a few of the many commands claimed to be available in the library are actually accessible. Namely the basics needed for the calculator example (that being a subset of a subset) . Beyond that the calls fail to find the commands.


    I have resorted to coding the solution in AutoIt directly, converting that to an EXE and then calling that from Robot. I use a ini file to pass data back and forth.

    If you have any pointers that would allow me to convert this hack to use AutoItLibrary I would be very happy to hear it.



Your email address will not be published. Required fields are marked *