Integrating AutoIt, Sikuli, and other tools with Selenium when running tests in Selenium Grid

22 Jan

A true integration for tools like AutoIt, Sikuli, etc. with Selenium would be when they are able to run under Selenium Grid configuration just like Selenium. That would probably first require that they can run over WebDriver API, JSONWireProtocol as with AutoItDriverServer. But when Grid automatically selects which node to run your Selenium tests with, would it not do the same for AutoIt, except if you specifically request a certain capability set that let’s you know which exact node the test will go to. But anyhow, it might get confusing how to map/correlate which grid node runs the Selenium part of test vs the AutoIt/Sikuli part, etc. So maybe having AutoIt/Sikuli run under Grid configuration isn’t the whole answer either.

Regardless if AutoIt/Sikuli ran under Grid configuration or not, you could still deploy your Selenium tests that use AutoIt/Sikuli, etc. under Grid. I did blog about this a while back but never gave concrete examples. In StackOverflow tradition, people seem to want specific examples. So I’ve finally worked up a simple illustrative example.

To have AutoIt/Sikuli work with Selenium while it’s running under grid mode, you need to find a way to determine the node host that runs the Selenium tests at runtime, so that you can then call/run AutoIt/Sikuli code on the same target node host, so that everything runs on the same correct machine (rather than the dilemma novices encounter where Selenium executes on the remote node but AutoIt/Sikuli runs on the localhost, causing mismatch and test fails).

The easiest example solution is to extract out the node host information from the Selenium Grid API by providing the WebDriver session ID that you first need to extract. Once you have that information, you can then use various ways to remotely execute AutoIt, Sikuli against the node host that you’ve extracted. All this is presented in this Github gist:

of all the various ways in the sample gist, I personally prefer calling AutoIt, Sikuli over WebDriver API, where possible rather than to resort to using PSExec.exe, SSH, etc. I’d go for web service route as alternative to WebDriver API. Sticking with PSExec.exe type option as last resort. But you are free to pick whichever option works best for you.

11 Responses to “Integrating AutoIt, Sikuli, and other tools with Selenium when running tests in Selenium Grid”

  1. Andrew January 26, 2015 at 11:49 am #


    I have error when try to launch


    File “”, line 182
    print “script2exec: %s” % script
    SyntaxError: invalid syntax

    • autumnator January 26, 2015 at 10:08 pm #

      What is your system environment? It runs fine for me using Python 2.7.2 (32-bit) on Windows 7 64-bit, running the latest version from GitHub.

      • Andrew January 27, 2015 at 3:13 am #

        Last version from git. Python 3.4,1, Windows 7 64-bit

      • autumnator January 27, 2015 at 4:28 am #

        I’ll have to look into that then. I haven’t tested with Python 3.x. Only 2.7.x. Could likely be an issue with Python 3. Do you know if your Python install is 32-bit or 64-bit? Either will install on a 64-bit OS.

      • Andrew January 27, 2015 at 4:32 am #

        Thank you! I ran it too on Python 2.7.2

      • Andrew January 27, 2015 at 4:33 am #

        it was ActivePython-

    • Andrew January 29, 2015 at 5:13 am #

      Hi again!
      I have a problem with find activeX element in IE

      I try to find element and for example get coordinates, but i have error

      WebElement table = autoIt.findElement(By.cssSelector([CLASS:PMTableSelf, INSTANCE:2]”));

      org.openqa.selenium.WebDriverException: AutoIt failed to get element [CLASS:PMTableSelf, INSTANCE:2] location coordinates.

      • autumnator January 29, 2015 at 7:21 am #

        It sounds like element not found. Have you worked with AutoIt separately before like AutoIt scripting or using AutoItX via COM?

        Your best debugging solution is to try to find that element natively in AutoIt (script or COM), which is what I mention on the Github wiki page for debugging.

        Try calling ControlGetPosX() and/or ControlGetPosY() from COM or ControlGetPos() from AutoIt script

        if error object/flag is 1, that element not found

        If it fails natively in AutoIt, it’s going to fail in AutoItDriverServer. And in that case, the answer/solution, if there is one, is to find the “correct” locator string value to use. What that value is, is for you to find/try out. Sometimes there isn’t, and you have to resort to workarounds, as AutoIt isn’t the best/perfect desktop GUI automation tool, though it is a good one.

        For workarounds, there may be some or not depending on what you need to do. For getting element location (or text of element, etc.), you might be out of luck. For clicking element and such, you can always use the mouse click operations based on x,y coordinates rather than click specific element.

    • autumnator February 11, 2015 at 9:29 am #

      The Python 3 issue has been fixed. The server and sample code should be runnable on Python 3.3+ now. They might not run on Python 3.1-3.2. I only tested against 3.4 though. Let me know if one still sees issues.

  2. Vladimir October 20, 2016 at 4:51 pm #

    hi David,
    i tried to run the samples provided with AutoItDriverServer for java.
    AutoItDriverServer is up and running according to your instruction.
    I have console confirmation:
    Listening on

    But CalculatorTest and other samples throw
    org.openqa.selenium.remote.UnreachableBrowserException: Could not start a new session.
    It occurs for the following code line:
    driver = new RemoteWebDriver(new URL(“http://localhost:4723/wd/hub” ), capabilities);

    What can be missed/misconfigured here?

    Thx in advance.

    • autumnator October 22, 2016 at 1:14 am #

      It could be an issue with address binding. Localhost does not equal Start up the server supplying address parameter of “” or “localhost” and that should likely work. Or edit the test file to use the 192 IP address rather than localhost. The current version of the server does not bind to multiple addresses/hostnames. When no address given, it defaults to trying a lookup of your hostname to IP address, and falls back to if the lookup fails.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Anastasia Writes

politics, engineering, parenting, relevant things over coffee.

One Software Tester

Trying To Make Sense Of The World, One Test At A Time

the morning paper

an interesting/influential/important paper from the world of CS every weekday morning, as selected by Adrian Colyer

RoboSim (Robot Simulator)

Visualize and Simulate the Robotics concepts such as Localization, Path Planning, P.I.D Controller


open notebook

a happy knockout mouse.

my journey into computer science

Perl 6 Advent Calendar

Something cool about Perl 6 every day


Inspire and spread the power of collaboration

Niraj Bhatt - Architect's Blog

Ruminations on .NET, Architecture & Design

Pete Zybrick

Bell Labs to Big Data

Seek Nuance

Python, technology, Seattle, careers, life, et cetera...


New Era of Test Automation

Der Flounder

Seldom updated, occasionally insightful.

The 4T - Trail, Tram, Trolley, Train

Exploring Portland with the 4T

Midnight Musings

Thoughts on making art

Automation Guide

The More You Learn The More You Play...!

The Performance Engineer



Thoughts related to software development

%d bloggers like this: