AutoIt, Sikuli, and other tools with Selenium Grid

22 Dec

I have seen several posts about this topic, though it’s not a very common question/request. But there’s been no talk of a solution yet.

Thought I’d share some my idea of a theoretical solution that just needs to be proven out by someone or group of people via a proof of concept prototype implementation.

So, here’s the idea in question: you’ve worked with AutoIt, and/or Sikuli, and/or other tool together with Selenium to perform your desired web automation, where Selenium by itself could not do. But now you need to scale tests up, ideally with Selenium Grid. But uh oh, you can’t seem to use AutoIt or other tool with Grid anymore. Why is that?

Short answer – your lack of understanding of how AutoIt or other tool really works compared to Selenium and Selenium Grid. They weren’t designed to operate under a “grid” mode, unlike Selenium. So how do you make it all work? The simple quick fix answer is to use some techniques to remotely execute AutoIt/Sikuli/etc. such as use of SSH, telnet, PSExec.exe, or use of custom XML-RPC/SOAP/web services. But for a truly grid scalable solution that minimizes hard coding or fixing test execution to specific nodes, read this:

Using specific browser environments option with Selenium Grid and build upon that. Start by reading this article: (update/note: it seems this page may be gone now, see if Google has a cached version or not)

Based on that, create startup scripts for the hub & nodes that can define the specific environment labeling, where order of “node 1” to N is calculated at startup. The mapping can then be saved to file or database like follows:

environment label name,SeleniumNodeHostnameOrIp,SeleniumNodePort

from there external tools (AutoIt, Sikuli, etc.) can be started up or set up similarly with additional fields to the mapping like

environment label name, SeleniumNodeHostnameOrIp, SeleniumNodePort, AutoItHost, AutoItPort, OtherAutoItDetailsAndFields

In example above, if hostname or IP same for Selenium and AutoIt on same machine, can just use same shared field instead of duplicating host field. Ports will obviously be different. OtherAutoItDetailsAndFields could be things like usename, password, etc. if using tools like PSExec.exe to call it remotely. Though I’d recommend calling AutoIt over XML-RPC server or other web service than to use PSExec/SSH/telnet in this grid setup. To elaborate, that means build an XML-RPC or other web service server that you can make API calls to that will then forward to AutoIt on same server machine to perform the requested AutoIt action (via API call).

Now with the mapping, just modify test scripts or test framework to do a lookup at runtime, where if passing or given environment label as runtime argument/parameter, look up via the mapping, the matching host and port to use for Selenium connection and matching host and connection info for other tool. Your test then doesn’t need to have hard coded values and can operate under a grid config. You just use template variables for hard coded values, that get replaced by the lookup at runtime.

This should work in theory, now just for people to try it out, I don’t have the time and priority to do this myself yet. Anyone done something like this before already? I’m guessing probably not yet.  I’d love to hear what you think or how your progress on this is going.

And if you still don’t understand my theoretical solution, just contact me,  and perhaps you might also need to learn more about distributed network computing as that is what this is based on.

Update 1/7/2012: This solution presented is for Selenium RC Grid, for Selenium Grid2, the method doesn’t quite apply as the “requesting specific environment” option is not in Selenium Grid2. But one could derive a similar solution in code by extracting the host (and maybe port) info of the node the test is currently executed on, and map this the same way I’ve presented for the RC solution, you use same host (but different port) to access the AutoIt/Sikuli server that’s also running on same Selenium node. Example of how to get the host and port info for Selenium Grid2 node in Java:

Update 4/2/2012: another option for Grid2 using special parameters (like applicationName) from capabilities object.

Update 5/9/2012: or custom capability matcher or custom grid plugin

Update 6/10/2012: not an update regarding Grid, but info on using Sikuli like Selenium RC, check out this project:

Update 11/23/2012: this topic might also be facilitated now by using new tools like Appium and iOS Driver, but porting those tools to drive other automation like AutoIt/Sikuli instead of iOS UIAutomation. Then either drive 2 driver instances in code (1 for WebDriver, and 1 for this driver), or merge the functionality into a single driver to do both WebDriver and this non-Selenium automation. However, in both cases, it’s still a lot of work to do.

Update 01/12/2015: following on my previous update, I actually took the initiative to show you that you can wrap AutoIt, Sikuli, etc. around WebDriver APIs to better facilitate integration with Selenium and grid deployment. They are a work in progress, but be sure to follow these projects, and try them out when code/tool available:

Update 09/27/2021: forgot to link a related post to this for using Grid2 node extraction method for the node a test is running on to then integrate with AutoIt using various methods for a grid-supported remote execution:

3 Responses to “AutoIt, Sikuli, and other tools with Selenium Grid”

  1. Denis MACHARD May 12, 2015 at 9:32 am #

    A new tool ExtensiveTesting based on sikuli and selenium (best of the two world in one tool) : it’s a generic and open source test framework with a test recorder and testcase modelisation.
    Take a look

  2. Teodor September 27, 2021 at 9:50 am #

    Hi, I there any update on this?
    is there a tool or library capable of working with selenium grid?

    Thank you.

    • autumnator September 27, 2021 at 6:04 pm #

      As far as I’m aware, there is likely no real updates in this area since my post. Also I haven’t actively followed changes in Selenium, so the interface to figure out the node a test is running on in Selenium Grid for Grid 3 and Grid 4 is likely different than Grid 2 and 1/RC.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

pictures & ponderings

adventures in gratitude


Surf here for Programming Ideas, Online Tricks and much more. Stay connected to explore new things. Mail your queries to ""

The Counter-top Guy's Blog

All about the wonderful world of Countertop Maintenance, Repair & Installation

Chris Tech Blog

IT problems and solutions

Random ASCII - tech blog of Bruce Dawson

Forecast for randomascii: programming, tech topics, with a chance of unicycling

Automation Panda

A blog for software development and testing


Various musings on automation and more...


Lee Badman's *Mostly* Wi-Fi Blog- opinions are my own, and I speak only for me.

Adventures in Remodeling

Misdirected remodeling


ericlaw talks about the web and software in general

Bayu Prasetio's Weblog

Exploring the Possibilities

Recoding my career

My journey as an aspiring software tester

Christian Huitema

Keeping working on this Internet thing...

PowerShell, Programming and DevOps

Musings and mischief on PowerShell, Programming and DevOps.

Station HYPO

Celebrating the Past, Present and Future of Navy Cryptology

a guy, named urban jack

Android, Electronics, DIY, HowTo, WiFi and much more


Testing. With Java, Selenium, TestNG, Maven, Spring, IntelliJ and friends.

%d bloggers like this: