AutoIt, Sikuli, and other tools with Selenium Grid

22 Dec

I have seen several posts about this topic, though it’s not a very common question/request. But there’s been no talk of a solution yet.

Thought I’d share some my idea of a theoretical solution that just needs to be proven out by someone or group of people via a proof of concept prototype implementation.

So, here’s the idea in question: you’ve worked with AutoIt, and/or Sikuli, and/or other tool together with Selenium to perform your desired web automation, where Selenium by itself could not do. But now you need to scale tests up, ideally with Selenium Grid. But uh oh, you can’t seem to use AutoIt or other tool with Grid anymore. Why is that?

Short answer – your lack of understanding of how AutoIt or other tool really works compared to Selenium and Selenium Grid. They weren’t designed to operate under a “grid” mode, unlike Selenium. So how do you make it all work? The simple quick fix answer is to use some techniques to remotely execute AutoIt/Sikuli/etc. such as use of SSH, telnet, PSExec.exe, or use of custom XML-RPC/SOAP/web services. But for a truly grid scalable solution that minimizes hard coding or fixing test execution to specific nodes, read this:

Using specific browser environments option with Selenium Grid and build upon that. Start by reading this article: (update/note: it seems this page may be gone now, see if Google has a cached version or not)

Based on that, create startup scripts for the hub & nodes that can define the specific environment labeling, where order of “node 1” to N is calculated at startup. The mapping can then be saved to file or database like follows:

environment label name,SeleniumNodeHostnameOrIp,SeleniumNodePort

from there external tools (AutoIt, Sikuli, etc.) can be started up or set up similarly with additional fields to the mapping like

environment label name, SeleniumNodeHostnameOrIp, SeleniumNodePort, AutoItHost, AutoItPort, OtherAutoItDetailsAndFields

In example above, if hostname or IP same for Selenium and AutoIt on same machine, can just use same shared field instead of duplicating host field. Ports will obviously be different. OtherAutoItDetailsAndFields could be things like usename, password, etc. if using tools like PSExec.exe to call it remotely. Though I’d recommend calling AutoIt over XML-RPC server or other web service than to use PSExec/SSH/telnet in this grid setup. To elaborate, that means build an XML-RPC or other web service server that you can make API calls to that will then forward to AutoIt on same server machine to perform the requested AutoIt action (via API call).

Now with the mapping, just modify test scripts or test framework to do a lookup at runtime, where if passing or given environment label as runtime argument/parameter, look up via the mapping, the matching host and port to use for Selenium connection and matching host and connection info for other tool. Your test then doesn’t need to have hard coded values and can operate under a grid config. You just use template variables for hard coded values, that get replaced by the lookup at runtime.

This should work in theory, now just for people to try it out, I don’t have the time and priority to do this myself yet. Anyone done something like this before already? I’m guessing probably not yet.  I’d love to hear what you think or how your progress on this is going.

And if you still don’t understand my theoretical solution, just contact me,  and perhaps you might also need to learn more about distributed network computing as that is what this is based on.

Update 1/7/2012: This solution presented is for Selenium RC Grid, for Selenium Grid2, the method doesn’t quite apply as the “requesting specific environment” option is not in Selenium Grid2. But one could derive a similar solution in code by extracting the host (and maybe port) info of the node the test is currently executed on, and map this the same way I’ve presented for the RC solution, you use same host (but different port) to access the AutoIt/Sikuli server that’s also running on same Selenium node. Example of how to get the host and port info for Selenium Grid2 node in Java:

Update 4/2/2012: another option for Grid2 using special parameters (like applicationName) from capabilities object.

Update 5/9/2012: or custom capability matcher or custom grid plugin

Update 6/10/2012: not an update regarding Grid, but info on using Sikuli like Selenium RC, check out this project:

Update 11/23/2012: this topic might also be facilitated now by using new tools like Appium and iOS Driver, but porting those tools to drive other automation like AutoIt/Sikuli instead of iOS UIAutomation. Then either drive 2 driver instances in code (1 for WebDriver, and 1 for this driver), or merge the functionality into a single driver to do both WebDriver and this non-Selenium automation. However, in both cases, it’s still a lot of work to do.

Update 01/12/2015: following on my previous update, I actually took the initiative to show you that you can wrap AutoIt, Sikuli, etc. around WebDriver APIs to better facilitate integration with Selenium and grid deployment. They are a work in progress, but be sure to follow these projects, and try them out when code/tool available:

One Response to “AutoIt, Sikuli, and other tools with Selenium Grid”

  1. Denis MACHARD May 12, 2015 at 9:32 am #

    A new tool ExtensiveTesting based on sikuli and selenium (best of the two world in one tool) : it’s a generic and open source test framework with a test recorder and testcase modelisation.
    Take a look

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Sue Connelly's KIT List Blog

Job Search and Career Management Tips and Strategies

The Networker

Bassem Aly



xlcwu's Blog

Lifestyle, Computer, Ubuntu, Linux, Apple, Mac, iPhone, Game, Programming, Java, C, ObjC and More ...

Multiple States Knowledge Base

Multiple States is a digital design, development and content creation studio, specialising in beautiful, clean and functional websites with powerful content.

journey of quality

make it perfect!

Marc Guillemot's blog

Java, Groovy, HtmlUnit, WebTest, NekoHTML and more. Contact me for rates.

North American Sea Glass Association official website

NASGA is a non-profit organization positively supporting sea glass collectors and the sea glass community with festivals, information, educational opportunities, commercial membership and more. The primary goal of NASGA is to establish a community of informed collectors and sellers of sea glass that are educated on the characteristics and significance of genuine sea glass.

Abiral Shakya's Blog

Learning, Improving and Sharing

Tea in Eighteenth-Century Britain

History of Tea Project at Queen Mary University of London

Phimosis - A Simple Cure (That is working for me)

No nonsense, no adverts, no sales pitch, just honest information.

Glass Onion Blog

Cheat sheets, post-its and random notes from the desk of a programmer

Abode QA

A Hub For Testing Minds...

The Test Therapist

Performance & Security Testing Blog


a programmer's hub

Let's Not Crash and Burn

it makes your brain tingle


“Incinerate Ignorance”

Anastasia Writes

politics, engineering, parenting, relevant things over coffee.

One Software Tester

Trying To Make Sense Of The World, One Test At A Time

%d bloggers like this: