A Selenium IDE alternative for other browsers and another record & playback method

I see quite a few posts about Selenium IDE support for IE, etc.

While there’s no such thing and as such record & playback support, just wanted to mention an alternative:

It does require you to know Selenium RC or WebDriver API to make use of. But it also allows a form of record & playback as well.

Read my other posts for background reference:

Using Selenium with interactive interpreter shells?

Developing and debugging in Java via an interpreter

Using the interpreter shell method, you can try things out in shell (~using Selenium IDE but with RC/WebDriver commands) and when you have what you want, copy & paste to file and clean up to make into a script (Python, Ruby, Perl, PHP) or then compile code (Java, .NET/C#) (~record), and use that output to run commands again (~playback).

This method will work for whatever browser Selenium supports making it sort of an IDE that will work for the non-FF browsers.

And this method also allows you test/verify Selenium locators. You can’t find them using this method (need Firebug, or other tools for IE and other browers) but you can test them using this method to make sure locators work. And you can also run the Firebug and other plugins while using this method so combine those tools with this method and you can find and test locators at same time and know Selenium will work with them, with exception of timing problems that don’t show up under this method (compared to scripts/code).

One other benefit of this method, is it can help you find problems in other browsers like quirks for locators, windows, and other things, if you develop in the other browsers using this method rather than always starting with Firefox.

Tags: Automation, Java, Selenium

Firefox profiles for Selenium automation

Most automation testers probably know how to work with Firefox profiles for Selenium automation. But thought I’d point out some tips and bring up a discussion question.

This is a good article for how to create a Firefox profile for automation (or even other purposes): http://girliemangalo.wordpress.com/2009/02/05/creating-firefox-profile-for-your-selenium-rc-tests/

And here’s a tip on how to best use Firefox profiles on Selenium 2 / WebDriver particularly if you are migrating or migrated from Selenium RC, to rather not start over from scratch building the profile the new way (from code):

use code to load an existing profile (directory), a sample in Java is presented

FirefoxProfile profile = new FirefoxProfile(new File(“C:\\PathTo\\CustomProfileDirectory”));
driver = new FirefoxDriver(profile);

And now a discussion question based on the profile creation method descripted in the referenced link/post:

What I’ve noticed, at least for FF 3.6 profile manager is that there is no option to import and export an existing FF profile. So you can only create new ones and edit existing ones on your FF installation (i.e. existing profiles you’ve set up on the machine).

As a hack, to do an import, I’ve create a dummy profile with the manager, then open FF with it, and exit FF. Then replace/overwrite content in that profile (directory) with the desired profile to import. Then open FF again with that modified profile and make any additional changes as needed, then exit. Then can manually copy back that profile folder elsewhere as export step. This method allows you to update a FF profile created on another machine.

Is there any better method to do an import/export other than my hack? Or use newer version of FF?

Tags: Automation, Selenium

Automation helps you find additional bugs

Perhaps a very keen and alert QA/developer will catch the bugs, but even the best of us sometimes overlook or miss things.

Automating tests can help you find some additional bugs that you can overlook during manual and exploratory testing, particularly in the field of web applications, for browser differences:

• unexpected behavior of some browsers
• different formatting in some browsers

Those two cases have happened to me. Would have missed them if not for automation, due to oversight in visual inspection and not regression testing everything across all browsers. It was actually easier to spot by automating to run for all browsers and catch it failing when running against the problem browser(s).

Though I assume some of you are already aware of this benefit…

Tags: Automation, Selenium

Automation alerts and the good old system beep

When you make automation, for testing and other things, it can be a pain waiting for the task to complete and get results. Or be alerted for some condition.

The standard mechanism usually is email notification. Depending on if you use existing framework solutions (commercial or free/OSS) or build your own, other options might include SMS/text messaging, instant messaging, paging, or perhaps an automated phone call.

I like all those options, each has its usefulness depending on frequency of alerts and type of alert. And infrastructure and cost factors. Email is rather bland and we get too many emails these days.

One great alert mechanism if you happen to be in the vicinity of the machines doing the work is the good old computer beep. Most computers can beep even without sound card installed, though not guaranteed. And some behave intermittently sometimes beep, sometimes don’t. But the old obsolete workstations with low CPU and memory from a decade or more ago still work like a charm with the beep.

The beep is supported by most programming languages as well as can be called from command line/shell for DOS/Windows and *nix if you know the technique.

A beep is rather bland and not unique so I tend to use a beep pattern to inform me of my alerts like 2x, 3x, or 4x consecutive beeps as an alert. Has worked well for me in my jobs.

A paging (public address, speaker) system or instant messages may be nice too, but haven’t had a chance to implement those types of alerts yet for my work.

Tags: Automation

Developing and debugging in Java via an interpreter

This is written from QA/automated testing perspective, but can be applied to general Java programming.

While the traditional and correct way to develop and debug in Java is using a good IDE with debugger and set breakpoints and watch variables and object references, etc., it is also useful to dynamically work and test with Java at runtime using an interpreter shell session.

You can say I got bit by the “bug” or am spoiled by such feature from Python, Ruby, etc.

I find it useful this way to test thing out quick w/o having to use full featured bloated IDE, setting breakpoints, and monitoring stuff. Just try some code, don’t work try something else from the shell.

This is particularly useful for quick test of code snippets and can be useful in QA for testing Selenium / WebDriver API calls and see if things work or not, or if you got the correct element locator or not.

With Java, you don’t really get an interpreter, but there are some options like interface to Java using Jython, JRuby, BeanShell. Don’t know all the options, but I believe BeanShell gives the closest approximation of using Java in a shell, so this article describes a bit of using it.

As an example of using Beanshell to test/debug/develop WebDriver test code, you can do the following:

Start up Beanshell with reference to Selenium library, I show Windows method, but Linux/Mac OS X method is similar:

java -cp .;selenium-server-standalone-2.15.0.jar;bsh-2.0b4.jar bsh.Console

or

java -cp .;selenium-server-standalone-2.15.0.jar;bsh-2.0b4.jar bsh.Interpreter

select the option you like best starting up an IDE like GUI console shell (bsh.Console) or use existing command prompt shell of your OS (bsh.Interpreter).

From there, you just use normal Java code like:

import org.openqa.selenium.*;
import org.openqa.selenium.firefox.*;

WebDriver driver = new FirefoxDriver();
driver.get(“http://www.yourdomain.com”);
driver.findElement(By.id(“signin”)).click();

There may be some caveats from using interpreter shell as I’ve found for using WebDriver sendKeys() method, you have to write the command differently in the shell:

driver.findElement(By.id(“email”)).sendKeys(new String[]{“yourusername@gmail.com”});

see reference post here on reasoning for that:

http://groups.google.com/group/webdriver/browse_thread/thread/be568157b38ee47b/f83818c0d28afc49?lnk=gst&q=beanshell#f83818c0d28afc49

Once you got code you like, you can then copy & paste into your real Java code in IDE to complete it and run formal test.

Happy playing with Java interpreter shell now…particularly with WebDriver and Selenium!

Tags: Automation, Java, Selenium

AutoIt, Sikuli, and other tools with Selenium Grid

I have seen several posts about this topic, though it’s not a very common question/request. But there’s been no talk of a solution yet.

Thought I’d share some my idea of a theoretical solution that just needs to be proven out by someone or group of people via a proof of concept prototype implementation.

So, here’s the idea in question: you’ve worked with AutoIt, and/or Sikuli, and/or other tool together with Selenium to perform your desired web automation, where Selenium by itself could not do. But now you need to scale tests up, ideally with Selenium Grid. But uh oh, you can’t seem to use AutoIt or other tool with Grid anymore. Why is that?

Short answer – your lack of understanding of how AutoIt or other tool really works compared to Selenium and Selenium Grid. They weren’t designed to operate under a “grid” mode, unlike Selenium. So how do you make it all work?

Using specific browser environments option with Selenium Grid and build upon that. Start by reading this article:

http://selenium-grid.seleniumhq.org/how_it_works.html

Based on that, create startup scripts for the hub & nodes that can define the specific environment labeling, where order of “node 1″ to N is calculated at startup. The mapping can then be saved to file or database like follows:

environment label name,SeleniumNodeHostnameOrIp,SeleniumNodePort

from there external tools (AutoIt, Sikuli, etc.) can be started up or set up similarly with additional fields to the mapping like

environment label name,SeleniumNodeHostnameOrIp,SeleniumNodePort,AutoItHost,AutoItPort,OtherAutoItDetailsAndFields

In example above, if hostname or IP same for Selenium and AutoIt on same machine, can just use same shared field instead of duplicating host field. Ports will obviously be different. OtherAutoItDetailsAndFields could be things like usename, password, etc. if using tools like PSExec.exe to call it remotely. Though I’d recommend calling AutoIt over XML-RPC server or other web service than to use PSExec in this grid setup. To elaborate, that means build an XML-RPC or other web service server that you can make API calls to that will then forward to AutoIt on same server machine to perform the requested AutoIt action (via API call).

Now with the mapping, just modify test scripts or test framework to do a lookup at runtime, where if passing or given environment label as runtime argument/parameter, look up via the mapping, the matching host and port to use for Selenium connection and matching host and connection info for other tool. Your test then doesn’t need to have hard coded values and can operate under a grid config. You just use template variables for hard coded values, that get replaced by the lookup at runtime.

This should work in theory, now just for people to try it out, I don’t have the time and priority to do this myself yet. Anyone done something like this before already? I’m guessing probably not yet.  I’d love to hear what you think or how your progress on this is going.

And if you still don’t understand my theoretical solution, just contact me,  and perhaps you might also need to learn more about distributed network computing as that is what this is based on.

Update 1/7/2012: This solution presented is for Selenium RC Grid, for Selenium Grid2, the method doesn’t quite apply as the “requesting specific environment” option is not in Selenium Grid2. But one could derive a similar solution in code by extracting the host (and maybe port) info of the node the test is currently executed on, and map this the same way I’ve presented for the RC solution, you use same host (but different port) to access the AutoIt/Sikuli server that’s also running on same Selenium node. Example of how to get the host and port info for Selenium Grid2 node in Java:

http://groups.google.com/group/webdriver/browse_thread/thread/415b17c853a5ccbc

https://gist.github.com/1766772

Update 4/2/2012: another option for Grid2 using special parameters (like applicationName) from capabilities object.

http://groups.google.com/group/selenium-users/browse_thread/thread/3d1b0405c6e93653

Update 5/9/2012: or custom capability matcher or custom grid plugin

http://groups.google.com/group/selenium-users/browse_thread/thread/4806d0f1ec9dca8a/0e551e64586493fb?hl=en

http://groups.google.com/group/webdriver/browse_thread/thread/1eec95202243f9e7

Tags: Automation, Selenium

SauceLabs vs other cloud providers like Amazon EC2

It never occurred to me until today, so making a post for my own reference and others.

For running Selenium tests in parallel distributed fashion across a larger set of nodes than what your internal environment allows, one has two or a few choices depending on how you view it:

• SauceLabs.com
• Other cloud providers (expand the list rather than define generically, and you would have more than two options)

Cloud virtualization services/providers:

• Amazon EC2
• Microsoft Windows Azure
• Rackspace Cloud
• GoGrid.com
• Other small players

With SauceLabs, it may be cheaper to use the service, or not. And you don’t have to manage/maintain cloud servers (e.g. turn on, off, etc.). It’s all managed for you and on demand. Those are the advantages.

Disadvantages are that you’re limited to programming languages that they support, can only scale to whatever their max is, and lack of customization like if you need custom tools to integrate with Selenium that need to go on the browser host machine.

Clearly using cloud providers gives you more power and customization. Everything SauceLabs does for automation, you can do with cloud providers. Only problem is you have to build the control and maintenance infrastructure yourself (unless you plan to manually start/stop servers and kick off tests). Until someone releases prebuilt frameworks to use with the cloud providers, you have to “roll your own”.

Given time and resource allocation, I’d prefer the cloud provider route, but if you need simple quick solution, SauceLab might be the way to go (short term, anyways).

Tags: Automation, Selenium