SafariDriver is too fast

Of all the browsers I’ve worked with SafariDriver is too fast. Fast is good but too fast has some issues too.

Don’t know about Opera, but Chrome and/or Firefox is next in fast speed. I don’t test Chrome enough to say if faster than FF but think so, with IE being slowest.

With all the other browsers, FF/Chrome/IE, you can see the visual activity happen in most cases. But SafariDriver does it so fast, you don’t get to visually monitor & verify that what’s automated actually working well other than whether test passes or you get some exception from SafariDriver.

I’d prefer to be able to see the automated actions during a test run…

Testing XPath and CSS locators FirePath style across browsers

FirePath (for Firebug on Firefox) is a nice tool for finding and testing XPath and CSS selector locators. Firebug alone (and similar developer tools/console on other browsers) can only inspect element(s) but can’t give you the XPath/CSS to it nor allow you to directly test a given XPath/CSS locator value to see if it matches/finds any element to see if your locator is correct or not.

Well, I recently came up with a workaround that works across browsers. I’d still use FirePath on Firefox, but go with the workaround for all other browsers (until someone comes up with a FirePath port on those browsers).

Here’s the technique. You simply inject/execute some javascript code in your desired browser’s javascript (or error) console. Once done, you can then query for elements by XPath and CSS. Note that this isn’t a perfect workaround, it may have issues, but in general seems to work.

Here’s the javascript code snippet to run in the browser console:


document.getElementByXPath = function(sValue) { var a =
this.evaluate(sValue, this, null,
XPathResult.ORDERED_NODE_SNAPSHOT_TYPE, null); if (a.snapshotLength >
0) { return a.snapshotItem(0); } }; document.getElementsByXPath = function(sValue){ var aResult = new
Array();var a = this.evaluate(sValue, this, null,
XPathResult.ORDERED_NODE_SNAPSHOT_TYPE, null);for ( var i = 0 ; i <
a.snapshotLength ; i++ ){aResult.push(a.snapshotItem(i));}return
aResult;}; document.getElementByCssSelector = document.querySelector; document.getElementsByCssSelector = document.querySelectorAll;


To then query by XPath or CSS, simply do:

document.getElementByXPath(“value here”);
document.getElementByCssSelector(“value here”);

for getting multiple locators/elements, use the plural versions getElementsBy…

be sure to escape the double & single quotes as needed

and note that you can directly query by CSS w/o executing the code snippet above. You simply call document.querySelector(“css value here”); and document.querySelectorAll(“multiple element css value here”); the above code snippet simply create an alias to the native methods to keep the naming convention closer to Selenium/WebDriver APIs.

Now the results show as HTML tags in the javscript console and when you hover over the result, it should highlight in the browser. You’d get nothing returned if there was no match (or undefined, null, etc.)

Neat trick don’t you think?

As for find element by ID, name, class, tag, we already have those natively in browser as
document.getElementById(), getElementsByClassName(), getElementsByName(), getElementsByTagName(). Note that only by ID returns a single element, the others here return a multiple but if it only matches 1, you then have a javascript array of 1 returned. You can then access directly like document.getElementsByName(‘uniqueName’)[0].

The code snippets here are derived from Selenium discussion thread: https://groups.google.com/d/topic/selenium-users/PTPWFU2ho8Y/discussion

Update: it looks like the workaround doesn’t seem to work in IE9 (assume same for older IE versions). Not sure about IE10. But it works fine for Chrome and Safari (tested on Windows for both).

Manually setting cookie value in browser for testing and automation

Sometimes you have to set/use cookie for testing, and you may not have control to force set cookie on server side application/site, so you have to set it on the client side.

But what’s a good option to set cookie that’s cross platform across browsers and operating systems? I assume probably javascript, as not all browsers have a cookie editor section like Firebug related tools for Firefox. Most browsers only seem to have a cookie viewer built in with the dev tools unless I’m mistaken.

With javascript, you can set the cookie for use with Selenium (or use Selenium’s API to set cookie instead of pure javascript), and you can also use it for manual testing as well.

Here’s one example of how to do it adapted from http://www.w3schools.com/js/js_cookies.asp


var doSetCookie = function setCookie(c_name,value,exdays){ var exdate=new Date(); exdate.setDate(exdate.getDate() + exdays); var c_value=escape(value) + ((exdays==null) ? "" : "; expires="+exdate.toUTCString()); document.cookie=c_name + "=" + c_value; }; doSetCookie("cookieName","cookieValue",1);


For manual testing, simply open up the javascript/error/developer console of the browser developer tools and then paste & execute the above code. May then have to refresh page for cookie to take effect.

I do notice that setting cookies this way for Selenium doesn’t persist as well as doing it via native Selenium APIs, but this is a workaround should you have issues with the native approach like with Selenium issue 5503.

And one last thing to end this post, anyone have a more optimized javascript version of the above code to execute? What I have above I’m assuming is just rudimentary proof of concept and could be fine tuned.

Selenium automation with execution of custom site specific javascript

Expanding on my previous post: Javascript is your ally for Selenium / WebDriver

I am wondering if anyone has used WebDriver’s javascript execution facility to execute custom javascript code (not general code used for browser/Selenium limitation workarounds) but javascript code that’s specific to their website or web application under test.

I asked a colleague within our organization and he said his department recommends avoiding such use. While I understand that it is best not to dig into specific detailed functionality of the app under test for UI automation it does have some uses:

Limitations of the UI and element locators (IDs, class names, name attributes or other DOM attributes, visual text on screen) in providing a good way to define locators in an object oriented way programmatic parameterized way so that your locators are more dynamic and not static. For example have a method that selects a color and we “select/click” the element (finding it) based on color name rather than some less user friendly/understandable strategy. In some cases, that’s not possible, but might be if you execute app-specific javascript code to find out the info you want that is not exposed via the UI. Same goes for checking application state that might not be fully exposed visually in the UI. I’ve found it helpful in my case.

The other useful case may be for more white box javascript API & unit testing integrated within Selenium UI tests for extended test coverage. This might be useful compared to issues that may come up with pure javascript API/unit tests alone per this blog: http://watirmelon.com/2013/02/09/why-i-dont-like-jasmine/.

What are your thoughts on this?

Javascript is your ally for Selenium / WebDriver

First a background introduction / discussion

When you run into issues or limitations for Selenium, particularly WebDriver, whether it be in Selenium itself or a browser specific limitation, javascript is your ally.

Though javascript isn’t all power and limitless, there’s much that can be done with javascript provided you know what to do with it and how to use it.

With Selenium RC, you have access to the browserbot object that gives you javascript-based access to windows and the HTML Document Object Model (DOM) via Selenium. You can do web search on Selenium browserbot to learn more about it or find usage examples. One of my posts show how to use it in interesting ways (under the DOM examples): Special element state validation with Selenium and CSS and DOM

With Selenium 2 / WebDriver, you have even more power with javascript as you’re given direct access to javascript with the only security restrictions being that of the browser itself (e.g. no system file access, no cross domain AJAX calls w/o workarounds), whatever you can do with javascript, you can execute with the WebDriver JavascriptExecutor. Again, you can web search for some examples.

Now the goodies:

Unfortunately, most Selenium users are not well versed in javascript (not front end web developers), and fortunately, Selenium is good enough 80-95% of the time that you don’t need to resort to javascript. But for the 5-20% of the time where you need a workaround, javascript might be it. So first, it is good idea to learn javascript programming so it can help you be better with Selenium. Outside of Selenium, it also helps you better debug/troubleshoot front end web application bugs.

In this blog post, I’ll list some useful javascript related content links that apply towards Selenium. And I’ll update this post as I find more. Sort of acting as a central repository of useful Selenium javascript workaround hacks. Since it’s not easy scouring the web for such solutions. Feel free to contact me with additional hacks you find.

NOTE: to understand how these hacks work, you have to learn more about javascript…

List of useful Selenium workaround javascript hacks

Verifying images - “really” verifying an image is rendered/displayed on the browser, not just a broken link
http://watirmelon.com/2012/11/27/checking-an-image-is-actually-visible-using-webdriver/

Checking scrollbars – checking whether an element (div, iframe, textarea, etc.) that holds other content has scrollbars (vertical and/or horizontal). The solution here is for Selenium RC in Java, but can be adapted for WebDriver and other languages. A good javascript learning exercise on how to adapt it.
http://selenium-automation.blogspot.com/2010/10/selenium-automation-problems.html

Manipulating (or clearing) HTML5 local & session storage
https://gist.github.com/anonymous/3972875
Can add/remove, check for data, etc. but in terms of automated testing, we’re more likely just wanting to clear storage cache like clearing cookies. The above link is a Java example for session storage, but can be adapted to other languages, and local storage has the same API, just window.localStorage instead. And read this for more info on working with HTML5 local & session storage: http://diveintohtml5.info/storage.html

Drag & Drop using javascript instead of native interaction / Actions API
http://ynot408.wordpress.com/2011/09/22/drag-and-drop-using-selenium-webdriver
In case the native actions drag & drop fails to work for any given browser, or where not available like SafariDriver

Mouse over using javascript instead of native interaction / Actions API
http://code.google.com/p/selenium/issues/detail?id=2067, comment #60 & other related comments (before & after that are relevant). This method useful when the native method fails for any given browser, and currently is only option for SafariDriver.

Mouse click using javascript instead of native interaction / Actions API or WebElement.click()
See the mouse over link above for code. You then just need to modify the code to replace “mouseover” with “click” and “onmouseover” with “onclick”. How is this useful? When the regular WebElement.click() method fails to cause the expected click action, and in case the native interactions / Actions API fails to work either (or where not available like SafariDriver).

Force setting value to a (Web)Element – sometimes, you can’t seem to manipulate a WebElement as action does not seem to take effect or it is not allowed because the element is not visible/displayed (e.g. hidden). One trick is to use javascript to set the value (e.g. value of a form input field, or value of a hidden input field that is actually used by web app and set via javascript when you perform action against some other element that’s usually near it in the DOM tree). For an example of this, see my blog post (WebDriver update at the end): Special element state validation with Selenium and CSS and DOM

Mouse wheel zoom action
http://stackoverflow.com/questions/6735830/how-to-fire-mouse-wheel-event-in-firefox-with-javascript
This currently doesn’t appear to be available in Selenium, even with the native interactions / Actions API? Or I overlooked it. Here’s a possible javascript solution. I haven’t fully tested whether it works or not. As the link is to web developer/application code, you do have to adapt it to work within Selenium. Not simply copy, paste, and run.

more to come…

Performance and usability testing Facebook integration

Just wanted to blog about this as there doesn’t seem to be much discussion about this online or I’m not searching right.

I am assuming either of the following is occurring at present:

• nobody has really implemented performance and usability testing around Facebook integration
• people/organizations have done so (or to some extent) but have decided for company policy not to share their experiences

I write this because this is an interesting area of testing. In terms of load/performance testing, and also automation, there’s a fine balance between doing accurate and valid testing and abusing Facebook’s terms of service (for actual user accounts anyways). And on the usability performance side, it may be a pain to “set up” a Facebook account (real or not) with thousands of friends, or photos/albums, or posts, or likes, etc.

From my research, there’s only 2 ways of doing this:

• using real Facebook user accounts, perhaps sometimes painstakingly set up (unless someone programmed a bot to do all the set up work). And being careful not to create too much activity that Facebook would ban/delete the account.
• using Facebook’s test users API, which requires significant amount of investment in reading up on the documentation, and developing API client calls to interface with Facebook to do all the set up work for the accounts (unless done manually later) then to use the APIs with Selenium/UI automation and HTTP/web service/API load/performance testing.

See these links for references:

http://stackoverflow.com/questions/8845614/testing-a-facebook-connect-application-using-selenium

http://developers.facebook.com/docs/test_users/

For me, I’m not looking to getting the details and code from others on how they automated Facebook for usability/performance testing but rather hearing experiences from others on how things worked out. What worked, what didn’t, pitfalls to avoid, etc. Like whether Facebook’s test users API is all that useful for rich functional automated testing or for high volume performance testing. For example, here are some Facebook integration examples in terms of from your site/service access Facebook for:

• photos, where user has thousands of photos or albums
• login, and you want to test what happens if you had a lot of users on your site/service login via Facebook connect at a given time period (burst or ramped up)

I find it also interesting that Facebook doesn’t mention anything about this area to help themself or other companies with regard to testing usabillity/performance with Facebook integration, since it affects them to some extent as well.

Taken further, this could also apply to other service integrations like OAuth login with Google, Yahoo, WordPress, etc.

Your  thoughts?

Getting window handle on existing windows in javascript by use of window name

I came across this recently. Don’t recall if I found the basis for technique off the web or not (think I did, forgot the source URL), but adapted it for use in javascript code.

In rare cases, could also be used for Selenium/WebDriver, if you’re not using their built in window handling APIs.

//get handle to some existing window
var someWinHdl = window.open(null,”some existing window name if you know it”);

//get handle to self (current window)
var selfWinHdl = window.open(null,window.name);

granted getting the current window handle this way isn’t all that useful though since you could also just use “window.self”. But it is useful to gain access to an existing window if you know it’s “window.name” value.

This same technique can also be applied for opening new windows to keep a handle on the newly opened window, just provide a real URL instead of null to window.open().

With this window handle in place you could access it like any window object such as closing it, using the stored handle variable example above:

someWinHdl.close();