Safari 5 on Windows may not be able to perform multiple file uploads in parallel

8 Jul

Just wanted to point this out for those who do testing and test automation. Although we’re supposed to be going off Safari on Windows and test directly Safari on Mac. But I’m sure there are still some people who use Safari 5 on Windows to do interim/tentative testing due to lack of resources of a Mac or because it’s handy to have around.

As such, just wanted to point out that Safari 5 on Windows appears to have an issue with uploading multiple files in parallel via the HTML5-based multiple file upload support. At least in the implementation of a site I test.

You won’t run into this issue with Safari on Mac. So just something to be aware of.

Debugging mobile and tablet websites on iOS on Windows or locally on device

13 Jun

I was doing some research and came across these useful tools one can use to debug and analyze mobile/tablet versions of websites/web applications in the case one doesn’t have a Mac handy to do the remote Safari debugging.

Remote Safari debugging on Windows

You can try option of using Telerik AppBuilder (Windows client) as a replacement on Windows for Safari debugger on Mac when remote debugging. There’s a nice blog post about the steps to do it in link below.

http://blog.falafel.com/Blogs/josh-eastburn/2014/03/04/ios-web-inspector-on-windows-with-telerik-appbuilder

The process of using it is much the same as with Safari remote debugging. Have the app open, connect device to Windows via USB, have Safari open on device, then browse device in the app, and open debugger in the app. The tool also will require you to create a Telerik account, and either load/use the default app or create a blank (javascript?) app just to use for debugging mobile Safari on your device.

The tool requires a license or you can use the trial, which becomes a starter edition afterwards. I think the starter edition will still allow you to do the debugging.

NOTE: using this method may require you to have the iOS device drivers to be able to interact with it from AppBuilder. You can get the device drivers by installing iTunes, but if you want to avoid that bloat, you can try to get just the drivers by searching online or downloading iTunes and extracting out the drivers from the installer so you only install the drivers rather than the whole iTunes package, both of which are not covered in this blog post.

Local device debugging on iOS/iPhone/iPad

You can also try these iOS apps below, you can find them in the iTunes App store. They give you built in developer tools like feature (right on iOS, no remote debugging needed) that mobile Safari doesn’t offer. However, they are not the native mobile Safari view/app, though I think it might use the internals of mobile Safari/Webkit to be close enough. These apps are useful when you don’t have a Mac or Windows PC around to do remote debugging with, so you can debug locally on the device through the apps.

MIH Tool – basic edition

HTTPWatch Basic – HTTP Sniffer and Debugger

I gave them a try and they’re at least better than the mobile Safari you get on iOS, unless one needs to target full mobile Safari compatibility. I’m guessing the pro/paid editions of those apps give you more/better features, I confirmed that was the case for HTTPWatch.

Testing with your own environment vs a shared environment

27 May

I do wonder, within QA among all the different organizations/companies, does QA generally test on a shared (test) environment, or have their own testbed environment to work with, or both?

It may be organization/industry specific, as certain fields/areas work better where you have individual testbeds/environments to work with and some are better with a shared environment. The contraints and decision are often based on:

  • resource efficiency/utilization – will people be more productive testing with their own environment or a shared one? Can the environment be shared easily for testing? Is it easy to deploy & manage the environment individually by anyone or is it easier to manage centrally as a shared environment? Cost to deploy individual vs shared environment
  • corporate policy – what are the organization’s IT policy/policies that affect environment setup for testing?
  • test data setup/sharing, reproducibility of bugs – whether better with individual environment or shared
  • codebase sync of test/release branch code with the environments with scheduling/agile in mind – easier with individual vs shared environment
  • hmm…any others that come to mind?

But what’s interesting to me though, is that regardless of the QA test setup, developers will usually have their own environment rather than a “shared” development environment. Why is that? More productive & effective to develop individually before you merge all code to a central codebase & environment for testing? But I could be wrong with this assumption as I’ve not worked at too many places but so far have noticed devs do get their own environments at the places I’ve worked. So perhaps QA should get their own environments too if devs get their own. A hybrid approach can be useful though, shared use environment and individual testbed environments, so you get the best of both, though it does cost more this way.

What are your thoughts on this? I personally prefer the individual environment or hybrid option. It’s nice to have more control of the environment that you test, as long as you are smart & responsible on how you configure and test the system.

From personal experience I can provide the following insight:

  • if the industry/organization deals with hardware, you are likely to get your own testbed environment to configure & manage & test. There may be a shared environment as well where multiple people use it, perhaps in real world type scenario (e.g. PBX phone system with all organization members having an account, phone, and access to that system to test and use what they build themselves)
  • if the industry/organization is about website and/or web applications/services, you are likely to get a shared test environment, individual might not be likely though is awesome if you do get that. The shared environment may or may not be managed by QA, it could be managed by IT, DevOps, or Release Engineering, etc.

Selenium page object modeling challenge – are you up for it?

11 Feb

Having seen some good & bad page object model code & especially after having read Clean Code book, I sure wish there was some book, presentation, blog post, etc. that went over in more detail about the page object model beyond the simple examples presented in various online articles and the official reference example.

Something that presents case studies of good implementations, bad ones, complex ones (ones that you’d expect novices would never implement or have trouble implementing), and examples/tips on how to fix bad implementations into good ones. For bad/complex ones, would be nice to cover things like:

  • shared page widgets, global (or somewhat global) headers, footers, etc.
  • templated/master/skeleton pages where other pages & sites inherit from (similar flow & UI but where the final sites may differ from one another in minor aspects on some flow or UI element implementation/definition like CSS/JS specification)

if such good one presentations already exist, please point me to them. If not, I urge my fellow Selenium bloggers (especially those that follow my blog), fellow Selenium users/contributors/developers, my blog followers/readers to try and take on this challenge (find such a presentation or come up with one to present online, at some meetup, a code camp, or upcoming official Selenium conference).

I would think such a presentation will greatly benefit the Selenium user community, particularly the novice users who lack the initial software development/architectural design skills to appropriately model page objects with the right level of abstraction.

I do understand such a task may not be easy as it requires finding/sharing code that might be confidential or intellectual property (corporate code) unless reworked as generalized code that could be shared publicly. Unless of course we can find some good open source Selenium test code that happens to be bad or complex (or good) implementations to show as case studies. We need to find and present such code in a central location (as presentation, etc.) because it’s labor intensive to scour the internet OSS codebase for Selenium code examples as case studies (for each person to do individually).

What are your thoughts on this? My ideal thoughts are we need a Clean Code type book for Selenium that covers page objects in detail. A sort of “Clean Selenium Page Objects” or “Clean Selenium Code” book.

On that note, maybe you might have code you can share to use for such a case study presentation? Or maybe you could try to get your organization to approve a generalized version of your Selenium page object code (not all of it but some example excerpts) to use for a public case study?

Thoughts on a Selenium interactive exploratory test and debug tool

24 Jan

I came across a nice Selenium-based tool recently called SWD Recorder. You can find more about it here or watch this video. But for me, the topic to discuss today is how it could be extended to offer features that I had been planning to build into a tool that would use the Selenium API. But since a similar tool already exists now, I can build off of it rather than create from scratch. But in case I don’t get to it yet or never do, I’m blogging my thoughts on what I would build upon this current tool.

The tool is great but I had been looking for specific features (to have), which don’t necessarily align with this tool’s original goal and thus I’d fork it for my own purposes. In general, I’ve been looking to build a cross browser Selenium interactive debugging tool that has the following features:

  • A Selenium IDE or Selenium Builder like command executor. But cross browser compatible and doesn’t run as browser extension but separate tool that drives browser via Selenium APIs. Basically the tool provides a set of standard API commands to select from and user supplies the locator value and then you can click execute/run to execute the selected command against specified locator. I’m not looking for option to dynamically add a list of commands to execute like a test sequence/scenario/case or be able to save that to disk as a test case/suite file that Selenium IDE/Builder offers. Rather it would be more for exploratory test/debug session to see if certain Selenium commands work against specified element in given browser, etc. But those additional options I mention that are not of interest to me could be extended/added by other folks for a type of Selenium IDE/Builder that actually works across browsers (not just for running tests but recording actions or for manually defining/creating the action list/set). Examples of commands to execute: click, sendKeys/type, getText, getAttribute, drag & drop.
  • Expand the previous bullet to also offer a list of javascript emulated/simulated action commands (not native actions/interactions API from Selenium but pure javascript events & emulation of actions) to execute. This would be nice addition that even Selenium IDE/Builder doesn’t offer I think. It serves as a way to see if javascript workarounds for some Selenium commands will work against given browser or not. Without such feature, the only way to test this is to actually wrap & executed the needed javascript code within your test framework to then test & try out, which also requires trial test runs of some test case with setup & teardown that takes time, or using a Python/Ruby shell to execute it but which requires you to type it up or load a pre-written library. But why waste all that time when you can test out quickly in a special debugger tool like this. Examples of javascript wrapped commands: drag & drop, mouse click, mouse over, mouse up, mouse down, scroll mouse wheel. See this blog post about javascript workarounds for ideas.
  • Add alternate options for finding & testing locators. For example buttons to inject Selector DetectorSelector Gadget, and Super Selector among other similar tools without need to set up bookmarklets. Enhance the tool’s current WebElement explorer with option (by default or not) to generate a matching CSS selector (if applicable) for the given XPath that is generated. In this case, for Windows we’d have to have a pre-compiled binary of CSSify Python script for Windows, while on other OSes, we could perhaps just run the script natively. Or better yet, we instead screen scrap the CSSify public page/service to call the underlying “web service” (an HTTP POST call) to translate the XPath to CSS without havng to shell execute a Python script. This collection of goodies would then finally give Selenium users a cross browser element locator finder & tester tool to find elements (their XPath and/or CSS) and then modify the default XPath/CSS value and see if it still works, etc. A tool comparable to FirePath and Firebug. Granted it won’t be a perfect equivalent since it doesn’t quite show the HTML DOM source relative to the inspected element like Firebug & FirePath do at this point.
  • Add alternate debugging options across browsers. For example, have button to inject & load FirebugLite without having to set up bookmarklet.
  • Add (sort of) cross browser javascript error collector functionality for debugging with Selenium. Have buttons to inject the javascript error collection code snippet, and button(s) to check/display/retrieve the collected javascript errors (error count and the specific list of messages). Or instead of button(s) to check errors, it could be automatically dumped out in the tools display after the injection whenever error occurs. This functionality of course would not support javascript page load errors (only after page load) and would not persist across pages, have to manually inject on every page desired. Unless we enhance the tool by modifying the Selenium source code (.NET binding?) to auto inject on every page load. Now this might kind of seem pointless, but it’s sort of a cross browser solution as well as nice alternative to IE’s not so nice developer console or alert dialogs of javascript errors. It’s also a way to test out how well such a solution would work when you actually implement the same into your test framework without you have to do that first by evaluating it in an interactive debugging mode.
  • Have a tab section where you can inject arbitrary javascript source files (via HTTP URL) into the current page. It’s a lot easier than having to manually write the javascript code snippet to inject the script element with src set to the HTTP URL then to execute that javascript snippet with Selenium command in say an interactive Python/Ruby shell. Just paste the URL to the GUI tool’s text field, then click inject script.
  • Followng on previous bullet, also nice to have a section for a cross browser Selenium javascript console, whee you can execute any desired javascript code snippet via Selenium WebDriver’s JavascriptExecutor. You paste or type the code snippet in a textarea field and click execute. Any return value is cast as a String and dumped back in a results area for user to see in the tool. This provides a javascript console equivalent to the browser’s native developer tools but one in which the code is executed by Selenium rather than directly by the browser. It would be a nice way to test out whether certain javascript can be executed or whether it works well with Selenium before you actually code it into your test framework or test. I used to and currently do this over a Python interactive shell but it’s more simpler to do this over a GUI tool, especially for novice users.

Now from all the features mentioned above, to summarize, I’m looking to have an interactive GUI-tool based Selenium exploratory test & debugging tool. One in which you can test out code snippets, locators, and Selenium commands cross browser before you actually code it into test framework. Others might prefer the direct approach but I personally prefer interactive test & debug first as it is a lot more insightful and faster this way than to put everything in framework and a throwaway code test script that also takes much more time to execute through the test flow or having to set breakpoint in debugger and debug from that point. Having such a tool, you can easily combine manual & automated steps in one to see how things work. Such a tool is essentially a GUI version of what I talk about in previous posts:

How to debug test and try selenium code with interactive shells

Using selenium with interactive interpreter shells

A selenium IDE alternative for other browsers and another record playback method

Update 01/26/2014:

It recently occurred to me, that it would be nice to be able to inject jQuery and/or Sizzle into Selenium with such a tool via some buttons to click. To be able to test out whether can locate elements defined with jQuery syntax or for the non-standard CSS provied by Sizzle. This of course is only needed in the case of jQuery if the site under test doesn’t already use jQuery.

Update 04/14/2014:

I recently came across a similar too, this one is not .NET based but Java as a JAR file. Love it when the community comes up with new supporting 3rd party tools.

https://github.com/dmolchanenko/LookingGlass

Robot Framework Test Automation book review

9 Dec

I recently volunteered to review a book about Robot Framework (RF) in exchange for an eBook copy. As a user & fan of RF, a previous technical subject matter reviewer for Packt publishing (publisher of the book), and one who loves free stuff, I volunteered to review the book to see what it had to offer. Here’s my review, reposted from my Amazon review below. And first, here’s a link to the book:

http://www.packtpub.com/robot-framework-test-automation/book

I think some of the reviewers may have been a bit harsh in their reviews, so I’ll be one of the few to provide a balanced assessment. However, do note that I’ve read most of the book but skimmed over parts of it and have not looked at the accompanying source code that you have to download from the internet. Therefore, the review is not a complete review of the whole content of the book including the source code.

This appears to possibly be the first book about Robot Framework (excluding the framework’s user & quick start guides), which I think is a positive thing regardless of the quality of the book. As a whole, the book is nicely written for someone not familiar with Robot Framework (RF) and for non-technical people. It provides a good introduction about RF, delves into some of capabilities and benefits of RF, and is a good stepping stone to use & learn more about RF. It is a good bridge or companion to the RF project’s existing User Guide and Quick Start guide, which provide more technical detail and information not covered by this book. For more advanced coverage of RF, look elsewhere, hopefully there will be such a book to cover that in the future.

Having said that, the book does have its downsides. As pointed out by others, there are some typos in the current/first edition of the book, not a whole lot, but a few here and there as I’ve seen. Understandable, but sadly that these were missed before publication. The organization of the book is decent, but could be improved, and the chapter titles don’t reflect well against the actual topics in the chapters, at least from a technical user for what they expect to see based on the title alone as the content vs title doesn’t quite fit descriptively. Only by reading the details of the chapter summary at beginning of chapter or in the Preface do you see what the chapter is really about (compared to its title).

I also found the book a bit lacking in some areas in terms of subject matter or content. The section about Data Driven testing and Behavior Driven Development (BDD) testing could have benefited from elaboration with some actual test case examples and/or code implementation (in the book, not as external accompanying source code samples). The BDD information I felt was too brief in coverage. In same area, the book mentions “DSL” without actually defining the abbreviation (DSL = Domain Specific Language), which is bad form in technical writing as we don’t expect the user to already know abbreviations. The book also mentions RF generically without mentioning which version of RF is being covered at the time of publication/writing.

I may be incorrect in assumption, but the author also might not be active with and well informed of the RF user and developer community. I say this because while the book does mention some common & useful test libraries, there are some omissions whether by choice or ignorance. For example: AutoItLibrary (for desktop UI testing with free AutoIt tool), ranorex-robot-library (for desktop UI testing with Ranorex commercial tool) SimpleSikuli (Java test library alternative version of Sikuli integration that was covered in the book), SSH library. Another thing the author missed was the fact that the Remote Library interface/API of RF provides for much more than what was mentioned in the book, particularly that it can be used to interface RF to other languages, tools, platforms not natively supported by RF (Python/Jython/IronPython) and as such to execute tests in those areas as well. For example: (pure/native) Java (as opposed to through Jython), pure .NET as opposed to via IronPython, Ruby, Perl, PHP, and more. Furthermore, the book mentions that only the Python and Ruby versions of the (generic) remote server are implemented for users to make use of for remote libraries, and for anything else, one would have to create it themselves, though it is not necessarily hard to do so. But in reality, as of this review (and at least about 1-2 years before it), there have been other (generic) remote libraries server implementations already available (Java, .NET/C#, Perl, PHP, Clojure, node.js), so it is truly really easy to use remote libraries and for other platforms. The author should have also put in some references that for more technical information and whatever not covered by the book, the reader can look to the RF user guide and discussion with the RF user/developer community via the online Google Groups, providing the links to them with the references.

So given all this, to an (or a more) advanced/technical user, the book is not very helpful, and for what you can get for free with online searching, the RF user/developer community discussion group, and the framework’s existing user guide, the price of the book may also seem extravagant. But I think it is a useful book that can be used to help try and convince upper management, non-technical business stakeholders to adopt RF, etc.

I’d like to end the review with some positive points at least. Unless one is well familiar with RF, there are some useful things that can be learned or be reminded of from this book. For me, I hadn’t noticed that RF offered a randomization feature when executing test cases and test suites. And the coverage of the test configuration/data files like variable files was a helpful reminder to me. Last, I like the fact (at least what is stated in the book) that Packt publishing has a “Packt Open Source Royalty Scheme, by which Packt gives a royalty to each Open Source project about whose software a book is sold”. Which would mean RF should get monetary donations from Packt for each copy of this book bought by someone. I wonder what happens in the case of refunds/returns by customers though. Does that get removed from the donation amount or Packt still passes that on to OSS projects for the “initial” sale.

Maybe a future edition of the book will be more improved. And/or look forward to seeing an advanced coverage version of the book for RF for more advanced/technical readers.

Selenium WebDriver – extracting an image on page by use of take screenshot and cropping

4 Dec

Normally, if we wanted to get an image off the web page under test, we’d download it externally using the URL extracted off the image element source attribute. Unfortunately, in today’s AJAX based web apps, that doesn’t always work. Sometimes, the elements appear like images visually to the user but are rendered by javascript into DOM elements that are not “image” elements, composed of sprites, etc. on the server side.

In such cases, you have to go an alternate route…

The technique to do it is explained here:

https://groups.google.com/d/msg/selenium-users/DjoEjKwvIN0/sI-siW0gOA8J

and an example implementation in Java here:

https://groups.google.com/d/msg/selenium-users/8atiPIh39OY/Gp9_KEXnpRUJ

Side comment 1 – it would be nice if others in the community can contribute other language implementations to the Java source code example above so others can use it. Python, Ruby, C# perhaps? And a sprinkle of PHP, Perl…I may contribute code snippets when I have time.

After you’ve obtained the image, you can do whatever you want with it. Binary/MD5/SHA-1 hash compare the image against a known benchmark one?

For my case, that’s not sufficient as the image isn’t always a single image but a collection formed as one and dynamically generated. So we do some flexible fuzzy logic image comparison with our internal in house image comparison solution.

And on that topic, side comment 2:

I’ve noticed that taking screenshot then cropping to desired element location & size isn’t 100% cross platform compatible. So blogging this for reference to others and to see if anyone else have same issue. The problem here is that different browsers crop with slightly different coordinates and/or width/height ending up with cropped images that differ slightly across browsers. Now I haven’t tested across all browsers, but my findings reveal that Safari provides the most exact desired cropping, while Firefox and IE have slight variations that are not desirable. I didn’t test Chrome, nor mobile Safari with Appium.

With that in mind, you definitely can’t do hash/binary compare of the cropped images across browsers due to differences, unless you have a different benchmark image per browser. So with a single benchmark image, you’d need some flexible fuzzy logic image comparison technique. There’s multiple options here, but none readily available, other than maybe ones like Sikuli. The in house solution we use is based off ImageMagick and does the dirty work of calculating the parameters for the image comparison for you wrapped in a nice REST API.

Now anyone have experiences they can share around the topics discussed above?

Let's BDD PHP

Just another WordPress.com site

muthutechno

All about coding!

Watir WebDriver

the most elegant way to use webdriver with ruby

AzevedoRafaela

Automated Tests and other stuffs

The Guide On Asian Male Interracial Dating

RedStack

Musings on Integration with Oracle Fusion Middleware

TestO'Matic

A blog on Test Automation, Technical QA, Testing Tools & Techniques, Test Architecture and more…

AutomationRocks.com

Making the QA World a Better Place!

jdhnet

Notes about testing, web development, etc...

Burdette's Test Blog

All about how I test software.

VBA 4 all

programming in VBA

Correct Toes

Align. Strengthen. Restore.

The Informed Healthnut

Science-based health information, recipes, and life!

Don't Make the Same Mistake Twice

best and worst development practices

Adventures in QA

Software Testing, Mobile Testing and Agile Development by Daniel Knott

Muses Of IT Engineer

The Learnings of a QA engineer

c089

some nerd writing about computery thingies

Assert Selenium

Selenium Automation in a Right Way

Geoff Varosky's Blog

SharePoint, Office 365, and Other Musings...

Follow

Get every new post delivered to your Inbox.

Join 43 other followers