Archive | Frameworks RSS feed for this section

A page object representing both desktop and mobile views or website and mobile app

2 Mar

It just occurred to me today, from reading a Selenium forum post, that a past blog post of mine can also apply to the following cases:

  • you have both desktop and mobile views for a website or web app (e.g. responsive design, etc.)
  • you have a website and a native mobile app that offers similar service/functionality

and as such in both these cases there is shared logic (user/app/site workflow, element locators – even if the actual values differ, but the “logical” representation is the same such as login button on both web site and mobile app, etc.).

Using the techniques defined in that other post, you can share locator references/variables and share page object methods and have a single page object represent both mobile & web versions. For sharing of methods with branching logic, instead of checking which A/B flow to go through, you check what platform you’re on and go to appropriate branch logic based on that.

Also on some specifics, I haven’t tested this type of implementation before, but in theory should work out even for cases where you have website and a native mobile app:

  • You just have to use the right driver instance in the page object methods (or instantiate the page object with the right driver instance), etc. based on the platform (e.g. Appium/ios-driver/etc. vs Selenium WebDriver).
  • For shared locators, XPath may work best, using the multi-valued locator approach with pipes “|”. That is because I know Appium (at least) supports XPath based locators. Not sure about CSS selectors. So, I’m hoping/assuming Appium supports the multi-valued XPath functionality. Don’t know about the other tools lke ios-driver, etc. If this doesn’t work out, you would need separate locator references/variables then.

Some of you might not agree, but this is one way to do it with code reuse via some manageable complexity. The alternative is to have separate page objects with separate methods and locators. That keeps things simpler but give you extra files, extra code, and some redundancy when some of that code and locators are kind of similar. In the end of course, choose whatever works best for you.

Advertisements

To re-invent the wheel or not? And testing frameworks…

16 Sep

Was reading a slide presentation today from another blog post: http://www.slideshare.net/abagmar/automate-across-platform-os-technologies-with-taas. Brought back memories of earlier work I did of defining a test automation framework: http://yadiytaf.sourceforge.net. I still think my YADIYTAF specification is a nice read. Although I never did create a reference implementation of the spec. I ended up prototyping what I needed at the time against the existing RobotFramework (RF). From that effort, I ended up expanding the functionality/feature set of RF by expanding it’s interoperability with Java, .NET, Perl, PHP via their remote library server interface implementing servers in those languages. So RF is a lot more powerful now since I first evaluated it years back.

In the end, makes me think, should one re-invent the wheel or not at times? Or see what’s already available and build upon that. I guess it depends on what works for you or not.

Complex test scenario example to truly evaluate test frameworks?

6 Jun

This came to mind in a discussion I read a while back, so thought I’d mention it. Not sure if I already mentioned it or not. But the general way we evaluate test frameworks, is rather simple. Or supposedly our system/application is not so complex to test after all, at least in terms of interactions among different component types, which themselves need to be UI/functional/user acceptance tested. For example, these days, most systems to test involve a website front end and mobile app client front ends, both of which talk to a server back end. And the website and mobile app clients don’t often need testing where they interact with each other (or rather say you test something on one client and verify on the other for example). Instead we simply test them standalone, independent of each other (or maybe mock the other side). And/or we got lucky in that it happens that the different clients (web, mobile) happen to be able to use the same platform (e.g. Java, Selenium-API) so that we can use the same framework to test them.

But truly testing the capabilities of a test framework is when you have a complex scenario that involves multiple components that need to be tested in integration (whether you mock them or not), and for the sake of simplicity in this argument, let’s say if you had to mock them, you mock them in the same language as the original component (not mocking to the platform of the desired test framework, etc.). Thus in a truly complex scenario/world, you won’t be able to have all things in Java, .NET, or whatever.

One example showcasing testing a system that comprised of the following components would be ideal: a website or web application/client, an iOS mobile app client, an Android mobile app client, a desktop GUI client, a SIP/VoIP client, perhaps a terminal/console client talking to server back end, perhaps networking communication w/ server on backend (some type of TCP/IP or other network communication), REST API web services utilized by the web/mobile/desktop clients, some native Java/.NET/etc. APIs to call. And let’s assume each of these components are built with different technology stacks and/or the tools to test them (not the framework) use different technology stacks.

Such scenarios may void certain test frameworks out of the box, unless one doesn’t mind writing custom glue logic to bind the different testing (components, tools) together within the single test framework. But a few frameworks do have better support built-in that allow you to integrate different tech stacks and tools to the framework, granted it’s not necessarily plug & play, but it minimizes the amount of effort in glue logic to integrate. I’ll leave it as an open thought on what frameworks are useful for such complex scenarios and which aren’t so much.

WebDriver API and JSONWireProtocol is not just for web and mobile applications testing, it can be for desktop too!

19 Jan

And so, here’s a blog post about just that. I’ve found that the WebDriver API in general, with regards to common location strategies of ID, name, class, tag, and kind of XPath, along with element manipulations–such as click, type/sendKeys, get/set text, get attributes, mouse operations, key up/down, taking screenshots, finding elements & validating properties (enabled, visible, selected)–all that being common across web, mobile, and desktop.

Selenium/WebDriver started off for web applications. Then came along Appium, ios-driver, etc. which expanded it to the mobile space, first iOS then Android, and now somewhat even Windows Phone.

But there has been very little in the desktop area. The first for it was Appium for Mac. And we’ve seen nothing else since. Though if you consider unofficial Selenium/WebDriver-like APIs, then maybe there’sa few more: Twin and sikuli-remote-control. But now, I’ve worked out some proof of concept prototypes (based off the old Appium Python implementation) to give you more options to desktop automation using WebDriver API and for Windows, not Mac!

https://github.com/daluu/AutoItDriverServer
https://github.com/daluu/AutoPyDriverServer

Check them out. Pretty interesting. The AutoIt one has a Selenium integration demo showing automation of AutoIt and Selenium together against websites using 2 drivers – a “web” driver and an AutoIt driver. They’re not Selenium Grid compatible yet, something to look into for the future. But at least they work for remote deployment. Though there is a way to make it work unofficially with Selenium Grid deployment of Selenium tests (but where these desktop UI servers are just not officially part of Grid as nodes) – this solution will be presented in a future blog post.

And last, I had wanted to complete with a three’s company big bang, but I couldn’t get Sikuli working just yet, maybe in the future…

https://github.com/daluu/SikuliDriverServer

So try them out, submit feedback, send pull requests with enhancements and bug fixes, etc.! 😉

P.S. all this made possible by the great work of others. Like the Appium team for the server base I used. And if/when I work on the Java server version – the Selenium team or ios-driver team, and .NET server version – Jim Evans for his Strontium server implementation. As well as the work of those who build great free or open source tools like AutoIt, Sikuli, AutoPy.

Update 6/5/2015: since my post, I found a new solution that’s better and more recent than Twin for desktop Selenium-style UI automation – Winium.

Selenium page objects beyond pages like a cart object?

23 Sep

Selenium page objects is a design pattern to help you model test code better. But one doesn’t have to follow the full guidelines of the design pattern.

Some people have used it to model parts of pages as well (headers, footers, navigation, templates, widgets, etc.).  But perhaps it can be useful for more than that, and some people might have already done this or not, as I couldn’t really search anything up or don’t know what/how to search specifically for this. If people have already done this, they haven’t widely publicized it.

What I have found so far is this: https://github.com/cheezy/page-object/wiki/Indexed-Properties

It’s an interesting piece to review. I had this similar thought in mind recently and decided to blog about it:

A shopping cart page doesn’t really do much really. It contains cart items, and offers a visual call to action (click button) that then takes you to checkout. That along with standard site (header/footer/navigation) template actions (login, logout, links to other areas of site).

The core functionality in the shopping cart page really belongs to the cart items and what you can do with them. So in my mind, having the shopping cart page object manipulate cart item actions doesn’t seem quite appropriate for object oriented design.

For example, this would be how you might typically implement the cart page in basic page object model:

cartPage.updateQuantity(cartItemIndex, quantity);
cartPage.removeCartItem(cartItemName);
cartPage.getCartItemName(cartItemId);
//obviously access cart item by index, text string name of item, or by some unique ID

However, perhaps you can extrapolate the cart items outside of the page object to manipulate individually as a collection or set of related WebElements (name, quantity field, remove button, etc.) or for more advanced usage as an encapsulated cart item object model itself.

Both modeling options are presented here below (since I find it hard to list code in a “basic” WordPress blog)

https://gist.github.com/daluu/8e4c43d653c07eb0c57a

I would note that modeling cart items in such a way, while being more object oriented, can make implementing the cart page object (particularly the getCartItems method) and associated procedure to locate and group the related cart item elements together functionally more complicated. Because often, the web application will not have an easily implemented UI that has all the related elements easily related and uniquely identifiable to each other, especially on a cart page with N number of cart items.

Usually that may require sophisticated use of CSS and XPath patterns to locate and relate the set of elements for N cart items on the page. So this this whole approach is not something novice page object model and Selenium users can easily tackle. It takes time and skill to do. But worth trying out.

In the long run, I feel this type of approach is more maintainable, scalable, and makes the tests more readable. It just requires more thought in architectural design and more work upfront to implement. However, the complexity to implement could be reduced if you can get the developers to make the element locator values easily defined w/o resorting to custom CSS and XPath, and make it work for N cart items, and X related cart item elements (e.g. item color, item description, item this, item that, for every cart item)

If you ask me personally which cart item object model I prefer, it is the latter one that resembles a page object rather than the one that is simple a container of WebElements for a cart item.

What are your thoughts on modeling things or objects on a page like a page object. Cart item is the one that tends to come to mind, but there are others, for which they can be thought of as objects but not widgets, nor headers, footers, or navigation for page object modeling. Some other possible examples include a search result, a category listing, etc.

Also, please do inform me if you come across other articles about using page objects for things like cart items, search results, etc. where we’re not working with a page but some other object per se.

A/B testing and Selenium automation

12 Sep

Don’t know how many people have to deal with this particular topic. But I had to research and deal with it before, so thought I’d mention a few things and share some links.

Whatever the case, unless you need to specifically “test” the A/B testing system/framework in terms of how it allocates/routes users to one flow or another, statistical reporting, and other things, it is best & simplest in terms of UI test automation to just be able to go through both A and B flows during testing.

Some people have mentioned to disable A/B test (e.g. go with A/old/base flow always), or workaround A/B testing. You can’t always work around it, and unless you can simply or easily disable A/B testing, it is simply just going through one of the 2 flows alwasy. In which case, it’s technically not that hard to set up the test framework to be able to switch flows.

Usually the A/B test framework/system has a mechanism to assign flows including always forcing to a particular flow, but each framework/system implementation is different. You would utilize that capability to select the flow to test in Selenium. By the way, usually the implementation involves, cookies, javascript, or URL manipulation. The way I’ve done it so far is to define a table/spreadsheet that defines the flow for each A/B test of interest to specify to Selenium test which flow to force going through. You can have variations of the spreadsheet (different files, different Excel worksheet tabs, different SQL DB tables, etc.). It would also be helpful to build into your framework the ability to query for what the defined/specified flow is for going through so the test knows what to do, as well as query for what the actual flow (as returned by the site/system) is so as to confirm the automation has correctly force set the desired A/B test flow. How all that is done is dependent on your A/B test system.

As for how you implement handling the A/B test flows, it is easiest to tackle if you use page objects. I would recommend encapsulating all/most of the A/B flow checking within the page object class methods, so that the tests themselves and the test author doesn’t need to know the details. That way the test can implicitly support both A/B flows as it is written. The only case you detract from that is when the actual workflow changes between A/B flows (e.g. a functional/workflow change not a UX/visual/icon change) or when you need to verify specifically UX elements like images, icons, and text between the flows. In such case you would then put A/B handling logic in the test (or test utility/parent classes) itself.

Handling A/B flow is simply control and branching logic (if/else/switch/case statements) in checking what flow is to be traversed and handle appropriately based on the given flow.

A tip on handling UI element locators in A/B testing is that one can minimize creating additional locators for an A element and B element, by instead using multi-value locators. This does then force you to use CSS selectors or XPath instead of simply by ID, name, etc. These 2 types of location strategies support defining multiple values and as long as one value matches, then you have found a match for a WebElement.

CSS: a flow value, b flow value

XPath: a flow value | b flow value

as you see in example above, the multi value separator in CSS is a comma and in XPath is the horizontal pipe character.

Also when handling A/B testing within page object methods and attempting to not change a test to support both flows, that means you typically modify a page object method to internally check what flow it should go through and handle it, so that the caller of the method (usually a test, or another page object method) does not need to know about A/B test.

Following my suggestions, it will be easier to manage the A/B testing in progress with minimal code changes across files. And the cleanup work will be easier (when you decide to stick with one flow 100 percent). Cleanup is simply then a removal of branching flows and the obsolete branches and deleting one of the values in the multi-valued locator variables/constants. The locator variable/constant (names) and the page object method signatures all remain the same, and tests themselves may require very little if any update at all.

And now here’s some links to topics on A/B testing that I have seen online that you might find useful:

https://groups.google.com/forum/#!msg/selenium-users/5t3vnPoUXzA/3glQ6kT9oZkJ

http://stackoverflow.com/questions/5269027/how-do-you-handle-testing-with-selenium-when-you-are-running-ab-tests

http://stackoverflow.com/questions/12078675/writing-integration-tests-against-external-resources-which-are-a-b-testing

http://elementalselenium.com/tips/12-opt-out-of-ab-tests

https://speakerdeck.com/neurites/accessibilitytesting

http://sauceio.com/index.php/2011/07/easiest-ever-ab-split-tests/

Update 3/1/2016: it just occurred to me today, from reading a Selenium forum post, that this blog post of mine can also apply to a case where you have both desktop and mobile views for a website or web app (e.g. responsive design, etc.) or a case where you have a website and a native mobile app, and as such in both these cases there is shared logic (user/app/site workflow, element locators – even if the actual values differ, but the “logical” representation is the same such as login button on both web site and mobile app, etc.). Using the techniques defined here, you can share locator references/variables and share page object methods and have a single page object represent both mobile & web versions. For sharing of methods with branching logic, instead of checking which A/B flow to go through, you check what platform you’re on and go to appropriate branch logic based on that.

IDEs and fancy development tools can be bad for you

23 Aug

Especially for testers (at least those without a good developer/hacker type skillset), you become reliant on them. And when you want to do something outside of them, you are lost.

I’ve seen quite a few posts from testers about how to execute Java (Selenium/TestNG) tests outside of Eclipse, and they usually have maven set up with Eclipse too.

As someone who likes to have options & builds tools & utilities from (glue) scripts and custom mini applications, whether for testing or not, it irritates me to see those questions pop up.

Either the poster is ignorant & an idiot for not knowing how to search up the solution (one search may not give you the entire solution, but search up the components that will make up your solution individually and you can piece together the puzzle to solve), or they are plain lazy to just want someone to solve it for them.

Those who work with Java toolset should know how to compile Java from scratch on command line, how to execute maven from command line, how to execute JUnit and TestNG tests from the command line test runner.

Those who work with .NET/C# should know how to compile the code from scratch on command line, how to execute NUnit/etc. tests from command line test runner, and the similar equivalents to what you do in Java.

Those who work with scripting languages should know how to run code from files on command line, run code from interpreter via command line (in GUI interpreter mode or pure command line).

Those who work with any language should know how to use libraries/packages/modules, installing, referencing them, etc. all without the fancy IDE – just command line compilation, installation, etc. Should also know how to use methods & functions of libraries/packages/modules without auto-complete in IDE, rather simply by looking up quick references and API documentation instead.

Those who work on Windows should know how to work with batch files and the Windows task scheduler. Even better to learn WSH, WMI, VBScript and/or Powershell.

Those who work on *nix should know shell scripting and cron jobs at least.

Pete Zybrick

Bell Labs to Big Data

Seek Nuance

Python, technology, Seattle, careers, life, et cetera...

TELLURIUM

New Era of Test Automation

Der Flounder

Seldom updated, occasionally insightful.

The 4T - Trail, Tram, Trolley, Train

Exploring Portland with the 4T

Midnight Musings

Thoughts on making art

Automation Guide

The More You Learn The More You Play...!

The Performance Engineer

Code.Test.Tune.Optimize.

humblesoftwaredev

Thoughts related to software development

Yi Wang's Tech Notes

A blog ported from http://cxwangyi.blogspot.com

Appium Tutorial

Technical…..Practical…..Theoretically Interesting

LinuxMeerkat

I swear! Meerkats can do Linux

PacketsDropped

Requeuing the packets dropped in my memory.

Two cents of software value

Writing. Training. Consulting.

@akumar overflow

wisdom exceeding 140 chars.