Tag Archives: Testing

iOS mobile testing tips

6 Jun

These tips may be found elsewhere on the web already, I’m just archiving this info I put together a while back into a single repository in the form of a blog post.

Tips for both iOS devices and iOS simulator

Firebug for iOS

Set up a javascript bookmarklet to Firebug Lite, so that you can invoke the lite version of Firebug in mobile Safari on actual device or simulator. See Martin Kool’s article for details, if you don’t know how to set up javascript bookmarklets. NOTE: on simulator, you will lose the bookmark when you do an iOS “Reset Content and Settings…” command. No issue on device unless you specifically delete bookmark or do a factory reset.

View Source of a page

iOS 8+: install an iOS 8 extension like the one called View Source. There are several extensions to choose from. The View Source one may be best as most popular, well known. Or you can also choose the other option for older iOS versions.

iOS 8+, 7.1, and earlier : use this Javascript bookmarklet trick/tip, or this version called Snoopy. NOTE: on simulator, you will lose the bookmark when you do an iOS “Reset Content and Settings…” command. No issue on device unless you specifically delete bookmark or do a factory reset.

Tips for iOS simulator

Set up camera roll to be able to use photos and videos for testing

See: How to import photos into the iPhone Simulator — Aptogo. In general, drag & drop photo into simulator and then tap & hold to save image in simulator will add to camera roll for later use. See the linked article if you need to bulk import photos into simulator. NOTE: on simulator, you may lose the photos when you do an iOS “Reset Content and Settings…” command. No issue on device unless you specifically delete photo or do a factory reset.

For videos, instead of photos, see How do I get video into the iPhone simulator camera roll in iOS 4.2? – Stack Overflow. NOTE: on simulator, you may lose the videos when you do an iOS “Reset Content and Settings…” command. No issue on device unless you specifically delete video or do a factory reset.

Tips for iOS devices

Debugging mobile and tablet websites on iOS – on Windows or locally on device

Tips on mobile and tablet device testing shared usage and remote testing outside corporate network

Sharing bookmarks and photos across devices

If you use an Apple/iCloud account across the devices (This might work on simulator iOS 7/7.1+ if you sign in to iCloud from simulator and then do a Debug > Trigger iCloud sync, although I haven’t tested it):

The photos on each device (if not existing photos, then new photos taken) will be accessible across devices via the Photo Stream section of Photos app. And deleting photo locally in camera roll of a device won’t delete it from the Photo Stream (for that device and all other shared iOS devices), so you have to specifically delete from Photo Stream to delete across devices, the camera roll is treated separately from Photo Stream on each device, so you will notice copies of what you have locally on camera roll also in the Photo Stream that’s available across devices. NOTE: that the Photo Stream (for iOS 6-7.x) is temporary storage, and to make it permanent on each device, you have to save it to local system (device/simulator) before it expires in 30 days (I forget, but the system may copy the photos you’ve previewed to your device after expiration rather than let it disappear from the stream). The iCloud Photo library in iOS 8 is different though with it being permanent storage.

Creating a bookmark on one device will automatically sync (if enabled, by default) them to the other devices using same account, so you don’t have to set up bookmarks/bookmarklets individually on each device.

Recording actions against or test session of iOS physical device into a video

For iOS 8+ only, but see Record your iOS Test Session with QuickTime | Daniel Knott

Tips for simulating iOS device in browser via user agents, etc.

Device screen sizes for testing mobile and tablet views of a site and user agent tools

Tips on mobile and tablet device testing, shared usage, and remote testing outside corporate network

6 Jun

Here’s some useful tips for development and QA testing of websites or apps on mobile/tablet devices, if you didn’t already know.

Tip on wireless network login (accounts) when sharing between people

This tip is for those who use a corporate wireless network that has user accounts per person without a common shared account (or guest access to internal resources).

Rather than ask the previous user for their network login password if/when prompted by the (iOS) device, here’s how to resolve that issue of sharing between users. The tip here is specific to iOS but should be similar for Android, Windows Phone, Windows, Mac, etc.

Option 1 – login with your own network login

You only need to do this if ever prompted to login with only a password (or prefilled username) rather than the whole username & password.

  1. Clear the cached user login info.
    1. Settings > Wi-Fi > find network (name), tap the wifi network’s settings/properties icon (a blue circled arrow on iOS 6, an “i” circled tooltip on iOS 7).
    2. Choose Forget this Network
  2. Find the network (name) again in wifi settings
  3. Select that network. You’ll now be prompted for the login username & password, and not just a password to cached login
  4. If you don’t like “your” login cached, then “forget” the network after using the device.

Option 2 – use other available network but then connect to internal network

This method won’t require you to keep swapping user logins when device shared between people. This only works if you have a wireless access for guests (that perhaps only grants access to internet and some internal resources but not all), or some public wireless access but not corporate wireless access.

  1. Connect to the guest or public wireless network
  2. Connect to internal network using VPN app on the mobile/tablet/device
  3. Disconnect VPN when done testing

Tips on remote testing at home or outside local network

When you’re at home or outside the local network, you don’t have access to internal resources. Here’s some options.

Option 1 – device VPN

Use VPN on the device to connect internally.

Option 2 – shared computer VPN

Use VPN on your computer and route network requests from device through a proxy server running on your computer, assuming both computer & device are on same network. This method allows you to share VPN from computer to device.

Tips on remote debugging of mobile, tablet devices

For Windows, see Debugging mobile and tablet websites on iOS – on Windows or locally on device

For Mac, search the web for iOS remote debugging for details.

Tips on HAR file capture on mobile/tablet devices

For iOS: https://github.com/andydavies/ios-har-builder

For Chrome mobile (Android)?: https://github.com/cyrus-and/chrome-har-capturer

Device screen sizes for testing mobile and tablet views of a site and user agent tools

6 Jun

These tips may be found elsewhere on the web already, I’m just archiving this info I put together a while back into a single repository in the form of a blog post.

Table quick reference for device screen sizes

Device Type Screen Size (width x height, in portrait view, landscape = swapped values)
Old iPhones up to iPhone 4S 320×480
iPhone 5 series 320×568
iPhone 6 375×667
iPhone 6+ 414×736
iPad (any model so far) 768×1024

 

Online references for these screen sizes:

CSS Media Queries for iPads & iPhones | Stephen Gilbert

http://www.emailonacid.com/blog/details/C4/adapt_your_media_queries_for_apples_new_iphone_6_and_6

css3 – iPhone 6 and 6 Plus Media Queries – Stack Overflow

Some desktop browser tools to assist with testing mobile and tablet views of a website (there are more you can lookup yourself)

User-Agent Switcher for Chrome – Chrome Web Store

Window Resizer – Chrome Web Store

User Agent Switcher :: Add-ons for Firefox

Firesizer :: Add-ons for Firefox

Safari has bulit in user agent switcher between iPhone, iPad, and normal Safari desktop. Not sure about resizing window to match user agent though.

Complex test scenario example to truly evaluate test frameworks?

6 Jun

This came to mind in a discussion I read a while back, so thought I’d mention it. Not sure if I already mentioned it or not. But the general way we evaluate test frameworks, is rather simple. Or supposedly our system/application is not so complex to test after all, at least in terms of interactions among different component types, which themselves need to be UI/functional/user acceptance tested. For example, these days, most systems to test involve a website front end and mobile app client front ends, both of which talk to a server back end. And the website and mobile app clients don’t often need testing where they interact with each other (or rather say you test something on one client and verify on the other for example). Instead we simply test them standalone, independent of each other (or maybe mock the other side). And/or we got lucky in that it happens that the different clients (web, mobile) happen to be able to use the same platform (e.g. Java, Selenium-API) so that we can use the same framework to test them.

But truly testing the capabilities of a test framework is when you have a complex scenario that involves multiple components that need to be tested in integration (whether you mock them or not), and for the sake of simplicity in this argument, let’s say if you had to mock them, you mock them in the same language as the original component (not mocking to the platform of the desired test framework, etc.). Thus in a truly complex scenario/world, you won’t be able to have all things in Java, .NET, or whatever.

One example showcasing testing a system that comprised of the following components would be ideal: a website or web application/client, an iOS mobile app client, an Android mobile app client, a desktop GUI client, a SIP/VoIP client, perhaps a terminal/console client talking to server back end, perhaps networking communication w/ server on backend (some type of TCP/IP or other network communication), REST API web services utilized by the web/mobile/desktop clients, some native Java/.NET/etc. APIs to call. And let’s assume each of these components are built with different technology stacks and/or the tools to test them (not the framework) use different technology stacks.

Such scenarios may void certain test frameworks out of the box, unless one doesn’t mind writing custom glue logic to bind the different testing (components, tools) together within the single test framework. But a few frameworks do have better support built-in that allow you to integrate different tech stacks and tools to the framework, granted it’s not necessarily plug & play, but it minimizes the amount of effort in glue logic to integrate. I’ll leave it as an open thought on what frameworks are useful for such complex scenarios and which aren’t so much.

What the f**k to test?

23 Feb

Was reading my emails and some blog posts today and came across these:

http://lifehacker.com/5911385/seven-helpful-sites-that-tell-you-what-the-fk-to-do

https://github.com/soulwire/WTFEngine

it made me think, as a tester, it would be interesting to have/see a site about “What the f**k to test” or something similar. It could be useful to the entry level and novice testers just getting into testing profession. And it could be useful to all of us testers when we happen to feel lazy or have “tester’s block” not being able to test or decide how/what to test.

Or do we actually have something like that already? If so, let me know.

If not, anyone interested in helping build such a site/service? It would be a site that randomly decides to give you the tester a tip on what to test for your site/application/system (in general terms that are not specific to your site/application/system).

Thoughts on a Selenium interactive exploratory test and debug tool

24 Jan

I came across a nice Selenium-based tool recently called SWD Recorder. You can find more about it here or watch this video. But for me, the topic to discuss today is how it could be extended to offer features that I had been planning to build into a tool that would use the Selenium API. But since a similar tool already exists now, I can build off of it rather than create from scratch. But in case I don’t get to it yet or never do, I’m blogging my thoughts on what I would build upon this current tool.

The tool is great but I had been looking for specific features (to have), which don’t necessarily align with this tool’s original goal and thus I’d fork it for my own purposes. In general, I’ve been looking to build a cross browser Selenium interactive debugging tool that has the following features:

  • A Selenium IDE or Selenium Builder like command executor. But cross browser compatible and doesn’t run as browser extension but separate tool that drives browser via Selenium APIs. Basically the tool provides a set of standard API commands to select from and user supplies the locator value and then you can click execute/run to execute the selected command against specified locator. I’m not looking for option to dynamically add a list of commands to execute like a test sequence/scenario/case or be able to save that to disk as a test case/suite file that Selenium IDE/Builder offers. Rather it would be more for exploratory test/debug session to see if certain Selenium commands work against specified element in given browser, etc. But those additional options I mention that are not of interest to me could be extended/added by other folks for a type of Selenium IDE/Builder that actually works across browsers (not just for running tests but recording actions or for manually defining/creating the action list/set). Examples of commands to execute: click, sendKeys/type, getText, getAttribute, drag & drop.
  • Expand the previous bullet to also offer a list of javascript emulated/simulated action commands (not native actions/interactions API from Selenium but pure javascript events & emulation of actions) to execute. This would be nice addition that even Selenium IDE/Builder doesn’t offer I think. It serves as a way to see if javascript workarounds for some Selenium commands will work against given browser or not. Without such feature, the only way to test this is to actually wrap & executed the needed javascript code within your test framework to then test & try out, which also requires trial test runs of some test case with setup & teardown that takes time, or using a Python/Ruby shell to execute it but which requires you to type it up or load a pre-written library. But why waste all that time when you can test out quickly in a special debugger tool like this. Examples of javascript wrapped commands: drag & drop, mouse click, mouse over, mouse up, mouse down, scroll mouse wheel. See this blog post about javascript workarounds for ideas.
  • Add alternate options for finding & testing locators. For example buttons to inject Selector DetectorSelector Gadget, and Super Selector among other similar tools without need to set up bookmarklets. Enhance the tool’s current WebElement explorer with option (by default or not) to generate a matching CSS selector (if applicable) for the given XPath that is generated. In this case, for Windows we’d have to have a pre-compiled binary of CSSify Python script for Windows, while on other OSes, we could perhaps just run the script natively. Or better yet, we instead screen scrap the CSSify public page/service to call the underlying “web service” (an HTTP POST call) to translate the XPath to CSS without havng to shell execute a Python script. This collection of goodies would then finally give Selenium users a cross browser element locator finder & tester tool to find elements (their XPath and/or CSS) and then modify the default XPath/CSS value and see if it still works, etc. A tool comparable to FirePath and Firebug. Granted it won’t be a perfect equivalent since it doesn’t quite show the HTML DOM source relative to the inspected element like Firebug & FirePath do at this point.
  • Add alternate debugging options across browsers. For example, have button to inject & load FirebugLite without having to set up bookmarklet.
  • Add (sort of) cross browser javascript error collector functionality for debugging with Selenium. Have buttons to inject the javascript error collection code snippet, and button(s) to check/display/retrieve the collected javascript errors (error count and the specific list of messages). Or instead of button(s) to check errors, it could be automatically dumped out in the tools display after the injection whenever error occurs. This functionality of course would not support javascript page load errors (only after page load) and would not persist across pages, have to manually inject on every page desired. Unless we enhance the tool by modifying the Selenium source code (.NET binding?) to auto inject on every page load. Now this might kind of seem pointless, but it’s sort of a cross browser solution as well as nice alternative to IE’s not so nice developer console or alert dialogs of javascript errors. It’s also a way to test out how well such a solution would work when you actually implement the same into your test framework without you have to do that first by evaluating it in an interactive debugging mode.
  • Have a tab section where you can inject arbitrary javascript source files (via HTTP URL) into the current page. It’s a lot easier than having to manually write the javascript code snippet to inject the script element with src set to the HTTP URL then to execute that javascript snippet with Selenium command in say an interactive Python/Ruby shell. Just paste the URL to the GUI tool’s text field, then click inject script.
  • Followng on previous bullet, also nice to have a section for a cross browser Selenium javascript console, whee you can execute any desired javascript code snippet via Selenium WebDriver’s JavascriptExecutor. You paste or type the code snippet in a textarea field and click execute. Any return value is cast as a String and dumped back in a results area for user to see in the tool. This provides a javascript console equivalent to the browser’s native developer tools but one in which the code is executed by Selenium rather than directly by the browser. It would be a nice way to test out whether certain javascript can be executed or whether it works well with Selenium before you actually code it into your test framework or test. I used to and currently do this over a Python interactive shell but it’s more simpler to do this over a GUI tool, especially for novice users.

Now from all the features mentioned above, to summarize, I’m looking to have an interactive GUI-tool based Selenium exploratory test & debugging tool. One in which you can test out code snippets, locators, and Selenium commands cross browser before you actually code it into test framework. Others might prefer the direct approach but I personally prefer interactive test & debug first as it is a lot more insightful and faster this way than to put everything in framework and a throwaway code test script that also takes much more time to execute through the test flow or having to set breakpoint in debugger and debug from that point. Having such a tool, you can easily combine manual & automated steps in one to see how things work. Such a tool is essentially a GUI version of what I talk about in previous posts:

How to debug test and try selenium code with interactive shells

Using selenium with interactive interpreter shells

A selenium IDE alternative for other browsers and another record playback method

Update 01/26/2014:

It recently occurred to me, that it would be nice to be able to inject jQuery and/or Sizzle into Selenium with such a tool via some buttons to click. To be able to test out whether can locate elements defined with jQuery syntax or for the non-standard CSS provied by Sizzle. This of course is only needed in the case of jQuery if the site under test doesn’t already use jQuery.

Update 04/14/2014:

I recently came across a similar too, this one is not .NET based but Java as a JAR file. Love it when the community comes up with new supporting 3rd party tools.

https://github.com/dmolchanenko/LookingGlass

Candidate interviews should include/have a debugging test

11 Oct

I had some experiences where some QA folks are not that quick or sharp at debugging problems in web applications, and automation test failures. Granted you should be intimately familiar with the application to better debug it, or with the automated test framework to know where and what failures happen.

But finding failure then tracking it back is a general principle that’s not that hard to do if you practice and do it enough. And in automation test failures, the exceptions or error message stacktrace is good starting point to backtrack against the test code. Combined with rerunning the test (in debug mode or not) and manually observing what happens, you can get an idea and track down the failure. But I’ve seen some folks not quite get the hang of it at tracking down the exact failures to an automated test. Or how to find further details of a failure when testing manually (by monitoring the AJAX, HTTP calls, or the javascript console logging of errors).

So it got me thinking that a “debug” test in an interview is useful for interviewing candidates. Purposely build or create a broken website/application and have the candidate find the cause or at least details to the failure than just that they seem something is broken functionally or visually in the UI (find out more why it is so). And for test automation. Same thing, purposely have a test that tests against old/obsolete functionality requirements that have since changed and/or a test that is valid but website/app is broken, hence test fails. Then have the candidate debug the test failure to figure out why the test fails.

Such questions will be useful in seeing how th candidate fares in actual work when debugging is needed. Or for QA candidates & manual/exploratory testing – to ensure the candidate can look into details on the failure and give that to the developer rather than just say I can reproduce and here’s what I see visually, I have no idea what might be causing failure underneath, you take a look.

What are your thoughts on this?

TELLURIUM

New Era of Test Automation

Der Flounder

Seldom updated, occasionally insightful.

The 4T - Trail, Tram, Trolley, Train

Exploring Portland with the 4T

Midnight Musings

Thoughts on making art

Automation Guide

The More You Learn The More You Play...!

The Performance Engineer

Code.Test.Tune.Optimize.

humblesoftwaredev

Thoughts related to software development

Yi Wang's Tech Notes

A blog ported from http://cxwangyi.blogspot.com

Appium Tutorial

Technical…..Practical…..Theoretically Interesting

LinuxMeerkat

I swear! Meerkats can do Linux

PacketsDropped

Requeuing the packets dropped in my memory.

Two cents of software value

Writing. Training. Consulting.

@akumar overflow

wisdom exceeding 140 chars.

Lazy Programmer's Shortcut

Java, J2EE, Spring, OOAD, DDD & LIFE! .......all in one :)

Testing Mobile Apps

www.SoftwareTestingStudio.com

Photofocus

education and inspiration for visual storytellers

No, Seriously...

Freeing up some mind cache!