UI test automation of virtual app/browser delivery to local desktop

7 Jul

This thought just came to mind as I read through forum posts recently regarding automation scenarios like the following:

  • a web app or site is delivered to the user via a browser presented within a browser. e.g. via Citrix virtualization solutions delivered over a browser for desktop & browser-based apps.
  • a web or desktop app delivered to the user remotely via virtualization but made to feel as if it was installed, running, and deployed locally on the local machine, delivered via tools like MS Application Virtualization Client, Citrix, etc.

How do we test for these apps/cases, particularly say if we wanted to test against the exact deployment setup of how they are used? It would seem from the current infrastructure, this is likely a manual testing activity. And for automated testing, we are relegated to simplifying the test setup and assuming the simplified setup will behave the same as actual deployment, and just cover any differences with manual testing.

I bring this up as I know not of any existing automation solutions, commercial or free/open source, but particularly the latter, where these kinds of special deployment setups are supported for automation testing. As these deployments are delivered through virtualization technology, unless the test automation tools are integrated in that virtualization delivery (e.g. hosted and running in the same context as the app that’s delivered over virtualization rather than locally on the client receiving the virtualized app), the test tools won’t be able to detect and automate the virtual app. That integration is likely not feasible or easy to do due to platform isolation and security restrictions and sandboxing. It’s not quite the same as just running everything (the app under test & the automation tools) on a remote host and accessing it via remote desktop/VNC to see it in action.

Perhaps this isn’t much a concern for the community/industry until more and more people use this type of app delivery infrastructure where it is ubiquitous like mobile devices/apps are today. Still, something to ponder about for the future of testing. Wonder what is to come in this space for test automation. Or have I just been ignorant and we already have some tooling for this today?


4 Responses to “UI test automation of virtual app/browser delivery to local desktop”

  1. Kevin Cui July 7, 2016 at 4:52 pm #


    Your interesting post remember me my old project when testing a medical system. The idea is to render 3D images from server side and then send the final model to client browser. Why the company decided to use this kind of technology? The main reasons are:
    1. Client’s local machine is not powerful enough to rental 3D images
    2. Clients want to have multiple accesses to the final model at the same time. They can collaborate remotely but together
    3. Client doesn’t want to have complicated local setup

    In short, old school + ambitious + lazy.

    The challenge for me is how to implement test automation. The tool/framework I used is Sakuli, which provides a possibility to recognize the image and then manipulate the user actions. Then we use image comparison tool, like ImageMagick, to compare the result with the reference (generated manually). For text check, there are a lot of OCR tools.

    But in the end, the accuracy is too bad. It needs many maintenance and supports. The worse part was non functional tests: performance and robustness. It’s nearly impossible to get a stable results.

    It sounds sad but I mean it’s an unusual experience learnt.

    • autumnator July 8, 2016 at 6:15 am #

      Indeed, it seems image recognition and OCR tools are the last resort here for test automation. Sadly they don’t work flawlessly and are not quite effective unless used in a well controlled/constrained environment.

  2. Jim Hazen July 8, 2016 at 3:02 pm #

    You are absolute correct about dealing with a “hosted” environment (desktop) or application via a virtualization solution like Citrix. In order for an automation tool to “see” and “hook” the software under test (SUT) properly it has to be in the same memory space (context). Otherwise you are automating against the “client” which is represented as a graphical image of the application/desktop running on the remote server. Basically the client is a big graphical “blob”. Citrix is notorious for this, and is a nightmare to automate against it. I know, I’ve done it (to some extent). This causes a situation like the early days of GUI automation where mouse/keyboard/bitmap capture/playback is used, no messaging/object level recognition/interaction is possible. Tools like Selenium, UFT, etc. that are for GUI level automation will not work.

    The only thing I’ve seen that can break the brick wall is Loadrunner with its Citrix protocol and having the Citrix Agent (sits on the server) installed. Even then it gives you limited object recognition and capabilities. Again, I know as I’ve been there and done that.

    The only real solution is to test/automate the target SUT outside of the hosted environment, or put the tool into the same context/memory space and work with it. If you do the former then you can do things like you normally would and then after it passes those tests you then put it under the hosted environment and re-run by hand the tests that are necessary to prove it is viable/running properly. You don’t do a complete regression run, you only test for things that are affected by hosting (if you know what they are). All-in-all it takes a different strategy to solve/manage the problem.

    Jim Hazen

  3. Ed Linac October 20, 2017 at 9:36 pm #

    great post mate!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Tea in Eighteenth-Century Britain

History of Tea Project at Queen Mary University of London

Phimosis - A Simple Cure (That is working for me)

No nonsense, no adverts, no sales pitch, just honest information.

Glass Onion Blog

Cheat sheets, post-its and random notes from the desk of a programmer

Abode QA

A Hub For Testing Minds...

The Test Therapist

Performance & Security Testing Blog


a programmer's hub

Let's Not Crash and Burn

it makes your brain tingle


“Incinerate Ignorance”

Anastasia Writes

politics, engineering, parenting, relevant things over coffee.

One Software Tester

Trying To Make Sense Of The World, One Test At A Time

the morning paper

a random walk through Computer Science research, by Adrian Colyer

RoboSim (Robot Simulator)

Visualize and Simulate the Robotics concepts such as Localization, Path Planning, P.I.D Controller


open notebook

a happy knockout mouse.

my journey into computer science

Perl 6 Advent Calendar

Something cool about Perl 6 every day


Inspire and spread the power of collaboration

Niraj Bhatt - Architect's Blog

Ruminations on .NET, Architecture & Design

Pete Zybrick

Bell Labs to Big Data

%d bloggers like this: