Selenium WebDriver – extracting an image on page by use of take screenshot and cropping

4 Dec

Normally, if we wanted to get an image off the web page under test, we’d download it externally using the URL extracted off the image element source attribute. Unfortunately, in today’s AJAX based web apps, that doesn’t always work. Sometimes, the elements appear like images visually to the user but are rendered by javascript into DOM elements that are not “image” elements, composed of sprites, etc. on the server side.

In such cases, you have to go an alternate route…

The technique to do it is explained here:

https://groups.google.com/d/msg/selenium-users/DjoEjKwvIN0/sI-siW0gOA8J

and an example implementation in Java here:

https://groups.google.com/d/msg/selenium-users/8atiPIh39OY/Gp9_KEXnpRUJ

Side comment 1 – it would be nice if others in the community can contribute other language implementations to the Java source code example above so others can use it. Python, Ruby, C# perhaps? And a sprinkle of PHP, Perl…I may contribute code snippets when I have time.

After you’ve obtained the image, you can do whatever you want with it. Binary/MD5/SHA-1 hash compare the image against a known benchmark one?

For my case, that’s not sufficient as the image isn’t always a single image but a collection formed as one and dynamically generated. So we do some flexible fuzzy logic image comparison with our internal in house image comparison solution.

And on that topic, side comment 2:

I’ve noticed that taking screenshot then cropping to desired element location & size isn’t 100% cross platform compatible. So blogging this for reference to others and to see if anyone else have same issue. The problem here is that different browsers crop with slightly different coordinates and/or width/height ending up with cropped images that differ slightly across browsers. Now I haven’t tested across all browsers, but my findings reveal that Safari provides the most exact desired cropping, while Firefox and IE have slight variations that are not desirable. I didn’t test Chrome, nor mobile Safari with Appium.

With that in mind, you definitely can’t do hash/binary compare of the cropped images across browsers due to differences, unless you have a different benchmark image per browser. So with a single benchmark image, you’d need some flexible fuzzy logic image comparison technique. There’s multiple options here, but none readily available, other than maybe ones like Sikuli. The in house solution we use is based off ImageMagick and does the dirty work of calculating the parameters for the image comparison for you wrapped in a nice REST API.

Now anyone have experiences they can share around the topics discussed above?

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

ÜberDev

open notebook

a happy knockout mouse.

my journey into computer science

Perl 6 Advent Calendar

Something cool about Perl 6 every day

technolinchpin

Inspire and spread the power of collaboration

Niraj Bhatt - Architect's Blog

Ruminations on .NET, Architecture & Design

Pete Zybrick

Bell Labs to Big Data

Seek Nuance

Python, technology, Seattle, careers, life, et cetera...

TELLURIUM

New Era of Test Automation

Der Flounder

Seldom updated, occasionally insightful.

The 4T - Trail, Tram, Trolley, Train

Exploring Portland with the 4T

Midnight Musings

Thoughts on making art

Automation Guide

The More You Learn The More You Play...!

The Performance Engineer

Code.Test.Tune.Optimize.

humblesoftwaredev

Thoughts related to software development

Yi Wang's Tech Notes

A blog ported from http://cxwangyi.blogspot.com

Appium Tutorial

Technical…..Practical…..Theoretically Interesting

%d bloggers like this: