This is a concise personal list of questions I came up with to ask prospective QA candidates for roles involving Selenium, web API/REST web service testing, and general website/web application testing. Thought it might be worth sharing for:
- interviewers of QA candidates
- those who are interviewing for a QA job position (interviewees)
But first a background on the topic, before we get to the questions list…
In my brief experience interviewing QA candidates, and given the limited interview time period, I’ve found it generally futile to ask about one’s career experience in areas of my interview interest mentioned in this post. And it is also rather futile to ask Selenium and other (REST/API/web service test) tool specific questions. And also possibly futile to ask questions specific to Selenium test development methodologies (e.g. handling popups, AJAX synchronization, waiting for elements, specific Selenium commands, writing a sample test, etc.). I’ll kind of skip on thoughts on web service API/REST questions since the positions we were trying to fill didn’t really require heavy testing of those and more focused on Selenium UI testing in terms of automation & programming aptitude. And my beliefs here are in regards to finding an ideal QA candidate with skills on par to a developer (at minimum, a junior /entry level developer), but who specializes in testing, a.k.a. a “Software Development Engineer in Test” or SDET. Not your average QA tester with some basic scripting/programming experience or mostly manual testing and using some test tools or use of record & playback automation tools.
So why do I think so above? It’s hard to pry out a candidate’s overall experience and aptitude from just talking in an hour or less. Programming and technical questions might help, but aren’t perfect, especially given the short time period. Asking tool specific questions or questions on Selenium test methodology/processes are useful, but aren’t helpful. Reason is candidate can find out the needed info if they’re good at research and bothered to research them. They’re not hard to learn, and they either know it or they don’t. And to conclude on that, they may appear knowledgeable but after you hire them (or observing other people who have knowledge like your candidate), you might notice that they lack certain skills or ways of thinking to develop good object-oriented and optimized code/tests, as well as come up with creative tips & tricks to work around problems (besides the typical Google search approach for answers in discussion groups and StackOverflow site). So it’s hard to weed out really good candidates the traditional way.
And so I avoid standard Selenium and other stock QA interview questions and came up with a set of my own. I don’t provide answers to these questions nor do I give insight to the what the questions assess. I leave that to the reader to figure out. If you’re smart, you’d have ideas on what they’re supposed to gauge from the candidate. And they’re all meant to be open ended to gauge a candidates skill/experience level rather than a simple finite answer. To me, these questions are ones that can possibly reveal a candidate’s real skill and creativity (or lack thereof).
I’ll update this blog post with additional questions as I come up with them.
(in no particular order of relevance or priority, and each paragraph is a question):
If we wanted to define locators for WebDriver as RC style locators (“id=”, “name=”, “xpath=”, “css=”, etc.), where we pass in to some method an RC style locator and the method’s code will dynamically return a By object or a WebElement(s) object to work with (exposing it to user or not), how would you implement this dynamic conversion or lookup of RC locator to WebDriver? Optionally follow up with how they would handle RC locator where we omit the locator type (id, name, etc. leaving Selenium to figure out the actual locator type).
How would you design a scalable load (or hint at REST/API based) test from a manual test scenario or Selenium/UI automated test that doesn’t stress/test/involve the UI? Depending on candidate response, also mention how would they handle session state where it is required. If they mention use of proxy for record & playback script generation, ask how they might do it w/o proxy or where proxy not feasible.
While performing manual tests and monitoring automated test runs, whether there are any obvious visible UI errors seen or not, how can you monitor for issues, hidden or otherwise, on both the client and server side?
For Selenium RC, can not directly find and manipulate multiple elements so will have to iterate through them in a list (of repeated single element finds), etc. For WebDriver, can work with multiple elements but let’s be creative for fun and try alternate ways to find and work with multiple locators than the native WebDriver findElements option. What alternate methods are there to validate multiple elements or sets of data by using only 1 call to get (root) element and then work with it as a set of data?
When testing a web app manually or via automation, how can you test for (client side) usability and performance?
Given any website or page, using Selenium (RC and/or WebDriver) I want to get counts of the total number of internal links and external links on the page. Internal links are those that are on same domain as the site/page, external ones have a separate domain (or hostname). How would you solve this?
Given a login page, what (types of) test cases or test coverage can you come up with? (Can extend/modify question to some other page like a checkout page. Note the focus on types of tests not number of tests nor all possible test scenarios).
Ask if they’ve dealt with file uploads, downloads, and browser popup HTTP authentication dialogs. If yes, how did they deal with them? Also check to see if they have or know how to do it cross browser not just Firefox and/or Chrome, etc.
If they’ve used AutoIt, Sikuli or other 3rd party tools with Selenium, ask if those tools can be used with Selenium Grid, and if yes, how to deploy them with Selenium Grid. If they’ve come to conclusion that can not use, then why and how might we be able to use them together?
What type of web service/REST API call can you make from a browser without any browser plugins/extensions or creating a web form to send the web service request?
How do you reverse engineer AJAX calls and (REST) web service API calls made by a website or web application (which you can’t see via the UI)?
Ask a question around modeling some page (or section of a page), widget, etc. as a page object in Selenium. Particularly how they would implement particular/specific page object methods for the given “page” (the method signature, and general idea of method implementation). Page locators are of less concern since that’s just filling in values of variables that map to UI elements. Also try to ask/gauge them on templatizing the object for reuse by other pages, tests, etc. One example is if there’s a common page template that other pages derive from (e.g. mobile and desktop versions of site page derive off a master template with similarities for the two different versions but also differences in final implementation).
Ask question(s) on defining complex XPath or CSS selectors for element location strategy. Things like use of ancestor/descendent/sibling node referencing, matching by (partial) string value or regular expression, multiple element matching, XPath functions, etc.