How do you qualify a tester? Look at your customers.

In an earlier post I discussed a job listing for a QA Manager that was described as "perfect for a new grad." I questioned whether a new grad was qualified to manage anything, particularly QA? Let's step back from the management aspect, and consider what qualifies someone for a QA role. What should you look for in a prospective software test engineer in your company? My advice would be to look at your customers.

Sure, there are certain characteristics you want to look for in a candidate, as an employee of your company, a member of your team, and in particular a software tester. But what I want to focus on is the utility of comparing your prospective hire to your current customers based on two criteria: Technical skill level and domain knowledge.

Technical skill level. You want someone who is capable of not only becoming knowledge and facile with your particular products, but also of using them in all of the myriad ways that your customers will. Does the product run on both Windows and Unix/Linux? A candidate who is "most comfortable with Windows" may not be prepared for the command line, scripts, and system utilities that are common in a Unix house.

Domain knowledge. Face it, if your product under test is an immersive RPG (role playing game), you want a hard core gamer putting it through its paces. Is it a tool for musicians, or an accounting package? It's hard to see how you can test either without specialized knowledge, beyond what might be available in a spec or user documentation. But if we are talking about a lightweight user interface, casual game, or general consumer application, the main concern is the skill of the tester, rather than their background.

In my experience with engineering software it has been important to know the size and content of typical data sets; to understand, for example, that due to Moore's Law the "large" circuit testcase you built six or seven years ago is now a "medium" sized circuit. You want to be familiar with the other tools they will be using in conjunction with yours, and the file formats that implies. Engineers, particularly EEs, are very technical - and notorious hacks. Chances are they are loading the data into the tool via script, kicking off runs with cron jobs, integrating the tool into their environment using API programming, customizing interfaces, and parsing results with their own reporting utilities. If you support it, you need to test it. So you need testers who can think and work like your users.

The point is, you want to find the bugs before your customer does. And you never want to have to say, "Gee, how did they do that?"