Should All Testers Be Automators? The Argument I’m Tired Of, But I Argue Anyway

It has come up recently, in several conversations – what is the ideal tester skillset on a team? Some folks are in the “make them all automators” camp, or “teach them all programming”, or even “get rid of manual testers”. I find that I have strong feelings in this argument, but perhaps my thoughts aren’t all together enough to make my case during these conversations. So here I’ll be spelling out my arguments, and the supposed counter-arguments, about the ideal tester skillset.

This is, of course, my opinion. You are certainly free to use the comments to pose yours 😀

What is the ideal project delivery team?

I feel like this is a good place to start, as it lays out my overall viewpoint on an entire team’s structure.

My ideal project delivery team (Agile of whatever flavor, because Waterfall is nowhere near ideal for me):

• 3-5 developers
• 1-2 business analysts
• 1 tester focused on automation (test engineer, software developer in test, whatever you want to call it)
• 1 tester focused on manual (“analyst”)
• PM, PO, etc. etc.

Argument 1 – you don’t need two testers, the automator can do manual testing too

My problem with this argument is that when you have one tester on a team, there is hardly any time for automation. I, for instance, was the sole tester on a greenfield project. There was so much prototyping, trying things out, trying another newer technology out, etc. that there was no time to do automation. Nothing was automatable because it was going to have to change the next iteration anyway, so why even start?

If the project is in a state where the automator can actually write code, that’s where the majority of their time will be spent. Probably by choice. If I’m writing code, I feel like I’m getting more done – something much more tangible than a test case passed. And automation suites are in constant need of updating and maintenance, especially in a continuously changing codebase.

The fact is, if you want one tester to be on the team, they’re going to split their time between automation and manual testing. They’re going to do manual testing enough to get their automation test cases written. Which means that the amount of exploratory testing that should be done will not get done. Yes, automation helps to be able to do more exploratory testing. But if you’re working on that automation, you’re not doing as much exploratory testing either.

My Stand: two testers, each with their own focus, means you’re getting full attention on automation and on manual testing at the same time. One tester doesn’t have enough time to do both to a high enough degree of attention.

Argument 2 – well, can’t a BA pick up some of the manual testing?

Sure, a BA can do some testing. I hope they already are! However, a BA is someone that is closer to the business or to the client the software is for. They should be testing on a UAT kind of level. I wouldn’t expect a BA to know all of the various types of testing they can perform against a piece of software to the depth that a tester would.

Absolutely a BA’s experience is a benefit to the testing of the software being developed. But a BA also has other obligations – they have to research and write stories, collect acceptance criteria, and interface with the client/end-user. They also don’t have time to do two peoples jobs!

My Stand: a BA should do the job of a BA – yes they should do testing, on a UAT level. Testers are valued for their ability to think differently than other roles on project teams. Each role should be able to do its job.

Argument 3 – at least teach all of the testers how to code!

While I agree that anyone in a software testing role should understand how code is written, I feel that a “basics of programming” and “data structures and algorithms” classes would be enough. I don’t think that everyone has to know how to code. Not everyone wants to, and not everyone is adept at it and THAT IS OK! If a manual tester can’t code, that absolutely does NOT mean that they aren’t progressing in their career. If they are keeping up with technology, they understand the base architecture of the software project they’re working on, then they don’t need to know how to code to test!

What does any tester need to know, in my opinion?

• basics of programming
• data structures and algorithms
• how web services/api’s work at a base level
• how to use the browser debugger (if testing a web-based app)
• how to troubleshoot various applications (debug logs, Fiddler, etc)
• the technologies being used to create the application they’re testing
  • what they are at a basic level
  • how they’re being implemented
• how to find out what code got built or pushed to their test environment – what changed, what to test
• how to ask questions when they don’t know something
• what questions to ask
• how to just fucking google it sometimes
• of course, standard testing stuff (exploratory, boundary, security, etc etc etc)

The important things for a tester to know are how to talk to the people around them, and various ways they should test the software they’re testing. Yes, understanding how a developer works, or what they need to do to go from requirement to finished software, is incredibly important for anyone working in software to understand. But they don’t need to know the difference between a List and an Array in C#, and when to use each.

Hey, understanding the limits of an int and how you can test a field that has an int datatype is great. Knowing how to see in the database that the field for a phone number has a float datatype and understanding that you can just put in a bug without testing the frontend because wtf were they thinking – that’s invaluable.

My Stand: So yeah, the more you know, the better. Knowledge is power, after all. But you don’t have to be able to write code to be a good tester. Period.

Argument 4 – we should just automate everything and get rid of manual testers

Lets go back to that “standard testing stuff” and “basics of programming”. Testers have a job because developers are human, and they make mistakes. Also, writing code is fucking complicated.

You can automate a lot of things. But you can’t automate a brain, curiosity, a set of eyes that see that weird thing happening on the taskbar that your automation didn’t catch because it’s not a HUMAN and you didn’t tell it to look there so IT DIDN’T!

A computer will do exactly what you tell it to. It can’t explore, it can’t see those subtle differences, or even the big honkin’ bugs unless you code it to do that. But then there’s always something else that your code is not specifically asked to do so it won’t do it.

I personally still feel squeamish about CI. I don’t like the thought of code going straight to production without a set of eyes on it. I will never be ok with CI.

Please, if you haven’t read it or you need a refresher, check out James Bach and Michael Bolton’s definition of testing and checking (automation = checking, btw, not actually testing): http://www.satisfice.com/blog/archives/856

My Final Stand: We need manual testers. We need automators for the regression checks, unit and integration checks, etc. But we will never NOT need people whose sole job is to manually test an application.

So there you go. We should absolutely not get rid of manual testers. Manual testers are not “at the end of their careers”. Manual testers, that are good at what they do and continue to learn, are indispensable.

Arguments? Further points? Let’s chat 😀