Should All Testers Be Automators? The Argument I’m Tired Of, But I Argue Anyway
It has come up recently, in several conversations – what is the ideal tester skillset on a team? Some folks are in the “make them all automators” camp, or “teach them all programming”, or even “get rid of manual testers”. I find that I have strong feelings in this argument, but perhaps my thoughts aren’t all together enough to make my case during these conversations. So here I’ll be spelling out my arguments, and the supposed counter-arguments, about the ideal tester skillset.
This is, of course, my opinion. You are certainly free to use the comments to pose yours š
What is the ideal project delivery team?
I feel like this is a good place to start, as it lays out my overall viewpoint on an entire team’s structure.
My ideal project delivery team (Agile of whatever flavor, because Waterfall is nowhere near ideal for me):
- 3-5 developers
- 1-2 business analysts
- 1 tester focused on automation (test engineer, software developer in test, whatever you want to call it)
- 1 tester focused on manual (“analyst”)
- PM, PO, etc. etc.
Argument 1 – you don’t need two testers, the automator can do manual testing too
My problem with this argument is that when you have one tester on a team, there is hardly any time for automation. I, for instance, was the sole tester on a greenfield project. There was so much prototyping, trying things out, trying another newer technology out, etc. that there was no time to do automation. Nothing was automatable because it was going to have to change the next iteration anyway, so why even start?
If the project is in a state where the automator can actually write code, that’s where the majority of their time will be spent. Probably by choice. If I’m writing code, I feel like I’m getting more done – something much more tangible than a test case passed. And automation suites are in constant need of updating and maintenance, especially in a continuously changing codebase.
The fact is, if you want one tester to be on the team, they’re going to split their timeĀ between automation and manual testing. They’re going to do manual testing enough to get their automation test cases written. Which means that the amount of exploratory testing that should be done will not get done. Yes, automation helps to be able to do more exploratory testing. But if you’re working on that automation, you’re not doing as much exploratory testing either.
My Stand: two testers, each with their own focus, means you’re getting full attention on automation and on manual testing at the same time. One tester doesn’t have enough time to do both to a high enough degree of attention.
Argument 2 – well, can’t a BA pick up some of the manual testing?
Sure, a BA can do some testing. I hope they already are! However, a BA is someone that is closer to the business or to the client the software is for. They should be testing on a UAT kind of level. I wouldn’t expect a BA to know all of the various types of testing they can perform against a piece of software to the depth that a tester would.
Absolutely a BA’s experience is a benefit to the testing of the software being developed. But a BA also has other obligations – they have to research and write stories, collect acceptance criteria, and interface with the client/end-user. They also don’t have time to do two peoples jobs!
My Stand: a BA should do the job of a BA – yes they should do testing, on a UAT level. Testers are valued for their ability to think differently than other roles on project teams. Each role should be able to do its job.
Argument 3 – at least teach all of the testers how to code!
While I agree that anyone in a software testing role should understand how code is written, I feel that a “basics of programming” and “data structures and algorithms” classes would be enough. I don’t think that everyone has to know how to code. Not everyone wants to, and not everyone is adept at it and THAT IS OK! If a manual tester can’t code, that absolutely does NOT mean that they aren’t progressing in their career. If they are keeping up with technology, they understand the base architecture of the software project they’re working on, then they don’t need to know how to code to test!
What does any tester need to know, in my opinion?
- basics of programming
- data structures and algorithms
- how web services/api’s work at a base level
- how to use the browser debugger (if testing a web-based app)
- how to troubleshoot various applications (debug logs, Fiddler, etc)
- the technologies being used to create the application they’re testing
- what they are at a basic level
- how they’re being implemented
- how to find out what code got built or pushed to their test environment – what changed, what to test
- how to ask questions when they don’t know something
- what questions to ask
- how to just fucking google it sometimes
- of course, standard testing stuff (exploratory, boundary, security, etc etc etc)
The important things for a tester to know are how to talk to the people around them, and various ways they should test the software they’re testing. Yes, understanding how a developerĀ works, or what they need to do to go from requirement to finished software, is incredibly important for anyone working in software to understand. But they don’t need to know the difference between a List and an Array in C#, and when to use each.
Hey, understanding the limits of an int and how you can test a field that has an int datatype is great. Knowing how to see in the database that the field for a phone number has a float datatype and understanding that you can just put in a bug without testing the frontend because wtf were they thinking – that’s invaluable.
My Stand: So yeah, the more you know, the better. Knowledge is power, after all. But you don’t have to be able to write code to be a good tester. Period.
Argument 4 – we should just automate everything and get rid of manual testers
Lets go back to that “standard testing stuff” and “basics of programming”. Testers have a job because developersĀ are human, and they make mistakes. Also, writing code is fucking complicated.
You can automate a lot of things. But you can’t automate a brain, curiosity, a set of eyes that see that weird thing happening on the taskbar that your automation didn’t catch because it’s not a HUMAN and you didn’t tell it to look there so IT DIDN’T!
A computer will do exactly what you tell it to. It can’t explore, it can’t see those subtle differences, or even the big honkin’ bugs unless you code it to do that. But then there’s always something else that your code is not specifically asked to do so it won’t do it.
I personally still feel squeamish about CI. I don’t like the thought of code going straight to production without a set of eyes on it. I will never be ok with CI.
Please, if you haven’t read it or you need a refresher, check out James Bach and Michael Bolton’sĀ definition of testing and checking (automation = checking, btw, not actually testing):Ā http://www.satisfice.com/blog/archives/856
My Final Stand: We need manual testers. We need automators for the regression checks, unit and integration checks, etc. But we will never NOT need people whose sole job is to manually test an application.
So there you go. We should absolutely not get rid of manual testers. Manual testers are not “at the end of their careers”. Manual testers, that are good at what they do and continue to learn, are indispensable.
Arguments? Further points? Let’s chat š
Agile and Beyond 2015 recap
Storified here:Ā https://storify.com/g33klady/agile-and-beyond-2015-recap
Please forgive the giant image at the top – I can’t find a way to change it š¦
It’s a loooooong post, a couple hundred tweets, so be prepared!
KalamazooX 2015 recap
I put together my tweets and thoughts of KalamazooX 2015 on Storify here:Ā https://storify.com/g33klady/kalamazoox-2015-recap
Introversion, social anxiety, and conferences
Just a warning that this is gonna get pretty honest and open about my introversion and depression. I’ve touched a bit on my social anxiety before, but not like this.
I’m an introvert. To some of you who’s only experience with me is on Twitter, this may come as a surprise. According to Meyers-Briggs, I’m an INFJ. Even if you think Meyers-Briggs is bullshit, some of it still applies.
Yes, I’ve been trying to work on myself more lately by speaking at conferences and running a user group. But that doesn’t change how much social situations give me anxiety and sometimes depression.
As an introvert, I find it terribly draining to be at a conference. Just attending the sessions, walking the halls in between, makes me feel brain dead and useless at the end of the day. And if I’m attending alone, any of the social stuff I’ll usually skip – I just can’t deal with it. Lately I’ve been attending with my fiance, so with him as my social crutch I’ll attend these social events. Yeah, I know plenty of people at the conference but I always feel so awkward and alone, surrounded by hundreds or thousands of people that seem to be doing just fine.
If alone, and someone reaches out to a group I belong to asking if anyone wants to hang out, I’ll usually ignore it – surely they can’t mean me, too. If I am asked directly to hang out etc., I’ll reluctantly accept unless I’m too overwhelmed with social interactions already. I have once or twice initiated these kinds of social sessions at conferences or when I’m hanging out in someone else’s city. Not usually, though – that introduces a lot of stress.
Usually if I have a drink or two in me, I’ll loosen up a bit. But it’s the getting to the bar and getting that drink that’s often the problem.
Recently, I attended CodeMash. I was the Testing track chair for the conference, and was excited to see what I had helped happen in that track. There were some great, and not-so-great, talks, lots of social situations, and generally hanging out with folks I know. All seemed great to the outside world, I’m sure. I did some hiding…
On the way home, I started to feel the dragging pull of depression. Not sadness that the conference was over and I’d miss my friends. Actual depression. My social anxiety was so overactive during the 4 days of the conference that it turned directly to depression.
I’m just now, a week later, feeling back to normal. I have depressive cycles once in a while, and this one had a definitive cause. I haven’t had such a lasting issue from a conference before… After each of the user group meetings I host, I have a day or so of feeling terribly drained and unable to function after; the time leading up to the meeting, especially the day of, is full of anxiety.
This won’t keep me from attending conferences in the future, of course, or hosting my user group. I just hope folks understand if I sit out the social meetups or just want to be alone in between sessions.
Are you an introvert or have social anxiety? How do you deal with conferences? User groups?
So much failure. So many lessons to learn.
As my CEO has said, it’s ok to fail as long as you learn from it. So I’m analyzing this gigantic failure to try to get as much knowledge from it as possible. I don’t want to fail this hard again…
Let me start from the beginning of this wonderful lesson. I was asked to lead a project team for Ann Arbor GiveCamp. In case you haven’t read all of my posts on GiveCamp, it’s a weekend-long event where technical volunteers come together to help non-profit organizations. Generally the volunteers give just their weekend and walk away from the event with no obligations.
I hadn’t lead a project team before, but someone had to step up so we could help more non-profits. So I stepped up.
Failure 1: I didn’t ask anyone for advice on handling a project at GiveCamp. I know plenty of folks that have run successful projects at past GiveCamps, but I figured “I’ve got this, I’ve done enough GiveCamps.”
Lesson 1: Nope, you don’t “got this”. Ask for advice, ask for lessons learned. Ask folks that have lead teams before what do they wish they had done differently.
I got my project prior to GiveCamp like most leaders, read through what the non-profit wanted, and had a phone meeting with the non-profit rep. I had a good general idea of what needed to be done.
Failure 2: I didn’t get specific enough.
Lesson 2: Get specific details. Get the Trello board set up prior to GiveCamp. Lay out all of the work that needs to be done – Every. Single. Bit of work. Lay out the must-haves and the nice-to-haves and make sure that the non-profit agrees with everything.
I met my team on the first day of GiveCamp. After the opening ceremonies (I was trying to help as an organizer at this point, getting people to the ceremony location, etc.), my team gathered. I had us all say our names, and what we do.
Failure 3: I didn’t know my team well enough. There just isn’t enough time to get to know everyone – their strengths, weaknesses, what they can and can’t do, what they will and won’t do.
Lesson 3: Try to glean as much from each person as you can. Not just what they do for a living, but all of the above – what they can/can’t/will/won’t do. Express the importance of honesty and openness on the team – if you can’t do something (or don’t want to), let us all know so that someone can. Also, understand everyone’s schedule – some folks are only there for the first night, and rather than finding that out just when they’re about to leave and they could have been doing something much more important or helpful, get everyone’s schedule out in the open at the beginning.
Underline the importance of all of the work getting done – no room for laziness. This is for the non-profit, not for your resume.
We had a trello board with a general list of things that needed doing – each card had a checklist on it of the various things that each item needed to be done to make that thing “done”.
Failure 4: Oh man the disorganization. Not everything was on the board. Not everything was in each card. Not everyone was using it properly.
Lesson 4: Definitely need a better way to track work. Getting the board up and populated before GiveCamp would have been helpful here. Rather than having checklists, have a card for *everything* that needed to be done, including testing. And make sure everyone is using it! Check in every hour to see what’s in the Done column, what’s assigned to who, make sure we’re not spinning our wheels. Hell, maybe have a team standup every hour or so.
We find out, late Saturday, that there’s a bug in the checkout page of the store on the site we’re working on.
It’s a critical bug, we can’t go live with it.
Failure 5: It took far too long to find the bug. Why?
Lesson 5:Ā Work, and more importantly testing, wasn’t defined properly. The site was being tested but from a user that was already logged in to WordPress – but most of the end-users of this site won’t be. And that’s where the bug was.
The person who was responsible for it said they were too lazy to set up a PayPal developer account in order to test checkout and donation (or to find out if someone else had one and ask them to test it). I didn’t flip out on this person, smacking them and telling them that if they want to be lazy get the fuck out of GiveCamp, this isn’t for lazy people this is to help a fucking charity, stop being an asshole and help for fuck’s sake. I didn’t flip out on them, but I should have. It’s inexcusable, and as a leader I shouldn’t just roll my eyes and try to pick up the slack, which is what I did. Part of that is my social anxiety/not wanting conflict thing.
We keep getting the rest of the stuff done, still trying to find the cause of the bug and trying to fix it off and on.
Failure 6: We needed a plan B sooner. But we kept spinning our wheels. And subequently, myself and one of the members of the team (who volunteered to do this) kept trying, off and on, to fix it for several weeks after. It was determined that the theme was the cause of the bug, so this other person rebuilt the theme from the ground up. For several more weeks, off and on, we tried to get this new site back to the non profit’s specifications.
Lesson 6: Be more agile – if it’s not working, fucking fix it. Ask for help during the standups. I let it get drawn out for weeks and weeks and weeks. I had suggested we go live with the site without the store, while we try to fix that bug. The non-profit didn’t want to do that. The store was integral.
Finally, months later, the non-profit asks for all of the files we have. They have to pay for someone to finish it, they can’t wait for us anymore. They need the store up and running to generate revenue.
When I get this email, I’m devastated. I knew this was coming – how could it not? We couldn’t keep spinning our wheels forever. I should be relieved, that this isn’t on me anymore. But I’m not. I’m angry with myself for leading the project so poorly. For failing so hard.
I thought that writing this out, walking through all of the lessons I’ve learned in this, would help me to work through the pain of failure. To put a positive spin on it – “hey, look, I’ve gotten better through this!”
I’m not sure that it’s helped me work through the pain of failure, but at least I know what I can do better next time, and how to not fail at GiveCamp again.
Same place, new adventures
Note: I’m not going to address the “QA” terminology thing here – we call our software testers “QA” where I work, but this isn’t about that. Sorry – we can talk about it later š
On Tuesday, I started my first day in my new position as an “SQA Architect” at Quicken Loans – moving from an Engineer position. What that means where I work is that my focus is on:
- assisting project teams in determining processes, procedures, and standards around quality
- mentoring and training any team member on testing-related stuff (tools or even the basic “agile testing 101”)
- building the “quality” community
Essentially, that whole “rising tides raise all boats”, and I’m helping to raise the tide of quality.
I’m very lucky to work where I do, because my team leader encouraged me to go after what I wanted. I was willing to compromise a bit, and wait until we had some testing support on the team. I am very thankful to Joe Guadagno for his continued support in being an awesome leader, and good friend. It’s not a rare thing there, and why I think I’ll be there for the long haul – lots of support, and you can really write your own ticket to success.
As our CIO said today at our holiday party, don’t be the thing that’sĀ getting in the way of your own success.
So I’m not! Here’s to an awesome new year ahead with many opportunities to exceed our own expectations!
Why user groups and networking are important
Starting a user group is probably the biggest challenge I’ve taken upon myself in a long time. Getting sponsorship and paying for it, finding a place to have it and an amazing speaker – that’s been the easy part. The hard part is creating that community. Getting people to understand that a few hours after work each month are worth it. I understand people have families and other important things to take care of after work. What I don’t understand is when the priorities of all of those other things come before bettering yourself, furthering your career, and networking with like-minded people every.single.time.Ā Do you think you’re the smartest person at work? LEAVE THAT COMPANY. If you don’t think there’s anything to learn… hell I don’t know what to tell you. I feel sorry for you, actually. There’s ALWAYS something to learn and ways to grow personally and professionally.
Michael Dell: Never Be The Smartest Person In The Room
Try never to be the smartest person in the room. And if you are, I suggest you invite smarter people ā¦ or find a different room. In professional circles itās called networking. In organizations itās called team building. And in life itās called family, friends, and community. We are all gifts to each other, and my own growth as a leader has shown me again and again that the most rewarding experiences come from my relationships. ~University of Texas, 2003
I’m not a model citizen when it comes to user groups and networking meetups – I don’t make the meetings every month without fail. Other things happen – I totally understand. But I keep up with folks on Twitter in between, and go every few months to .NET user groups (like MIGANG) and networking meeups (like IT in the D). And I’m starting a tester user group to get that part of my passion and career rounded out (Motor City Software Testers). There’s also online networking groups, like Software Testing Club, to keep me up to date and networky.
Why are user groups important?
So many reasons! You’re keeping up your knowledge about your industry. You’re meeting new people/catching up with old friends – all that you can share ideas with, or just bitch. You get free food and swag (usually). Maybe you’re getting some public speaking experience, and sharing your passion. You can’t go wrong! Maybe the speaker’s topic doesn’t apply to you – big deal! You still learn, you still broaden your horizons, you still meet new people or just socialize – you still get amazing opportunities to grow in your chosen field.
Also, going to the bar after with the group is the best. Socializing, sharing ideas over a drink, NETWORKING!
Why are networking groups important?
As the IT in the D guys say, if you start networking when you’re desperateĀ for a job, it’s too late. You need to start building those relationships early, and keep them up. I got my first job in the software industry by going to a Women in Tech networking lunch. At that first lunch, just chatting with one of the ladies, she says “you’d be a great QA, just by your personality. We have an opening at my company…”. It doesn’t happen this way for everyone – I was very lucky to have found the career I love by going to one networking lunch. BUT! The important thing is that I didn’t stop going to those groups. I keep up with networking, I keep up with folks on twitter. That’s why I routinely get messages from respected peopleĀ in the industry offering me a job – not just recruiters that found me on LinkedIn.
I don’t worry about losing my job (even though it’s very secure, there’s always the possibility) – I feel like I would be able to get into a new jobĀ very easily. Because of networking. Because of user groups. Because I’ve kept up those relationships.
Just go already!
THAT is why user groups and networking are important. It’s a small price to pay to spend time away from family and other things. But if your career is a priority – if being financially secure is a priority – it’s totally worth it.
Tester Drinkup in Columbus 6/15
Hi folks! I’ll be in Columbus speaking at Nationwide and Per Scholas on 6/16, but I’ll be getting into town 6/15 so let’s have a Testers Drinkup! Suggestions on a place? I’m staying in Dublin, so near there would be preferable…
What: Testers Drinkup
When: 6/15 8pm
Where: some bar near Columbus, OH
Why: to hang out and ease the nerves!
