Teams at my company are moving to agile, and all of us will go eventually. I’ve been learning about how testing works in agile, and I’ve been talking a lot with people about it, both inside my company and outside. One thing that has come up is the perception that manual testing is not a big piece in agile, and that all of the testers will need to learn automation to continue to succeed. While I agree that testers should know enough to at least identify areas for automation, or better yet, do it, I think there will always be a place in agile for manual testing. Some things just cannot be automated in the first place. Also, manual testing is quick and robust, and the cost can be about the same as automation. Let’s break those down.
Quick: Testing in agile needs to be quick. If something is broken, it needs to be discovered and fixed quickly or put the sprint at risk. Manual testing is an efficient way to make sure that something is functioning. Stories can be very small to quite large, and, especially for the smaller stuff and GUI items, being able to quickly look at something, play around with it, and see whether it is working the way it’s supposed to is best done with manual testing. This is especially true for a smaller piece of a work in progress, when not everything is connected, but the developers want to make sure they’re on the right track.
Robust: Manual testing is robust. Deviations from a script are expected. If a URL or menu changes, or if a button moves around, a manual test can go forward. An automated test stops and throws an error and has to be inspected for what went wrong. If an application is very stable and unlikely to change anything, automation may be helpful, but if features are being added or moved around, as happens frequently in agile, manual testing will be able to continue more easily than an automated test.
Cost: Automation is a large up-front expense. What takes five minutes of manual testing may take thirty minutes to code an automated test. If the same test is going to be run over and over again over multiple sprint, automation may be more cost-effective. But manual testing is more cost-effective to determine functionality for initial tests and one-off tests.
Automation does have its uses. Automation takes some human error out of testing (though possibly introduces human error in the coding for testing itself). If testers want to be involved with writing unit tests or helping with TDD, automation is a good way to go. If it’s a long-term project with core functionality built at the beginning, regression tests can be automated to ensure that future features don’t break the main stuff. Integration testing of earlier features can be built too. But, especially for the first sprint of a feature, before it’s fully realized and integrated, manual testing is the way to go. I know of some companies who do entirely manual testing in agile, which poses its own set of challenges.
For a discussion of how agile changes the tester’s role, and for briefly touching on different benefits of manual vs automated testing in agile, see Ulf Eriksson’s blog post here. From the same site, here is a mind map about testing in agile, and section 6 deals with manual testing and why it’s necessary.
I ran this past people who know what they’re talking about to make sure I’m not saying anything heretical, but I understand there may be differences of opinion. I’d welcome a discussion about this!