a documentation argument

I’m testing some vendor-supplied software, and I’ve been testing a variety of functions and integrations. I’ve been more deliberate in my testing on this project, less monkey-at-a-keyboard and more creative thinking about how it might be broken. I’m testing things that aren’t necessarily documented in the requirements. I got some push-back about it, that we should only be documenting (and even running) tests that have written requirements. That’s basically telling me that I shouldn’t do a thorough job because someone else didn’t do a thorough job. And that’s not how I want to work.

5 thoughts on “a documentation argument

  1. Hello, Rachel. I’m one of your coworkers (Colby) and overheard someone mentioning your blog.

    I have a few ideas that might help. Most of them from James Bach (I know you’re a fan).

    1) In arguing that there can be bugs outside of the requirements, maybe try giving an example to your project team. The requirements may not explicitly say that a web application have a certain uptime, for example, but if it was down 23 hours a day, I think everyone would agree that’s a bug.

    2) In dealing with the politics of bugs, one of James’ definitions for a bug (and it’s one I more or less came to on my own over the years as well) is useful to remember. A bug is “something that bugs someone whose opinion matters.” As testers our opinions don’t matter very much. It’s important that we identify potential bugs for our project teams, but if no one whose opinion really matters (the business, etc.) feels it’s a bug, then it isn’t.

    3) You can always call out potential bugs in different ways. James has two terms for the extreme opposites of bug reports, MIPing and Black Flagging. Black Flagging is essentially pulling the fire alarm. Calling an immediate meeting to discuss a severe bug would be Black Flagging. To MIP a bug is to mention it in passing. This would be saying “Hey, how this looks bugs me. Does it bother you?” on a weekly project call or when working directly with a project team member. If it doesn’t bother any one whose opinion matters, you can then leave it at that. MIPing might be the direction to take with your potential bugs that are outside of the requirements.

    4) When trying to convince others of potential bugs that might be subjective, the idea of consistency can be a valuable tool (another idea from James). Pointing out that something is not consistent with other web-bases applications or GUIs or even within an application itself and might confuse a customer can go along way to convincing others.

    I hope some of this helps, and good luck!

    1. That’s something that I’m slowly coming to terms with – that I’m not someone who is entitled to an opinion. But I do think James Bach said that it’s important to explain the full extent of the bug, even if someone whose opinion matters still dismisses it. And that’s where I think I may be falling down. I’ve tried the consistency approach and the MIP approach so far. But I’m not sure if I’m doing a clear enough job of explaining why the bugs might be important. Is that the wrong approach still?

      As far as documentation goes, I agree that writing cases is the worst part of the job. I’ve received push-back on bugs that I have found that I think deserve test cases though (otherwise how will we do adequate regression testing in future releases?). Before coming here, I was solely responsible for testing, so writing things down didn’t matter, but now, I can’t rely on others to pick up on my (brilliant, right?) brain waves.

      1. I’m betting you’ve done all you can do. You are obviously a clear communicator. Eventually we just have to let some bugs go (if it hasn’t happened to you already, someday, someone will be bugged in the field by one of your pet bugs that got dismissed, and you will at the very least get to feel that superior I-told-ya-so feeling).

        One thing I’ve struggled with at times is separating my ego from my bug reports. Early on I cared a lot about them, that they were taken seriously. I don’t as much anymore. I care about presenting my case, as you clearly do, and if it’s not to be, I try to let it go. It still occasionally get caught up in it though (I admit to having done so as recently as last week).

        My impression is you are doing exactly what you need to do, and you will also, over time, get used to the specific flavor of our environment. The communication works a little differently, but you find how it works pretty quickly. One thing I always recommend to testers—especially at our work—is always speaking in terms of we and us with project teams. We could do this better, we did this well, we may have a bug here, etc.

        1. I totally hear you about separating ego from the reports. I need to separate ego from a lot of things though, not just reports. 🙂

          Thanks for the vote of confidence!

  2. Oh, and on writing test cases, why not take advantage of only having to write as many as the project team requires? Documentation is expensive and the worst part of the job (at least for me). If you have extra testing you want to do, beyond the requirements, and you have time to do it (and the project still has the money for you to do so), you are probably fine to do it without officially documenting it. If you encounter a bug while doing this testing and the project team agrees it’s a bug, you could then go back and add a test case for it.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.