Tags
Collaboration
pair testing adventures
Last week, I finally did what I’ve been wanting to do for months (years) and engaged in pair testing three times with some differing results. what is pair testing? Pair testing is a lot like pair programming, where you have two sets of eyes and two minds engaged in the same problem. There can be a lot of benefits to it (Lisa Crispin talks about it here ), such as increased creativity, better quality, and more exploration, but, just like pair programming, it can be difficult to start and requires a lot of focus and energy. I wanted to learn strong-style pairing , as it seems to involve the most engagement from both people.
Jan 2017a transition to agile
I joined a team at work that has moved to agile from waterfall. It was a high-performing team in waterfall, and now the team is figuring out how to get that same level of performance while doing agile. It’s been a struggle. I wasn’t with the team before, so I’ve only seen them in this transition phase. There’s lots of talk like, “This is how we’ve always done it, and we’ve been just fine,” and “We don’t need to be told how to do our jobs.” People are frustrated, and those of us who are trying to be cheerleaders and evangelists get shouted down frequently. One of the principles that has fallen by the wayside is the idea of face-to-face communication being the best way to get things done. We are theoretically co-located. One guy works out of a different office, but the other 14 of us are in the same room. However, we all work from home on Fridays, and most people take a second day at home too, on different days, and then when the weather is bad, people work from home, and if their kid is sick, they work from home, and if their elbow hurts, they work from home, and on and on. Only for first and last days of the sprint is everyone in the same room. When the idea of video calls has been raised, the response has been swift and negative. Emails, IMs, and the occasional Skype call are said to be sufficient, and the suggestion that more information can be transmitted by being able to see facial cues has been deemed irrelevant. They say, “Our communication has always been good, why do we need to change it?” like I mentioned above. (note: voice deliberately passive in this paragraph to obfuscate my own embarrassment) We’re struggling to figure out how to make our testing the most effective. On the one side, there are a couple who are all about automation, who see it as a point of pride that they haven’t touched a mobile device (which is what we test) for a long time. On the other side, there are people who don’t want anything to do with automation, who don’t want to do any kind of programming. And in the middle, there I am. I think automation is a means to an end. That if you have good enough automation, you can spend more time touching devices and doing session-based exploratory testing, to try to find bugs that would be difficult or impossible to find with automation. I think our coach basically believes this too, but the automation enthusiasts only hear that we need more automation, not that it is meant to facilitate better manual testing. It’s almost a culture war. I guess that one of the effects (benefits?) of agile is that you’re always going to be a little annoyed, a little frustrated, a little driven to (hopefully) improve. It comes from interacting with people so regularly, from needing to rely on working as a team. Differences in approach, in opinion, in work ethic, all come to the forefront, where they can theoretically be discussed. Perhaps one of the next phases of our agile development development (ha, see what I did there?) is to have those open and honest conversations without devolving into raised voices and accusations. That’s a difficult step that requires a lot of maturity from everyone involved, and even then, emotions and egos can still impair the productivity of those conversations.
Oct 2016hand-waving and subject matter expertise
From this recent project I’ve been discussing, I have one more point. Our customers can make transfers into other accounts. The vendor told us that it would “hard fail” if the customer tried to go over their balance. “Hard fail” sounded fancy, and we were assured that it would prevent penalties. However, I was able to show that customers could go over their balance once the fees were included, and it wouldn’t fail. There were actually a number of ways a person could transfer more than they had to give. Turns out “hard fail” wasn’t anything fancy, and it only happened at one point in the process, rather than it being a continuing thing. Business got involved, said the internal people had control over whether to assess the penalties and that they “probably” wouldn’t assess those penalties. Finally, a person who had done that kind of work spoke up to assure us that he had never seen a penalty not assessed. It’s another battle I lost. (I lost a lot of battles on this project, but there were also just lots of battles.) The financial institution assumes that people won’t be that foolish, or perhaps the institution just likes those penalties. I learned a lesson about thoroughly understanding processes. Before the true meaning of “hard fail” came to light, and before a subject matter expert spoke up, it was just hand-waving. We tested it with our assumptions that there were fail-safes involved, when there weren’t really. It took pushing the issue and not giving into the hand-waving that we were able to make an informed decision about the process. The project also taught me not to trust anyone, particularly vendors, when they talk about behaviors. When they say something is “as designed”, they often mean “no design” or “designed without thought”. The product was exhilarating to test, because it was so very broken, but it was also immensely frustrating to be stonewalled by the vendor at most steps.
Sep 2016why testers need to be involved from the beginning
On a recent project, the vendor’s software was going to be embedded in our website through an iframe. After a quick tutorial from my husband (a software architect who had done a lot of research on the security of iframes not too long before that), I thought it would be interesting to poke around and see what I could find in their system and see if I could break into it, though I’m not a security tester. I found some of what I expected to find - they were using security headers that are a little dated but the best that work on Safari (there’s more secure stuff for the other browsers - I don’t know why Apple is behind the times on that). But there was also a header that was misspelled when it should have been automatically generated by the system. This gave me pause. There was a possibility that something got a little funky with the server, but it was also possible that they were hard-coding things on a homegrown server rather than using a commercial server that is regularly patched. I filed a bug. And pushed it. A lot. I got the Information Security team involved. The response finally came back (after more than a month of asking) that they were not using Apache, but they couldn’t tell us more about it, that they were keeping up with security issues and regularly testing their server. This was another issue. The software we were given was so poorly written and tested, that I wasn’t sure if we could trust their word that they were adequately testing their own servers, much less keeping up with OWASP vulnerabilities or doing anything security-related. InfoSec was aware of this. But when I asked about the results from their penetration testing, the project manager told me that they weren’t doing any testing. It was in our contract with this vendor that we wouldn’t do any testing of their server. Seriously? My response to that in a meeting was, “Someone is touching their servers, and I’m concerned that it’s not us.” I lost the battle. I looked through the contract to see if there was any way we could say they breached the contract (hooray, law background!), but the vendor had written it so wishy-washy that it was basically a “best effort” contract without anything binding. So frustrating. In addition to the gaping potential security issues, it was a big problem to me that testers weren’t involved in the decision-making process. I’m not sure if anyone technical was involved, though I really hope so. But a tester could have raised the issue about not testing their servers instead of lawyers and business people just agreeing to it without considering the implications. It should have been a major red flag that they didn’t want us to try to break into their server. Testers are not just a checkmark. We have experience and specific knowledge that can assist with product decisions and can end up protecting the business’s interests.