racheljoi.com

I’ve been delv­ing into the more tech­ni­cal side of test­ing, that of using tools to exer­cise code pre­cise­ly rather than galumph­ing through UI, which real­ly is my pre­ferred method. Our mobile app uses a light­weight ser­vice to con­nect to the oth­er mov­ing pieces at the com­pa­ny and our ven­dors, so we can test those calls direct­ly through SOAP, which, for you non-soft­ware peo­ple out there, is also called API test­ing.

The devel­op­ers of our soft­ware took kind of a blan­ket approach to the envelopes for the ser­vice calls, so there are tons of super­flu­ous fields for each of the calls, and it’s kind of a guess­ing game as to which are required for any par­tic­u­lar ser­vice call. My devel­op­ers did­n’t cre­ate all the ser­vice calls, so I’ve actu­al­ly been able to fig­ure things out and share my envelopes with them (so, you know, I feel pret­ty cool).

Once I fig­ure out which fields are nec­es­sary to get a call work­ing, then it’s time to fig­ure out an asser­tion. These are what the test­ing soft­ware checks against to make sure things are work­ing appro­pri­ate­ly. In our test­ing, they’re usu­al­ly relat­ed to the con­tent of a field, whether it is a “suc­cess” or “fail­ure” mes­sage, or, in a case I was work­ing on recent­ly, whether the name of a doc­u­ment start­ed with a spe­cif­ic year or not.

With this case, we mod­i­fied the call to pull doc­u­ments one year at a time, so the call for each year need­ed to return only doc­u­ments for that year and none oth­ers. I end­ed up doing a check that the count of docs that start­ed with some­thing oth­er than the year I was look­ing for was 0. This may sound sim­ple, but it took quite awhile to get the xpath right, and I had some (a lot of?) help from a devel­op­er.

The  SOAP tests are noth­ing with­out the cor­rect asser­tions. You can go ahead and get all the infor­ma­tion you want, but with­out an auto­mat­ed check on the infor­ma­tion, the tests have very lit­tle advan­tage over UI test­ing in terms of speed, though they do have the abil­i­ty to cut down on some of the noise that is inevitable with UI test­ing.

API test­ing is anoth­er tool in the box. It can­not be the only thing, and it should not be ignored. It’s great for get­ting pre­cise infor­ma­tion, for mak­ing sure you exer­cise calls at a low­er lev­el, and it can be faster than UI test­ing, but it is not a sub­sti­tute for a full com­ple­ment of test­ing, includ­ing explorato­ry test­ing (galumph­ing), script­ed tests run man­u­al­ly, and script­ed tests run through automa­tion. And, of course, if your devel­op­ers unit test their code, that helps too. 🙂