Polymorphic Rants

Piergiuliano Bossi wrote an interesting feedback about AgitarOne, the new product from Agitar Software.

I’d like to add some thoughts from my experience with Agitator, AgitarOne’s parent.

I had used Agitator for an year; what Agitator used to do is invoke one method several times(one hundred times or something like that…don’t remember the default setting) using random combination of the input parameters.
This way of “testing” a method had some implications:

• if the method was fairly complex (mainly if the input parameters weren’t only primitive types and Strings but other objects) and the design wasn’t nearly perfect the configuration of Agitator was absolutely a pain and the obtained results weren’t really reliable (lot of uncovered snippets of code and some expected behaviours not tested).

• I am often interested in testing an happy path e some well-known corner cases, not all the available combinations! Agitator instead tried every single combinations(unless you start trying to configure its “factories”, a sort of “trigger from hell” with a level of usability of 1 in a 1 to 10 range). In this scenario you test several useless combinations and this takes time, often a lot of time (yes, Agitator was quite greedy about RAM an CPU cycles).

During a whole year of (desperate) tentatives of using the tool, I spotted other goodies, like bugs in the way the tool determined the code coverage: in a word, the coverage changes due to comment/uncomment of a snippet of code that should be totally ininfluent for Agitator’s code strategy. Obviously every software has bugs, but things like that just reduced my confidence with a tool that was frustating my eagerness in using it in the proper way.

I had a quick look at the web demo available on the JUnitFactory website, and AgitarOne looks smarter than Agitator: AgitarOne tries to do some sort of syntactic analisys in order to understand what is really important to test and what is not.

I’d like to see it at work with some real word code, where the design is not perfect and the complexity of methods is a bit higher than paySalary(double basicSalary, int age).

A part from this pretty technical details, it’s also quite interesting to see how Agitar’s marketing on this product is drammatically changed: Agitator was the alternative to JUnit, and Agitar claimed that was possible to do TDD with that tool. AgitarOne generates unit tests, they’re highlighting the importance of the the tool specially for legacy code and they’re saying that the tool helps you in completing the suite of tests you obtained from TDD.

Quite a big change, uh ?