Improving Your Unit Testing Skills

Unit testing is hard!  I came to this sad realization when my code which had a large test suite with near 100% code coverage, with every thought-of requirement unit tested, failed during integration testing.

How is that possible?  I thought to myself.  Easy…my unit tests were incomplete.

Writing software is pretty complex stuff.  This is pretty evident in the level of difficulty in determining how good a software developer is.  Do you measure them based on lines of code?  Can you measure on time to completion of tasks?  Or maybe the feature to bugs coming back ratio?  If writing features alone can be this complicated, surely unit testing is just as (or more) complicated.

First, you must be in a place where you can even start testing your code.  Are you using dependency injection?  Are your components decoupled?  Do you understand mocking and can you use mocking frameworks?  You need an understanding of all these concepts before you can even begin to unit test effectively.  Sure, you can unit test without these prerequisites, but the result will likely be pain and more pain because the you’ll be spending 90% of your time setting up a test rather than running it.

But back to the point of this post.  Unit testing is hard because it’s one of those things where you’ll never really know how to do it properly until you’ve tried it.  And that assumes you’ve even made the step to do testing at all.

I’m at the point where my experience with writing unit tests allows me to set them up quickly with relative ease, and perform very concise tests where anyone reading the unit test would say “oh, he’s testing this requirement.”  However, the problem is that there’s still 2 problems that are hard to fix:
a) Is the test actually correct?
b) Do the tests cover all scenarios?

(a) is obvious.  If the test is wrong then you might as well not run it all.  You must make it absolute goal to trust your unit tests.  If something fails, it should mean there’s a serious problem going on.  You can’t have “oh that’s ok, it’ll fail half the time, just ignore it.”

(b) is also obvious.  If you don’t cover all scenarios, then your unit tests aren’t complete.

In my case, I actually made both errors.  The unit test in question was calculating the time until minute X.  So in my unit test, I set the current time to 45 minutes.  The target time was 50 minutes.  5 minutes until minute 50, test passed.  Simple right?  Any veteran developer will probably know what I did wrong.  Yep…if the current time was 51 minutes, I ended up with a negative result.  The test was wrong in that it only tested half of the problem.  It never accounted for time wrapping.  The test was also incomplete, since it didn’t test all scenarios.

Fixing the bug was obviously very simple, but it was still ego shattering knowing that all the confidence I had previously was all false.  I went back to check out other scenarios, and was able to come up with some archaic scenarios where my code failed.  And this is probably another area where novice coders will do, where experienced coders will not: I only coded tests per-requirement.  What happens when 2 requirements collide with each other?  Or, what happens when 2 or more requirements must happen at the same time?  With this thinking I was able to come up with a scenario which led to 4 discrete requirements occurring at the same time.  Yeah…not fun.

Basically, in summary:
a) Verify that your tests are correct.
b) Strive to test for less than, equal, and greater than when applicable.
c) Cross reference all requirements against each other, and chain them if necessary.

TDD is Design…Unit Testing Legacy Code

I’ve been listening to a bunch of podcasts lately and I came across a gem that was pretty darn useful.  It’s pretty old I suppose in technology standards, since it was recorded January 12, 2009, but it’s still worth a listen.  Scott Hanselman talks with Scott Bellware about TDD and design.  Find it here.
I’m merely reiterating what Scott has already said in the podcast, but it’s different when I can personally say that I’m joining the ranks of countless others who have seen the benefits of TDD.

Now I can’t say that I’m a 100% TDD practitioner since I’m still learning how to write tests before code effectively, but I’ve already seen the improvements in design in my code many times over just by merely writing unit tests. At this point I'd say half of my tests are written first, and the other after are after the fact.

I’ve been through many phases of writing unit tests, it it took me a long time to get it right (and I’m sure I have tons more to go).  It takes some experience to figure out how to properly write unit tests, and as a by-product how to make classes easily testable.  Code quality and testability go hand-in-hand, so if you find something difficult to test, it’s probably because it was badly designed to start.

The first time I was introduced to unit testing in a work setting was when I was writing C++…and man it was ugly, not because it was C++, but because everything was tightly coupled and everything was a singleton.  The fixture setup was outrageous, and it was common to ::Sleep(5000) to make tests pass.  Needless to say, my first experience with unit testing was extremely painful.

After a job switch and back to the C# world, I started reading more blogs, listening to more podcasts, and experimenting more.  I was given a project to prototype, and for phase 1 it was an entirely DDD experience with I got the opportunity to experiment with writing unit tests the “proper” way with dependency injection and mocking frameworks.

Unit tests are easy to write when you have highly decoupled components.

Prototype was finished, and now I was back on the main project, and given a critical task to complete in 2 weeks.  I had to modify a class which was 10,000 lines long, which has mega dependencies on everything else in the project.  Anything I touched could potentially break something else.  And of course…no unit tests whatsoever.  I like challenges and responsibility – but this was close to overkill.  This thing produces revenue for the company daily so I really don’t want to mess anything up.

First thing I realized was that there’s no way I could possibly write any unit test for something like this.  If the class file was 10,000 lines long, you can imagine the average line count for methods.

And of course, the business team didn’t make things easy on me by asking that this feature be turned off in production, but turned on for QA.  So, the best option was to refactor the existing logic out to a separate class, extract an interface, implement the new implementation, and swap between the 2 implementations dynamically.

After 4 days of analyzing and reading code to make sure I have a very detailed battle plan, I started extracting the feature to a new class.  The first iteration of the class was UGLY.  I extracted out the feature I was changing to a separate class, but the method names were archaic and didn’t have any good “flow” to them.  I felt that I had to comment my unit tests just so whoever’s reading them could understand what’s happening, which brings up a point.

If you need to comment your unit tests, you need to redesign the API

It took many iterations and refactoring of the interface to get it to the point where I found acceptable.  If you compared the 1st batch of unit tests to the last batch of unit tests it is like night and day.  The end result were unit tests which typically followed a 3-line pattern of setup/execute/verify.  Brain dead simple.

The last thing to do was to reintegrate the class into the original class.  For this I was back to compile/run/debug/cross-fingers, but I had full confidence that whenever I called anything in the extracted class it would work.

An added benefit is that I didn’t add another 500 lines of code to the already gigantic class.  Basically, in summary:

• Get super mega long legacy code class files
• Extract feature into separate file as close to original implementation as possible (initially will be copy-paste)
• Run and make sure things don’t die
• Write unit tests for extracted class (forces you to have a better understanding of the code modifications, and the requirements)
• Make sure unit tests cover all possible scenarios of invocation from original class
• Start refactoring and redesigning to better testability, while maintaining all previous tests
• Done!  Repeat as necessary!