Auto Mocking NSubstitute with Castle Windsor

I was debating whether to make this blog post because it’s so damn simple to implement, but hey, if it saves someone else time, I did some good.

First of all, register an ILazyComponentLoader into Windsor:

var c = new WindsorContainer();
c.Register(Component.For<LazyComponentAutoMocker>());

Then, the implementation of LazyComponentAutoMocker is simply this:

public class LazyComponentAutoMocker : ILazyComponentLoader
{
  public IRegistration Load(string key, Type service, IDictionary arguments)
  {
    return Component.For(service).Instance(Substitute.For(new[] { service }, null));
  }
}

And you’re done!  Here’s a simple unit test example using only the code from above:

[Test]
public void IDictionary_Add_Invoked()
{
  var dict = c.Resolve<IDictionary>();
  dict.Add(1, 1);
  dict.Received().Add(1, 1);
}

That was almost too easy.

That Immutable Thing

Do you have some sort of ImmutableAttribute in your domain that you use to mark classes as immutable?  Have you ever needed to enforce that contract?  Checking for readonly fields isn’t enough?  Well, this weekend I had a code spike that helped solve this problem in my current project.

For this project, I’m using the NoRM driver for MongoDB, and one of the limitations of the serializer is that all types must be classes, must have a default constructor, and all properties have a public setter.  So, now the domain has a bunch of classes like this:

public class UserCreatedEvent : IEvent
{
  public string Name { get; set; }
  public UserCreatedEvent() { }
  public UserCreatedEvent(string name) { Name = name; }
}

That God for code snippets (or Resharper templates).  With so many classes like this that need to get serialized, I wanted to extra sure that no code ever calls the setter method for the Name property.  Thankfully, with some help of Mono.Cecil, it’s possible.

First off, you need to define ImmutableAttribute and that add that do classes, and in my case, it is historical domain events that get serialized to an event store.

Then, you just write a unit test which leverages the power of Mono.Cecil.  It turned out to be pretty simple.  Here’s the code:

using System.Collections.Generic;
using System.Linq;
using Mono.Cecil;
using Mono.Cecil.Cil;
using NUnit.Framework;
namespace blingcode
{
    [TestFixture]
    public class ImmutabilityTests
    {
        private static readonly MethodDefinition[] _setterMethods;
        private static readonly AssemblyDefinition[] _assemblies;
        static ImmutabilityTests()
        {
            _assemblies = new[]
            {
                AssemblyDefinition.ReadAssembly(typeof(Something).Assembly.Location),
            };
            _setterMethods = _assemblies
                .SelectMany(a => a.Modules)
                .SelectMany(m => m.Types)
                .Where(t => t.CustomAttributes.Any(attr => attr.AttributeType.Name.Contains("ImmutableAttribute")))
                .SelectMany(t => t.Properties)
                .Where(p => p.SetMethod != null)
                .Select(m => m.SetMethod)
                .ToArray();
        }
        [Test]
        public void ClassesWith_ImmutableAttribute_ShouldNotUse_PropertySetters()
        {
            AssertForViolations(_assemblies
                                    .SelectMany(a => a.Modules)
                                    .SelectMany(m => m.Types)
                                    .Where(t => t.IsClass)
                                    .SelectMany(t => t.Methods));
        }
        [Test]
        public void ThisFixtureActuallyWorks()
        {
            var assembly = AssemblyDefinition.ReadAssembly(typeof(ImmutabilityTests).Assembly.Location);
            var type = assembly.Modules.SelectMany(m => m.Types)
                .Where(t => t.IsClass && t.FullName.Contains(GetType().FullName)).First();
            try
            {
                AssertForViolations(type.Methods);
            }
            catch (AssertionException)
            {
                Assert.Pass();
            }
        }
        private static void AssertForViolations(IEnumerable<MethodDefinition> potentialMethods)
        {
            foreach (var method in potentialMethods.Where(m => m.HasBody))
            {
                foreach (Instruction ins in method.Body.Instructions.Where(ins => ins.OpCode == OpCodes.Callvirt))
                {
                    MemberReference mr = ins.Operand as MemberReference;
                    if (mr != null)
                    {
                        var result = _setterMethods.FirstOrDefault(m => m.FullName == mr.FullName);
                        if (result != null)
                        {
                            throw new AssertionException(result + " was invoked by " + method + ", even though the type has the Immutable attribute.");
                        }
                    }
                }
            }
        }
        private void InvokeCardSetters()
        {
            // this only exists to test that the test does indeed work
            var c = new SomeImmutableClass();
            c.SomeImmutableValue = 123;
        }
    }
}

Nothing too complicated.  The main thing to look for is the callvirt method, which the C# compiler always generates for classes.  Then, you match the operand to the method definition and viola!

How to Advocate Test Driven Development (TDD)

It seems that everywhere you read you will find blog posts asserting that it is difficult to convince others to use TDD, and that everyone talking about TDD is already a believer preaching to other believers.  So what's the best way convince others?

I recently convinced a good buddy of me to starting doing TDD in his web development.  It took a while, but finally he "saw the light."  He tried to relay the information to his coworker, and his coworker didn't react as positively, citing the common "I don't want to waste time writing test code when I know my stuff works."

There are many many MANY hurdles of overcome to begin writing software using TDD.  I won't go into great detail since you can easily find relevant posts on the web, but basically, if you have successfully transitioned to TDD that also implies that you have in one way or another adopted S.O.L.I.D. principles in software design.

The reason I say this is because it is literally impossible to do TDD otherwise.  It is just plain painful.  I've been there.  It is so painful you shout in your head WTF are people thinking, how does this make my job easier?!

Usually when this happens, you'll probably notice that most (or all) of the SOLID principles have been violated one way or another.

<sarcasm>

• Single Responsibility Principle
• You mean 1 class to rule them all right?
• Open Closed Principle
• Does this have something to do with sockets?
• Liskov Substitution Principle
• I thought the point of Object Oriented Programming is that everything is an object!
• Interface Segregation Principle
• What's an interface?
  
• Dependency Inversion Principle
• I don't get it...I only have 1 super object anyway, I don't have dependencies.

</sarcasm>

Notice that even though this post was supposed to be about how to advocate TDD, I ended up talking about SOLID instead for a bit.  What am I trying to get at?

TDD spans from the individual to the team to the organization.  You need to be able to tackle it from all points of view.  The obsession of the word 'test' has hurt TDD quite a bit.  Anyone who's spent any reasonable amount of time will assert that tests are just a means to drive the design.  Tests should read like a story (well to programmer eyes), and yes, I said read, as in readable!

At the individual level, you must convince them it forces you to design better code, because testing against badly designed code is not worth the pain/annoyance.  You must assert that better designed code is good because it also means being more maintainable and less bugs.

At the team level, you must convince that collective code ownership is a good thing.  There should never be a situation where "Bob's on vacation, we'll need to put a hold on Project A because only he knows it".  Tests serve as a way on how to interact with a specific service, and are a jump-start to figuring out how to actually use a particular service.  Moreover, collective code ownership is a means of sharing and distributing knowledge.

At the organizational level, you must convince managers that you are a professional software developer, and that it is your job to produce high quality work.  And if what is required to produce high quality work involves writing proper unit tests, following SOLID principles, and even other agile practices like pair programming, then you should be allowed (and optimally encouraged) to do so.  Now TDD might not work for everyone, or every organization, but I think everyone owes it to themselves to give it a fair chance.

Improving Your Unit Testing Skills

Unit testing is hard!  I came to this sad realization when my code which had a large test suite with near 100% code coverage, with every thought-of requirement unit tested, failed during integration testing.

How is that possible?  I thought to myself.  Easy…my unit tests were incomplete.

Writing software is pretty complex stuff.  This is pretty evident in the level of difficulty in determining how good a software developer is.  Do you measure them based on lines of code?  Can you measure on time to completion of tasks?  Or maybe the feature to bugs coming back ratio?  If writing features alone can be this complicated, surely unit testing is just as (or more) complicated.

First, you must be in a place where you can even start testing your code.  Are you using dependency injection?  Are your components decoupled?  Do you understand mocking and can you use mocking frameworks?  You need an understanding of all these concepts before you can even begin to unit test effectively.  Sure, you can unit test without these prerequisites, but the result will likely be pain and more pain because the you’ll be spending 90% of your time setting up a test rather than running it.

But back to the point of this post.  Unit testing is hard because it’s one of those things where you’ll never really know how to do it properly until you’ve tried it.  And that assumes you’ve even made the step to do testing at all.

I’m at the point where my experience with writing unit tests allows me to set them up quickly with relative ease, and perform very concise tests where anyone reading the unit test would say “oh, he’s testing this requirement.”  However, the problem is that there’s still 2 problems that are hard to fix:
a) Is the test actually correct?
b) Do the tests cover all scenarios?

(a) is obvious.  If the test is wrong then you might as well not run it all.  You must make it absolute goal to trust your unit tests.  If something fails, it should mean there’s a serious problem going on.  You can’t have “oh that’s ok, it’ll fail half the time, just ignore it.”

(b) is also obvious.  If you don’t cover all scenarios, then your unit tests aren’t complete.

In my case, I actually made both errors.  The unit test in question was calculating the time until minute X.  So in my unit test, I set the current time to 45 minutes.  The target time was 50 minutes.  5 minutes until minute 50, test passed.  Simple right?  Any veteran developer will probably know what I did wrong.  Yep…if the current time was 51 minutes, I ended up with a negative result.  The test was wrong in that it only tested half of the problem.  It never accounted for time wrapping.  The test was also incomplete, since it didn’t test all scenarios.

Fixing the bug was obviously very simple, but it was still ego shattering knowing that all the confidence I had previously was all false.  I went back to check out other scenarios, and was able to come up with some archaic scenarios where my code failed.  And this is probably another area where novice coders will do, where experienced coders will not: I only coded tests per-requirement.  What happens when 2 requirements collide with each other?  Or, what happens when 2 or more requirements must happen at the same time?  With this thinking I was able to come up with a scenario which led to 4 discrete requirements occurring at the same time.  Yeah…not fun.

Basically, in summary:
a) Verify that your tests are correct.
b) Strive to test for less than, equal, and greater than when applicable.
c) Cross reference all requirements against each other, and chain them if necessary.

TDD is Design…Unit Testing Legacy Code

I’ve been listening to a bunch of podcasts lately and I came across a gem that was pretty darn useful.  It’s pretty old I suppose in technology standards, since it was recorded January 12, 2009, but it’s still worth a listen.  Scott Hanselman talks with Scott Bellware about TDD and design.  Find it here.
I’m merely reiterating what Scott has already said in the podcast, but it’s different when I can personally say that I’m joining the ranks of countless others who have seen the benefits of TDD.

Now I can’t say that I’m a 100% TDD practitioner since I’m still learning how to write tests before code effectively, but I’ve already seen the improvements in design in my code many times over just by merely writing unit tests. At this point I'd say half of my tests are written first, and the other after are after the fact.

I’ve been through many phases of writing unit tests, it it took me a long time to get it right (and I’m sure I have tons more to go).  It takes some experience to figure out how to properly write unit tests, and as a by-product how to make classes easily testable.  Code quality and testability go hand-in-hand, so if you find something difficult to test, it’s probably because it was badly designed to start.

The first time I was introduced to unit testing in a work setting was when I was writing C++…and man it was ugly, not because it was C++, but because everything was tightly coupled and everything was a singleton.  The fixture setup was outrageous, and it was common to ::Sleep(5000) to make tests pass.  Needless to say, my first experience with unit testing was extremely painful.

After a job switch and back to the C# world, I started reading more blogs, listening to more podcasts, and experimenting more.  I was given a project to prototype, and for phase 1 it was an entirely DDD experience with I got the opportunity to experiment with writing unit tests the “proper” way with dependency injection and mocking frameworks.

Unit tests are easy to write when you have highly decoupled components.

Prototype was finished, and now I was back on the main project, and given a critical task to complete in 2 weeks.  I had to modify a class which was 10,000 lines long, which has mega dependencies on everything else in the project.  Anything I touched could potentially break something else.  And of course…no unit tests whatsoever.  I like challenges and responsibility – but this was close to overkill.  This thing produces revenue for the company daily so I really don’t want to mess anything up.

First thing I realized was that there’s no way I could possibly write any unit test for something like this.  If the class file was 10,000 lines long, you can imagine the average line count for methods.

And of course, the business team didn’t make things easy on me by asking that this feature be turned off in production, but turned on for QA.  So, the best option was to refactor the existing logic out to a separate class, extract an interface, implement the new implementation, and swap between the 2 implementations dynamically.

After 4 days of analyzing and reading code to make sure I have a very detailed battle plan, I started extracting the feature to a new class.  The first iteration of the class was UGLY.  I extracted out the feature I was changing to a separate class, but the method names were archaic and didn’t have any good “flow” to them.  I felt that I had to comment my unit tests just so whoever’s reading them could understand what’s happening, which brings up a point.

If you need to comment your unit tests, you need to redesign the API

It took many iterations and refactoring of the interface to get it to the point where I found acceptable.  If you compared the 1st batch of unit tests to the last batch of unit tests it is like night and day.  The end result were unit tests which typically followed a 3-line pattern of setup/execute/verify.  Brain dead simple.

The last thing to do was to reintegrate the class into the original class.  For this I was back to compile/run/debug/cross-fingers, but I had full confidence that whenever I called anything in the extracted class it would work.

An added benefit is that I didn’t add another 500 lines of code to the already gigantic class.  Basically, in summary:

• Get super mega long legacy code class files
• Extract feature into separate file as close to original implementation as possible (initially will be copy-paste)
• Run and make sure things don’t die
• Write unit tests for extracted class (forces you to have a better understanding of the code modifications, and the requirements)
• Make sure unit tests cover all possible scenarios of invocation from original class
• Start refactoring and redesigning to better testability, while maintaining all previous tests
• Done!  Repeat as necessary!

Getting code coverage working on TeamCity 5 with something other than NUnit

I've been experimenting lately with TeamCity 5 EAP and so far it's been a pretty awesome experience. I was up and running within minutes and I was swarmed with beautiful graphs and statistics with specifics even per-test. Getting something like that up with CC.NET is not a trivial task.

Anywho, with TC5 code coverage is one of the cool new features added for .NET, but unfortunately only NUnit is supported. Not that that's a bad thing, but some people prefer to use other testing tools. Two notable contenders are xUnit.net and MbUnit.

I like the fact (pun intended) that xUnit.net makes it a point to prevent you from doing bad practices (like [ExpectedException]), and I like how MbUnit is so bleeding edge with useful features (like [Parallelizable] and a vast availability of assertions).


And with that I set up to figure out how to get TC working with Gallio, but the following should work with any test runner.

It certainly was a pain to set up because it took a lot of trial and error but eventually I figured it out.  I analyzed the build logs provided in each build report and noticed something interesting...specifically:

[13:51:23]: ##teamcity[importData type='dotNetCoverage' tool='ncover' file='C:\TeamCity5\buildAgent\temp\buildTmp\tmp1C93.tmp']

and

[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageL' value='94.85067']

[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageM' value='97.32143']

[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageC' value='98.68421']

[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='921.0']

[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageAbsMCovered' value='218.0']

[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageAbsCCovered' value='75.0']

[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='971.0']

[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageAbsMTotal' value='224.0']

[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageAbsCTotal' value='76.0']

The first message happens after NCover.Console is done its thing.  After NCoverExplorer is done its thing, the statistics are published.  I set out to mimic this functionality with Gallio, but what's described here should work with any test runner.

1) Disable code coverage in TC.  We're doing it manually instead.
2) In your build script, run your unit tests with NCover and generate a coverage.xml report.
3) Run NCoverExplorer on coverage.xml and generate reports ncoverexplorer.xml and index.html.
4) Create a zip file of index.html and name it coverage.zip.
5) Configure coverage.zip to be an artifact in your TC configuration (this is to enable the tab).
6) Parse out ncoverexplorer.xml with XPath and output the statistics.

Certainly a lot of things to do just for the sake of pretty statistics reporting....but it was the weekend and I was bored.  With the help of MSBuildCommunityTasks, the zip file and XML parsing was made a lot easier.

After that, viola!  Code coverage + Gallio on TeamCity 5!!!

Unfortunately, NCoverExplorer's report only reports the # total of classes and nothing about unvisited or covered, so for those values I set to 0/0/0 (BTW, you need all values present for the statistics to show).  A task for next weekend!!!

(Edit: I suppose I could should also mention that you could technically avoid all the trouble above and hack it with this:
#if NUNIT
using TestFixtureAttribute = NUnit.Framework.TestFixtureAttribute;
using TestAttribute = NUnit.Framework.TestAttribute;
#endif
And it'll work just fine as well)