bling.github.io

This blog has relocated to bling.github.io.

Friday, November 6, 2009

Multiple Inheritance in C# with Castle DynamicProxy Mixins

Having some significant experience with C++, every once in a while I’ll really miss features available in C++ that just aren’t possible in C#.  Sure, linking takes forever and if you could calculate the percentage of time for a C++ developer spent on linking time I’m sure it’d be around 50%.  Nonetheless, there’s a certain seductive pull to stepping through the debugger and the sheer speed of a C++ program (arguably debugger is being made obsolete with TDD).  If you’re only used C# for a long time, you probably didn’t even know that C++ was THAT much faster until you try this out.  Even a simple hello world program: you can tell a difference.
But that’s not the point of this post.  This post is about mixins!
So what is a mixin?  You can wikipedia it, but basically it’s a way to add functionality to an existing object.  In a way, it’s like multiple inheritance, but not exactly.  If you wanna go CS101 terms, this is not a “is-a” relationship, but more of a “can-do” relationship.  Both methods promote code reuse, and of course, both can be abused.
In a way, C# extension methods are a form of mixin, but really it’s just syntactic sugar.  It’s a static method but looks like a instance method.  True mixins are actual instances.
I’ve been playing around with Castle.DynamicProxy lately, and some of it is going into production code soon.  This is honestly kind of scary, because I really don’t want to mess up production code which generates daily profit for the company.  However, if NHibernate’s using dynamic proxy, and that has been proven to work in countless production environments, I’m sure the complexity of the proxies I’m doing pales in comparison to what NHibernate does.
Most of the examples available on the web show you how a mixin works.  Basically, you create a target, and then you mixin extra stuff.  Basic stuff…but there’s a really cool trick you can do to make all consuming code a lot easier to use by removing the need of consuming code to cast to a mixin instance.  In my case, the system is a very complex piece of legacy code where there are threads all over the place locking things all over the place.  I was given the task of keeping track of who owned the lock.
Originally I started with an extension method, but I soon realized that even though it’s nice that along with IDisposable, I could do something like this:
using (anyInstance.Lock()) {
anyInstance.DoStuff();
}

Where Lock() is an extension method which returns an IDisposable, which when Dispose() is invoked will call Monitor.Exit.  This is all nice and dandy use of the ‘using’ syntax, but as you can see this is all just syntactic sugar.  At the end of the day, this is still a public static method can is accessible by thousands of other objects.  And once you need to keep track of all those owners, you need to create a dictionary of object hashcodes, and locking on that thing for each incoming lock on an instance is…well…very very bad.

Anyways, so with DynamicProxy2, I decided to mix in this lock tracking logic.  Now, the interface is simple:
public interface ILockMixin {
IDisposable Lock(string tag);
IDisposable TryLock(string tag);
}

Now, the particular thing I wanted to target happened to be a hashtable, so with a little interface trick, you could do this:
public interface ILockableHashtable : IDictionary, ILockMixin { }

Finally, any code that references hashtable needs to be replaced with ILockableHashtable, but the beauty of this is that none of the existing code had to change, and now all proxied hashtables have the Lock() method as well.  And alongside Windsor, this is brain-dead simple to wire up:
kernel.Register(
Component.For<IDictionary, ILockableDictionary>()
.ImplementedBy<Hashtable>()
.Proxy.AdditionalInterfaces(typeof(ILockableDictionary))
.Proxy.MixIns(new Locker()));

And that’s it!  BTW, I realize I could have easily achieved the same effect and with a lot less effort if I just inherited from Hashtable directly (and avoid the negligible proxy/performance cost), but if I needed this locking mechanism for any other code, I would have to resort to either copy/pasting a lot of code, or a lot of copy/paste/redirect to a static class.

Edit: There is a bug in this last bit, check out my latest entry about this topic.

Monday, October 19, 2009

Improving Your Unit Testing Skills

Unit testing is hard!  I came to this sad realization when my code which had a large test suite with near 100% code coverage, with every thought-of requirement unit tested, failed during integration testing.

How is that possible?  I thought to myself.  Easy…my unit tests were incomplete.

Writing software is pretty complex stuff.  This is pretty evident in the level of difficulty in determining how good a software developer is.  Do you measure them based on lines of code?  Can you measure on time to completion of tasks?  Or maybe the feature to bugs coming back ratio?  If writing features alone can be this complicated, surely unit testing is just as (or more) complicated.

First, you must be in a place where you can even start testing your code.  Are you using dependency injection?  Are your components decoupled?  Do you understand mocking and can you use mocking frameworks?  You need an understanding of all these concepts before you can even begin to unit test effectively.  Sure, you can unit test without these prerequisites, but the result will likely be pain and more pain because the you’ll be spending 90% of your time setting up a test rather than running it.

But back to the point of this post.  Unit testing is hard because it’s one of those things where you’ll never really know how to do it properly until you’ve tried it.  And that assumes you’ve even made the step to do testing at all.

I’m at the point where my experience with writing unit tests allows me to set them up quickly with relative ease, and perform very concise tests where anyone reading the unit test would say “oh, he’s testing this requirement.”  However, the problem is that there’s still 2 problems that are hard to fix:
a) Is the test actually correct?
b) Do the tests cover all scenarios?

(a) is obvious.  If the test is wrong then you might as well not run it all.  You must make it absolute goal to trust your unit tests.  If something fails, it should mean there’s a serious problem going on.  You can’t have “oh that’s ok, it’ll fail half the time, just ignore it.”

(b) is also obvious.  If you don’t cover all scenarios, then your unit tests aren’t complete.

In my case, I actually made both errors.  The unit test in question was calculating the time until minute X.  So in my unit test, I set the current time to 45 minutes.  The target time was 50 minutes.  5 minutes until minute 50, test passed.  Simple right?  Any veteran developer will probably know what I did wrong.  Yep…if the current time was 51 minutes, I ended up with a negative result.  The test was wrong in that it only tested half of the problem.  It never accounted for time wrapping.  The test was also incomplete, since it didn’t test all scenarios.

Fixing the bug was obviously very simple, but it was still ego shattering knowing that all the confidence I had previously was all false.  I went back to check out other scenarios, and was able to come up with some archaic scenarios where my code failed.  And this is probably another area where novice coders will do, where experienced coders will not: I only coded tests per-requirement.  What happens when 2 requirements collide with each other?  Or, what happens when 2 or more requirements must happen at the same time?  With this thinking I was able to come up with a scenario which led to 4 discrete requirements occurring at the same time.  Yeah…not fun.

Basically, in summary:
a) Verify that your tests are correct.
b) Strive to test for less than, equal, and greater than when applicable.
c) Cross reference all requirements against each other, and chain them if necessary.

Wednesday, October 7, 2009

TDD is Design…Unit Testing Legacy Code

I’ve been listening to a bunch of podcasts lately and I came across a gem that was pretty darn useful.  It’s pretty old I suppose in technology standards, since it was recorded January 12, 2009, but it’s still worth a listen.  Scott Hanselman talks with Scott Bellware about TDD and design.  Find it here.
I’m merely reiterating what Scott has already said in the podcast, but it’s different when I can personally say that I’m joining the ranks of countless others who have seen the benefits of TDD.

Now I can’t say that I’m a 100% TDD practitioner since I’m still learning how to write tests before code effectively, but I’ve already seen the improvements in design in my code many times over just by merely writing unit tests. At this point I'd say half of my tests are written first, and the other after are after the fact.

I’ve been through many phases of writing unit tests, it it took me a long time to get it right (and I’m sure I have tons more to go).  It takes some experience to figure out how to properly write unit tests, and as a by-product how to make classes easily testable.  Code quality and testability go hand-in-hand, so if you find something difficult to test, it’s probably because it was badly designed to start.

The first time I was introduced to unit testing in a work setting was when I was writing C++…and man it was ugly, not because it was C++, but because everything was tightly coupled and everything was a singleton.  The fixture setup was outrageous, and it was common to ::Sleep(5000) to make tests pass.  Needless to say, my first experience with unit testing was extremely painful.

After a job switch and back to the C# world, I started reading more blogs, listening to more podcasts, and experimenting more.  I was given a project to prototype, and for phase 1 it was an entirely DDD experience with I got the opportunity to experiment with writing unit tests the “proper” way with dependency injection and mocking frameworks.
Unit tests are easy to write when you have highly decoupled components.
Prototype was finished, and now I was back on the main project, and given a critical task to complete in 2 weeks.  I had to modify a class which was 10,000 lines long, which has mega dependencies on everything else in the project.  Anything I touched could potentially break something else.  And of course…no unit tests whatsoever.  I like challenges and responsibility – but this was close to overkill.  This thing produces revenue for the company daily so I really don’t want to mess anything up.

First thing I realized was that there’s no way I could possibly write any unit test for something like this.  If the class file was 10,000 lines long, you can imagine the average line count for methods.

And of course, the business team didn’t make things easy on me by asking that this feature be turned off in production, but turned on for QA.  So, the best option was to refactor the existing logic out to a separate class, extract an interface, implement the new implementation, and swap between the 2 implementations dynamically.

After 4 days of analyzing and reading code to make sure I have a very detailed battle plan, I started extracting the feature to a new class.  The first iteration of the class was UGLY.  I extracted out the feature I was changing to a separate class, but the method names were archaic and didn’t have any good “flow” to them.  I felt that I had to comment my unit tests just so whoever’s reading them could understand what’s happening, which brings up a point.
If you need to comment your unit tests, you need to redesign the API
It took many iterations and refactoring of the interface to get it to the point where I found acceptable.  If you compared the 1st batch of unit tests to the last batch of unit tests it is like night and day.  The end result were unit tests which typically followed a 3-line pattern of setup/execute/verify.  Brain dead simple.

The last thing to do was to reintegrate the class into the original class.  For this I was back to compile/run/debug/cross-fingers, but I had full confidence that whenever I called anything in the extracted class it would work.

An added benefit is that I didn’t add another 500 lines of code to the already gigantic class.  Basically, in summary:

  • Get super mega long legacy code class files
  • Extract feature into separate file as close to original implementation as possible (initially will be copy-paste)
  • Run and make sure things don’t die
  • Write unit tests for extracted class (forces you to have a better understanding of the code modifications, and the requirements)
  • Make sure unit tests cover all possible scenarios of invocation from original class
  • Start refactoring and redesigning to better testability, while maintaining all previous tests
  • Done!  Repeat as necessary!

Sunday, September 27, 2009

Dark Color Refresh

There you have it!  I started with Visual Studio’s default settings and configured each and every color one by one.  3 hours later I was finished and satisfied with what I have…but I think it was time well spent considering this is what I’m looking at for >= 8 hours a day.

This was a complete re-do of my previous dark theme.  This one is even more colorful than the previous, mainly because Resharper does an amazing job of detecting additional code elements like mutable vs unmutable, or unused code, in additional to many other things.

By the way, you should definitely spend the time to configure your own colors rather than using someone else’s template.  The reason being is that each and every person is different, and obviously will have preferences towards different colors.  I go through different moods and I’ll tweak the theme to be “blue” or “red” for a period of time.

Another reason why you configure it manually is that it drills into your brain more of what each color represents.  If you just used my template you would probably never notice that orange means input parameter, and green is a local variable.

However, one thing you can and should take from this blog post is not to use pure black or pure white as your background (did you notice my current line indicator is darker than my background?).  Set your background to (10,10,10) or (245,245,245).  Your eyes will thank me.

Saturday, September 26, 2009

Autofac’s extremely powerful and flexible ContainerScope

I need to show some love and support for my favorite IoC tool because it’s most powerful feature needs more explaining.  It’s not that the main site doesn’t provide a good explanation, because it does, but because most people don’t really understand what the solution is solving.
The following is on Autofac’s front page:
var container = // ...
using (var inner = container.CreateInnerContainer())
{
  var controller = inner.Resolve<IController>();
  controller.Execute(); // use controller..
}
The first time I saw this I basically glanced right passed it.  Honestly I didn’t think anything of it, and initially the reason I tried out Autofac was for its very slick lambda registrations.  I didn’t realize I was in a world of surprises when I finally realized the power and flexibility of Autofac’s container scope.

If benchmarks and feature comparisons are not enough show off Autofac’s features, this blog post hopes to show how to solve “complex” problems elegantly.

Let’s start with the typical NHibernate use-case.  Create 1 singleton, create many sessions-per-request.  Here’s a solution with Ninject (not that I’m picking on Ninject, because I love its very slick contextual binding, but because most other IoC containers have a similar solution, like per-session with WCF & Windsor).

Basically, the solutions mentioned above will following an approach like this:

a) Hook into the beginning of a request, and create the ISession from the ISessionFactory.
b) Set it in the HttpContext.Current or OperationContext.Current’s dictionaries.
c) Get this property in all the dependencies that need it.
d) At the end of the request, Dispose() the ISession.

OK, pretty simple and straightforward solution, but there’s one key thing that really bugs me is that by doing this we have introduced a dependency…that is, HttpContext.Current[].  That, or you could wrap that around a class, like SessionManager, again, basically coupling to a dependency under a different name.  With Autofac, we can bypass steps b and c entirely and only worry about the beginning and end of a request.

To start off, here's the basic wiring needed:
1:   var cb = new ContainerBuilder(); 
2:   cb.Register(x => CreateSessionFactory())
       .As<ISessionFactory>()
       .SingletonScoped(); 
3:   cb.Register(x => x.Resolve<ISessionFactory>().OpenSession())
       .As<ISession>()
       .ContainerScoped(); 
4:   IContainer c = cb.Build(); 
5:   
6:   Assert.AreSame(c.Resolve<ISessionFactory>(), c.Resolve<ISessionFactory>()); 
7:   Assert.AreSame(c.Resolve<ISession>(), c.Resolve<ISession>()); 
8:   
9:   var inner1 = c.CreateInnerContainer(); 
10:  Assert.AreSame(c.Resolve<ISessionFactory>(), inner1.Resolve<ISessionFactory>()); 
11:  Assert.AreNotSame(c.Resolve<ISession>(), inner1.Resolve<ISession>());
That’s the configuration.  And that’s it!  Nothing more.  No additional SessionManager class.  No need to use HttpContext.Current to store the session.  Just pass ISession in with regular constructor/property injection.

Here’s how it works:

Line 2: ISessionFactory is created from CreateSessionFactory().  This is a singleton so there will always be one and only one instance of it within the container (and all child containers).

Line 3: This is where it’s interesting.  We’re saying “whenever I need an ISession, resolve ISessionFactory and call OpenSession() on it”.  Also, by specifying ContainerScope, we only get 1 instance per-container.

And this is where it’s sort of confusing with the terminology.  You can think of Autofac as a tree of containers.  The root container (variable c in this case), can create children containers (inner1 in this case, and inner1 could create an inner2, and so on).  So when something is Singleton scoped, that means that the root container, and any child containers (and child’s children) will only have 1 instance of a service.  With ContainerScope, each “node = container” in the tree gets 1 instance.

So back to the unit test above, in line 6 we verify that there is only 1 instance of ISessionFactory.  We resolve ISession twice as well, which shows that we get the same instance.

Line 9, we create an inner container, and here we see that ISessionFactory is the same for both the container c and inner container inner1.  However, the ISession resolved is different between the two.
Thus, by specifying ContainerScope, you can very easily group multiple services and dependencies together as one unit.  Implementing the Unit of Work pattern is insanely easy with Autofac.  Create services A, which depends on B, which depends on C, which all the previous depends on D.  Resolve A within a new inner container, and B, C, and D will always be the same instances.  Resolve A in another inner container and you will get a new set of instances.

Last but not least, Autofac will automatically call Dispose() on all resolved services once the container is disposed.  So for the above, once inner1.Dispose() is called, ISession.Dispose() is automatically called.  If you needed to, you can very easily hook into this mechanism and implement things like transactions and rollbacks.

I hope this blog post clears things up a little bit about Autofac’s ContainerScope!

Sunday, September 13, 2009

Getting code coverage working on TeamCity 5 with something other than NUnit

I've been experimenting lately with TeamCity 5 EAP and so far it's been a pretty awesome experience. I was up and running within minutes and I was swarmed with beautiful graphs and statistics with specifics even per-test. Getting something like that up with CC.NET is not a trivial task.

Anywho, with TC5 code coverage is one of the cool new features added for .NET, but unfortunately only NUnit is supported. Not that that's a bad thing, but some people prefer to use other testing tools. Two notable contenders are xUnit.net and MbUnit.

I like the fact (pun intended) that xUnit.net makes it a point to prevent you from doing bad practices (like [ExpectedException]), and I like how MbUnit is so bleeding edge with useful features (like [Parallelizable] and a vast availability of assertions).


And with that I set up to figure out how to get TC working with Gallio, but the following should work with any test runner.

It certainly was a pain to set up because it took a lot of trial and error but eventually I figured it out.  I analyzed the build logs provided in each build report and noticed something interesting...specifically:


[13:51:23]: ##teamcity[importData type='dotNetCoverage' tool='ncover' file='C:\TeamCity5\buildAgent\temp\buildTmp\tmp1C93.tmp']
and

[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageL' value='94.85067']
[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageM' value='97.32143']
[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageC' value='98.68421']
[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='921.0']
[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageAbsMCovered' value='218.0']
[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageAbsCCovered' value='75.0']
[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='971.0']
[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageAbsMTotal' value='224.0']
[13:51:30]: ##teamcity[buildStatisticValue key='CodeCoverageAbsCTotal' value='76.0']

The first message happens after NCover.Console is done its thing.  After NCoverExplorer is done its thing, the statistics are published.  I set out to mimic this functionality with Gallio, but what's described here should work with any test runner.

1) Disable code coverage in TC.  We're doing it manually instead.
2) In your build script, run your unit tests with NCover and generate a coverage.xml report.
3) Run NCoverExplorer on coverage.xml and generate reports ncoverexplorer.xml and index.html.
4) Create a zip file of index.html and name it coverage.zip.
5) Configure coverage.zip to be an artifact in your TC configuration (this is to enable the tab).
6) Parse out ncoverexplorer.xml with XPath and output the statistics.

Certainly a lot of things to do just for the sake of pretty statistics reporting....but it was the weekend and I was bored.  With the help of MSBuildCommunityTasks, the zip file and XML parsing was made a lot easier.

After that, viola!  Code coverage + Gallio on TeamCity 5!!!

Unfortunately, NCoverExplorer's report only reports the # total of classes and nothing about unvisited or covered, so for those values I set to 0/0/0 (BTW, you need all values present for the statistics to show).  A task for next weekend!!!

(Edit: I suppose I could should also mention that you could technically avoid all the trouble above and hack it with this:
#if NUNIT
using TestFixtureAttribute = NUnit.Framework.TestFixtureAttribute;
using TestAttribute = NUnit.Framework.TestAttribute;
#endif
And it'll work just fine as well)

Monday, September 7, 2009

Member injection module for Autofac

Inevitably as a project gets more complicated, you will need to start using more features of your IoC container. Autofac has built-in support for property injection by hooking into the OnActivating or OnActivated events, which basically set all public properties (or only those which are unset). However, I didn't really like this because once you start using properties, it's not as clear cut as constructors that you are using injection. It becomes hard to manage later on when you have many properties which some should be injected and others should be manually set in code. Autofac's inject all or only-null approach doesn't fit the bill when a class gets a little more complicated. I set up to fix this by writing a custom module, and it turned out to be very very simple. With attributes, we can mimic functionality that's used in Ninject. Here's the module code:
public class MemberInjectionModule : IModule
{
  public void Configure(IContainer container)
  {
    container.ComponentRegistered += (oo, ee) =>
      ee.ComponentRegistration.Activated += (o, e) =>
    {
      const BindingFlags flags = BindingFlags.Instance |
                                 BindingFlags.NonPublic |
                                 BindingFlags.Public;
      var type = e.Instance.GetType();
      foreach (var field in type.GetFields(flags))
      {
        if (field.GetCustomAttributes(typeof(InjectedAttribute), true).Length > 0)
          field.SetValue(e.Instance, e.Context.Resolve(field.FieldType));
      }
      foreach (var prop in type.GetProperties(flags))
      {
        if (prop.GetCustomAttributes(typeof(InjectedAttribute), true).Length > 0)
        {
          if (prop.GetIndexParameters().Length > 0)
            throw new InvalidOperationException("Indexed properties cannot be injected.");

          prop.SetValue(e.Instance, e.Context.Resolve(prop.PropertyType), null);
        }
      }
    };
  }
}
[AttributeUsage(AttributeTargets.Field | AttributeTargets.Property)]
public class InjectedAttribute : Attribute { }
And that's it! Pretty basic reflection here stuff here, but as you can see...iterate through all fields and properties (including privates), and try to resolve them. Now, you can just RegisterModule(new MemberInjectionModule()), and inside your services you simply annotate your properties/fields with the [Injected] attribute, like so:
public class SomeService : ISomeService
{
  [Injected]
  protected ISomeOtherService SomeOtherService { get; set; }
}
And now, it's a very clear cut way of explicitly defining which properties (and fields) should be injected. Also, it's also a simple string property away defined in the attribute if you want to specify a named service, which I'll leave the the reader to implement :)