BLing on Software Pragnitism

My Thoughts on MEF

2011-10-31T15:45:00.001-07:00

Ever since MEF was conceived, despite the authors saying that it is not an IoC container, it has since evolved to become one of the more popular IoC containers. I’ve always avoided it because I disagree with using attributes, and I’ve had no reason to use it over Autofac or Windsor.

Recently, I found a reason to use it – Metro-style applications only support MEF so far. My Twitter client ping.pong uses Autofac as the IoC container. It uses some very basic functionality like factories and hooks. To my surprise, MEF has no support for either of these.

Coming across these limitations solidifies my opinion that MEF is a plugin container, not an IoC container.

First let’s take a look at automated factories. What I mean is that by registering Foo, like so:

container.RegisterType<Foo>();

the container will automatically provide us a Func<Foo> without explicitly having to register it. This can be useful when you want to create an instance of Foo some time in the future rather than at constructor time. You can do this with MEF via an ExportFactory<T>, but it’s limited because you cannot override dependencies at resolve time.

For example, let’s say Foo has a constructor of Foo(Bar1, Bar2, Bar3). With MEF, you have no control at resolution time what the Bars are. A container that has support for automated factories (like Autofac and Castle Windsor), will let you resolve a Func<Bar1, Foo>, which lets you override Bar1 at resolve time. Similarly, you can resolve a Func<Bar1, Bar2, Bar3, Foo> and override all dependencies. Any dependencies not overridden fall back to their configuration in the bootstrapper. This is a very useful feature, and coupled with the scoping features for automatic disposal it opens up many doors for elegant solutions for what otherwise are complicated problems.

On to the second point; MEF has limited extension points. This one sounds odd since MEF is all about designing decoupled plugins so surely it should have extension points! The problem here is that MEF is designed as an explicit API (attributes are required) rather than an implicit API. In Autofac, you can scan an assembly and register every type. In MEF, every class needs to have an [Export] on it. It also baffles my mind why [ImportingConstructor] is required even when there’s only one constructor. All this explicitness means you lose a bunch of “free” extension points that typical IoC containers have, like this:

b.RegisterAssemblyTypes(GetType().Assembly)

  .OnActivated(x => x.Context.Resolve<IEventAggregator>().Subscribe(x.Instance));

What the code above is saying that every time any component is activated, it will subscribe to the event aggregator. If the component doesn’t IHandle<> any messages, it’s a no-op and continues on. If the instance does IHandle<> messages, this will ensure it’s hooked up.

The closest thing I could find in MEF was IPartImportsSatisfiedNotification (yes, an interface, more explicitness!). It contains a single method OnImportsSatisfied() which gets called when the part is created. Needless to say, the one line of code from Autofac would translate into a method for every implementation of IHandle<>, and since OnImportsSatisfied() contains no contextual information, every component will need IEventAggregator injected just to be able to call Subscribe.

To fully complete this example, Autofac has the following methods when registering a component: OnRegistered, OnPreparing, OnActivating, OnActivated, and OnRelease. Each of these methods gives you complete contextual information at the time it is called like access to the current scope of the container, the instance (if applicable), which component which requested the dependency, etc. This makes it almost too easy to extend the container.

For MEF, the only real extension point is an ExportProvider. It is pretty low level (all it does is parse attributes for you) so to write anything similar for MEF requires a lot more code. To further illustrate this point, compare the interception modules from AutofacContrib and MefContrib. The Autofac implementation is a single file with a couple extension methods. The MEF implementation is an entire namespace, over multiple classes, not the mention that it also relies on other infrastructure code in MefContrib. Basically, the guys that wrote MefContrib had to write a mini-container within MEF.

MEF is great for building extremely loosely coupled applications. I don’t think it has any business in an application where you know and own all of the dependencies; there are simply better libraries for that.

ping.pong Twitter Client

2011-10-09T10:32:00.001-07:00

I have a whole series dedicated to blogging about how I wrote a Twitter client from scratch, but if I want anyone to actually use it, I better do some advertising :-D.

ping.pong is a fast and lightweight Twitter client written in Silverlight. As of this moment it targets v4 but will likely target v5 whenever that is released.

Here are some highlights…

Visually Pleasing

The UI is based on your typical column-based design. The column widths will automatically resize to take up all available horizontal space. There is no horizontal scrolling.

Access to the Streaming API

No more rate limits! Your timeline is connected directly with Twitter’s streaming API, which means whenever someone you follow tweets you will know about it in almost near real-time. Searching is also done through the streaming API.

Fast & Lightweight

ping.pong was built to run fast with low CPU utilization. Even when stream searching for “e” (yes, the letter E), the CPU usage stays under 15%. The maximum number of tweets that Twitter sends appears to be 50 per second.

Conversations

ping.pong will quickly show an entire tweet conversation by navigating reply tags back to the original tweet that started it all.

Free & Open Source

The full source code for ping.pong can be found on GitHub. You can compile it, make modifications as you please, and run it yourself. The only thing missing is the consumer keys which uniquely identifies this client from another. You can generate them through Twitter once you have a developer account.

Installer

The latest and greatest can be quickly installed from here. Check back periodically for updates!

Push Driven Development with Reactive Extensions

2011-09-29T19:04:00.001-07:00

This is going to be the last post that concludes my series on building a real-time push app with Silverlight. Any additional posts would likely be outside the context of writing a push app and more about how I’m adding features to ping.pong, my Twitter app, so I think this is a good place to wrap up and talk generally from a top down overview of building a push-style application.

Here’s a recap of everything discussed so far:

Part 1: Basics – Creating an Observable around a basic HTTP web stream against Twitter’s streaming API

Part 2: Subscription and Observation of Observables

Part 3: Basics of UX design with a look at shadows and gradients.

Part 4: Integrating with 3rd party libraries, notably Caliburn Micro and Linq2Twitter and how to achieve polling with observables.

Part 5: A minor hick up with Linq2Twitter.

Part 6: Taking advantage of transparencies to improve the design and reusability of UX.

Part 7: A summary of all things encountered so far, replacing Linq2Twitter with Hammock, first release of code to GitHub, and a binary released capable pulling and streaming tweets from Twitter.

Part 8: Examples of using Caliburn Micro to easily resolve data bindings that otherwise would be much more effort.

And that leads us to this post…

PDD (Push Driven Development)

One of the main goals of this series is to create a performant Silverlight app based on push principles, as opposed to more traditional pull principles. To that effect, ping.pong has performed remarkably well and is limited only by Twitter’s throttling, which currently appears to be maximum of 50 tweets per second via the streaming API.

Writing the application from a push-driven mindset was definitely unintuitive at first, and I had to refactor (actually rewrite is more accurate) the code many times to move closer to a world where the application is simply reacting to events thrown at it (as opposed to asking the world for events).

To be absolutely clear on what I mean on the differences between push and pull, here’s a comparison:

Pulling	Push
var e = tweets.GetEnumerator(); while (e.MoveNext()) // is there more? { e.Current; // get current DoSomething(e.Current); }	IObservable<Data> data = /* get source / // whenever data comes, do something* data.Subscribe(DoSometing);

On the pulling side, the caller is much more concerned with the logic on how to process each message. It needs to repeatedly ask the world, “hey, is there more data?”.

On the push side, the caller merely asks the world, “hey, give me data when you get some”.

Twitter is a perfect example because their APIs have both a pulling and pushing models. Traditional clients poll continuously all the time, and many had configurable options to try to stay under the 200 API calls per hour limit. Most of Twitter’s API still consists of pulling, but the user’s home line and searching can be streamed in near real time via the streaming API, aka. push. Streaming tweets effectively removes the API call limit.

Push and Pull with Reactive Extensions

The beauty of Rx is that regardless of whether it is actually pushing or pulling, the API will look same:

IObservable<Tweet> tweets = _client.GetHomeTimeline();

tweets.Subscribe(t => { /* do something with the tweet */ });

As far as the caller is concerned, it doesn’t care (or needs to know) whether the GetHomeTimeline method is polling Twitter or streaming from Twitter. All it needs to know is when a tweet comes it will react and do something in the Subscribe action.

In fact, Subscribe simply means “when you have data, call this”, but that could also be immediately, which would be analogous to IEnumerable.

However, if that was the only thing Rx provided it wouldn’t be as popular as it is, because other pub/sub solutions like the EventAggregator already provide a viable asynchronous solution.

Unlocking Rx’s power comes with its multitude of operators. Here’s an example:

public static IObservable<Tweet> GetStreamingStatuses(this TwitterClient client)

  return client.GetHomeTimeline()

      .Merge(client.GetMentions())

      .Concat(client.GetStreamingHomeline());

GetHomeTimeline and GetMentions initiate once-only pull style API calls, while GetStreamingHomeline will initiate a sticky connection and stream subsequent tweets down the pipe.

The Merge operator is defined as this: “Merges an observable sequence of observable sequences into an observable sequence.”

I think a better description would be “whenever there is data from any of the sources, push it through”. In the example above, this would translate to whenever a tweet comes from either the home timeline or the mentions timeline, give me a Tweet (first-come-first-push), followed by anything from the streaming timeline.

And there lies one of the greatest beauties of Rx. All of the complexity lies solely on setting up the stream and operators. And that, also, is its disadvantage.

Rx Complexity

Let’s take a look at the Concat operator, defined as: “Concatenates two observable sequences.” In the remarks sections it states this: “The Concat operator runs the first sequence to completion. Then, the second sequence is run to completion effectively concatenating the second sequence to the end of the first sequence.”

Let’s try it out:

var a = Observable.Interval(TimeSpan.FromSeconds(1)).Select(x => x.ToString());

var b = Observable.Interval(TimeSpan.FromSeconds(1)).Select(x => "s" + x);

a.Concat(b).Subscribe(Console.WriteLine);

// output: 0, 1, 2, 3, 4...

As expected, only numbers are printed because the first sequence never ends, so it won’t concatenate the second one. Let’s modify it so that it does finish:

int count = 0;

var a = Observable.Interval(TimeSpan.FromSeconds(1))

    .TakeWhile(_ => ++count < 5)

    .Select(x => x.ToString());

Note, that using Observable.Generate is preferred because it doesn’t introduce an external variable, but I stuck with Interval so the code looks similar to the second observable. As expected again, it will print “0, 1, 2, s0, s1, s2”.

OK, let’s spice things up. Let’s make b a ConnectableObservable by using the Publish operator, and immediately call Connect.

int count = 0;

var a = Observable.Interval(TimeSpan.FromSeconds(1))

    .TakeWhile(_ => ++count < 5)

    .Select(x => x.ToString());

var b = Observable.Interval(TimeSpan.FromSeconds(1)).Select(_ => "s" + _).Publish();

b.Connect();

a.Concat(b).Subscribe(Console.WriteLine);

What do you think the output of this will be? The answer is “0, 1, 2, 3, s5, s6, s7, …”

Despite using the same Concat operator, the result can be very different depending on the source observables. If you use the Replay operator, it would have printed “0, 1, 2, 3, s0, s1, s2, …”

Years and years of working in synchronous programming models have trained us to think in synchronous ways, and I picked Concat specifically because Concat also exists in the enumerable world. Observable sequences are asynchronous, so we never know exactly when data comes at us, only what to do when it does. And because streams occur at different times, when you combine them together there are many many ways of doing so (CombineLatest, Merge, Zip, are just a few).

The greatest hurdle to working in Rx is to know what the different combinations do. This takes time and practice. RxTools is a great learning tool to test out what all the operators do.

Unit Testing

Last but not least, Rx can make it easier to write unit tests. The concept is easy: take some inputs and test the output. In practice this is complicated because applications typically carry a lot of state with them. Even with dependency injection and mocking frameworks I’ve seen a lot of code where for every assert there is 10 lines of mock setup code.

So how does it make it easier to test? It reduces what you need to test to a single method, Subscribe, which takes one input, an IObservable<T>.

Conclusion

Rx is a library unlike any other you will use. With other libraries, you will add them to your solution, use a method here or there, and go on with your life. With Rx, it will radically change the way you code and think in general. It’s awesome.

Building a Real-time Push App with Silverlight: Part 8

2011-09-21T20:25:00.001-07:00

Exploring Caliburn Micro

As I hinted in earlier posts, Caliburn Micro has some wicked conventions that makes for writing MVVM super easy, and it also have a very convenient syntax for hooking up events. For example, the following:

<Button Content="R">   <i:Interaction.Triggers>       <i:EventTrigger EventName="Click">           <i:InvokeCommandAction Command="{Binding ReplyCommand}" CommandParameter="{Binding}" />       </i:EventTrigger>   </i:Interaction.Triggers></Button>

Can be rewritten like this:

<Button Content="R" cal:Message.Attach="[Reply($dataContext)]" />

There are some smarts going on here. Caliburn Micro will default to the Click event for buttons. For a full syntax, it would be cal:Message.Attach=”[Event Click] = [Reply($dataContext)]”. As you can imagine, that will call the Reply method and pass in the current data context. You can also pass in other things like $this, $source, or $executionContext for full access to anything and everything Caliburn Micro itself has access to.

The coolest thing about this is it gives you some wicked control over how your data context gets set. Ever struggled with popup windows or data grids and using weird hacks to get the binding correct? Caliburn Micro makes this very easy. Here’s an example.

I have a DataTemplate which renders the UI for the model Tweet.
Tweet is just a simple class which holds only properties.
Inside the DataTemplate, I have some buttons that when the user clicks will reply, retweet, quote, or direct message.

The Tweet class is purely for modeling data, so adding any methods would be bad practice. Also, since I’m in a DataTemplate I can’t easily reference another control with ElementName (in this case I need the containing parent’s DataContext). And to add insult to injury, Silverlight 4 doesn’t have RelativeSource ancestor type. So how do I solve this?

<StackPanel VerticalAlignment="Bottom" cal:Action.TargetWithoutContext="shell" Orientation="Horizontal">

    <Button Content="R" cal:Message.Attach="[Reply($dataContext)]" />

    <Button Content="RT" cal:Message.Attach="[Retweet($dataContext)]" />

    <Button Content="Q" cal:Message.Attach="[Quote($dataContext)]" />

    <Button Content="DM" cal:Message.Attach="[DirectMessage($dataContext)]" />

</StackPanel>

The secret is the attached property TargetWithoutContext. As the name implies, it will set the target for all the ActionMessages attached to all the buttons, without setting the context. If I used the Target attached property, it would set all of the Buttons’ data context to the same object – not what we want. Since the Button’s data context remains intact, we can call “Reply($dataContext)”, which calls the Reply method on the target object (set on the StackPanel) and pass in the Tweet. “shell” is the key of the service that I registered into the container.

Originally I wanted this entire series to be able writing a fast push data app with Silverlight and Rx, and now I’m finding that I’m writing an entire Twitter client because it’s so much fun :-).

I’m going to make another release soon. While the first release was merely experimental, the next one will be useful enough to potentially use full time. As you can probably tell with this blog post, it supports all the actions mentioned previously (and it’ll appear on mouse hover):

The tweet box is much improved and shows you how many character you have left:

And it’s smart enough to auto wrap http links via Twitter’s t.co service, and the counter takes that into account. Some interesting things to note is that in the future all links will be wrapped t.co. Looks like Twitter is trying to eat up bt.ly or something.

Clicking on @users and #topics will automatically open a new timeline and subscribe to those tweets. It is almost full featured enough to become my main Twitter client. There are certain features still missing, and it’s purely based on when I have time to port them over.

As always, you can install directly from here, or you can grab the code on the GitHub page!

Next post will be about Rx from a very top level perspective and how it influenced my code from beginning to be experienced and all refactorings in between. Stay tuned!

Building a Real-time Push App with Silverlight: Part 7

2011-09-16T06:26:00.001-07:00

Infrastructure Refactor

A lot of things changed internally, and I mean….a lot….

From an infrastructure standpoint, I decided to remove the dependency on LinqToTwitter, and I replaced it with Hammock. A couple things led me to this decision, one being the Silverlight support wasn’t as good as I’d hoped, and the streaming API implementation was limited. After reading the Twitter documentation I realized that the REST API was super simple and I’d be better off writing a simple interface to it.

I heard good things about Hammock, so I decided to give that one a try (I wasn’t going to go as far as reimplementing OAuth). It was pretty easy to set up and in the end I was able to get Twitter working again and with less lines of code compared to the beginning of the refactor.

Goals

I had a couple goals for this project:

Learn: I was a complete newbie to Reactive Extensions when I started but now I understand it enough to hit the ground running with it. I’m still learning about more conventions available to Caliburn.Micro.
UX: I wanted to learn a little more about interface design. I wanted to know how little changes to gradients, shadows, colors, etc. could have a radically effect in the end result.
Performance: It should be fast. It should be able to react to real-time data. And it should do it with low CPU utilization.
Concise: I am a huge advocate for KISS. I like convention over configuration. I like implementing something in 2 lines of code rather than 20 (assuming it’s not cryptic). As I was writing the app and refactoring, if there was an opportunity to remove a line of code, I did it. The result is that the app currently consists of less than 500 lines of code as of this post (excluding XAML).

Tidbits

What are some interest things I learned?

System.Json is an amazing assembly. All you need to do is invoke JsonValue.Parse on a string and it will create a JsonValue for you, which will be a dictionary of key/value pairs. What’s more, by doing something like “string s = json[“text”]” will do an explicit conversion and unescape JSON characters, and only via the explicit operator. Calling ToString(), even though converting it to a string, will not unescape. This was completely undocumented and only found when I looked at the source code via Resharper’s external sources feature.
Rx is awesome. When I ran into performance problems of trying to stream tweets from the world that contained the letter ‘a’ all I had to do was add an operator to improve the performance (in this case it was Buffer). It should be noted that it is very important to understand what Rx is doing underneath the hood to realize its benefits. Rx lets you refactor 30 lines of async code into 1 operator, but it’s still doing that 30 lines of code – you just don’t see it.
I really, really, like the conventions available from Caliburn. Some of the features that come out of the box from this very small library saves me from writing a lot of boilerplate code like commands, triggers, and evening bindings (Caliburn will auto bind x:Name to a property).
Twitter’s documentation for user streams currently sucks and some trial and error was required to get it working.

What is the end result of all this effort? We have a styled Twitter app that can update your status, pull your home/mentions timeline, and most importantly will stream all subsequent tweets. There’s no pulling and no limits. You will get a tweet of everyone you follow in real-time as it happens.

Moreover, there’s a feature to connect to the Streaming API to search Twitter for anything. To get an idea of what we’re talking about, here’s a full screenshot of it:

You read that right. I’m streaming any tweet in the world that has the words ‘and’, ‘the’, ‘yes’, or ‘no’ in them. This is streaming around 400kB/s continuously and CPU utilization is under 25%. The tweets are coming so fast it’s impossible to read them (at a rate of 50 tweets/second), so ideally you’d want to specify realistic search terms.

Moreover, the majority of the performance cost is actually downloading all the profile images. If I take took out pictures I could stream any tweet in the world that has the letter ‘e’ in it at under 10% CPU. It looks like Twitter limits the rate of tweets to 50 tweets/second because that was the rate for this one as well.

Features are minimalistic. You can update your status, but you can’t DM, you can’t RT, you can’t do any of the normal things. My original goal was not to write another Twitter client, but it’s actually quite fun to do so, so I’ll probably eventually get all features in.

And as promised, it’s up on GitHub, and version 0.0.0.1 alpha (yes! expect bugs!!) is available in the downloads section. Or, here’s a direct link to the XAP file on my Dropbox. Have fun!

Building a Real-time Push App with Silverlight: Part 6

2011-09-13T07:00:00.000-07:00

Back to the UI!

For this post I’m going to restyle the tweets. Recall that they currently look like this:

The gradient background is currently #FFEEEEEE to #FFDDDDDD. For this post I’m going to talk about a very powerful tool in a designer’s arsenal: transparency.

Your first may be to think “big deal”, but just like programmers can use base classes and injection to share common code, designers can use transparency to achieve a similar effect.

Let’s change change the color to be black, and tweak only the alpha. I’m going to set the colors to be #11000000 to #22000000 on a white background. This is the result:

Looks almost identical doesn’t it? However, by doing this we have dramatically improved the reusability of the gradient. Here’s what happens when I change the background to be a different color:

I just changed one variable to do that. If I wanted to provide different themes for my application it would be extremely easy to do that if all the data templates were built with transparencies. In fact, I just slap on a ColorPicker and I’d be done!

Even though it’d be easy to do this, any application that is dominated by one color gets boring very quickly. Most applications that look nice tend to focus around two dominant colors that contrast well against each other. Black and white are very common because they contrast well with a large variety of colors, but you can also have things like blue/green, purple/orange, etc. As always, as long as you’re consistent you’ll likely have a good result.

Now, the flip side of the equation is also possible. This is where you have something that exists already and then you put a transparent layer on top of it, creating a lightening or dimming effect. In my experience I’ve found this to be inferior because it tends to wash out colors. In the example above, if I applied a slightly transparent layer over top of the tweet, my picture and text would be negatively affected. This is nonetheless a very useful trick, like with mouse over effects where you want a quick and cheap way of conveying information to the user.

Now, let’s take a big detour and restyle the entire application and go with a completely different theme. I also wanted try something besides Apple and Microsoft inspired designs, which was more difficult than expected because I guess I’m not as creative as I thought I was :-). Coming up with a good design takes a long time, and frequently you need some sort of inspiration. Twitter in general is a very simple application, so the best designs are simple as well.

In an attempt to try to come up with something “cool” and “unique”, I started with the idea of elevated boxes layered on top of each other. Here’s a before and after once I was done:

The redesign went through many iterations. I showed it to friends and colleagues and got mixed feelings. Some liked it. Some thought it was too noisy. And herein revealed a problem with complex designs – they are hard to get right! That, and they tend to divide audience into those that really like it, and those that really don’t.

Anyways, the beauty of XAML is that I can try something else entirely without any changes to the code, so I’ll try another theme in the future.

I’m about 90% ready to release code to GitHub along with the first public alpha version. Stay tuned!

Building a Real-time Push App with Silverlight: Part 5

2011-09-08T18:26:00.001-07:00

I planned on this post to be about UI, but I’m going to defer that until the next post. I said from the start of this series that I would document about everything about building the application from scratch, including my struggles.

And with that I want to mention something that got me scratching my head one too many times. It was with how I used LinqToTwitter. Here is the source code which you can immediately copy/paste into a blank project to reproduce:

   1: public partial class MainPage : UserControl   2: {   3:    private readonly TwitterContext _context = new TwitterContext();   4:    private readonly ViewModel _vm1, _vm2, _vm3;   5:     6:    public MainPage()   7:    {   8:        InitializeComponent();   9:        _vm1 = new ViewModel(_context);  10:        _vm2 = new ViewModel(_context);  11:        _vm3 = new ViewModel(_context);  12:        _vm1.Callback += () => Debug.WriteLine("Callback of VM1: " + _vm1.LocalState);  13:        _vm2.Callback += () => Debug.WriteLine("Callback of VM2: " + _vm2.LocalState);  14:        _vm3.Callback += () => Debug.WriteLine("Callback of VM3: " + _vm3.LocalState);  15:    16:        _vm1.Start();  17:        _vm2.Start();  18:        _vm3.Start();  19:    }  20: }  21:    22: public class ViewModel  23: {  24:    private readonly TwitterContext _context;  25:    public event Action Callback;  26:    27:    public int LocalState;  28:    29:    public ViewModel(TwitterContext context)  30:    {  31:        _context = context;  32:    }  33:    34:    public void Start()  35:    {  36:        var query = (from s in _context.Status  37:                     where s.Type == StatusType.Public && s.Count == 10  38:                     select s);  39:        Debug.WriteLine("Hash code of ViewModel: " + query.GetHashCode());  40:        query.AsyncCallback(statuses =>  41:        {  42:            LocalState++;  43:            Debug.WriteLine("Hash code inside callback: " + GetHashCode());  44:            Callback();  45:        }).FirstOrDefault();  46:    }  47: }

Now, if you run this, you will see that only one of the view models will get its state updated. Huh?!

How is that possible? I started getting paranoid so I even added the local state variable “just in case.”

Well, I had to look into the source code of LinqToTwitter to figure out exactly what happened. Here is the code for AsyncCallback:

public static IQueryable<T> AsyncCallback<T>(this IQueryable<T> queryType, Action<IEnumerable<T>> callback)

     (queryType.Provider as TwitterQueryProvider)

         .Context

         .TwitterExecutor

         .AsyncCallback = callback;

     return queryType;

See what happened? The callback gets overwritten every time you call this method. Even though the call to FirstOrDefault() causes all 3 expressions to evaluate, only the last view model will get values because that’s the with the callback attached.

Lesson of the day: The AsyncCallback extension method for LinqToTwitter is not thread-safe.

So…the question is, how do we make it thread safe? I just replaced wrapped the AsyncCallback with another extension method:

private static readonly AutoResetEvent _twitterEvt = new AutoResetEvent(true); public static void AsyncTwitterCallback<T>(this IQueryable<T> twitter, Action<IEnumerable<T>> callback){    Observable.Start(() =>    {        _twitterEvt.WaitOne();         twitter.AsyncCallback(results =>        {            try            {                callback(results);            }            finally            {                _twitterEvt.Set();            }        }).FirstOrDefault();    });}

Nothing complicated – just a simple wait handle to ensure only 1 thread can go through at a time.

Hopefully upstream fixes this, or at least documents it.

Building a Real-time Push App with Silverlight: Part 4

2011-09-05T14:26:00.001-07:00

Originally I wanted to avoid bringing in external libraries to keep the app as lean as possible, but then I realized that I would spend too much time reinventing the wheel. Twitter is deprecating basic authentication in the near future, which makes OAuth no longer optional. Rather than writing yet another Twitter client (if you’re curious I found a great reference here), I fired up NuGet and brought in LinqToTwitter, and while I’m there I brought in Autofac and Caliburn.Micro as well.

Naturally, LinqToTwitter will work nicely with Rx because as name implies it uses LINQ heavily. Caliburn.Micro is a MVVM library which I’ve always wanted an excuse to try because of features like this:

<ListBox cal:Message.Attach="[Event Loaded] = [LoadList($dataContext)]" />

That’s only scratching the surface of what Caliburn can do, so it will be a fresh breath of air to see what else it can do.

By default, Caliburn uses MEF to wire up its bootstrapper. After adding a couple [Import]s and [Export]s, I knew it wasn’t for me. It works well for writing plugins, i.e. external dependencies because of its built-in assembly scanning capabilities, but for injecting internal dependencies, other IoC containers do a much better job of that. I used Castle Windsor in past projects, but for a change I’m going to use Autofac which I haven’t used since v2 came out.

When this was all said and done the View was the only thing that didn’t change. Everything underneath either changed radically or was deleted altogether (because LinqToTwitter provided it). I added OAuth support and registered my application with Twitter, and with that was the birth of Ping Pong.

This took much longer than expected. Silverlight 5 RC just came out and it broke pretty much any container (including MEF) for OOB because of a TypeLoadException. I haven’t been using too many v5 features, so for the time being I downgraded to v4 to get the project working until RC2 comes out.

Integrating LinqToTwitter was a challenge. The project site has a lot of good documentation, but most of it was for desktop, not Silverlight, and because of that I banged my head a couple times. I wish I grabbed the source code earlier because it’s there where you’ll find hundreds of working examples (in code!) to do everything with the library (and in Silverlight).

After all that, PingPong now has 3 columns (home, public, sampling) that dynamically resizes (it’s surprising that MetroTwit is the only client that does this….) to the window size.

Oh, and there’s pictures now! The streaming time line takes significantly more CPU now that it has to load images, but we’re still sitting at around 5-10% for what is continuously streaming data and loading pictures. Not too shabby! (It took a couple tries to get a PG-13 screenshot from the public/streaming time lines…)

To conclude this post in the series, I’m going to talk about converting an asynchronous operation into an Observable that does not follow any predefined pattern.

Creating an Observable

One of Silverlight’s limitations is that almost everything needs to be an asynchronous call. In regards to LinqToTwitter, something like this will fail (but work on desktop):

var tweets = (from t in context.Status

              where t.Type == StatusType.Public

              select t).ToArray();

On Silverlight you will get a single empty element. To get it working, there is an extension method that comes with the library, and you use it like this:

(from t in context.Status

 where t.Type == StatusType.Public

 select t)

  .AsyncCallback(tweets => { /* do something with it */ })

  .FirstOrDefault();

Code is self-explanatory. The FirstOrDefault() exists only to initiate the expression, otherwise it wouldn’t do anything. So now the question is how do we convert that into an Rx Observable?

Every time I write an Rx query I try to use the least amount of state as possible. This helps to keep the number unexpected anomalies to a minimum. In the following section of code, I was able to get it down to 2 fields: _sinceId, and Context. There is probably some operator that will let me save the sinceId variable from one observable to the next but I wasn’t able to figure it out. In any case, I came up with this:

_subscription =    Observable.Create<Tweet>(        ob => Observable.Interval(TimeSpan.FromSeconds(60))                  .StartWith(-1)                  .SubscribeOnThreadPool()                  .Subscribe(_ =>                  {                      ulong sinceId;                      (ulong.TryParse(_sinceId, out sinceId)                           ? Context.Status.Where(s => s.Type == statusType && s.Count == 200)                           : Context.Status.Where(s => s.Type == statusType && s.Count == 200 && s.SinceID == sinceId))                          .AsyncCallback(statuses =>                          {                              foreach (var status in statuses)                              {                                  ob.OnNext(new Tweet(status));                                  _sinceId = status.StatusID;                              }                          })                          .FirstOrDefault(); // materalize the results                  }))        .DispatcherSubscribe(SubscribeToTweet);

That contains some custom code:

Context: is a TwitterContext from LinqToTwitter
DispatcherSubscribe: is a helper extension method which Subscribes on the ThreadPool, Observes on the Dispatcher, and then Subscribes with the specified action
SubscribeToTweet: a method in the base class which adds to a ObservableCollection so the UI gets updated

To translate the code, here is a basic flow of what’s happening:

Observable.Create wraps the subscription of another Observable. It provides access to an IObserver ob which lets you explicitly invoke OnNext().
Observable.Interval will raise an observable every 60 seconds.
The subscription of Observable.Interval will query the TwitterContext for the next set of tweets.
Inside the AsyncCallback, it invokes ob.OnNext as well as keeps track of the ID so the next time it queries it only gets newer tweets.
Finally, DispatcherSubscribe will take the Tweet object and add it to an ObservableCollection<Tweet>, which notifies the UI.

As always, you should “clean up your garbage”. In this respect I was pretty impressed with Rx as it was able to clean up the entire chain of observables with a single call to _subscription.Dispose(). Nice!

In the next post I’m going to switch back to UI and completely restyle the application. The code will hit GitHub soon as well (I promise!). Stay tuned…

Building a Real-time Push App with Silverlight: Part 3

2011-08-28T15:21:00.001-07:00

In this part we’re going to fire up Expression Blend (the trial for version 5 can be found here) and do some UI work.

In part 2, I created a simple Twitter client which connected to the streaming API, and connected to the sampling request which brings back random tweets. Here is the data template:

<DataTemplate x:Key="TweetDataTemplate">    <Grid DataContext="{Binding}">        <Grid.RowDefinitions>            <RowDefinition />            <RowDefinition Height="Auto" />        </Grid.RowDefinitions>        <TextBlock FontFamily="{StaticResource FontFamily}" FontSize="12" Text="{Binding Text}" TextWrapping="Wrap" />        <TextBlock Grid.Row="1"                   HorizontalAlignment="Right"                   VerticalAlignment="Bottom"                   FontFamily="{StaticResource FontFamily}"                   FontSize="13.333"                   Foreground="BlueViolet"                   Text="{Binding ScreenName}" />        <TextBlock Grid.Row="1"                   HorizontalAlignment="Left"                   VerticalAlignment="Bottom"                   FontFamily="{StaticResource FontFamily}"                   FontSize="9.333"                   Foreground="DarkCyan"                   Text="{Binding CreatedAt}" />    </Grid></DataTemplate>

This renders into something like this:

The text is randomly generated from Blend’s sample data capability, which is totally awesome as it allows designers to see what they’re working with, and keeps the sample data separate from the real data.

While design is a matter of personal taste, and you’re bound to get disagreements between different people, if you follow some basic rules you’ll satisfy a greater audience.

Subtle gradients and small shadows
- If you take a look at all the nice interfaces, they tend to use very slight gradients and small shadows. Most of the time you don’t even notice unless you look closely.
- I think Microsoft’s Metro design is beautiful. Reason? It emphasizes text over decorations (like gradients and shadows). This tends to lead to very clean design because there’s very little opportunity to abuse gradients and shadows.
Realism and light sources
- Continuing on with gradients and shadows, they should be realistic. Look at your design from a 3D point of view. Apply a light source from a certain angle, and then apply your shadows relative to that light source.
- Convey distance properly
Consistent theme
- This one seems obvious but nothing is worse than having a nice looking application bring up an unskinned dialog.
Usability
- If the design doesn’t serve a purpose to make it more usable, it shouldn’t be there. Even something as simple as black on white follows this – you do that so you can read text. However, even something as simple as that can be improved. Take a look at why the Kindle is so successful. The readability is better because of the lower contrast between the black and light-brown background.

With these starting points, let’s redesign the data template.

<DataTemplate x:Key="TweetDataTemplate">   <Grid>       <Grid.Background>           <LinearGradientBrush StartPoint="0.5,0" EndPoint="0.5,1">               <GradientStop Color="#FFDADADA" />               <GradientStop Offset="1" Color="#FFC8C8C8" />           </LinearGradientBrush>       </Grid.Background>       <Grid.RowDefinitions>           <RowDefinition />           <RowDefinition Height="Auto" />       </Grid.RowDefinitions>       <TextBlock FontFamily="{StaticResource FontFamily}" FontSize="12" Text="{Binding Text}" TextWrapping="Wrap" />       <TextBlock Grid.Row="1"                  HorizontalAlignment="Right"                  VerticalAlignment="Bottom"                  FontFamily="{StaticResource FontFamily}"                  FontSize="13.333"                  Foreground="BlueViolet"                  Text="{Binding ScreenName}" />       <TextBlock Grid.Row="1"                  HorizontalAlignment="Left"                  VerticalAlignment="Bottom"                  FontFamily="{StaticResource FontFamily}"                  FontSize="9.333"                  Foreground="#FF003D8F"                  Text="{Binding CreatedAt}" />       <Border Grid.RowSpan="2" BorderBrush="#FF999999" BorderThickness="0,0,0,1" />       <Border Grid.RowSpan="2" BorderBrush="White" BorderThickness="0,1,0,0" />   </Grid></DataTemplate>

After these changes, it looks like this:

Did you notice the gradient? You might think after seeing it here to adjust the gradients more so you can see it. That would be a mistake. See below.

To the right is the exact same thing, but stacked vertically three times. When this happens the subtle difference between the top and bottom of the control is more pronounced, so it looks like multiple panels are aligned together.

However, there’s still a little touch you can add. The white and gray borders are only 1 pixel high, but that’s the little touch needed to make it look crisp.

Finally, let’s see the before and after (or eh…rather after and before, because I took the screenshot backwards :P):

Building a Real-time Push App with Silverlight: Part 2

2011-08-27T14:16:00.001-07:00

Let’s review the main Rx code from last time:

public IObservable<string> GetJsonStreams(){  var request = GetRequest();  return Observable.FromAsyncPattern<WebResponse>(request.BeginGetResponse, request.EndGetResponse)()    .Select(wr => wr.GetResponseStream())    .Select(str => Observable.FromAsyncPattern<byte[], int, int, int>(str.BeginRead, str.EndRead))    .SelectMany(ParseJson);}

One thing I didn’t like about this was that the web request object was created regardless of whether the Observable gets a subscription or not. This is potentially wasted resources, and I wanted to refactor this to be completely lazy.

And with this I started to run into my first “huh?” moments with Rx: I blocked the UI thread. How did I do that? I started down the path of exploring some more of the Rx methods, which lead me to Create, which lets you manually call OnNext. With this train of thought, I came up with something like this:

return Observable.Create<string>(obs =>{  var request = GetRequest();  var response = Observable.FromAsyncPattern<WebResponse>(request.BeginGetResponse, request.EndGetResponse)().First();  var str = response.GetResponseStream();  var reader = Observable.FromAsyncPattern<byte[], int, int, int>(str.BeginRead, str.EndRead);  foreach (var json in ParseJson(reader))      obs.OnNext(json);   obs.OnCompleted();
  return str;});

Great! The initialization of the web request only occurs when subscribed! And it will even dispose the stream (by returning str) upon unsubscription. I ran the app and the UI thread immediately blocked. What happened?

Rx has the concept of subscription and observation, and provides a way to subscribe and observe on different threads. Here is the original code that subscribed:

s.GetJsonStreams()

   .ObserveOnDispatcher()

   .Subscribe(x => Text = x);

Can you spot the error? I explicitly told Rx to observe on the dispatcher thread, because I want the action inside Subscribe to be invoked on the UI thread, but I didn’t specify where I want to set up the subscription. Since I left it out, it uses the current thread, which happens to be the UI thread. To solve this, it’s as simple as doing this:

s.GetJsonStreams()

  .SubscribeOn(Scheduler.ThreadPool)

  .ObserveOnDispatcher()

  .Subscribe(x => Text = x);

That’s it! Easy! This also follows one of the most important guidelines when using Rx: Subscription and Observation should be done as late as possible, typically just before the Subscribe. Anything more and you’ll likely make Rx spawn more threads than are necessary or some other nasty bugs. KISS!

Now with that out of the way, let’s replace the boring TextBlock with something more usable. First, I need to parse all the JSON streams I’m getting into bindable models. To do that, I upgraded my StreamReader component and threw in System.Json for some basic parsing:

public class TweetParser{    private int _stack;    private readonly StringBuilder _sb = new StringBuilder();     public IEnumerable<Tweet> Parse(byte[] buffer, int count)    {        for (int i = 0; i < count; i++)        {            var current = (char)buffer[i];            _sb.Append(current);             if (current == '{') _stack++;            else if (current == '}') _stack--;             if (_stack == 0 && _sb.Length > 0)            {                Tweet tweet;                var value = JsonValue.Parse(_sb.ToString());                 if (value is JsonObject && Tweet.TryParse((JsonObject)value, out tweet))                    yield return tweet;                 _sb.Clear();            }        }    }}

Nothing overly complicated. Next, the Tweet object:

public class Tweet{    private readonly JsonObject _json;     public static bool TryParse(JsonObject value, out Tweet tweet)    {        if (value.ContainsKey("text") && value.ContainsKey("user"))        {            tweet = new Tweet(value);            return true;        }        tweet = null;        return false;    }     private Tweet(JsonObject json)    {        _json = json;    }     public string Text    {        get { return _json["text"].ToValueString(); }    }     public string ScreenName    {        get { return _json["user"]["screen_name"].ToValueString(); }    }} internal static class TweetEx{    public static string ToValueString(this JsonValue s)    {        return s.ToString().Trim('"');    }}

To keep things simple I’m only extracting the screen name and text. I won’t bore you setting up the views since it’s just simple ListBox bound to an ObservableCollection<Tweet>, and a DataTemplate for Tweet. When it’s all said and done, we see something like this:

Performance is still good at 2-5% CPU, even though we’re scrolling through 1000 items in near real-time.

Stay tuned for part 3, when we introduce Expression Blend and go into basics of UI design. Also, most of this will hit GitHub very soon.

Building a Real-time Push App with Silverlight: Part 1

2011-08-26T19:54:00.001-07:00

This is the beginning of a multi-part series where I’ll be building an interactive application with Silverlight 5 (still beta as of this post). It will be built from the ground up and designed predominantly from a “push data” point of view, where the application is reacting to events in real-time, rather than a more traditional “pulling” point of view. This type of application has exploded with the popularity of social networking and has made a lot of the traditional methods of building applications obsolete.

The best example of this shift is from Twitter. When it first came out, it was (and still is predominantly) a “pull” model. You have 200 API requests an hour, and you pull Twitter whenever you want to check if there are tweets of people you follow. That is changing with the Streaming API where data is “pushed” to you as it comes. This changes the way you write your code and requires a mind shift much like from for loops to LINQ.

The primary purpose of this series is build a real-time push application from beginning to end, and to show any problems I run along the way, and how I managed to solve them. This will be my first attempt at building a Silverlight application as well as learning Reactive Extensions (Rx), so I’m bound to run into newbie mistakes. If you have any tips or pointers please let me know!

Also, rather than doing this from a purely technical point of view, I’m also going to put on my designer hat and talk about UI design, and make it look good with Expression Blend when I get to designing the UI in later parts of this series. I feel that this is often overlooked and can result in a lot of wasted work and lead to frustration.

This first post will be to get up and running and connected to Twitter with the Streaming API. This was chosen primarily because it is so much data can be pushed through and it can be used to demonstrate how to write a high performance Silverlight application. It’s one thing to write something maintainable, and another altogether to make it run fast as well. I’ve worked on too many WPF projects where performance took a back seat, and in WPF particularly this tends to create some massive technical debt. The Silverlight/WPF technology stack is an exception to the rule and you should definitely think about performance from the start.

Well let’s get started! We will connect to Twitter, convert it to an Observable, and then display tweets on the UI.

First, let’s create our TwitterStream class:

   1: public class TwitterStream   2:     {   3:         static TwitterStream()   4:         {   5:             // this allows http authentication to work   6:             WebRequest.RegisterPrefix("http://", System.Net.Browser.WebRequestCreator.ClientHttp);   7:         }   8:     9:         protected readonly StreamParser parser = new StreamParser();  10:    11:         public string Username { get; set; }  12:         public string Password { get; set; }  13:    14:         protected HttpWebRequest GetRequest()  15:         {  16:             var request = WebRequest.CreateHttp("http://stream.twitter.com/1/statuses/sample.json?delimited=length");  17:             request.UseDefaultCredentials = false;  18:             request.Credentials = new NetworkCredential(Username, Password);  19:             return request;  20:         }  21:    22:         public IObservable<string> GetJsonStreams()  23:         {  24:             var request = GetRequest();  25:             return Observable.FromAsyncPattern<WebResponse>(request.BeginGetResponse, request.EndGetResponse)()  26:                 .Select(wr => wr.GetResponseStream())  27:                 .Select(str => Observable.FromAsyncPattern<byte[], int, int, int>(str.BeginRead, str.EndRead))  28:                 .SelectMany(ParseJson);  29:         }  30:    31:         private IEnumerable<string> ParseJson(Func<byte[], int, int, IObservable<int>> reader)  32:         {  33:             var buffer = new byte[256];  34:             int bytesRead;  35:    36:             while ((bytesRead = reader(buffer, 0, buffer.Length).Single()) > 0)  37:             {  38:                 foreach (var json in parser.Parse(buffer, bytesRead))  39:                     yield return json;  40:             }  41:         }  42:     }

And that’s all there is to it! The most complicated part is probably parsing of the JSON documents themselves, which I refactored into its own class. The algorithm in the parser is pretty primitive, as it just looks for opening { and closing } and calls that a document – nothing more, nothing less. Note that this uses the deprecated basic HTTP authentication. Sooner or later it will be shut down and OAuth will be required, so I will upgrade when that happens as I want to keep the amount of written code to a minimum.

Perhaps the more interesting bit is this part with Rx:

Observable.FromAsyncPattern<WebResponse>(request.BeginGetResponse, request.EndGetResponse)()  .Select(wr => wr.GetResponseStream())  .Select(str => Observable.FromAsyncPattern<byte[], int, int, int>(str.BeginRead, str.EndRead))  .SelectMany(ParseJson);

The API takes some getting used to, but once you “tune in” things start to make sense. The FromAsyncPattern essentially wraps the APM pattern’s Begin/End calls into a single Func<>, which when invoked gets you the result as if you called End(), however without all the tedious AsyncCallback implementation.

The nice thing about this wrapper Func is that the code starts to look synchronous, even though it is using the ThreadPool behind the scenes. Next, the response is projected into its response stream, which again is converted to another Observable for reading bytes from the stream until finally it goes into the ParseJson method which projects string results. This is both good and bad. Good in that it can make asynchronous code look short and succinct (in this example I’ve wrapped 2 APM pattern calls in 2 lines of code), bad in that anyone inexperienced with Rx will get lost pretty fast.

Rx thus far has been a fairly high learning curve. Maybe it’s just me. When I read about it from others, or from seminars, it all makes sense and I generally don’t have any questions. Trying to use it directly is another story, as there are so many overloads it’s easy to get overwhelmed. I feel like Intellisense is making things worse!

Carrying on, the code above actually doesn’t do anything yet. First, you must Subscribe to an observable before it does something, similar to you must foreach on a IEnumerable before it starts pulling data.

To finish off, let’s create a simple Window with a TextBlock bound to a Text property. This is the constructor:

public MainPage(){   InitializeComponent();   DataContext = this;    var s = new TwitterStream();   s.Username = "blingcoder";   s.Password = "1234567";   s.GetJsonStreams()       .ObserveOnDispatcher()       .Subscribe(x => Text = x);}

Perhaps one of the nicest features of Rx is automatic thread switching between background threads and UI threads. Above, there is a call to ObserveOnDispatcher, which as the name implies it observes to events on the Dispatcher thread :-). Rx automatically handles switching to the UI thread so when Subscribe occurs we’re on the UI thread.

Performance is also very good so far, only maxing out at 5% CPU for what appears to be a continuous flood of tweets from random people.

And there you have it! A completely asynchronous streaming Twitter client in roughly 20 lines of code (minus the parsing). Stay tuned for part 2….

Injecting Logger Instances Via Convention With Unity

2011-08-19T10:22:00.000-07:00

Let's say you have something like this:

public void MyClass {
  public ILog Logger { get; set; }
}

And you simply want Unity to inject an instance of ILog such that its name is requestingDependency.GetType().FullName, which in this case would be Full.Namespace.MyClass.

I'll leave it up to the exercise of the reader, but if you search for 'log4net and Unity' you will find some working solutions which IMO are way too long to solve this. I have obviously taken for granted the features of Windsor and Autofac too much because I expected something like this to be easily done in 2-3 lines of code.

After a lot of cursing I've come up with something that's almost as short and gets the job done:

    public class LoggingExtension : UnityContainerExtension
    {
        protected override void Initialize()
        {
            this.Context.Strategies.AddNew(UnityBuildStage.PreCreation);
        }
        private class LogBuilderStrategy : BuilderStrategy
        {
            private readonly Stack buildKeys = new Stack();
            public override void PreBuildUp(IBuilderContext context)
            {
                var type = ((NamedTypeBuildKey)context.BuildKey).Type;
                if (type == typeof(ILog))
                {
                    if (this.buildKeys.Count < 1)
                        throw new InvalidOperationException("Log instances cannot be resolved directly.");
                    context.Existing = LogManager.GetLog(this.buildKeys.Peek());
                }
                this.buildKeys.Push(type);
            }
            public override void PostBuildUp(IBuilderContext context)
            {
                this.buildKeys.Pop();
            }
        }
    }

If you noticed, I had to cast to NamedTypeBuildKey, which unfortunately means I'm still stuck with Unity 1.2...

Adding Scripting to your Tool Belt

2011-08-16T07:51:00.000-07:00

One of the most important things for any developer is to be well rounded. They should know about functional, imperative, object-oriented, aspect-oriented programming, among others. They should know the difference between dynamic and static languages. They should know the difference between scripting, interpreted, compiled languages.

Why? It opens your mindset. It lets you see things in a different light, which ultimately changes the way you will write your code, and for the better.

The best example is when C# introduced lambdas and LINQ. All of a sudden, the masses saw the benefits of functional programming style and how it greatly makes code succinct.

Here, I give an example of what knowing a little about Powershell did to save me a lot of time.

I use git-tfs to make my day job a little less painful every day, because I cannot stand TFS, at all. In their latest release, they decided not to tag every changeset. I like my things consistent, so I decided to go back in history to untag all the changesets.

Unfortunately, git doesn't have a mass untag, and you must untag each branch individually with "git tag -d name_of_tag".

This is how you do it with Powershell:

50000..40000 | % { git tag -d "tfs/default/C$_" }

50000..40000 creates an list of numbers from 50,000 to 40,000, then it gets piped (|) into a foreach loop (%) to run the git tag command by generating the tag name with the current loop item ($_).

Why was this awesome? I didn't have to open up Visual Studio, or a text editor, or anything. I just typed it into the shell.

Tools I Use To Do What I Do

2011-06-23T17:55:00.001-07:00

It’s been a while since I last blogged, but I want to get back into it….I just haven’t thought about what I wanna blog about yet. I’m thinking of doing some sort of multi-part series on a topic but haven’t decided on what yet. Nonetheless, just to get the ball rolling again here’s a quick blog post on some of the tools I use to do my every day job that’s in addition to your typical .NET developer tool chain.

General

StrokeIt: Funny name aside, on any clean install of Windows this is the first thing I install. I load up IE and I download this, followed by Chrome/Firefox. It is a mouse gesture program that affects all of Windows. By holding down the right mouse button, and making gestures you can do common tasks like minimize (drag SW), maximize (drag NE), close (draw a C), and others. It’s much faster than clicking the minimize/maximize buttons. For web browsing, creating and close tabs are drag SE and drag NW respectively. Back is W. Forward is E. Working this way is faster than any keyboard shortcut.

Console2: It’s not a necessity, but it’s nice to be able to use a console with a better looking font. You can easily customize to run another shell, and in my case I have Powershell as the shell.

VirtuaWin: Ironically I don’t use any virtual desktops, but this app creates a nice middle click on any window that lets you mark it as ‘always on top’.

Vim: It was an awful, painful 2 weeks to learn this thing but I’m sure glad I did.

Development/Visual Studio

Resharper: This one doesn’t need much mentioning as anyone serious about programming in .NET has either heard of it, use it, or use one of its competitors like CodeRush.

AllMargins: This hidden gem is an awesome extension has brings a bunch of very useful features to the editor. It brings structural highlighting, highlighting matching words under caret, and RockScroll inspired scroll bar, and it does it all with little to no cost in performance.

Productivity Power Tools: There’s only 2 that I use here: document tabs, and moving lines. All the rest are either eye candy, negligible enhancements, or just plain too slow to be any use in large solutions (solution navigator I’m looking at you!!).

XAML Intellisense Presenter: This is a must-have for anyone who works with XAML in VS regularly. Resharper 6 may soon make this obsolete, however…

ILSpy/dotPeek: Decompilers. ‘nuff said.

Snoop: Staple tool for WPF developers to inspect the visual/logical tree. There’s also WPF Inspector which has a much nicer UI, but unfortunately doesn’t perform as fast.

Profiling

ANTS/dotTrace/YourKit/etc: I’ve used many performance/memory profilers, and those 3 are the ones I like the most. Unfortunately, they all do things the others don’t, and all these things are very useful, so you still end up needing to use all of them to fix the particular problem you’re working on.

WinDbg: It isn’t every day I use this, but I’m sure glad to know that such a thing exists and the amount of power it gives me.

Scripting

PowerGUI: Best tool I found for editing PowerShell. The intellisense is great and has all the things you were expect when debugging through scripts.

Miscellaneous

BareTail: Amazing program which watches log files in real time and can highlight rows based on a search pattern.

PowerShell/Cygwin/MSYS: I used to be a Cygwin/MSYS user because back in the day I liked to install a different Linux distribution every week so I learned some basic Bash commands and that carried over to Windows. Since PowerShell now exists, we have a real shell in Windows, so I’ve switched over to learn it, since as a Windows developer it’s more applicable to the kind of work I do.

And there’s more but that’s enough for now!

Taking a Quick Look at .NET Decompilers

2011-05-20T19:57:00.001-07:00

Decompilers in the .NET space has changed a lot since the announcement from Red Gate that they were going to start charging for Reflector, which was the defacto most important/useful tool in any developer’s tool belt.

This blog post will be a quick overview to take a look at all the new contenders that have cropped up in the .NET space to see if any of them can dethrone Reflector as the decompiler of choice. This is only taking a look at the tools as a stand alone product and excludes any IDE integration.

To start, let’s take a look at Reflector.

Red Gate Reflector (7.1.0.143)

Having been around for years, Reflector is a rock solid product that sets the standard for everything else. It has a simple straight forward interface and has a large following with many 3rd party addins. It has a search which lets you find types, and an analyzer which shows you incoming and outgoing calls for any method. It also has the most options for decompiling, giving choices from C# to VB and even Delphi.

Another useful feature is being able change the framework from .NET to WPF to Silverlight.

As of this version it looks like they added tabs, but it for me it only used one at a time…perhaps a limitation of the trial version.

ILSpy (1.0.0.822)

This open source offering at this point looks the same as Reflector did just before Red Gate took over. It has all the necessities, including search and analyze. It can only view one thing at a time, and doesn’t have support for addins yet. However, progress on this project is blazing fast and it’s only a matter of time before it reaches feature parity with the rest.

JustDecompile Beta (2011.1.516.2)

First impression was, damn, this is a nice pretty WPF app. Onwards, it has a predefined assembly list of .NET2, 4, and Silverlight, which is nice. What’s nicer is that you can save a predefined list for use later, a feature that’s absent from the others. It has search by type, and search by symbol, however, the search is noticeably slower than everything else I tried. Also, the actual decompiling is much slower than the rest as well. Like ILSpy, it can only view one thing at a time.

Decompiling to C# and VB are supported, but no IL yet.

dotPeek EAP (1.0.0.1219)

I tried an EAP build and I was blown away. I may be biased because I’m a Resharper user, but having almost all of the keyboard shortcuts I have in my muscle memory work in dotPeek was a welcome surprise. The keyboard navigation makes dotPeek by far the best tool for navigating. While the others have searches, they only do so at the type level. With dotPeek (just like Resharper), you can search by type, current type, and even private members. Inheritance searching like finding implementations, etc., all of that works. Simply amazing. Oh, and it also has full tab support.

It was also fast as well, which was a surprise considering it’s searching through the entire framework. Searching all symbols for “m_” brought a list up in 2 seconds, and subsequent keystrokes narrowing it down occurred within half a second.

The downside? No support for decompiling to IL, and lambdas aren’t fully supported yet. I’m sure these will be done by the time it’s released.

Conclusion

I guess the most important thing I should be looking at is the actual decompiler, rather than the bells and whistles. If that’s what I looked at, this would be a pretty boring post because only Reflector is able to decompile everything out there.

JustDecompile failed to decompile some of the assemblies I threw it at it. dotPeek at this stage doesn’t have good lambda support (if at all). ILSpy decompiles very well at this point and I’d have to go out of my way to find an example where Reflector does it better than ILSpy.

In conclusion, I love dotPeek just because of the awesomeness that is Resharper keyboard shortcuts. For everything else, ILSpy does the job.

I’m actually pretty thankful that Red Gate decided to charge money for Reflector, because that inspired so much competition and we already have 3 viable replacements, all of which are free -- something that wouldn’t have happened if Reflector remained free.

Simple App to Preview Fonts

2011-05-08T08:51:00.001-07:00

I spent a good few hours yesterday trying out a new programming font. It’s a personal thing. I like switch fonts every couple weeks, and while Consolas is an excellent font and I have nothing against it, to me, it’s….too simple. For most people, the defaults are just fine, and they can’t be bothered to try another font. But for me, I want my font to speak to me a little. But enough of that voodoo magic stuff.

The problem with sites that survey fonts is that it’s hard to compare quickly one with the next. For example, this excellent article comparing 42 fonts on CodeProject is one of the most comprehensive surveys, but even with that I can’t really tell if I prefer one font over another. So, since I’m going to be doing this from time to time, I wrote a quick WPF app.

The XAML is simple, only this:

<StackPanel Orientation="Horizontal">
  <ListBox IsSynchronizedWithCurrentItem="True" 
           SelectedItem="{Binding CurrentFont}"
           ItemsSource="{Binding Fonts}" />
  <TextBlock FontFamily='{Binding CurrentFont}' FontSize="14" Text="{Binding Code}" />
</StackPanel>

And the code is just as simple, only this:

public class ViewModel
{
  public static readonly DependencyProperty CurrentFontProperty = DependencyProperty.Register("CurrentFont", typeof(FontFamily), typeof(ViewModel));
  public FontFamily CurrentFont
  {
    get { return (FontFamily)GetValue(CurrentFontProperty); }
    set { SetValue(CurrentFontProperty, value); }
  }
  public List<FontFamily> Fonts { get; private set; }
  public string Code { get { return "insert what you want to see"; } }
  public ViewModel()
  {
    Fonts = new List<FontFamily>();
    Fonts.AddRange(System.Windows.Media.Fonts.SystemFontFamilies);
  }
}

And with this, you get something like this. The coding sample is taken from Coding Horror.

Yep, my current chosen font is BPMono.

For the lazy, here’s the project on GitHub.

Integrating DependencyPropertyWeaver Into Your Build

2011-04-17T13:22:00.001-07:00

See the introduction here.

The MSBuild task as it stands currently by default will attempt to weave everything that it finds. This is probably not optimal in most use cases, but any changes is a simple modification to the LINQ query in the code. For now, it’s only available in source format since there are bound to be bugs here and there, and once those get ironed out it’ll be easier to release a “point something” release.

An example of how its used can be seen in the unit tests, in the project DependencyPropertyWeaver.Tests.Models. If you open it up in your favorite text editor, you will see this crucial line:

<Target Name="AfterBuild">
    <Exec Command="C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe postbuild.proj /p:Files=DependencyPropertyWeaver.Tests.Models.dll" WorkingDirectory="$(OutputPath)" />
  </Target>

And the contents of postbuild.proj is simply this:

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Weave" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <UsingTask TaskName="DependencyPropertyWeaver.DependencyPropertyWeaverTask"
             AssemblyFile="DependencyPropertyWeaver.dll" />
  <Target Name="Weave">
    <ItemGroup>
      <Input Include="$(Files)" />
    </ItemGroup>
    <DependencyPropertyWeaverTask Files="@(Input)" />
  </Target>
</Project>

The AfterBuild target is a special target with MSBuild, which, as the name implies, will run after the target has been built. In this case, after the project is completed, it will invoke another MSBuild instance to perform the weaving. Invoking a new MSBuild process is required because the <UsingTask> will load the assembly into the current AppDomain, and once loaded it cannot be unloaded.

If you don’t mind the unloading part, you can simply add the <UsingTask> directly into the project file and invoked the <DependencyPropertyWeaverTask> there, and avoid the hassle of having a separate file.

Currently, the DependencyPropertyWeaverTask supports the TypePatternMatch and AttributePatternMatch, which as the names imply will use regular expression filtering on type names or properties with the attribute you want to weave. If these properties are null, they are ignored.

Next on the feature list is weaving attached properties ;-)

Introducing DependencyPropertyWeaver

2011-04-15T19:09:00.001-07:00

As part of my demonstration for the NYC .NET Meetup in the next couple of days, I prepared some material to show how to use Mono.Cecil to manually weave IL as a post-build step. It solves the problem of what I call “WPF Verbosity Hell”: the thing that makes your eyes bleed and makes God kill a kitten every time you declare a new property.

Since notifypropertyweaver already exists, I figured I’d release the other side of the coin, dependencypropertyweaver. I just pushed my changes onto GitHub.

There are a couple of reasons to use dependency properties rather than regular properties with INotifyPropertyChanged. If you’re using WPF they make more sense because they perform faster and they give you a lot of extra bells and whistles for free. However, the INPC route tends to be more popular because it doesn’t bring in the dependency of WPF.

IL post-build weaving solves the dependency problem because you can selectively choose to weave only the assemblies that get deployed with WPF, and you could even weave INPC into your service layer, and DPs into your client layer. That’s the power of post-build processes.

While this little project initially started out as a simple demonstration of using Mono.Cecil to perform low-level aspect oriented principles, it didn’t take much more work to tidy things up and make it ‘releasable’ into the wild, so I did just that, and open sourced it.

The concept is simple. You take something like this:

    public class Student
    {
        public string Name { get; set; }
        public int Age { get; set; }
    }

And DependencyPropertyWeaver will turn it into this:

         public class Student : DependencyObject
	{
		public static readonly DependencyProperty NameDependencyProperty = DependencyProperty.Register("Name", typeof(string), typeof(Student));
		public static readonly DependencyProperty AgeDependencyProperty = DependencyProperty.Register("Age", typeof(int), typeof(Student));
		public string Name
		{
			[CompilerGenerated]
			get
			{
				return (string)base.GetValue(Student.NameDependencyProperty);
			}
			[CompilerGenerated]
			set
			{
				base.SetValue(Student.NameDependencyProperty, value);
			}
		}
		public int Age
		{
			[CompilerGenerated]
			get
			{
				return (int)base.GetValue(Student.AgeDependencyProperty);
			}
			[CompilerGenerated]
			set
			{
				base.SetValue(Student.AgeDependencyProperty, value);
			}
		}
	}

Even though the class initially didn’t inherit from DependencyObject, the weaved version does. It will also continually go up the inheritance chain until it finds the abstract base class, and makes that inherit from DependencyObject.

Next on the list is obviously documentation (what open source project doesn’t need more documentation?!), however, I think I can slack off on this one since the core library is only 200-300 lines of code :-)

That Commenting Thing

2011-03-10T19:38:00.001-08:00

I’m working on a project right now that has near 99% of its code commented. Wow, that’s impressive! Actually, it’s more scary…even some of the private fields are commented….

At one point, someone decided that it was a good idea to impose some kind of standard rather than none, so they recommended StyleCop with all of its default settings, including requiring comments on everything.

This seemed like a good idea, and it probably was, until someone found GhostDoc. The plugin definitely gives off that “cool” factor the first time you use it. Wow, with a single key stroke it automagically documented all of my code!!! The problem, and it’s a pretty big one, is that it generates documentation based on the name of the properties, names, and class. Surely if it can do that, you, a much more sophisticated machine, can do that!

Alas, the code base ended up getting GhostDocced to death all over the place to the point where I was driven mad and insane that I spent the time to write a VS2010 extension to purposely hide these comments. I’m talking about comments like this:

The comments add absolutely no value and IMO makes it harder to read, and has the added deficit that if for whatever reason someone changes the name property, chances are they will forget to update the comments. It gets worse with WCF service contracts with many parameters.

So yes, it frustrated me enough that I took some spare time to write an extension to shrink those damn XML comments to such a small size it doesn’t bother me anymore, like this:

You can grab the code off GitHub.

Auto Mocking NSubstitute with Castle Windsor

2010-12-11T15:59:00.001-08:00

I was debating whether to make this blog post because it’s so damn simple to implement, but hey, if it saves someone else time, I did some good.

First of all, register an ILazyComponentLoader into Windsor:

var c = new WindsorContainer();
c.Register(Component.For<LazyComponentAutoMocker>());

Then, the implementation of LazyComponentAutoMocker is simply this:

public class LazyComponentAutoMocker : ILazyComponentLoader
{
  public IRegistration Load(string key, Type service, IDictionary arguments)
  {
    return Component.For(service).Instance(Substitute.For(new[] { service }, null));
  }
}

And you’re done! Here’s a simple unit test example using only the code from above:

[Test]
public void IDictionary_Add_Invoked()
{
  var dict = c.Resolve<IDictionary>();
  dict.Add(1, 1);
  dict.Received().Add(1, 1);
}

That was almost too easy.

Working in Git to Working in Mercurial

2010-12-05T14:17:00.001-08:00

I took the dive a couple weeks ago and learned how to use Git and fell in love with its simplicity. I don’t know what it is, but after using Git every day it actually started to make sense that git checkout is used for so many things, which is ironic because prior to using Git, every introductory tutorial I’ve read has always had me thinking, “checkout does what and what and what now??”.

So why did I switch to Mercurial? I need to push/pull from a Subversion repository, and I work on Windows. General day to day work is great, but when I needed to git svn rebase or git svn dcommit it would take so long that I simply left to get coffee. What’s worse, no matter what I set for core.autocrlf I would always get some weird whitespace merging error, when nothing was wrong. It became a regular part of my workflow to just git rebase –skip everything because that’s what fixed things. Scary.

The crappy Subversion and whitespace support (at least in Windows) led me to try Mercurial. After getting accustomed to Mercurial, they’re actually a lot more similar than they are different.

First things first, you need to add the following to your mercurial.ini file in your home directory:

[bookmarks]
track.current = True

Here’s a comparison of my typical Git workflow translated to Mercurial:

	Git	Mercurial
Work on a new feature/bug	git checkout –b new_feature	hg book new_feature
Hack hack hack	git add .
Commit	git commit –m “done!”	hg commit –m “done!”
Hack more and commit	git commit –a –m “more hacking”	hg commit –m “more hacking”
Sync with up stream	git pull master	hg pull
Merge	git checkout master	hg merge new_feature
	git merge new_feature	hg commit –m “merged”
Push	git push	hg push

Notice that they’re practically exactly the same? There are some minor differences. The obvious one being that you need to add into git’s index before committing. And the other I didn’t have to switch branches in Mercurial (more on that later). But aside from that, it’s pretty much the same, from a user’s point of view. Check this fantastic thread on StackOverflow for one of the best comparisons on the net if you want to dive into technical details.

So far, there are only two things that bother me having switched:
a) No fast-forward merge. You need to manually rebase and then commit.
b) No automatic commit after a conflict free merge.

These are fairly minor annoyances, but can be scripted/aliased away. Or if you don’t like those defaults in Git, you can override them in the command arguments or set them in gitconfig. Mercurial similarly can change default arguments in its hgrc/mercurial.ini.

One of the biggest advantages of Git is easily switching between all its local lightweight branches at lightning speed. Tangled source code is no more! I was confused with Mercurial’s bookmarks when I first used them because all it does is label your head with something. I don’t know why this is, but it is the same as git checkout –b, but for some reason I visualized Git “branching” the latest commit into two paths. But how do you do that if they’re the same? It should only branch after a commit which introduces changes, which is what Mercurial does. In this scenario, Mercurial is more explicit, whereas Git is implicit.

Using bookmarks, you can mimic Git’s workflow like this:

	Git	Mercurial
Create a bunch of ‘working areas’	git branch feature1	hg book feature1
	git branch feature2	hg book feature2
Switch/commit/hack/commit	git checkout feature1	hg up feature1
	git commit –a –m “feature1”	hg commit –m “feature1”
	git checkout feature2	hg up feature2
	git commit –a –m “feature1”	hg commit –m “feature2”
Sync up stream	git pull master	hg pull
	git checkout master	hg up default
Merge in 1 feature	git merge feature1	hg merge feature1
		hg commit –m “merged”
Delete branch/bookmark	git branch –d feature1	hg book –d feature1
Push	git push	hg push –r tip
Switch back and hack again	git checkout feature2	hg up feature2

The –r tip switch on hg push might have raised an eyebrow. This is to tell Mercurial to push all changes that lead to the tip. This will include the changes in feature1 that we just merged in, but exclude all the ones in feature2. If you issue a hg push it will complain and warn you that you’re going to create multiple heads on the remote repository. This is not what you want since there may be unfinished features, experimental branches, etc. Of course, you can force it anyways, but that’s not a good idea.

At first, the tip was most confusing to me because I tried to associate it with Git’s master, which was simply not the case. Tip refers to the newest change that you know about. This can be on any branch or bookmark. Once I understood that, and stopped trying to create a Git master equivalent, everything was straightforward.

So let’s start with an example. First I create a bookmark feature1 after syncing, and then making a commit looks like this:

And then if you switch to feature2 and make a commit, it becomes like this:

Here’s where you start thinking, “I don’t have a master which tracks upstream changes, how do I separate my local changes?” And now here’s a situation where Mercurial does more black magic than Git. If there are up stream changes, after issuing hg pull, this happens:

Mercurial automatically split my local changes into their own separate branches (actually, heads is the accurate term). After this operation, tip is now tracking the changes that I just pulled in from upstream, instead of my bookmark.

From here, it’s simply hg up tip to switch to the “master” branch. Then a hg merge feature1; hg commit, and it becomes like this:

Then it’s simply hg push –r tip, and you’re good to go. Basically, if you bookmark every feature/bug you work on, then you should only have one branch that doesn’t have a bookmark, which is the same as the ‘master’ branch from Git.

What about Subversion?

Whoops, forgot why I switched in the first place. First, install hgsubversion, after that’s set up, simply:

hg clone http://path/to/subversion/repository

And then it’s just hg pull or hg push. Is it really that simple? Yes, yes it is.

CQRS: Building a “Transactional” Event Store with MongoDB

2010-12-04T14:46:00.001-08:00

As you all already know if you’re familiar with MongoDB, is that it does not support transactions. The closest thing we have is atomic modifications of a single document.

The Event Store in a CQRS architecture has the important responsibility of detecting concurrency violations, where two different sources try to update the same version of the aggregate. The one that gets it late should be denied changes into the store with an exception thrown. This ensures the integrity of the data.

Here is a very simple typical implementation of appending events into the event store:

public void Append(Guid id, long expectedVersion, IEnumerable<IEvent> events)
{
  try
  {
    _events.Insert(events.Select(x => ...)); // convert to storage type
  }
  catch (...)
  {
    if (E11000 duplicate key)
      throw new ConcurrencyException(...);
  }
}

Syntax is a mix of C#/pseudo code, but the basic concepts are the same. This assumes that you’ve set up an multi-index on the collection between the ID and the version. Thus, when you insert something that already has a matching ID/version, Mongo will tell you of a duplicate key violation, and all is good.

But wait! Operations are atomic per document! So what happens if you append 100 events, and it fails on the 43rd one? Events 1 through 42 will continue to exist in the data store, which is bad news.

Obviously, this solution is not going to work. The next step was to do something like this:

catch (...)
{
  if (E11000 duplicate keys)
  {
    foreach (var e in events)
      _events.Delete(new { _id = e._id });
 
    throw new ConcurrencyException(...);
  }
}

So, before inserting into the collection, each events gets a generated ObjectID, so that if it fails, the catch exception can simply tell the data store to delete everything.

At first glance this seems to fix everything, except for one glaring problem. What happens if you lose connection to the database before, or midway sending the deletes? Now you have a problem of ensuring that those deletes are guaranteed, and so then the question that arises from that is where would you store it? A local file? Another database? The problem is, at that moment, if another process in the system queries all events for the same aggregate it will return invalid data.

So, we’re back to square one. We need to simulate a transaction through a single insert.

The secret is in the schema design. Initially, we started out with a straight forward row-per-event schema. But since we’re operating with documents, we can model it as a batch of events.

Thus, instead of versioning every event individually, we version a batch of events. For example, originally we would insert 3 events, and the data saved would look like this:

{ _id = 1, aggregate_id = 1, version = 1, event = { … } }
{ _id = 2, aggregate_id = 1, version = 2, event = { … } }
{ _id = 3, aggregate_id = 1, version = 3, event = { … } }

In the new schema, it would look like this:

{ _id = 1, aggregate_id = 1, version = 1, events = [ { … }, { … }, { … }, { … } ] }

Now, a downside to this approach is you lose a bit of granularity of stored events, since you are grouping multiple events under a single version. However, I don’t see this as a huge loss since the main reason you want to use event sourcing in the first place is to be able to restore an aggregate to any state in its history, and we still retain that functionality.

In our case, this is working very well for us. When a command gets handled, it generates a bunch of events that get applied and then saved to MongoDB. I can’t think of any scenario where it’d want to replay to the middle of a half-processed command (but of course it’s possible anyways, just reply half of a batch of events). But that’s just asking for trouble. It’s most likely easier to just the re-process the command.

Now, you may be asking why go through the trouble of batching events when you can just store one document per aggregate, and then put all events in one document? Yes, that would solve the problem very effectively…until you hit the 4MB per document limit ;-)

CQRS: Auto register event handlers

2010-11-23T20:46:00.000-08:00

I’m not going to go into detail about what the deal is about event handlers in a CQRS architecture, since a quick Google/Bing search will give plenty of very good information. What this post is about is a solution to the “how do I quickly register something to handle a bunch of events” without copying pasting all over the place. There are other solutions out there, like this one. Here’s something I came up with (took some concepts from my post on Weak Events).

public class Aggregate
{
  private delegate void OpenEventHandler<in TTarget, in TEvt>(TTarget target, TEvt @event);
  private static readonly IDictionary<Type, OpenEventHandler<Game, IEvent>> _evtHandlers = new Dictionary<Type, OpenEventHandler<Game, IEvent>>();
   
  static Aggregate()
  {
    var methods = from m in typeof(Game).GetMethods(BindingFlags.NonPublic | BindingFlags.Instance)
    let p = m.GetParameters()
            where m.Name == "ApplyEvent" && p.Length == 1 && typeof(IEvent).IsAssignableFrom(p[0].ParameterType)
            select m;
    
    var registerForwarder = typeof(Game).GetMethod("RegisterForwarder", BindingFlags.NonPublic | BindingFlags.Static);
    foreach (var m in methods)
    {
      Type eventType = m.GetParameters()[0].ParameterType;
      var forwarder = registerForwarder.MakeGenericMethod(eventType).Invoke(null, new[] { m });
      _evtHandlers[eventType] = (OpenEventHandler<Game, IEvent>)forwarder;
    }
  }
  
  private static OpenEventHandler<Game, IEvent> RegisterForwarder<TEvt>(MethodInfo method)
  {
    var invoker = typeof(OpenEventHandler<,>).MakeGenericType(typeof(Game), typeof(TEvt));
    var forwarder = (OpenEventHandler<Game, TEvt>)Delegate.CreateDelegate(invoker, null, method);
    return (g, e) => forwarder(g, (TEvt)e);
  }
 
  private void ApplyEvent(EventHappened e)
  {
    _something = e.Something;
  }
 
  public void ApplyChanges(IEnumerable<IEvent> events)
  {
    foreach (var e in events)
    {
      _evtHandlers[e.GetType()](this, e);
    }
  }
}

A couple things:

The registration happens in the static constructor. This is important, because this relatively heavy cost of using reflection only happens once for the aggregate.
The filtering of methods is arbitrary. I chose “ApplyEvent” here as the convention, but of course you can choose whatever you like.
ApplyChanges simply invokes the event handlers dictionary directly. Assuming you’re being a good citizen with the code, accessing _evtHandlers doesn’t need a lock because once created it should never be modified.

So in summary, it finds all methods named ApplyEvent in the current class, and generates an “open delegate” which takes in an extra parameter which is the instance itself. In this case, the instance is the aggregate, as shown in the ApplyChanges method.

So there you have it! Excluding the lengthy LINQ query, roughly 10 lines of code to find and register all event handlers in the aggregate. And if you’re wondering, the performance cost is negligible because there’s no reflection involved in the invocation of the handlers. Awesome!

That Immutable Thing

2010-11-07T21:43:00.001-08:00

Do you have some sort of ImmutableAttribute in your domain that you use to mark classes as immutable? Have you ever needed to enforce that contract? Checking for readonly fields isn’t enough? Well, this weekend I had a code spike that helped solve this problem in my current project.

For this project, I’m using the NoRM driver for MongoDB, and one of the limitations of the serializer is that all types must be classes, must have a default constructor, and all properties have a public setter. So, now the domain has a bunch of classes like this:

public class UserCreatedEvent : IEvent
{
  public string Name { get; set; }
  public UserCreatedEvent() { }
  public UserCreatedEvent(string name) { Name = name; }
}

That God for code snippets (or Resharper templates). With so many classes like this that need to get serialized, I wanted to extra sure that no code ever calls the setter method for the Name property. Thankfully, with some help of Mono.Cecil, it’s possible.

First off, you need to define ImmutableAttribute and that add that do classes, and in my case, it is historical domain events that get serialized to an event store.

Then, you just write a unit test which leverages the power of Mono.Cecil. It turned out to be pretty simple. Here’s the code:

using System.Collections.Generic;
using System.Linq;
using Mono.Cecil;
using Mono.Cecil.Cil;
using NUnit.Framework;
namespace blingcode
{
    [TestFixture]
    public class ImmutabilityTests
    {
        private static readonly MethodDefinition[] _setterMethods;
        private static readonly AssemblyDefinition[] _assemblies;
        static ImmutabilityTests()
        {
            _assemblies = new[]
            {
                AssemblyDefinition.ReadAssembly(typeof(Something).Assembly.Location),
            };
            _setterMethods = _assemblies
                .SelectMany(a => a.Modules)
                .SelectMany(m => m.Types)
                .Where(t => t.CustomAttributes.Any(attr => attr.AttributeType.Name.Contains("ImmutableAttribute")))
                .SelectMany(t => t.Properties)
                .Where(p => p.SetMethod != null)
                .Select(m => m.SetMethod)
                .ToArray();
        }
        [Test]
        public void ClassesWith_ImmutableAttribute_ShouldNotUse_PropertySetters()
        {
            AssertForViolations(_assemblies
                                    .SelectMany(a => a.Modules)
                                    .SelectMany(m => m.Types)
                                    .Where(t => t.IsClass)
                                    .SelectMany(t => t.Methods));
        }
        [Test]
        public void ThisFixtureActuallyWorks()
        {
            var assembly = AssemblyDefinition.ReadAssembly(typeof(ImmutabilityTests).Assembly.Location);
            var type = assembly.Modules.SelectMany(m => m.Types)
                .Where(t => t.IsClass && t.FullName.Contains(GetType().FullName)).First();
            try
            {
                AssertForViolations(type.Methods);
            }
            catch (AssertionException)
            {
                Assert.Pass();
            }
        }
        private static void AssertForViolations(IEnumerable<MethodDefinition> potentialMethods)
        {
            foreach (var method in potentialMethods.Where(m => m.HasBody))
            {
                foreach (Instruction ins in method.Body.Instructions.Where(ins => ins.OpCode == OpCodes.Callvirt))
                {
                    MemberReference mr = ins.Operand as MemberReference;
                    if (mr != null)
                    {
                        var result = _setterMethods.FirstOrDefault(m => m.FullName == mr.FullName);
                        if (result != null)
                        {
                            throw new AssertionException(result + " was invoked by " + method + ", even though the type has the Immutable attribute.");
                        }
                    }
                }
            }
        }
        private void InvokeCardSetters()
        {
            // this only exists to test that the test does indeed work
            var c = new SomeImmutableClass();
            c.SomeImmutableValue = 123;
        }
    }
}

Nothing too complicated. The main thing to look for is the callvirt method, which the C# compiler always generates for classes. Then, you match the operand to the method definition and viola!

Memory Leak with WPF’s RichTextBox

2010-10-05T07:53:00.001-07:00

Apparently, setting IsUndoEnabled to false isn’t enough. You must also set UndoLimit=0 as well otherwise it’ll still keep track of undo history. Doh!

Yet Another Weak Event Implementation

2010-10-04T00:06:00.001-07:00

I gotta say that WPF is a complete pain in the butt when it comes to memory leaks. A simple google/bing search shows up more than enough results for weak events, so why am I making yet another blog post about another implementation of a weak event? Well, if someone else out there happens to have the same requirements/needs that I do right now, maybe this will save them a little time.

In my particular case, I needed to accomplish a couple goals:

Minimal performance hit. a.k.a minimal use of reflection.
Generic and easy to use.
Thread safe.
Support for explicit registration and unregistration.

One of the better implementations on weak events I found was from Dustin Campbell’s blog post. Unfortunately, this implementation has one major problem: you cannot explicitly unregister. In fact, the only way an attached listener no longer receives messages is if it gets garbage collected. Our application happened to use this for an especially special event on our base entity: the infamous INotifyPropertyChanged event. Needless to say, all the static WPF dependency objects left a whole bunch of leaked WeakReferences around (ironic?).

But wait, doesn’t WPF already have a IWeakEventManager that solves this problem? The problem with this is a couple things. It’s slow. It’s painful to use. It’s annoying to implement. Long story short, you need a static WeakEventManager for every unique delegate. Yes, sure, you can override a lot of the methods to make it more perform faster, but if you need to go that far you might as well write your own that does just what you need, and no more.

If you haven’t found it already, Daniel Grunwald’s Code Project article is yet another great resource. My implementation was a mix of this and the earlier mentioned link. Anywho, here it is:

    public interface IWeakEventEntry<TSender, TArgs>
    {
        bool IsAlive { get; }
        bool Matches(Delegate handler);
        bool Invoke(SynchronizationPriority priority, TSender sender, TArgs args);
    }
    public class WeakEventEntry<TTarget, TSource, TArgs> : IWeakEventEntry<TSource, TArgs> where TTarget : class
    {
        private delegate void OpenForwardingEventHandler(TTarget target, TSource sender, TArgs args);
        private readonly WeakReference m_TargetRef;
        private readonly OpenForwardingEventHandler m_OpenHandler;
        private readonly MethodInfo m_Method;
        public WeakEventEntry(Delegate handler)
        {
            m_OpenHandler = (OpenForwardingEventHandler)Delegate.CreateDelegate(typeof(OpenForwardingEventHandler), null, handler.Method);
            m_TargetRef = new WeakReference(handler.Target);
            m_Method = handler.Method;
        }
        public bool IsAlive { get { return m_TargetRef.IsAlive; } }
        public bool Matches(Delegate handler)
        {
            return handler.Method == m_Method && handler.Target == m_TargetRef.Target;
        }
        public bool Invoke(TSource sender, TArgs args)
        {
            TTarget target = m_TargetRef.Target as TTarget;
            if (target != null)
            {
                m_OpenHandler(target, sender, args);
                return true;
            }
            return false;
        }
    }
    public class WeakEvent<TSender, TArgs>
    {
        private static readonly IWeakEventEntry<TSender, TArgs>[] EMPTY_LIST = new IWeakEventEntry<TSender, TArgs>[0];
        private List<IWeakEventEntry<TSender, TArgs>> m_Events;
        private IWeakEventEntry<TSender, TArgs>[] m_InvokeList = EMPTY_LIST;
        public int Count { get { return m_InvokeList.Length; } }
        public void Add(Delegate handler)
        {
            if (handler.Target == null) // static method
            {
                Add(new StrongEventEntry<TSender, TArgs>(handler));
            }
            else
            {
                Type type = typeof(WeakEventEntry<,,>).MakeGenericType(handler.Target.GetType(), typeof(TSender), typeof(TArgs));
                Add((IWeakEventEntry<TSender, TArgs>)type.GetConstructors()[0].Invoke(new object[] { handler }));
            }
        }
        public void Add(IWeakEventEntry<TSender, TArgs> entry)
        {
            lock (this)
            {
                if (m_Events == null)
                    m_Events = new List<IWeakEventEntry<TSender, TArgs>>(8);
                m_Events.Add(entry);
                m_InvokeList = m_Events.ToArray();
            }
        }
        public void Remove(Delegate handler)
        {
            lock (this)
            {
                if (m_Events != null)
                {
                    for (int i = 0; i < m_Events.Count; i++)
                    {
                        if (m_Events[i].Matches(handler))
                        {
                            m_Events.RemoveAt(i);
                            break;
                        }
                    }
                    m_InvokeList = m_Events.ToArray();
                }
            }
        }
        public void Clear()
        {
            lock (this)
            {
                m_Events = null;
                m_InvokeList = EMPTY_LIST;
            }
        }
        public void Raise(TSender sender, TArgs args)
        {
            IWeakEventEntry<TSender, TArgs>[] events = m_InvokeList;
            bool removeDead = false;
            for (int i = events.Length - 1; i >= 0; i--)
                removeDead |= !events[i].Invoke(sender, args);
            if (removeDead)
                RemoveDeadReferences();
        }
        private void RemoveDeadReferences()
        {
            lock (this)
            {
                if (m_Events != null)
                {
                    for (int i = m_Events.Count - 1; i >= 0; i--)
                    {
                        if (!m_Events[i].IsAlive)
                            m_Events.RemoveAt(i);
                    }
                    m_InvokeList = m_Events.ToArray();
                }
            }
        }
    }

Nothing too special here. For performance, a copy of all events are stored in an array so that raising events doesn’t need to be in a lock. And while locking on “this” is usually bad practice, in this case I didn’t have any other local variable to use, and creating an object just for the sake of locking in my eyes was not worth it in this scenario.

I tried generating the OpenForwardingDelegate using DynamicMethod and IL, but the result was negligible because it doesn’t actually affect invoking speed, which is what we’re most concerned with.

Also, if you didn’t notice, this implementation also supports static methods attaching, hence why there is an IWeakEventEntry interface. Here’s the StrongEventEntry implementation:

    public class StrongEventEntry<TSource, TArgs> : IWeakEventEntry<TSource, TArgs>
    {
        private delegate void ClosedForwardingEventHandler(TSource sender, TArgs args);
        private readonly ClosedForwardingEventHandler m_ClosedHandler;
        private readonly MethodInfo m_Method;
        public bool IsAlive { get { return true; } }
        public StrongEventEntry(Delegate handler)
        {
            m_Method = handler.Method;
            m_ClosedHandler = (ClosedForwardingEventHandler)Delegate.CreateDelegate(typeof(ClosedForwardingEventHandler), null, handler.Method);
        }
        public bool Matches(Delegate handler)
        {
            return m_Method == handler.Method;
        }
        public bool Invoke(SynchronizationPriority priority, TSource sender, TArgs args)
        {
            m_ClosedHandler(sender, args);
            return true;
        }
    }

Last but not least, usage:

        private WeakEvent<object, PropertyChangedEventArgs> _propertyChanged = new WeakEvent<object, PropertyChangedEventArgs>();
        public event PropertyChangedEventHandler PropertyChanged
        {
            add { _propertyChanged.Add(value); }
            remove { _propertyChanged.Remove(value); }
        }

Performance is quite good. On my machine for 1,000,000,000 invocations, it takes the WeakEvent ~17.3 seconds, vs 4.2 seconds for a standard delegate, for roughly 4x invocation cost.

And there you have it! A simple, generic, and fast weak event in ~200 lines of code.

Contextual Lifestyle with Castle Windsor

2010-05-13T22:47:00.001-07:00

EDIT: As of version 3, scoped lifestyles are now a first class citizen supported out of the box (http://docs.castleproject.org/Windsor.Whats-New-In-Windsor-3.ashx)
EDIT: A much better implementation can be found at https://github.com/castleprojectcontrib/Castle.Windsor.Lifestyles

IMO, one of the big missing features of Castle Windsor is that it doesn’t come with a built-in way for dealing with contextual lifestyles. It handles transients and singletons fairly well, but once you get to other lifestyles it’s pretty heavily dependent on having some “state manager” handling the instances. For example, PerWebRequest uses the HttpContext, PerThread uses thread static variables, etc.
Contextual lifestyles is one of those things where it doesn’t seem all that useful at first, and then when you see the possibilities it’s like getting hit with a huge truck.
A question was posted to the Castle Google Group recently, which I follow, which illustrates a relatively common example of why someone would want to have a contextual lifestyle. Basically, you have a whole bunch of components you want to resolve, but only within a context.
Here’s some boiler plate code of the domain model:

public interface IRepository { ISession Session { get; } }
public interface ISession : IDisposable { bool IsDisposed { get; } }
public class Session : ISession
{
    public bool IsDisposed { get; set; }
    public void Dispose() { IsDisposed = true; }
}
public class Repository1 :IRepository
{
    public ISession Session { get; private set; }
    public Repository1(ISession session){ Session = session; }
}
public class Repository2 : IRepository
{
    public ISession Session { get; private set; }
    public Repository2(ISession session){ Session = session; }
}
public class Model1
{
    public IRepository First { get; private set; }
    public IRepository Second { get; private set; }
    public Model1(IRepository first, IRepository second) { First = first; Second = second; }
}
public class Model2
{
    public IRepository Second { get; private set; }
    public Model2(IRepository second) { Second = second; }
}

And here’s the unit test I want to pass:

[Test]
        public void ResolutionsByContext()
        {
            IWindsorContainer root = new WindsorContainer();
            root.Register(Component.For<Model1>().LifeStyle.Transient,
                          Component.For<Model2>().LifeStyle.Transient,
                          Component.For<IRepository>().ImplementedBy<Repository1>().LifeStyle.Transient,
                          Component.For<IRepository>().ImplementedBy<Repository2>().LifeStyle.Transient,
                          Component.For<ISession>().ImplementedBy<Session>().LifeStyle.PerContextScope());

            Model1 model1;
            Model2 model2;
            ISession session1, session2;
            using (var context1 = root.BeginLifetimeScope())
            {
                model1 = context1.Resolve<Model1>();
                session1 = model1.First.Session;
                Assert.AreSame(model1.First.Session, model1.Second.Session);
                Assert.AreSame(context1.Resolve<ISession>(), context1.Resolve<ISession>());

                using (var context2 = root.BeginLifetimeScope())
                {
                    model2 = context2.Resolve<Model2>();
                    session2 = model2.Second.Session;
                    Assert.AreNotSame(model1.First.Session, model2.Second.Session);

                    var anotherModel2 = context2.Resolve<Model2>();
                    Assert.AreSame(anotherModel2.Second.Session, model2.Second.Session);

                    Assert.AreSame(session2, context2.Resolve<ISession>());
                    Assert.AreNotSame(context1.Resolve<ISession>(), context2.Resolve<ISession>());
                }
                Assert.IsTrue(session2.IsDisposed);
                Assert.IsFalse(session1.IsDisposed);
            }
            Assert.IsTrue(session1.IsDisposed);
        }

I copied the name BeginLifetimeScope from Autofac, which inherently supports contextual scopes as a first-class citizen (of which the test passes). The question now, is how do we get Castle Windsor to do the same?
Initially, I took a look at ISubDependencyResolver and caching variables. Unfortunately, this didn’t work too well because sub resolvers never got hit if they were resolved from the container directly.
The next step I tried was with lifestyle managers, but alas, the CreationContext was always transient and I was unable to store any state that distinguished between different context resolutions.
After digging deeper into the Windsor codebase and getting into the subsystems and handlers, I found a solution that seems to work. It passes the test above, but that’s about it. Test well if you’re gonna use this in production code!!!
Here goes!
First, you have a lifestyle manager to distinguish between other lifestyles.

public class ContextualLifestyleManager : AbstractLifestyleManager
    {
        private object instance;
        public override object Resolve(CreationContext context)
        {
            return instance ?? (instance = base.Resolve(context));
        }
        public override void Dispose()
        {
        }
    }

And finally, the magic happens with this:

public static class ContextualExtensions
    {
        public static ComponentRegistration<T> PerContextScope<T>(this LifestyleGroup<T> group)
        {
            return group.Custom<ContextualLifestyleManager>();
        }
        public static IWindsorContainer BeginLifetimeScope(this IWindsorContainer parent)
        {
            var child = new WindsorContainer();
            var ss = (INamingSubSystem)parent.Kernel.GetSubSystem(SubSystemConstants.NamingKey);
            foreach (var handler in ss.GetHandlers())
            {
                if (handler.ComponentModel.CustomLifestyle == typeof(ContextualLifestyleManager))
                {
                    child.Kernel.AddCustomComponent(handler.ComponentModel);
                }
            }
            parent.AddChildContainer(child);
            return child;
        }
    }

First method is just a helper method to be a little more fluent in the registration for when you want many things to have contextual lifestyle. The second method is the guts. Long story short, we create a child container, and duplicate all component models of contextual lifestyle. Thus, whenever components are resolved, the “override” is found in the child and resolved. Anything else will be found in the parent.
I was initially pretty happy with this, until I profiled the performance. With Autofac, creating and disposing 100,000 contexts took 5ms on my computer. Doing the same with with Windsor took 3.8 seconds. Out of curiosity, I profiled again, but this time just creating child containers without copying handlers down: 1.9 seconds. So while this implementation works, it’s not as performant as I’d like it to be….
Maybe I’ll come up with another solution, but for now if the performance is acceptable maybe this would be useful for others!

Forcing Castle Windsor to Generate Class Proxies

2010-04-21T17:31:00.000-07:00

I don't know why it took me so long to come up with this idea, but to save others potential headaches...have you ever thought "hmmmm, I registered this as an interface with an implementation, but I want Windsor to use a class proxy, not an interface proxy, how do I do that?"

For me, initially I almost went as far as implementing my own ProxyFactory to force it to use class proxy no matter what, and then the light bulb hit me and it turns out that there's a much easier way to accomplish this.

c.Register(Component.For<ConcreteImpl, IService>().Interceptors<AnInterceptor>());

Tada! The actual service is now a concrete type, so Windsor will go, OK, I need to create a class proxy. But since it's forwarded to the interface as well, all your dependencies can simply use the interface and everything magically works.

Yay!

How to Advocate Test Driven Development (TDD)

2010-04-05T19:43:00.000-07:00

It seems that everywhere you read you will find blog posts asserting that it is difficult to convince others to use TDD, and that everyone talking about TDD is already a believer preaching to other believers. So what's the best way convince others?

I recently convinced a good buddy of me to starting doing TDD in his web development. It took a while, but finally he "saw the light." He tried to relay the information to his coworker, and his coworker didn't react as positively, citing the common "I don't want to waste time writing test code when I know my stuff works."

There are many many MANY hurdles of overcome to begin writing software using TDD. I won't go into great detail since you can easily find relevant posts on the web, but basically, if you have successfully transitioned to TDD that also implies that you have in one way or another adopted S.O.L.I.D. principles in software design.

The reason I say this is because it is literally impossible to do TDD otherwise. It is just plain painful. I've been there. It is so painful you shout in your head WTF are people thinking, how does this make my job easier?!

Usually when this happens, you'll probably notice that most (or all) of the SOLID principles have been violated one way or another.

<sarcasm>

Single Responsibility Principle

You mean 1 class to rule them all right?

Open Closed Principle

Does this have something to do with sockets?

Liskov Substitution Principle

I thought the point of Object Oriented Programming is that everything is an object!

Interface Segregation Principle

What's an interface?

Dependency Inversion Principle

I don't get it...I only have 1 super object anyway, I don't have dependencies.

</sarcasm>

Notice that even though this post was supposed to be about how to advocate TDD, I ended up talking about SOLID instead for a bit. What am I trying to get at?

TDD spans from the individual to the team to the organization. You need to be able to tackle it from all points of view. The obsession of the word 'test' has hurt TDD quite a bit. Anyone who's spent any reasonable amount of time will assert that tests are just a means to drive the design. Tests should read like a story (well to programmer eyes), and yes, I said read, as in readable!

At the individual level, you must convince them it forces you to design better code, because testing against badly designed code is not worth the pain/annoyance. You must assert that better designed code is good because it also means being more maintainable and less bugs.

At the team level, you must convince that collective code ownership is a good thing. There should never be a situation where "Bob's on vacation, we'll need to put a hold on Project A because only he knows it". Tests serve as a way on how to interact with a specific service, and are a jump-start to figuring out how to actually use a particular service. Moreover, collective code ownership is a means of sharing and distributing knowledge.

At the organizational level, you must convince managers that you are a professional software developer, and that it is your job to produce high quality work. And if what is required to produce high quality work involves writing proper unit tests, following SOLID principles, and even other agile practices like pair programming, then you should be allowed (and optimally encouraged) to do so. Now TDD might not work for everyone, or every organization, but I think everyone owes it to themselves to give it a fair chance.