Building a Real-time Push App with Silverlight: Part 7

Infrastructure Refactor

A lot of things changed internally, and I mean….a lot….

From an infrastructure standpoint, I decided to remove the dependency on LinqToTwitter, and I replaced it with Hammock.  A couple things led me to this decision, one being the Silverlight support wasn’t as good as I’d hoped, and the streaming API implementation was limited.  After reading the Twitter documentation I realized that the REST API was super simple and I’d be better off writing a simple interface to it.

I heard good things about Hammock, so I decided to give that one a try (I wasn’t going to go as far as reimplementing OAuth).  It was pretty easy to set up and in the end I was able to get Twitter working again and with less lines of code compared to the beginning of the refactor.

Goals

I had a couple goals for this project:

• Learn:  I was a complete newbie to Reactive Extensions when I started but now I understand it enough to hit the ground running with it.  I’m still learning about more conventions available to Caliburn.Micro.
• UX:  I wanted to learn a little more about interface design.  I wanted to know how little changes to gradients, shadows, colors, etc. could have a radically effect in the end result.
• Performance:  It should be fast.  It should be able to react to real-time data.  And it should do it with low CPU utilization.
• Concise:  I am a huge advocate for KISS.  I like convention over configuration.  I like implementing something in 2 lines of code rather than 20 (assuming it’s not cryptic).  As I was writing the app and refactoring, if there was an opportunity to remove a line of code, I did it.  The result is that the app currently consists of less than 500 lines of code as of this post (excluding XAML).

Tidbits

What are some interest things I learned?

• System.Json is an amazing assembly.  All you need to do is invoke JsonValue.Parse on a string and it will create a JsonValue for you, which will be a dictionary of key/value pairs.  What’s more, by doing something like “string s = json[“text”]” will do an explicit conversion and unescape JSON characters, and only via the explicit operator.  Calling ToString(), even though converting it to a string, will not unescape.  This was completely undocumented and only found when I looked at the source code via Resharper’s external sources feature.
• Rx is awesome.  When I ran into performance problems of trying to stream tweets from the world that contained the letter ‘a’ all I had to do was add an operator to improve the performance (in this case it was Buffer).  It should be noted that it is very important to understand what Rx is doing underneath the hood to realize its benefits.  Rx lets you refactor 30 lines of async code into 1 operator, but it’s still doing that 30 lines of code – you just don’t see it.
• I really, really, like the conventions available from Caliburn.  Some of the features that come out of the box from this very small library saves me from writing a lot of boilerplate code like commands, triggers, and evening bindings (Caliburn will auto bind x:Name to a property).
• Twitter’s documentation for user streams currently sucks and some trial and error was required to get it working.

What is the end result of all this effort?  We have a styled Twitter app that can update your status, pull your home/mentions timeline, and most importantly will stream all subsequent tweets.  There’s no pulling and no limits.  You will get a tweet of everyone you follow in real-time as it happens.

Moreover, there’s a feature to connect to the Streaming API to search Twitter for anything.  To get an idea of what we’re talking about, here’s a full screenshot of it:

You read that right.  I’m streaming any tweet in the world that has the words ‘and’, ‘the’, ‘yes’, or ‘no’ in them.  This is streaming around 400kB/s continuously and CPU utilization is under 25%.  The tweets are coming so fast it’s impossible to read them (at a rate of 50 tweets/second), so ideally you’d want to specify realistic search terms.

Moreover, the majority of the performance cost is actually downloading all the profile images.  If I take took out pictures I could stream any tweet in the world that has the letter ‘e’ in it at under 10% CPU.  It looks like Twitter limits the rate of tweets to 50 tweets/second because that was the rate for this one as well.

Features are minimalistic.  You can update your status, but you can’t DM, you can’t RT, you can’t do any of the normal things.  My original goal was not to write another Twitter client, but it’s actually quite fun to do so, so I’ll probably eventually get all features in.

And as promised, it’s up on GitHub, and version 0.0.0.1 alpha (yes! expect bugs!!) is available in the downloads section.  Or, here’s a direct link to the XAP file on my Dropbox.  Have fun!