Your windows service is watching an email inbox.

How often should it poll?

Once every 5 minutes? Every 10? Then of course you realise that it should be different for every customer… or maybe every mailbox. You need more config!

Or not.

The real answer, of course, is something completely different: it should poll a lot when a lot of emails are arriving, and not very much when they aren’t.

It took a lot longer than it should have done to get my maths brain back on, but with the help of my wife I eventually settled on this code for deciding the intervals between polls:

1
2
3
4
5
6

let interval i =
    let x = float i
    let maxWait = 60. * 10.
    let raisePower x = pown (x /10.) 4
    (maxWait * (raisePower x)) / (raisePower x + 1.)
    |> (*) 1000. |> int

The ‘i’ in this function is the number of times we’ve polled since the last time a new email was received (if one is received, we reset i to 0).

If you plot this out on a graph, you get something that looks like this:

You can play with the shape of the graph at Wolfram|Alpha if you're feeling really geeky :).

This gives us very aggressive polling for the first few minutes after discovering an email, then dropping off rapidly to close to the one every ten minutes mark that I decided was a reasonable background polling rate.

It's not truly adaptive in the machine learning sense, but it gives a very good first cut that is an awful lot better than any fixed value could be.

I’ve been wanting to write code like this in F#, and know that any exceptions within a bound expression in an audit { } block will not only get caught, but that an external auditing service will get notified that the operation has failed.

Unfortunately, it turns out my code in my post on error handling ( https//blog.mavnn.co.uk/playing-with-error-handling-strategies ) was flawed in its ability to handle errors. The irony has not escaped me.

The issue is with the eager evaluation of arguments to the TryFinally method of the builder. If it takes you a while to work out what that means, don’t worry: it took me about 2 days to wrap my head round it and work out how to correct the code to make it behave as I would have expected.

To make things work correctly, the type returned by the computational expression pretty much has to be a deferred function of some kind.

So, the Interface, now renamed IAuditBuilder, gains a couple of helper functions and becomes:

The implementation of the TestAuditBuilder (only logs to console on error) becomes:

So: many thanks to Johann Deneux for patiently pointing out to me what the flaw in the original code was. I hope this example of a lazy computational expression is useful to other starting out down this rabbit hole of monadic weirdness. At least the resulting code looks pretty nice and readable now that the builder is fixed.

In my previous post ( https//blog.mavnn.co.uk/playing-with-error-handling-strategies ), Johann Deneux asked me whether my implementation of TryFinally actually worked.

TD;LR: It works. But thank you Johann for making me check!

Editted TD;LR: It’s broken, but not for the reason I thought. See the comments for details, and corrected code here https//blog.mavnn.co.uk/corrected-error-handling-computational-expres

This gave me a bit of a pause, as it’s actually an implementation I took from another source. One I trust, but it was still embarrassing to realise I couldn’t answer this question with full confidence as I didn’t quite understand the code myself.

In the end (especially given I’m planning to actually use this code in production at some point), I decided to go the whole hog and de-sugar a couple of simple versions of the computational expression.

So, first, the audit monad as used in my example code from the last post with just a simple let binding:

This raised my confidence slightly - the code I’d written did what I expected. But, what happened when I tried to bind a disposable resource using use! rather than let!? Time for the moment of truth:

So there you have it. The code does actually behave as expected, protecting the use of the disposable resource and ensuring it’s disposal. And I can breath easy that my code is doing what I thought it should be. Hopefully this will also serve as a useful example for anyone else starting out on the happy road of using computational expressions.

Taking a break from Fake for a moment (I’ll get back to it, I promise!)

Note: this post assumes a basic understanding of computation expressions in F#; if you’re looking for a primer on them, http://en.wikibooks.org/wiki/F_Sharp_Programming/Computation_Expressions
is a good place to start.

Note 2: the code in this post is actually broken if you use a use! or try finally block within your audit block. Check out https//blog.mavnn.co.uk/corrected-error-handling-computational-expres for the corrected code.

I had a series of small services I needed to add error handling to today to polish them up from nice clean prototype to production ready status.

Thinking about the joys of re-implementing the same error handling strategy in each of them, it occurred to me that this was exactly the kind of thing that computational expressions were built for. But my requirements went a bit further than just log the error and continue: each of these services contain critical locations where if a error occurred, we didn’t just want it logged locally, but we wanted (in some cases) an alert sent to a remote auditing* server.

Of course, mocking out the auditing service for testing is a bit of a pain in the backside, and how auditing is achieved shouldn’t really be the responsibility of these services anyway.

So, I decided to experiment with a builder interface.

Meet the IErrorHandlerBuilder interface:

Should look awfully familiar if you’ve recently checked out the msdn page on computational expressions: http://msdn.microsoft.com/en-us/library/dd233182.aspx (except, you know, mine has the right signatures; that might want to be fixed at some point).

How do we use it?

Well, first we need an implementation. In fact, let’s have two. Our first candidate, the AuditBuilder, takes an unhealthy number of dependencies that (in turn) take an unhealthy number of dependencies that you almost certainly want to supply from an IoC container.

What does it do? Well, basically if any bound function throws it logs the error via log4net and our in house audit server and then returns None. Otherwise it returns Some result from the computation. Obviously, if a computation further up the chain has already thrown we already have a None and we just pass it along. TryFinally and Using are implemented to make sure that you can still use disposable resources as you would expect within the expression.

Our second candidate is the TestErrorBuilder:

This is almost identical to the implementation above, except that it takes no dependencies at all and if there is an error it just logs to the console via printfn. Very useful for debugging - you might even want a unit test version that logs nothing, depending on your unit tests.

And now, our preparation is complete. How do we use our new toy, you ask?

Something like this:

This little service persists interesting email messages to a document store, and sends the contents back to you when you request them. If an email is not persisted, or a request for an email fails, we want a remote alert to be triggered in production. Both the persist and content requests are handled as RabbitMq messages (using the EasyNetQ client).

So in the message handler callbacks, declare an audit { } block and let! bind our message. Then we simply pass that bound message on to a handling function or object and return the resulting value, safe in the knowledge that our hosting program will have supplied an appropriate IErrorHandlerBuilder for our current purposes. The beauty of all of this is that the service object does not need to know what the error handling strategy is, and the objects/functions doing the work do not need to worry about error handling at all.

This technique is especially useful in situations where you are making use of asynchronous programming techniques such as agents and you really don’t want a single failure taking out your whole agent or async expression. If for some reason it’s important to your code what the error is, as well as knowing it’s been handled, it would be easy to modify this technique to return a Choice or custom MaybeError discriminated union that you could pull the exception out of later.

It would also be helpful for managing things like SQL connections with multiple retry attempts: for testing you may not want that kind of complexity and you could pass in a simpler strategy builder, while testing your robust, multiple retry computational builder for use in actual deployed environments (test or production).

And, of course, just knowing that builder interfaces are possible raises up a whole new world of possibilities.

As always, I tend to write about stuff that is fresh and new to me (i.e. not fully considered and maybe fatally flawed). Comments and suggestions welcome.

* To avoid any confusion, I should note that the word audit is used loosely in this blog post, with a meaning more related to a namespace within our code than the normal meaning of the word ‘audit’.

This is the first post in what will be a series about how we moved to using Fake for our build and testing at 15below. Feel free to follow along for the ride. It’s partly there as a way of keeping the full development group within the company informed of what’s going on, so it will get pretty detailed and gritty in the later sections, but we decided that there wasn’t anything too company specific in here. So we’re sharing it with you all…

At my current employer, we have a wide range of code all the way from automatically translated vb6 (to vb.net) to brand spanking new F# and C#. At the end of the day though, everything runs on .net 4.0 (currently).

With the newer components, we could in theory be building and testing them independently: messaging based and developed in a TDD style they tend to have an existing test harness around them and be easy to add further tests to if it proves necessary.

The legacy code on the other hand, is a bit of a different matter. It’s still in the code base because it works. We know it works, because customers are using it and getting value from it. The bits that didn’t, or looked like they were getting flaky, are the bits that have been refactored as we went along. But they are old enough that what is now considered basic testing hygiene was still a little more than a twinkle in Kent Beck’s eye. (Ok, ok, so the book had been released. But it wasn’t exactly considered common best practice yet). So until the refactoring work is complete, we’re stuck with a system where the only feasible way of testing some aspects of it is to actually deploy it somewhere and run some jobs through.

So what does this all have to do with Fake?

Well, basically our continuous integration was getting out of control. To get the level of functionality and control we needed, we were ending up with somewhere in the region of 500 lines of re-entrant, recursive MsBuild scripts. I seriously considered starting the alt.ms.build.die.die.die newsgroup. A full build on a developer’s machine could take upwards of 45 minutes. Even more painfully, as the refactoring continued we ended up with TeamCity build configurations tightly coupled to specific versions of our software, and it was becoming clear that the whole CI system was going to rapidly collapse under it’s own weight unless things changed.

We needed a solution.

It had to be:

• Powerful. Programming in MsBuild is… not fun. Xml is for data.
• Storable in git along with our source code.
• .net based. We’re a .net shop, and our specialist knowledge is .net; rake wasn’t going to cut it. We also didn’t want the management overhead of even more tooling on the build agents.
• In active development.
• Preferably open source, so even if external development stopped we could continue to maintain it.

As both the company F# evangelist and one of the two people doing most of the CI work, I was very interested to hear about Fake, especially after having a chance to chat to it’s author (Steffen Forkmann) and having it recommended by Don Syme (who wrote F#).

It met all of the criteria above. It’s scripts are full F# – a language fully supported by Microsoft within it’s flagship Visual Studio product. The scripts can easily be stored in git. It’s actively being worked on, and accepting contributions as a fully open source product.

Time to give it a try.

Next post: how to set up your “HelloWorld.fsx” Fake script with your source, with all the niceties like intellisense that you would want.

Possibly my geekiest self Birthday present ever: the #NOOO! manifesto (http://notonlyoo.org) t-shirt, personalized with membership number (and a !).

I'm not exactly a fashion model, but I was asked for pictures, so there you go. My wife also appears to have decided I needed to be backlit for the St. Lambda effect…

Given the nature of the manifesto ('Options over nulls') I was especially amused to see note the fact that whatever software they're using for the packing skips handles blank lines incorrectly and print 'null' instead. Check the back text confirmation in the third picture.

Libraries like FsCheck ( http://fscheck.codeplex.com/ ) and  Pex always capture my imagination as fantastically clever and potentially useful tools. Along with things like strong type systems and code contacts they especially appeal to my long buried mathematical background.

Unfortunately, and probably because of said mathematical background, I sometimes struggle to think of ways to use them outside of maths and science problems. If I'm working on a maths library or I decide I want to write a sorting algorithm, trivially easy sanity checks of things like whether a function is cummutative or associative are immediately useful.

If I'm writing a service that polls an email address for bounced emails and out of office replies… I don't know. It feels like there should be things they can help me with: after all, over the years I've decided the strong type system is helpful even for this side of things, so surely the QuickCheck/proof side of things should be to. But the amount of work it requires to set up the test generators for complex data types (and then trying to formalize the acceptance criteria without just writing the actual implementation!) feels so large compared to the size of the project that I can never justify the time to actually use the tools.

So I'm left wondering: am I missing something? Is there a whole new level of treating ease and vigour I'm missing out on just because I don't have the imagination and experience to make good use of the tools right there in front of me?

Or are these tools only really useful in their own narrow field?

I suspect the middle ground; for the moment ideas are avoiding me in droves. Any good open source examples out there of these tools in use for the kind of 'business' code that most of us spend most of our time writing?

Although you do get proud of some of the strangest things. I particularly like the Welsh dragon.

What you can't see is that he's written the names of each country on the back of the respective flag.