Mavnn's blog

Stuff from my brain

An in Memory Message Bus in 100 Lines or Less

In reimplementing an EasyNetQ process manager one of the things I wanted to keep from the original project was an in memory message bus that could be used for testing without requiring a running RabbitMQ server. The code has ended up being pleasingly short and also uses a few techniques that seemed interesting, so I thought I'd document it here as part of the design process.

Please note we're not going for a full re-implementation of RabbitMQ in memory here, but this does give us enough to do some useful testing!

Trying .NET Core 2.0 With F# Today

Yesterday, I tried to use .NET Core for F# on day zero. A bit bravefoolish, I know, but v2 was supposed to be the one with all the bugs ironed out.

Short version: it's a lot better, but it's still easy to hit rough edges.

Longer version: be very careful that you don't hit issues with versions. On MacOSX, I hit a series of road blocks which made yesterday much more painful than it should have been.

  • If you're on a Mac, you'll need to update All The Things™ to get a reliable experience. And I don't just mean all the .NET Core things - full system update and brew upgrade are your friends
  • Don't use templates. Not many of them have been updated to 2.0 yet, you get no warnings about the ones which haven't, and enough has changed that it is very hard to update them manually unless you are a .NET Core expert already. (If you are, I suspect you're not reading this guide).
  • Don't try and update projects unless you know what you're doing; it cost me a lot of pain yesterday including bizarre internal compiler errors. On the happy news front, just copying across your actual code files works just fine.
  • Don't try and use Visual Studio (yet) - I'm not going to go into this one as I'm mainly talking to Mac users, but there has been issues there.

With all that said and done, if I skipped using any templates and stuck exclusively to the bundled project options, the actual experience of using dotnet is very pleasant.

For example, setting up a brand new solution with library and test project looks something like below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# Create solution file Project.New.sln in current directory
dotnet new sln -n Project.New

# Create library project in directory Project.New.Library
# Default proj name is Project.New.Library.fsproj
dotnet new classlib -o Project.New.Library --lang f#

# And again for test library
dotnet new console -o Project.New.Library.Tests --lang f#

# Add projects to solution (can combine to a single line)
dotnet sln add Project.New.Library/Project.New.Library.fsproj
dotnet sln add Project.New.Library.Tests/Project.New.Library.Tests.fsproj

# Set up test console app
cd Project.New.Library.Tests
dotnet add reference ../Project.New.Library/Project.New.Library.fsproj
dotnet add package Expecto
# Update Program.fs to run tests (see https://github.com/haf/expecto#testing-hello-world)

At this point, running dotnet run in the test directory should run your example test, and running dotnet build from the solution directory should successfully build your nice, portable, shiny, .NET Core 2.0 code.

Enjoy, and remember this post has a shelf life: hopefully issues like the template woes I had should disappear quickly as the eco-system catches up with the latest release.

Process Management in EasyNetQ

Back in 2015, I wrote about a process manager I'd written over the top of EasyNetQ. At the time it was released as open source, and I was pretty pleased with it. It allowed you to fairly quickly string together a managed work flow of services steps with built in state management for each work flow, and avoided many of the potential pitfalls of trying to build a request/response based system in situations where it isn't appropriate.

Two years on, I've learnt a lot about distributed system design and a lot about composing logic (cough monads cough) - and the original source is no longer available from my previous employers where it was written.

Despite that, I've had a lot of interest in the library in between, so I'm embarking on a full, clean room, rewrite incorporating everything I've learnt. This will also allow me to take advantage of the (very) recent move of EasyNetQ to be .netcore compatible to build the library against .NET Standard, providing a fully portable solution out of the box.

As with EasyNetQ itself, the major focus of this project will be providing the best possible developer experience. This means that it will provide sensible defaults and will be opinionated in places.

Where do you come into all of this? Well, we're looking for corporate sponsorship to help accelerate the development process and we're looking for testers to help build products with pre-release versions. In both cases you get to help to drive which opinions the library settles on, and as a corporate sponsor we'll also help you get up and running. Whether you're sponsoring or not, we'd love you to get involved.

And on a more general note, if you find you're pushing the boundaries of EasyNetQ in any way and you'd like some help, I'd be happy to set up training or consultancy for a bespoke solution for you as well. Drop a note to us@mavnn.co.uk.

Deliberate Poster: Fighting Imposter Syndrome

I first became aware of Imposter Syndrome via an excellent blog post by Scott Hanselman, but it immediately struck true. I'm a self taught programmer who dropped out of a Maths degree, and even now with years of experience that voice is there at the back of my head:

  • "You still make off by one errors, and you think you can train people?"
  • "How could you have let that bug slip by? It's obvious, I thought you knew what you were doing?"
  • "You want people to pay you for this? How exactly are you justifying that one to yourself?"

Normally I just try and tell it to shut up: everyone makes mistakes sometimes, I deliver stuff people get value from, etc.

But a few days back, something interesting happened. I'd hit a problem that looked like it should be simple, and I'd been bashing my head against it for a couple of days. The imposter critic was out in force: "call yourself a coder?". So I stepped back, took a deep breath, told the voice to stuff it and did the sensible thing. I asked my co-director (and wife) for help.

Now, this isn't asking a random person off the street for assistance. She has a high level post-graduate degree in mathematics, is ridiculously good at spotting patterns and logical deduction and has done some programming in the past herself (of the type you do during a maths degree).

It took me over an hour to explain what the actual problem was.

And as I dived through the OO patterns, domain specific knowledge and implementation constraints that built up into this "simple" problem it occurred to me that maybe I had learnt something over the years. That maybe I wasn't as much of an imposter as the voice was trying to tell me. That possibly, just maybe, not being able to solve this problem, right here, right now, didn't make me less valuable as a person or less competent as a professional. That the very fact I had this problem was actually evidence that I had a clue what I was doing, or else I wouldn't have been able to get here.

So, a technique for all the "imposters" out there when the voice fires up: be deliberate in reminding yourself that you're actually a poster. Don't fight the voice with generalities (even if they're true) - fight it with specifics.

Even if the specifics are "2 years ago I didn't know enough to get myself into this mess".

And you know the bonus extra of this technique? It fights Dunning-Kruger too…

I'll just be leaving this post lying around here so that my wife can post me a link to it next time I'm in the imposter blues.

Managing Mutable State With Computational Expressions

In mixed paradigm languages such as F# and Scala you frequently end up using mutable APIs in your "nice" pure functional code. It might be because you're using a 3rd party library, or it might be for performance reasons - but either way it's very easy to make mistakes with mutable constructs when you're in a functional mind space, especially if you want to compose operations on instances of a mutable type.

Let's have a look at one way of handling this issue: custom operations on computational expressions. We'll take the Provided Types API for building types within a type provider as an example of an API to use, and see what we can do to wrap it.

Return to the Ivory Tower Video

As mentioned on the blog already, I had the opportunity to talk at this year's F# Exchange at skillsmatter. As always, they've done an excellent job on the video production, so if you're interested in what makes F# different from other programming languages have a watch of Fixing Real Life Problems From the Ivory Tower (free log in required).

If you like the talk and would like to learn more, these are also the subjects that I'll be covering in my recently announced Level Up Your F# course, which will be running for 2 days in London on the 15th-16th June.

Level Up Your F# Skills

@mavnn ltd have always offered training as a service, but we're now pleased to announce our very first public training event. As readers of this blog (or people attending F# Exchange), you get first notification!

"Level Up Your F#" is going to be a two day course running in London on the 15th-16th June. We'll run through a syllabus of language features unique to F# and not shared by the other dotnet languages - features that offer unparalleled expressive power at the cost of a learning curve and the occasional sharp edge. In the course we'll give you a leg up that curve, and protect you from a few cuts along the way.

This course will be aimed at people who have spent some time coding in F# (possibly after taking a course like the "Fast Track to F#") and realise that their projects could benefit from a deeper understanding of F# special abilities. These techniques are especially useful for writing generic, reusable code - whether that's core internal code or widely used libraries.

It will be a small group, allowing for personalization and some flexibility in design, but at its core it will be build up through four major topics:

  1. Useful tricks: active patterns, member constraints, etc

  2. Quotations: what they are, and how and when to use them

  3. Type Providers: how to build them, and use them for type safety

  4. Computational Expressions: build powerful abstractions and then expose them in an easy to use way

We're aiming for a cost of £1,295.00 for the course with early bird (10 15% until the 15th May) and group discounts available (contact us). Due to the nature of the course, there will be a hard limit of 12 attendees. There's also a limited quantity of super-early bird tickets available if you're quick! Register your interest soon to secure a ticket, and if there's a waiting list we'll consult you about additional dates.

As always with @mavnn training courses, we're also willing to come and deliver this training on site and tailor it to your specific needs. Contact us at us@mavnn.co.uk to discuss requirements and quotes.

Returning to the Ivory Tower

But why should I learn F#?

I'm glad you asked! With multiple languages targeting the CLR it can seem just to be a matter of preference. Do you like curly braces, or significant white space?

But it soon becomes apparent there's a bit more to it than that; at first you spot pattern matching and discriminated unions. And then you start noticing a bunch of other things which look cool… but it's not quite so obvious how to use them or what to do with them. Last year I was able to give a lightning talk at F# Exchange on some of these language features and the response was positive enough that I'm back at F# Exchange 2017 to give the full version.

So: whether it's active patterns, computational expressions, type providers or quotations I'll be running through practical examples of how these features can be used, and how to get started creating your own examples.

And apart from being hyped to give my own talk, it's going to be great to meet up with the F# community again and meet some new faces. The opportunity to meet in person people I've exchanged ideas with online (hi Marcus Griep, Dave Thomas, …!) is invaluable. And even for someone who's been around F# for a while like me, there's always new things going on; Puritas looks fascinating.

Looking forward to meeting a bunch of you there, and you should start seeing a bit more appearing on this blog again after the conference. The last year has been pretty intense, but I'm hoping to make some announcements of interest to the F# community over the next couple of months.

Is this where I'm supposed to say "watch this space"?

Advent 2016

Each year I like to make my F# advent post centered around an aspect of the actual Christmas story, so this year I decided to look at the actual text of the Christmas story.

There are a couple of direct historical accounts recorded in the bible, in the Gospels of Mark and Luke. But Jesus's birth is a central point of the overall biblical story, with links to the Old Testiment books written before and referenced in places through the New Testiment.

Sounds like a graph to me, so lets see how far we can take some analysis.

Fortunately, someone has already produced a text file with a whole bunch of cross references in a nice regular format. So all we need to get started is a nice parser. We'll also want to pull in some metadata about the structure of the bible as a book in JSON format from the people at bibles.org.

Time to reference some dependencies to do our heavy lifting for us: FParsec for parsing, and FSharp.Data for the JSON type provider.

I'm writing this in the excellent Jupyter F# notebook (and then exporting it as markdown), so I'll use their Paket helpers to grab my dependencies (this should work on the Azure notebooks as well, although I've only tried it locally).