Mavnn's blog

Stuff from my brain

All Saints’ Day Sale

TL;DR: 10% off Building Solid Systems in F# until 7th November 2017

Lots of people these days seem to like giving Halloween sales, but historically and theologically, Halloween is really just the precursor to the real celebration: All Saints' Day.

So in the interest of getting the details right, we're having an All Saints' Day sale, starting today for 7 days. It's already live, get your 10% off your tickets now.

RouteMaster : Master Your Messaging Routes

I'm very pleased to announce the release of an initial alpha of RouteMaster.

What is it? Well, I'll let the README speak for itself:

RouteMaster is a .NET library for writing stateful workflows on top of a message bus. It exists to make the implementation of long running business processes easy in event driven systems.

There is also example code in the repository so you can see what things are starting to look like.

For those of you following along, this will sound awfully familiar; that's because RouteMaster is the outcome of my decision to rebuild a Process Manager for EasyNetQ. The first cut of that was imaginatively called "EasyNetQ.ProcessManager", but I decided to rename it for three main reasons:

  • On re-reading Enterprise Integration Patterns, it occurred to me that RouteMaster was an enabler for many of the other patterns as well as the "Process Manager"
  • The message bus RouteMaster uses is provided as an interface; the main dll has no dependency on EasyNetQ at all
  • The previous EasyNetQ.ProcessManager is still available as a Nuget package supplied by my previous employer, and they have both the moral and legal rights to the package given I wrote the original on their time

A pre-emptive few FAQs:

Is this ready to use?

No, not yet. I'm out of time I can afford to spend on it right now, get in touch if you can/want to fund future development.

If you want to play, the code as provided does run and all of the process tests pass.

Urgh! All the examples are F#!?

Yes, but there is a C# friendly API in the works. See the first question :)

What infrastructure do I need to run this?

At the moment, I'm using EasyNetQ (over RabbitMQ) and PostgreSQL (via Marten) for transport and storage respectively.

What about things like NServiceBus and MassTransit?

In some ways they fall in a similar space to RouteMaster, but with a different philosophy. Just as EasyNetQ is a focused library that supplies only part of the functionality you'd find in these larger solutions, RouteMaster is designed to work with your chosen transport abstraction not replace it.

Ask not what your RouteMaster can do for you, but what you can do for your RouteMaster!

I'd really like feedback, ideas, use cases and suggestions - leave comments here or ping an issue onto the repository. If you're feeling really brave and can try and actually experiment with it, but at the moment I'm mostly hoping for concrete use cases and, well, funding.

Quite a few people over the years have hit my website searching for an EasyNetQ process manager, and others have asked me if it's still available. I'd like to hear from as many of you as possible to build the tightest, simplest solution which will do the job.

POSTing to Freya

I've written about how nice Freya is as a library, but documentation is still a little light on the ground.

So here's a minimal implementation of a "microservice" Freya API, starting from which dotnet commands to run to install the Freya template, through to a running web service.

Make sure you have an up to date .NET Core SDK installed, and grab yourself the handy dandy Freya template:

1
dotnet new -i "Freya.Template::*"

Then create yourself a directory and go into it. The following command will set up a brand new Freya project using kestrel as the underlying webserver, and Hopac (rather than F# Async) for concurrency. Alternatively, you can leave both the options off and you'll get Freya running on Suave with standard Async.

1
dotnet new freya --framework kestrel --concurrency hopac

Your project should run at this point; dotnet run will spin up a webserver on port 5000 which will give a 404 at the root and text responses on /hello and /hello/name paths.

Api.fs is where all the magic of configuring Freya happens - KestrelInterop.fs contains boilerplate for making sure Routing information passes correctly between Kestrel and Freya, and Program.fs just starts Kestrel with our Freya API as an OWIN app.

Adding JSON

So, this is great and all, but we're building a microservice aren't we? That normally means JSON (or at least something more structured than plain text!).

Let's change things up so that as well as supplying the name to greet in the route, we can POST JSON with a name field to the /hello end point.

To respond in JSON, we need a Freya Represent record. We're sending a result with a fixed structure, so we don't need a serialization library or anything, we'll just construct the JSON by hand. Stick this near the top of Api.fs:

Workflow Alpha

Log of workflow test running

It's alive! The process manager code I've been reconstructing (see Intro and the in memory test bus) is slowly starting to take some shape.

As you can see, it comes with nice (no dependency) logging out of the box and it is async all the way down to the underlying transport.

This is still at the underlying plumbing phase in many ways: the code to construct a workflow like this is currently a boilerplate covered ugly mess - but it's all boilerplate which has been deliberately designed to allow powerful APIs to be built over the top.

Next up: a nice sleek API for creating "pipeline" workflows more easily. Then the real fun starts - pleasant to use abstractions over fork/join semantics…

Interested in seeing faster progress on this project? Drop [email protected] a line to talk sponsorship.

Kubernetes for the Masses

As part of a two day training course I'm going to be putting together a bunch of material on how to run .NET Core code on kubernetes.

It will include things like:

  • Setting up CI/automated builds
  • Configuring ingress points (including ssl via self updating Let's Encrypt certs)
  • Monitoring and metrics of the running system

If you'd have any interest in this material as a separate module (which wouldn't be F# specific), do get in touch. I'd reckon it will become a one day course, either in house or hosted.

As an aside, once the material has been battle tested a few times, I will also be putting together a "base" deployment repository - although it will have fair warning that Kubernetes is really still the kind of thing that requires you to understand it before you start pushing things onto it, not just cutting and pasting someone else's config.

After all, you're the one who's going to get the phone call at 03:00…

Functional Programming in the Wild

Last week was the annual Progressive .Net conference, hosted at SkillsMatter.

It was a bit of a strange conference for me; I was invited to speak, and very much enjoyed meeting up with everyone but I also had a bunch of other work I needed to get done. That means that unlike some other conferences I won't be able to give much speaker feedback!

However, there were a few sessions I managed to pay some attention to and some thoughts about my own that I'd like to record - so here goes!

Building Solid Systems in F#

Update: The original venue sold out - more tickets are available for the 8th - 9th March 2018 on EventBrite!

We are running another course, and I'm officially stoked! Read on for the details…

Building Solid Systems in F#

Writing code is only the first part of putting software into production - to run, maintain and scale your product, you'll need to understand and design the overall system.

This 2 day course will take you through best practice in both writing an F# based, distributed system and running it in production - including sensible inter-operation with components written in other languages.

To fit this into 2 days, we'll need to make some opinionated decisions (such as using F#!), but many of the ideas and concepts will be transferable.

We'll cover:

  • Writing reliable code using F#’s unique language features
  • Learn how to apply SOLID (like) principles in a functional style
  • How to instrument distributed services
  • Continuous Improvement:
    • Unit testing
    • Performance measurement
  • Running distributed systems in development
  • Deploying distributed systems to production
  • Good practice in dependency management and code organization

You'll come away with:

A git repository of your completed work, which will include:

  • Nicely instrumented, benchmarked and unit tested F# services
  • A scripted, deterministic deployment process for the overall distributed system
  • Real time centralized logging, metrics and health feedback from the system, whether running on the dev machine or in production
  • Zero down time continuous deployment for the overall system

Prerequisites:

You need to have:

  • a basic knowledge of F# syntax
  • a reasonable background knowledge of software development

You need to bring laptop with:

  • a relatively recent F# development environment (you’ll need to be able to build dotnet core 2.0 apps)
  • Minikube and kubectl installed

We’ll also ask you to pre-download some code and containers in advance so that we can hit the ground running on the first day; we’ll pass you the details of that before the event.

Is this a replacement for Level Up Your F#?

No; Level Up Your F# focussed on the details of the F# language, this course is focussed on building systems. We will be running Level Up Your F# again in the future.

Where/when will it happen?

At the Wellcome Collection Sold out!

The Skiff, in Brighton on the 8th and 9th Manch 2018

Where can I get tickets?

EventBrite or email [email protected] and we'll sort you out an invoice.

What have other people said about your courses?

Hassan Ezzahir, Lead developer (Contractor) at BNP Paribas

I’ve been trying to learn F# for several years now and got almost all the existing books on the subjects.

Yet, I felt there was a gap between my good understanding of the language and actually applying it on bigger “real” projects.

Michael’s great training skills has enabled me to quickly practice some advanced topics I was less familiar with.

With my newly acquired knowledge, I’m confident I will be able achieve some great (and fun) developments with F#

Alexander Battisti, Senior Software Developer (Machine Learning) at Freeletics GmbH

I found "Level Up your F#" a good course teaching advanced topics in F# that are commonly ignored or glossed over when you are on your own, but are useful enough so you shouldn't ignore them. The exercises were difficult in a good way and engaging. I definitely recommend the course for anybody after they took their first serious steps in learning F#.

System.Console Is Why We Can’t Have Nice Things

In writing a simple tutorial for this years Progressive .Net I thought I'd use the Console to allow some nice visual feedback with requiring a dependency.

TD;LR: System.Console (at least on dotnet core 2.0) is not as threadsafe as you'd hope, and means that writing any simple cross platform console UI is nearly impossible.

An in Memory Message Bus in 100 Lines or Less

In reimplementing an EasyNetQ process manager one of the things I wanted to keep from the original project was an in memory message bus that could be used for testing without requiring a running RabbitMQ server. The code has ended up being pleasingly short and also uses a few techniques that seemed interesting, so I thought I'd document it here as part of the design process.

Please note we're not going for a full re-implementation of RabbitMQ in memory here, but this does give us enough to do some useful testing!

Trying .NET Core 2.0 With F# Today

Yesterday, I tried to use .NET Core for F# on day zero. A bit bravefoolish, I know, but v2 was supposed to be the one with all the bugs ironed out.

Short version: it's a lot better, but it's still easy to hit rough edges.

Longer version: be very careful that you don't hit issues with versions. On MacOSX, I hit a series of road blocks which made yesterday much more painful than it should have been.

  • If you're on a Mac, you'll need to update All The Things™ to get a reliable experience. And I don't just mean all the .NET Core things - full system update and brew upgrade are your friends
  • Don't use templates. Not many of them have been updated to 2.0 yet, you get no warnings about the ones which haven't, and enough has changed that it is very hard to update them manually unless you are a .NET Core expert already. (If you are, I suspect you're not reading this guide).
  • Don't try and update projects unless you know what you're doing; it cost me a lot of pain yesterday including bizarre internal compiler errors. On the happy news front, just copying across your actual code files works just fine.
  • Don't try and use Visual Studio (yet) - I'm not going to go into this one as I'm mainly talking to Mac users, but there has been issues there.

With all that said and done, if I skipped using any templates and stuck exclusively to the bundled project options, the actual experience of using dotnet is very pleasant.

For example, setting up a brand new solution with library and test project looks something like below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# Create solution file Project.New.sln in current directory
dotnet new sln -n Project.New

# Create library project in directory Project.New.Library
# Default proj name is Project.New.Library.fsproj
dotnet new classlib -o Project.New.Library -lang f#

# And again for test library
dotnet new console -o Project.New.Library.Tests -lang f#

# Add projects to solution (can combine to a single line)
dotnet sln add Project.New.Library/Project.New.Library.fsproj
dotnet sln add Project.New.Library.Tests/Project.New.Library.Tests.fsproj

# Set up test console app
cd Project.New.Library.Tests
dotnet add reference ../Project.New.Library/Project.New.Library.fsproj
dotnet add package Expecto
# Update Program.fs to run tests (see https://github.com/haf/expecto#testing-hello-world)

At this point, running dotnet run in the test directory should run your example test, and running dotnet build from the solution directory should successfully build your nice, portable, shiny, .NET Core 2.0 code.

Enjoy, and remember this post has a shelf life: hopefully issues like the template woes I had should disappear quickly as the eco-system catches up with the latest release.

Thanks to Tim Preston for a minor correction to this post; our genius author had managed to copy and paste erroneous commands from his own command history…