Mavnn's blog

Stuff from my brain

Property Based Testing at NDC Oslo 2018

This summer I spoke at NDC Oslo. It's a fantastic event (if huge!), with good talks but also excellent facilities and shared spaces for meeting up with people.

Over the last few weeks, they've been publishing the videos of this year's talks, and mine appeared without me noticing. If you're interested in a deep dive session on using property based testing to test a templating library, you might find this interesting…

You can find all of the example code on GitHub: https://github.com/mavnn/ndcoslo2018

Working With NoRedInk

It's all change here at @mavnn land: as of last week, we've signed a full time, indefinite contract with NoRedInk to provide development services.

Given that we're a two person company, and only one of us is a developer, that means we won't be taking on any other work for the foreseeable future!

So, what's going on, why, and what does it mean for you?

Why?

The reasons come on a few levels, but they basically boil down to:

  • I think NoRedInk are doing something really worthwhile, and they really understand remote teams
  • My family may need to move on fairly short notice at some point (cough Brexit cough) and neither on-site work nor short term contracts give much security in that context

There are other reasons, and not necessarily minor ones:

  • They have an in-depth interview process which has convinced me I can work with the engineers there
  • They're using Elm on the front end, which I really like the look of
  • They're investing in tooling and support

I'm only just getting started with them (last week they had a 3 day retreat I was able to join them for, this week I'm at NDC Oslo), but so far I can tell you that they seem to be a great bunch of human beings with a clear, concrete, targeted plan on how to make (one aspect of) the world better. I can get behind that.

What does it mean for you?

Well, basically it means that @mavnn ltd will no longer be running on site training or ticketed training events, and we won't be available for bespoke development. It also means that I'll be a lot less active in the F# community; I have 3 new programming languages to learn and become productive with in fairly short order.

Having said that, you can probably expect some cross-ecosystem pollination talks at conferences in the future!

Anyway: enough about me. I now return you to your regular schedule of techy bloginess.

Full Stack With Freya

Yesterday night I was about to demo a quick server/client pair with Freya and Fable, and it all went a bit wrong. Some of the issues weren't related to what I did (computers, gotta love 'em) but others were just bits of configuration that I didn't have at my finger tips.

This means it's time for a little practice for me, and a mini-tutorial for you (and future me).

RouteMaster and the Tale of the Globally Unique Voters

RouteMaster is a process manager library I've been working on for simplifying the creation of complex work flows in message based systems.

One of the challenges RouteMaster faces is that once you have defined your "route" in RouteMaster, you generally want to run multiple instances of your process manager service in your distributed environment. This means that a lot of care has been taken to make sure that things like work flow state is handled safely, but it also causes a particular challenge for dealing with timeouts.

Cloud Native .NET

We're launching a new two day course this April (26th/27th), called "Cloud Native .NET". Despite the slightly pretentious name the industry has come up with, what we're really talking about here are the core engineering skills of building code that can be scaled and maintained. This course is a practical workshop using .NET Core and Kuberenetes so we can see what it all looks like in practice.

Full details below!

Getting Started With F# in Kubernetes

Author's note: This post is a quick start to help you get a single F# based service up and running on Kubernetes. If you want the full story on how to design a distributed system, we offer commercial training and consulting services to help you with that.

"Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications" - in other words, it will handle more deployment, health monitoring and service discovery needs out of the box, as long as you can turn your application into a container. So, let's have a quick look at how to do that with an F# application.

Logging Freya

Eugene Tolmachev asked in a comment on a previous post how I handle dependency injection with Freya.

So… my first, slightly annoying answer is that I try not to. Mark Seeman has written about this in a great series of blog posts which I won't try and repeat here.

Still, there are occasions where you want to quickly and easily do… something.. with a dependency making use of the context that being inside a Freya workflow provides. Let's quickly walk through how I inject a logger into a Freya workflow which "knows" about things like the request ID Kestrel has assigned to the current request.

I'm going to use Serilog as an example below, but you could also use any other structured logging library (I like Logary, but there isn't a .NET Core release at time of writing).

F# Through a Ruby Lens

I spent last week delivering a five day deep dive into F# for a group of (mostly) Ruby developers in Munich, and wanted to capture some of my thoughts before I lost them as well as give people an idea of the types of things internal training can give you.

I won't be mentioning personal, company or exact team names here as I've not been given explicit permission to do so; if the people who were on the course want to chime in I'll add their comments.

The background

Although mostly a Ruby on Rails shop, this company also relies on machine learning and expert systems to deliver some of its core services. The R&D department (who build the models) settled on F# for development as a good balance between:

  • familiarity of syntax (most have a background in Python and/or a ML language)
  • performance (Ruby had struggled here)
  • type safety
  • good "production" library support (logging, etc)

Having examined the available options in depth, they decided on a standard stack for creating F# microservices of:

  • Freya on Kestrel via .NET Core
  • Chiron for type safe JSON serialization/deserialization

They wanted to investigate the use of Hephaestus as a rules engine (Freya uses Hephaestus to process HTTP requests). Many of their machine learning models only work with quite constrained ranges of input values, and Hephaestus as a rules engine looked an effective way of routing decisions to the "correct" machine learning algorithm for a particular input range. This in turn would allow for the models to stay reasonably simple and testable.

The brief

Having made these decisions, the company needed to bring the production services team up to speed on what R&D were going to produce, especially because production had expressed an interest in having F# as an extra potential tool for their own projects.

My brief was to create 5 days of training, after which production needed to know enough about the F# libraries in use that they could work out what R&D's code was doing, and enough about running .NET code in production to feel confident adding error handling, logging, metrics, tests and all the rest of the "engineering" side of development which is not about the programming language but the surrounding ecosystem.

What we did

I knew that I had a lot of ground to cover in just 5 days, so there was no way that the team was going to come away with all of the new knowledge absorbed and at their finger tips. At the same time, it couldn't be an overwhelming flood of information.

I decided to split the training time between a deep dive in understanding a few key areas in depth (Freya's design, optics and testing), and providing worked examples for the rest which could be referred back to when they became needed. Although I had relevant training material on several of the areas already, it was all tailored in this course to fit a single theme: over the course of a week, we were going to build a microservice that did just one thing, and we were going to test the heck out of it.

The timetable ended up looking like this:

  • Monday AM: Introductions
    • high level microservice design
    • check everyone had all the software they needed installed
  • Monday PM: Freya overview
    • install the template
    • modify the hello world service to accept POSTs with a name
  • Tuesday AM: Optics
    • Chiron, Freya and Hephaestus all make heavy use of "Optics"
    • What are they?
    • Building our own
  • Tuesday PM: Handling external data
    • Using Chiron for translation, version handling and API design (using our new found knowledge of optics)
  • Wednesday AM: Start our actual microservice as a real project
    • how .NET solutions are (normally) laid out
    • using Paket for package management
    • add a test project with this set of property based tests
    • write our first bit of domain logic to pass these tests, and plug it into the Freya API
  • Wednesday PM: Start making our service production worthy
    • Spin up a docker "infrastructure" with Kibana and ElasticSearch
    • Adding logging to our service, plugged into Freya to automatically capture context like request IDs
    • Health endpoint
    • How to capture metrics
  • Thursday AM: interesting bits & answers to questions asked
    • How do computational expressions work?
    • How would I structure a functional UI?
  • Thursday PM: flexible rules engines with Hephaestus
    • rebuilt the logic from Wednesday AM reusing the same property tests
    • looked at how we can splice Hephaestus rules graphs together
  • Friday AM: BenchmarkDotNet
    • now we know it's correct - is it fast?
    • benchmarked our two implementations of the same logic together
  • Friday PM: Using it all in real life
    • code review of pieces of the existing code base, looking at adding what we'd learned

How it went

Overall the course seemed to go really well. At the end of it, the delegates were confident about the basics of building HTTP resources with Freya and Chiron, and happily building benchmarks and tests for their existing code base. For other areas (the boiler plate for plugging logging into Kestrel and Freya, for example) they understood the concepts and felt the course notes were sufficiently detailed they that could make use of them in other situations as needed. That was incredibly pleasing to hear from my point of view, as the course notes for these sessions are by far the most time consuming part of the process to create.

Although they missed some of the features of Ruby when writing F#, pattern matching with discriminated unions was a big hit and they liked the enforced discipline of Freya that required separating the logic of the various stages of handling an HTTP request - and how reusable that made components for handling concerns such as authentication.

Finally, all 3 of the core participants (there were other people around for certain parts of the course) came away saying that they'd really enjoyed it and found it interesting throughout - so that's a big win right there!

Can you do this for us?

Yes; this particular course was tailored for the specific circumstances, but I've also provided training on the more conceptual side (functional programming concepts) through to the gritty detail of DevOps (with both new and existing code bases).

We can also tailor delivery to match your availability; for this course I traveled to Munich to deliver it, and so it was delivered in a single 5 day unit. For other clients we can arrange regular shorter sessions or even remote workshops (group or individual) with tools such as Zoom.

And if you just want to turn up at a venue and get trained, check out Building Solid Systems in F# happening 31st Jan-1st Feb 2018 in London.

Get in touch with us at [email protected] if you have any ideas.

Advent 2017 - Reading From the Firehose With Fable

Each year, the F# programming community creates an advent calendar of blog posts, coordinated by Sergey Tihon his blog.

I really like the idea, and have taken part in 2016, 2015 & 2014.

Below is this year's post.

The plan: speed read Christmas

So; you want to find out what Christmas is about, where it really came from… but you don't have much time.

The solution is obvious: take the famous bible passages that churches read every year, and speed read them!

Let's build an app to help us with that.