Mavnn's blog

Stuff from my brain

Kubernetes for the Masses

As part of a two day training course I'm going to be putting together a bunch of material on how to run .NET Core code on kubernetes.

It will include things like:

  • Setting up CI/automated builds
  • Configuring ingress points (including ssl via self updating Let's Encrypt certs)
  • Monitoring and metrics of the running system

If you'd have any interest in this material as a separate module (which wouldn't be F# specific), do get in touch. I'd reckon it will become a one day course, either in house or hosted.

As an aside, once the material has been battle tested a few times, I will also be putting together a "base" deployment repository - although it will have fair warning that Kubernetes is really still the kind of thing that requires you to understand it before you start pushing things onto it, not just cutting and pasting someone else's config.

After all, you're the one who's going to get the phone call at 03:00…

Functional Programming in the Wild

Last week was the annual Progressive .Net conference, hosted at SkillsMatter.

It was a bit of a strange conference for me; I was invited to speak, and very much enjoyed meeting up with everyone but I also had a bunch of other work I needed to get done. That means that unlike some other conferences I won't be able to give much speaker feedback!

However, there were a few sessions I managed to pay some attention to and some thoughts about my own that I'd like to record - so here goes!

Building Solid Systems in F#

Update: The original venue sold out - more tickets are available for the 8th - 9th March 2018 on EventBrite!

We are running another course, and I'm officially stoked! Read on for the details…

Building Solid Systems in F#

Writing code is only the first part of putting software into production - to run, maintain and scale your product, you'll need to understand and design the overall system.

This 2 day course will take you through best practice in both writing an F# based, distributed system and running it in production - including sensible inter-operation with components written in other languages.

To fit this into 2 days, we'll need to make some opinionated decisions (such as using F#!), but many of the ideas and concepts will be transferable.

We'll cover:

  • Writing reliable code using F#’s unique language features
  • Learn how to apply SOLID (like) principles in a functional style
  • How to instrument distributed services
  • Continuous Improvement:
    • Unit testing
    • Performance measurement
  • Running distributed systems in development
  • Deploying distributed systems to production
  • Good practice in dependency management and code organization

You'll come away with:

A git repository of your completed work, which will include:

  • Nicely instrumented, benchmarked and unit tested F# services
  • A scripted, deterministic deployment process for the overall distributed system
  • Real time centralized logging, metrics and health feedback from the system, whether running on the dev machine or in production
  • Zero down time continuous deployment for the overall system

Prerequisites:

You need to have:

  • a basic knowledge of F# syntax
  • a reasonable background knowledge of software development

You need to bring laptop with:

  • a relatively recent F# development environment (you’ll need to be able to build dotnet core 2.0 apps)
  • Minikube and kubectl installed

We’ll also ask you to pre-download some code and containers in advance so that we can hit the ground running on the first day; we’ll pass you the details of that before the event.

Is this a replacement for Level Up Your F#?

No; Level Up Your F# focussed on the details of the F# language, this course is focussed on building systems. We will be running Level Up Your F# again in the future.

Where/when will it happen?

At the Wellcome Collection Sold out!

The Skiff, in Brighton on the 8th and 9th Manch 2018

Where can I get tickets?

EventBrite or email [email protected] and we'll sort you out an invoice.

What have other people said about your courses?

Hassan Ezzahir, Lead developer (Contractor) at BNP Paribas

I’ve been trying to learn F# for several years now and got almost all the existing books on the subjects.

Yet, I felt there was a gap between my good understanding of the language and actually applying it on bigger “real” projects.

Michael’s great training skills has enabled me to quickly practice some advanced topics I was less familiar with.

With my newly acquired knowledge, I’m confident I will be able achieve some great (and fun) developments with F#

Alexander Battisti, Senior Software Developer (Machine Learning) at Freeletics GmbH

I found "Level Up your F#" a good course teaching advanced topics in F# that are commonly ignored or glossed over when you are on your own, but are useful enough so you shouldn't ignore them. The exercises were difficult in a good way and engaging. I definitely recommend the course for anybody after they took their first serious steps in learning F#.

System.Console Is Why We Can’t Have Nice Things

In writing a simple tutorial for this years Progressive .Net I thought I'd use the Console to allow some nice visual feedback with requiring a dependency.

TD;LR: System.Console (at least on dotnet core 2.0) is not as threadsafe as you'd hope, and means that writing any simple cross platform console UI is nearly impossible.

An in Memory Message Bus in 100 Lines or Less

In reimplementing an EasyNetQ process manager one of the things I wanted to keep from the original project was an in memory message bus that could be used for testing without requiring a running RabbitMQ server. The code has ended up being pleasingly short and also uses a few techniques that seemed interesting, so I thought I'd document it here as part of the design process.

Please note we're not going for a full re-implementation of RabbitMQ in memory here, but this does give us enough to do some useful testing!

Trying .NET Core 2.0 With F# Today

Yesterday, I tried to use .NET Core for F# on day zero. A bit bravefoolish, I know, but v2 was supposed to be the one with all the bugs ironed out.

Short version: it's a lot better, but it's still easy to hit rough edges.

Longer version: be very careful that you don't hit issues with versions. On MacOSX, I hit a series of road blocks which made yesterday much more painful than it should have been.

  • If you're on a Mac, you'll need to update All The Things™ to get a reliable experience. And I don't just mean all the .NET Core things - full system update and brew upgrade are your friends
  • Don't use templates. Not many of them have been updated to 2.0 yet, you get no warnings about the ones which haven't, and enough has changed that it is very hard to update them manually unless you are a .NET Core expert already. (If you are, I suspect you're not reading this guide).
  • Don't try and update projects unless you know what you're doing; it cost me a lot of pain yesterday including bizarre internal compiler errors. On the happy news front, just copying across your actual code files works just fine.
  • Don't try and use Visual Studio (yet) - I'm not going to go into this one as I'm mainly talking to Mac users, but there has been issues there.

With all that said and done, if I skipped using any templates and stuck exclusively to the bundled project options, the actual experience of using dotnet is very pleasant.

For example, setting up a brand new solution with library and test project looks something like below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# Create solution file Project.New.sln in current directory
dotnet new sln -n Project.New

# Create library project in directory Project.New.Library
# Default proj name is Project.New.Library.fsproj
dotnet new classlib -o Project.New.Library -lang f#

# And again for test library
dotnet new console -o Project.New.Library.Tests -lang f#

# Add projects to solution (can combine to a single line)
dotnet sln add Project.New.Library/Project.New.Library.fsproj
dotnet sln add Project.New.Library.Tests/Project.New.Library.Tests.fsproj

# Set up test console app
cd Project.New.Library.Tests
dotnet add reference ../Project.New.Library/Project.New.Library.fsproj
dotnet add package Expecto
# Update Program.fs to run tests (see https://github.com/haf/expecto#testing-hello-world)

At this point, running dotnet run in the test directory should run your example test, and running dotnet build from the solution directory should successfully build your nice, portable, shiny, .NET Core 2.0 code.

Enjoy, and remember this post has a shelf life: hopefully issues like the template woes I had should disappear quickly as the eco-system catches up with the latest release.

Thanks to Tim Preston for a minor correction to this post; our genius author had managed to copy and paste erroneous commands from his own command history…

Process Management in EasyNetQ

Update! Check out RouteMaster for a progress update on this.

Back in 2015, I wrote about a process manager I'd written over the top of EasyNetQ. At the time it was released as open source, and I was pretty pleased with it. It allowed you to fairly quickly string together a managed work flow of services steps with built in state management for each work flow, and avoided many of the potential pitfalls of trying to build a request/response based system in situations where it isn't appropriate.

Two years on, I've learnt a lot about distributed system design and a lot about composing logic (cough monads cough) - and the original source is no longer available from my previous employers where it was written.

Despite that, I've had a lot of interest in the library in between, so I'm embarking on a full, clean room, rewrite incorporating everything I've learnt. This will also allow me to take advantage of the (very) recent move of EasyNetQ to be .netcore compatible to build the library against .NET Standard, providing a fully portable solution out of the box.

As with EasyNetQ itself, the major focus of this project will be providing the best possible developer experience. This means that it will provide sensible defaults and will be opinionated in places.

Where do you come into all of this? Well, we're looking for corporate sponsorship to help accelerate the development process and we're looking for testers to help build products with pre-release versions. In both cases you get to help to drive which opinions the library settles on, and as a corporate sponsor we'll also help you get up and running. Whether you're sponsoring or not, we'd love you to get involved.

And on a more general note, if you find you're pushing the boundaries of EasyNetQ in any way and you'd like some help, I'd be happy to set up training or consultancy for a bespoke solution for you as well. Drop a note to [email protected].

Deliberate Poster: Fighting Imposter Syndrome

I first became aware of Imposter Syndrome via an excellent blog post by Scott Hanselman, but it immediately struck true. I'm a self taught programmer who dropped out of a Maths degree, and even now with years of experience that voice is there at the back of my head:

  • "You still make off by one errors, and you think you can train people?"
  • "How could you have let that bug slip by? It's obvious, I thought you knew what you were doing?"
  • "You want people to pay you for this? How exactly are you justifying that one to yourself?"

Normally I just try and tell it to shut up: everyone makes mistakes sometimes, I deliver stuff people get value from, etc.

But a few days back, something interesting happened. I'd hit a problem that looked like it should be simple, and I'd been bashing my head against it for a couple of days. The imposter critic was out in force: "call yourself a coder?". So I stepped back, took a deep breath, told the voice to stuff it and did the sensible thing. I asked my co-director (and wife) for help.

Now, this isn't asking a random person off the street for assistance. She has a high level post-graduate degree in mathematics, is ridiculously good at spotting patterns and logical deduction and has done some programming in the past herself (of the type you do during a maths degree).

It took me over an hour to explain what the actual problem was.

And as I dived through the OO patterns, domain specific knowledge and implementation constraints that built up into this "simple" problem it occurred to me that maybe I had learnt something over the years. That maybe I wasn't as much of an imposter as the voice was trying to tell me. That possibly, just maybe, not being able to solve this problem, right here, right now, didn't make me less valuable as a person or less competent as a professional. That the very fact I had this problem was actually evidence that I had a clue what I was doing, or else I wouldn't have been able to get here.

So, a technique for all the "imposters" out there when the voice fires up: be deliberate in reminding yourself that you're actually a poster. Don't fight the voice with generalities (even if they're true) - fight it with specifics.

Even if the specifics are "2 years ago I didn't know enough to get myself into this mess".

And you know the bonus extra of this technique? It fights Dunning-Kruger too…

I'll just be leaving this post lying around here so that my wife can post me a link to it next time I'm in the imposter blues.

Managing Mutable State With Computational Expressions

In mixed paradigm languages such as F# and Scala you frequently end up using mutable APIs in your "nice" pure functional code. It might be because you're using a 3rd party library, or it might be for performance reasons - but either way it's very easy to make mistakes with mutable constructs when you're in a functional mind space, especially if you want to compose operations on instances of a mutable type.

Let's have a look at one way of handling this issue: custom operations on computational expressions. We'll take the Provided Types API for building types within a type provider as an example of an API to use, and see what we can do to wrap it.

Return to the Ivory Tower Video

As mentioned on the blog already, I had the opportunity to talk at this year's F# Exchange at skillsmatter. As always, they've done an excellent job on the video production, so if you're interested in what makes F# different from other programming languages have a watch of Fixing Real Life Problems From the Ivory Tower (free log in required).

If you like the talk and would like to learn more, these are also the subjects that I'll be covering in my recently announced Level Up Your F# course, which will be running for 2 days in London on the 15th-16th June.