Two Weeks With F#

And observations thereon, with optional extras.

I took a pretty hands-off approach to training my replacement at work. I told him in general terms what needed to be done, and I showed him how I had done it up to that point, and then I said, basically:

“Look, you’re not me and you don’t think the same way I do, so a lot of the little details of how I did this job will not matter to you, and may at worst hurt your ability to mold the role around yourself. So from this point forward, I’ll be here to unstick things if you get stuck, fix things in a hurry, offer tips and suggestions on request, and answer any questions you have. Beyond that, you’re pretty much on your own. Do it, in the words of Denethor, in whatever way seems best to you.”

It was a quiet couple weeks, and he picked everything up with admirable speed, which left me (as I had hoped) with a lot of time on my hands and nothing definite to do. Luckily, I had something in mind.

Up until yesterday (August 19th) I managed a fleet of some two hundred and a bit computers, made sure they were all up to date with our company’s software, all the internals were in working order, and shipped them out in batches of anywhere from two to seventy, to be used on a temporary basis by our clients and by other departments in the company and then shipped back, where I would be in charge of backing up everything on the hard drive, restoring it to a clean image of the OS and software, dealing with any error messages or other problems that had arisen on-site, and putting them back on the shelf for later reuse.

This involves a lot of repetition. Pull case from shelf, take computer out of case, plug everything in, turn it on, enact specific settings for the client in question, test it to make sure everything works as intended, double-check all necessary equipment is present and accounted for, pack everything back up, slap a shipping label on it, send it out the door. In a couple weeks, take it out of the case and set it up again, copy everything on the hard drive over to a remote server, wipe said hard drive, re-install OS and software, enact some small universal settings, check equipment again, pack up, put back on shelf. Five to twenty times a week on quiet weeks, forty to a hundred times a week during the busy season.

The physical acts of moving things around, plugging and unplugging, repacking equipment, etc, all take time, and there’s a limit to how fast they can safely be done, but when I arrived at this job, all of those software tasks had to be done by hand, every step enacted one at a time by the shipping coordinator — me. Now, I use Linux regularly and I’ve always been a bit of a computer nerd, and I know a programmer’s task when I see one, so after a couple months of this I decided it was time to cause as much of that stuff as I could manage to happen on its own, with as little interference from me or other pesky humans as possible.

I never managed quite what I had wanted (a full, scriptable command-line interface to our software) but I learned C# and Batch, multiplied my job efficiency by 1.5, and all-but eliminated typical human errors like typos and routine transcription failures during the setup-test-backup-reimage process. Then the new version of our software came out, and I watched all that work unravel because most of the method calls didn’t work anymore. Actually, it didn’t happen like that because for various reasons my department decided not to upgrade to the new version, and still have only just begun the process a year and a half (and a full version number) later. So as of right now, all the stuff I built is still mostly usable. But I could see it coming.

I knew more or less what to do about it, but the rhythms of the business had changed, and the days that used to just be completely empty during the quiet times now had things in them that I had to do. Not quite enough things to really call myself busy, but enough that it was no longer possible to plan those glorious days where I’d show up, sit down at the computer, start coding, and emerge six or seven hours later, having forgotten to eat lunch and utterly brain-dead as a result, but feeling extremely accomplished and satisfied with myself. There was always something else that needed doing in the middle of the day, and without being able to really dedicate myself to writing code, I found it all but impossible to make meaningful progress on the problem of version-based instability in the tools I’d built.

So there I was, my employment coming to a close, and about two weeks lay ahead of me with absolutely nothing in them. Time to get to work.

What I needed to do was write something that didn’t, in and of itself, depend on any code in the software it was trying to talk to. An interface with no introspection and no dependencies outside the basic core of whatever language I was using. I had to be able to enter an interactive environment, to mess around with commands in real-time and see what worked and what didn’t, but also be able to call the same .exe and have it run a single command, passed as arguments, and exit, so that it could be used in a Batch script with a minimum of extra work.

What I would do, I decided, was build an application that, when run from a given machine, would dynamically load all of the company code on that computer and turn around and expose all of it to the command-line. I had been given permission by the engineering team to browse and interact with company code, even though I couldn’t add to or alter it, and the easiest way to do that was through visual studio, which meant I had Visual Basic, J#, C#, C++, and F# immediately available. (For reference, my preference is to write Common Lisp wherever possible, which is almost nowhere. In this case, RDNZL, the only CL library I know of to interact with .Net code, hasn’t been updated since 2011 or so and has become non-functional.)

I hate Visual Basic, C++ is a monster and unsuited to such a small task, and I’ve never quite understood what J# is all about. This left me with C#, in which I’d done all my previous work on this task and which is annoying but usable, and F#, which I had never tried up to that point. I chose F#.

Why?

The big reason is that I wanted to eliminate as much boilerplate as possible in order to get the project done faster. C# is steadfastly object-oriented, and requires a lot of things that are great for enormous projects with massive code-bases, but are utterly useless when trying to do something small and streamlined, and end up just costing time in such cases. F# is functional and dynamic, and it allows, but does not force, all of of that infrastructural stuff that in a case like this is unnecessary. Plus, I’d never used it before, which made it inherently more interesting.

Ultimately, I did not finish the project. At 2:00 in the afternoon on August 19th, I realized that I didn’t know enough about using .Net in order to develop, in the two hours I had left, a method of storing code objects between executions of the program, which I needed in order to allow command line arguments (all String objects) to interact in any sophisticated manner with the underlying code base (which uses Strings sparingly where it uses them at all). Since doing that was the whole point of the project, I decided that was pretty much the end. But this is not a failure, except in a sort of surfacey kind of way. True, I didn’t finish the project, but I am still very proud of what I was able to accomplish in the time I had available:

• I learned F#, from zero prior knowledge to not perfect but useful fluency, in about three hours.
• I navigated “DLL Hell”, the process of dynamically loading libraries in dependency order with no outside knowledge of what those libraries are or what (or where on the system) their dependencies are likely to be; from zero knowledge to a solved problem in four days.
• I solved the problem (introduced by having to navigate DLL Hell) of having to open and examine every code library (file whose extension is “.dll”) on the system before being able to actually run the program, in my sleep over the weekend. Wrote it Monday, debugged it Tuesday morning.
• I built, on the first try, a repl that required four additional lines of code to convert it to a single-run execution if it was given command-line arguments.
• I figured out, within a few minutes of discovering it, how to solve the object permanence problem. Just didn’t have enough time to actually write and debug the code.
• I introduced my protege to the best tacos in town. From zero prior knowledge to eating beef tongue in the time it took to say, “I’ll have a taco with lengua please.”

That now established, here are my thoughts on what it was like to learn and use F#. For reference, you can try most of what I show you (except the bits that aren’t F#) at tryfs.net.

First things first, if I had to go back and make that choice again, whether to use C# or learn and use F# for a project that had a tight deadline, I would choose the latter every time. It wouldn’t even be a question. I found F# to be that nice to work with over its object-oriented compatriot.

The big easy differences can be got out of the way fairly quickly: F# is intended to be a primarily functional language, and its styling follows in the footsteps of Haskell:

let foo arg = 
    doThingsWith arg        \\doThingsWith isn't defined, so this 
                            \\will throw an error, but it still 
                            \\conceptually illustrates the point.

There are none of the standard block delimiters that you would expect if coming from one of the big players in the OO landscape. No moustache brackets, no semicolons.

let foo arg =
    doThingsWith arg
    andReturn value

Indentation replaces brackets and semicolons as the primary delimiter of block code. On the one hand, this is quite nice; not having to use Shift+[ and Shift+] constantly to tell the compiler where a given statement begins and ends does make writing the code both fast and convenient. On the other hand, Visual Studio hasn’t caught up with F#’s indentation requirements, so writing code is easy (new lines are automatically indented to the same point as the previous line) but debugging and refactoring it is tedious, because once you’ve written the code VS makes absolutely no effort to help you keep it all together when you change it. There’s a lot of individually adding and removing spaces on collections of lines as you change the names of variables, or switch from IF statements to MATCH blocks. The task of writing and maintaining F# code in Visual Studio would benefit a lot from something like emacs’ agressive-indent plugin.

F#’s predicate support is much more sophisticated, in fact it is as a language much more flexible, than its closest sibling, C#. In my experience, this comes down mostly to two things: anonymous functions and functions-as-data. Here’s what those things mean in practice:

In F# you can use the keyword “fun”, and in any language that supports anonymous functions, you can find something similar. In Lisp it’s “lambda”. In Clojure, it’s “fn”. In Haskell it’s “\”. In all cases, it begins a function definition:

fun x -> x + x        \\F#
(lambda (x) (+ x x))  \\Lisp
(fn (x) (+ x x))      \\Clojure
\x -> x + x           \\Haskell

The first thing you might notice about this function is that it doesn’t have a name:

let foo x = x + x
       \\is equal to
fun x -> x + x

This means there’s no way to reference it outside its own context

(fun x -> x + x) 5    \\This returns 10;
                      \\but after that line,
x                     \\x no longer has any meaning
--error!              \\and we have no way to reference "fun x" 
                      \\except by defining it again

We can reference it by returning it from a function and storing it in a variable:

let foo () = 
    fun x -> x + x           \\foo returns our fun x
let double = foo ()          \\doubler now = (fun x -> x + x)
let tenTimesTwo = double 10
tenTimesTwo                  \\20

This has some powerful applications. I can recreate an example from Paul Graham’s ANSI Common Lisp:

let makeAdder n = 
    fun x -> x + n
let addTen = makeAdder 10
addTen 5                    \\Returns 15

Or I can define small one-off functions on the fly without having to find a specific place or name for them in my code base. In fact, F# assumes you will do this, and all of its most powerful functions for dealing with data collections (sets, lists, arrays, maps, etc) take as one of their arguments another function:

let arr = [|1; 2; 3; 4; 5; 6|]         \\"[|" and "|]" are F#'s
                                       \\Array literal
let res = Array.map (fun x -> x + 5) arr
res                                    \\res = 
                                       \\[|6; 7; 8; 9; 10; 11|]

C# can only sort of do this. Some methods take predicates:

\\C# Code Here
var arr = {1, 2, 3, 4, 5, 6}
var res = arr.ForEach( element -> element + 5 )
                 \\adds 5 to each element of arr

But a C# predicate complains when you ask it to handle multi-line operations. In F#, a function defined anonymously has all of the same attributes and all the same flexibility as a function defined with a name, and as a result nested function definitions are perfectly normal in F#:

let foo args =                         //Code block begins here
    Array.map (fun x -> let y = x + 2    //Nested code block here
                        let z = x * x
                        z + y) args      //Both blocks end here

This is very powerful, and allows a much more exploratory style of coding. I generally go through it like this; I have an array of things. I need a given result from all of them, so I’ll start with “Array.map (fun x ->” and build “fun x” as a quick, rough draft. Once it works, I’ll go back and trim out unneeded bits (excessive variable declarations, for example) until I can get it down to as small as it will go. Between iterations, I can test and everything works just like it would if I were defining a top-level named function. The only difference the anonymous function that I write for the sake of a single “Array.map” statement and the named function that I define somewhere and use all over the place is that one has a name and the other does not. Otherwise, anything one can do, the other does just as well.

Parentheses function in F# almost exactly the way they do in math, with a few extras. Where the compiler can’t tell by reading how you use it what type a given variable is supposed to have, it will ask you to add a type annotation wherever the variable is defined:

let foo (args : Int[]) = 
    Array.map etc.

But the compiler is a little weird; it’s listed as a “single-pass left-to-right” compiler, and I don’t know what that means in theory. In practice it means that the function written above requires a type annotation, but if I write the same function using a pipe “|>” operator, which simply takes a value and passes it forward into a function, it no longer needs annotating:

let foo args =
    args |> Array.map etc.

All I’ve done is changed the order of statements, but now the compiler can infer what args is supposed to be, and even though it seems like it should be obvious, I don’t understand mechanically what makes the difference. Nevertheless, it encourages liberal use of pipe operators in order to avoid having to annotate everything you do, so F# code tends to start looking like this:

let foo args =
    args
        |> bar
        |> List.map baz
        |> bong
        |> bing
        |> cherries?

Where each of the statements preceded by a “|>” operator corresponds to a function defined elsewhere that takes one argument and returns a value. Each of the succeeding values is passed forward into the next function, and the final value, being the result of the final statement in the code block, is returned as the value of the function foo.

I like this a lot. A LOT. Laid out as it is above, it’s very easy to see at a glance what’s going on, and the order in which things happen, and what the return value is supposed to be, and building functions like that is intuitively easy because you can just do things in the order they are supposed to be done in. For reference, here’s the same function in Common Lisp:

(defun foo (args)
  (cherries? (bing (bong (mapcar #'baz (bar args))))))

It’s inside out. Now, Lisp has a lot of advantages, and can do things that would make F# cry in a corner, and many of those advantages come directly from that awkward-seeming structure, but there’s really no way around the fact that in Lisp you kind of write programs backwards. It’s nice to be able to write things in the order in which you think of them, without having to do a lot of variable declarations in between, as you would in order to pull off the same chain in C# or Java.

There is a lot to like about F#. It works hard in a lot of ways to help you write clean, concise, readable code, and many of those ways are the result of some very clever choices in the language design. But ultimately, F# is a .Net language, and the reason I used it was because I wanted an alternative to C# for interacting with .Net libraries. And here, it kind of falls down a bit. The thing is, F# has its own .Net codebase, but the vast majority of .Net is C#, and what isn’t C# is mostly C++ and Visual Basic, and what little is left of that is split between the remaining .Net languages (F# and…J#? I think? I really don’t know what J# is, so don’t quote me there.)

As a result, the whole time I was enjoying F#’s cool features, I was also limited in their use because every few lines I’d have to interact with wider .Net code, which doesn’t allow all that cool stuff that F# can do. Most importantly, pipes. For example, if I need to find the index of a given array element in F#, there are two ways I can do it:

\\given Array arr and Element elem:
let index = arr |> Array.findIndex (fun x -> x = elem)
\\or
let index = Array.IndexOf(arr, elem)

Neither of those is particularly pleasing. In the first case, I can’t just say “Array.findIndex elem” because findIndex expects a boolean function and will barf if you just feed it a random object; so I have to define that particular “fun x” every time I want to use it, which is a lot of typing for such a seemingly simple operation. In the second case, Array.IndexOf is not an F# function and as a result it doesn’t support piping; so it doesn’t fit into long function chains, and in many cases necessitates its own variable declaration, which wastes time, memory, and keystrokes. It also clutters up my code with surplus parentheses (“Array.IndexOf arr elem” throws an error), and it doesn’t look right surrounded by piped function calls.

Thus, F# shows its immaturity, and until that changes and you can treat all .Net code like F# code, F# code will almost always end up full of inconsistencies: sometimes you can pipe and sometimes you can’t. Sometimes you can call a function and separate it and its args with whitespace, sometimes you have to use parens and commas. Even some of the native F# functions (Array.get, for example) take arguments in the wrong order, so that piping the way you want to do it doesn’t work. That is really, really annoying, because it makes your code look sloppier than it is, and there’s absolutely nothing that you can do about it.

Don’t let that put you off the language, though. Even if you’re writing half F# and half C#, the F# half will (in my admittedly limited experience) be so much easier and faster to write that you’ll almost instantly make up the time spent learning how and when to switch back and forth between the two. As I said earlier, given the choice to go back and do it differently, I’d use F# every time, no question. It was that nice. Cheers!

PS: Note that although F# is intended as a primarily functional language, it does have a fully functioning Object-Oriented infrastructure, so you can define and use classes (types) in almost the same way you’re used to with C#. Note also that method overloading only works in OO F#; a minor annoyance that I soon forgot about because there’s no overloading in Lisp either, and I’m used to writing Lisp.

PPS: I need a job. I’m good at things. Hire me? :)