What Color is Your Function?

http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/

I don’t know about you, but nothing gets me going in the morning quite like a good old fashioned programming language rant. It stirs the blood to see someone skewer one of those “blub” languages the plebians use, muddling through their day with it between furtive visits to StackOverflow.

(Meanwhile, you and I, only use the most enlightened of languages. Chisel-sharp tools designed for the manicured hands of expert craftspersons such as ourselves.)

  1. Every function has a color. Each function—anonymous callback or regular named one—is either red or blue.
  2. There are no colorless functions in the language.
  3. The way you call a function depends on its color.
  4. You can only call a red function from within another red function. You can call a blue function from a red one.
  5. Red functions are more painful to call.

The obvious solution then is to never use red functions. Just make everything blue and you’re back to the sane world where all functions have the same color, which is equivalent to them all having no color, which is equivalent to our language not being entirely stupid.

One final thorn in our side: some core library functions are red.

Of course, I’m not really talking about color here, am I? It’s an allegory, a literary trick. The Sneetches isn’t about stars on bellies, it’s about race. By now, you may have an inkling of what color actually represents. If not, here’s the big reveal: Red functions are asynchronous ones.

When people talk about “callback hell” they’re talking about how annoying it is to have red functions in their language.

One technique that gets a bunch of people excited is promises, which you may also know by their rapper name “futures”.

Async-await solves annoying rule #4: they make red functions not much worse to call than blue ones. It is better. I will take async-await over bare callbacks or futures any day of the week.

If you peel back your compiler’s skull and see what it’s doing when it hits an await call you’d see it actually doing the CPS-transform. That’s why you need to use await in C#: it’s a clue to the compiler to say, “break the function in half here”. Everything after the await gets hoisted into a new function that it synthesizes on your behalf.

With callbacks, promises, async-await, and generators, you ultimately end up taking your asynchronous function and smearing it out into a bunch of closures that live over in the heap. Your function passes the outermost one into the runtime. When the event loop or IO operation is done, it invokes that function and you pick up where you left off. But that means everything above you also has to return. You still have to unwind the whole stack. This is where the “red functions can only be called by red functions” rule comes from. You have to closurify the entire callstack all the way back to main() or the event handler.

But if you have threads (green- or OS-level), you don’t need to do that. You can just suspend the entire thread and hop straight back to the OS or event loop without having to return from all of those functions.

  • This is a great analogy.

  • async/await creates a state machine that creates the illusion of “pausing” execution at the await invocation. Actually, the function is split in two (pre-await, post-await), the pre-await bit is called, the awaited operation is triggered, and the post-await bit is scheduled to run when the awaited operation is done, and the event loop keeps going.

  • This sounds a lot like go blocks in Clojure’s core.async:

    go is a macro that takes its body and examines it for any channel operations. It will turn the body into a state machine. Upon reaching any blocking operation, the state machine will be ‘parked’ and the actual thread of control will be released. This approach is similar to that used in C# async. When the blocking operation completes, the code will be resumed (on a thread-pool thread, or the sole thread in a JS VM). In this way the inversion of control that normally leaks into the program itself with event/callback systems is encapsulated by the mechanism, and you are left with straightforward sequential code. It will also provide the illusion of threads, and more important, separable sequential subsystems, to ClojureScript.

  • Does this imply that async/await is simply not possible in non-event loop models, like Java? This is possibly why Clojure go blocks are restricted to thread pools in Java; this gives the system a next thing to work on while a thread-of-execution is paused on a blocking operation.

Edit