Crafting “Crafting Interpreters”

http://journal.stuffwithstuff.com/2020/04/05/crafting-crafting-interpreters/

At some point, I realized that dropping readers straight into C was too unfriendly of an introduction. It’s hard to teach high-level concepts like parsing and name resolution while also tracking pointers and managing memory. OK, so we’ll build two interpreters. First a simple one in a high-level language, focused on concepts. Then a second bytecode VM in C to focus on performance and low-level implementation techniques.

So I wrote a separate script that instead of building the book, builds programs. For each chapter, it collects all of the snippets that appear in that chapter and the previous ones and writes them out to separate source files. In other words, it produces a separate interpreter, one for each chapter, containing only the code that readers have seen so far.

I made this problem harder for myself because of the meta-goal I had. One reason I didn’t get into languages until later in my career was because I was intimidated by the reputation compilers have as being only for hardcore computer science wizard types. I’m a college dropout, so I felt I wasn’t smart enough, or at least wasn’t educated enough to hack it. Eventually I discovered that those barriers existed only in my mind and that anyone can learn this. My main overarching goal of the book is to pass on that feeling, to get readers to understand there’s no magic in there and nothing keeping them out. To nail that conceit, I wanted to include every single line of code used by the interpreters in the book. No parser generators, nothing left as an exercise for the reader. If you type in all of the code in the book, you get two complete, working interpreters. No tricks.

I spent several weeks sketching out potential lists of chapters, sprinkling snippet markers throughout the code, and seeing if the result built. I’d get a compile error because a snippet in an early chapter tried to call a function in some later chapter and I would have to go back and reorganize things. I hand-drew dependency graphs between language features and tried to untangle them. Here’s an example of what this was like:

  1. To teach functions I want show that recursion works.
  2. But to have recursive functions I need control flow. Otherwise, every recursive function recurses infinitely without a base case. So control flow has to come before functions.
  3. For control flow, I need side effects so that I can show that a certain code path is not taken. The obvious way to do side effects is to have a print() function that displays output.
  4. But I don’t have functions yet. That’s a cycle. Crap.

I got through these four years and kept writing, but I paid a price. When I read the earlier chapters, they have a whimsy and light-heartedness that later chapters lack. We’re all going through dark times, and I don’t feel light. The past few years left a mark on me, and that mark shows up in the book. I miss the goofier person I used to be, sometimes. But I’d like to believe that maybe the person I am now is a little more honest. Maybe some of those jokes were a mask.

Sounds like a lot of work went into this book, and into making sure that it contains a full working interpreter. Should be worth a read , at the very least!

Look Up

  • Art of the meta object protocol
  • SICP
  • Writing an interpreter in go
  • Metaclasses
Edit