The Great CoffeeScript to Typescript Migration of 2017

https://dropbox.tech/frontend/the-great-coffeescript-to-typescript-migration-of-2017

  • Very detailed & specific overview of Dropbox’s switch from CoffeeScript to Typescript, that mirrors Heap’s in many ways.
  • They hit a roadblock with manual conversion (sheer volume, no one wants to work on this stuff full-time), and halted the project indefinitely.
  • Once decaffeinate became more mature, they used that across the board (to convert CS to ES6), and wrote an in-house ES6 -> TS converter to run subsequently.
  • ES6 is valid Typescript (so theoretically this converter is unnecessary); here’s why they needed it:
    • Convert AMD imports to ES6 module imports, including destructures. Ditto for exports.
    • Add type signatures (use any).
    • Convert all this. references inside a class to instance variables.
    • Some special-casing for React.

Introduction

In 2012, we were still a fairly scrappy startup of about 150 employees. The state-of-the-art in the browser was jQuery and ES5. HTML5 was still another two years away and ES6 was another three. As JavaScript itself appeared to be stagnant, we were seeking a more modern approach to web development.

At the time, the codebase consisted of about 100,000 lines of JavaScript. This was shipped as a single bundle by concatenating each file in a pre-specified order. While many of the company’s engineers touched this code, there were fewer than 10 who were working full-time on the web.

Simultaneously, our CoffeeScript codebase was proving difficult to maintain. As CoffeeScript (and vanilla JavaScript) are both untyped, it was very easy to break something without intending to.

Sensing the growing distaste for the language, in November 2015 we ran a survey of frontend engineers at Dropbox and discovered that only 15% thought we should stay on CoffeeScript, and 62% felt we should ditch

Migration

As an engineering organization, we had decided to set aside 20% of product team’s time for the year for “foundational work” and we thought that part of that blank check would be applied to this project.

we implemented a hard ban on new CoffeeScript files in our codebase. This didn’t stop people from editing existing CoffeeScript—as there was plenty of that around—but it did force most engineers to start learning TypeScript. The way we initially implemented this ban was to write a test that walked the codebase, found all the .coffee files, and asserted all the files that were found were on a whitelist. This list was populated with the paths of .coffee files that existed when the test was written. Our code review tools enabled us to require a Web Platform engineer review any changes to this test file.

In June, the TypeScript migration was postponed indefinitely. While it was still going to happen, there was no ETA for when it would actually be completed.

In hindsight this decision seems to have been inevitable given our initial migration strategy. Assuming a rate of roughly 1000 lines of code converted per engineering day (including testing and code review), it would have taken a year of one engineer’s time to complete the migration. That rate was actually very optimistic, as the actual reported rate of progress was more like 100 lines per day, which would have taken closer to 10 engineering years, or a full year of 10 engineers time.

Even if we split the difference and called it 3-5 engineer years, it was absurd to expect anyone to want this as their full time job for even a month or two.

To properly migrate our codebase we needed a multistep pipeline approach for any given file. First we ran decaffeinate to generate valid ES6. This code was untyped and even included pre-JSX React. Then we ran this ES6 through a custom ES6 to TypeScript Converter.

We learned that when writing code translators, you have to be rigorous, covering every corner case.

After the Fact

In the end, the auto-migration process took just about two months, with three engineers working on it and about 19 engineer-weeks spent; significantly better than the original 10 engineer-year estimate. Granted, the output was not idiomatic TypeScript, as most people had originally aimed for, but rather some very messy, any-laden TypeScript. This trade-off was worth it.

Edit