Sign in
Log inSign up
How we migrated a 200K+ LOC project to TypeScript and survived to tell the story

How we migrated a 200K+ LOC project to TypeScript and survived to tell the story

Kleo Petrov's photo
Kleo Petrov
·Feb 10, 2017

Statically-typed JavaScript exploded in use in 2016. More and more companies are leveraging the vast potential of types in JavaScript. Two major players were recognized in the field of typed JavaScript - TypeScript, and Flow. They have some differences, but try to solve the same problem - providing better organization, refactoring power, and confidence to all JavaScript developers.

Coherent Labs and TypeScript

We started exploring typed JavaScript last year when the project I currently work on - an HTML game UI Visual Editor, written mainly in JavaScript, passed 100K lines of code. With the high number of features we introduced, the code base became harder to maintain and refactor. That's why we decided to research and choose a possible solution to the arising problem.

After some time, we arrived at two choices - TypeScript and Flow. Both would give us static types for better control over the code, but that wasn't enough for us. We wanted to use the full power of ES6 and beyond. Flow is just a static type checker, which means that we would have to use a transpiler for all our ES6 code but with TypeScript, being a superset of JavaScript, we would have it all in one place - static types and support for most of the latest ECMAScript features.

That said, it wasn't hard to make the final decision - TypeScript it was!

The rise of the statically typed JavaScript

Let's take a step back and see what static types are and how they fit in the dynamic world of JavaScript.

The main difference between statically typed and dynamically typed languages are their way of performing type checking. The first does it at compile time, while the second performs the type checks at runtime. In other words statically typed languages require you to declare the data type before you use it, while dynamically typed languages do not.

JavaScript, being a dynamic language, allows us to use two (or more) different data types in assignments or statements. For developers coming from statically typed languages like Java or C++, this looks counter-intuitive and strange in most cases. This is basically a big headache in large enterprise projects. Developers often end up spending hours searching for "why my variable is NaN."

TypeScript and Flow were created to solve this problem. Flow, maintained by Facebook, gained popularity in last two years, but provides only the type checking. Nothing more, nothing less. TypeScript, on the other hand, is a superset of JavaScript. This means it not only gives you type checking but extends the language with future ES6 and ES7 features. They both have their compilers, which transforms the code to pure JavaScript, which can be used in every browser.

There are several benefits of using typed JavaScript:

  • It can make your code more readable
  • Can help you catch errors early
  • Makes room for more reliable refactoring
  • Better IDE support

Also, when deciding for or against typed JavaScript solution ask yourself the following questions:

  • Is the project critical for your business?
  • Does your project have many complex and moving parts?
  • Is there a chance you are going to refactor the project to suit your needs?
  • Do you have a large team, where the developers need to understand and learn the code base quickly?

If most of your answers are "yes," then you should consider migrating your project to TypeScript or Flow.

The Eternal Battle of the Giants - Typescript, and Flow

Like I said earlier, both Flow and Typescript introduce a compelling and in my opinion much-needed feature - a type system.

Typescript appeared first in 2012 when Anders Hejlsbeg first released it, and it's under the umbrella of Microsoft. It adds optional type annotations and support for a list of ES6 and ES7 features. It has its compiler, which checks and removes the annotations and outputs the generated JavaScript code.

Flow is developed and maintained by Facebook. The idea of Flow is to be a static type checker, designed to find errors in JavaScript applications quickly. It's not a compiler, but a checker. It can work without any type annotations, and it's excellent at inferring types.

Let's take a look at a simple example. We have the following function.

function greet = function(name) {
   return `Hello, ${name}!`;
}

In both TypeScript and Flow, the function will look something like this:

function greet(name: string): string {
   return `Hello, ${name}!`;
}

The words after the function's parameter are how we define a type in TypeScript and Flow. Types can represent primitive and nonprimitive data types. That means, that if we try to use an argument, different from the one specified, we will get an error. Let's take our greet function for example:

const message = greet(43);

We will call it using a number argument. When we compile the code, the following TypeScript error will occur:

Argument of type 'number' is not assignable to parameter of type 'string.'

Catching these kinds of errors in compile time can greatly benefit your overall productivity and sanity. Combine it with the ES6 and ES7 tools TypeScript provides like arrow functions, import/export, generators, async/await, classes, etc., and you are all set for an excellent JavaScript development environment.

So, let me outline how we finally migrated our existing 200K+ LOC project to TypeScript.

First step - the mindset

Our project was rapidly growing, and with that, the code base grew exponentially. At that time, TypeScript and static type JavaScript was gaining popularity, and we saw an opportunity we couldn't miss. After some research and a couple of meetings, we stopped at two potential solutions:

  • Use Flow and Babel for ES6 transpiling
  • Use TypeScript and its compiler for ES6/ES7 compilation

We chose TypeScript for several reasons:

  • TypeScript, at that time, had a wider community and the resources were easier to find.
  • We already had a build process that was complicated enough, and introducing two more (Flow compilation and Babel transpile) would slow us down even more. With TypeScript, we had to worry about only one.
  • TypeScript had better IDE support than Flow. Our whole team is using WebStorm, which provides excellent TypeScript support.
  • ES5 support - our core is built on WebKit and JavaScriptCore with some limitations in mind. Some of the standard ES5 features are not supported. So, the code compilation had a major part in our final choice.

After six months of using it, I can candidly say that choosing TypeScript was the right decision. The whole process of migrating our project to TypeScript had some rough parts, but nothing too difficult.

Second step - the beginning

After a long migration discussion with the whole team, we've decided to start with a small module and rewrote it. A great thing about TypeScript is that you can just rename your .js file to .ts and you are ready to go.

So, we set up our build tool (Grunt) using grunt-ts, and after a day and a half, we had our first TypeScript module done. Our biggest struggle back then was defining our global and external modules. TypeScript has a strict policy when it comes to types. As a result the TypeScript compiler wouldn't recognize our helper libraries like jQuery and Kendo UI and their methods. The result was over 200 compilation errors, stating that TypeScript couldn't find variable of type "$". The solution was pretty straightforward - we added declaration statements like:

declare var $;
declare var kendo;
...

The declare keyword is used for ambient declarations where you want to define a variable that may not have originated from a TypeScript file. For that time being, this was a plausible solution to our problem.

At this point we had migrated our first module successfully to TypeScript, and were ready to get our hands dirty & migrate the whole project to TypeScript.

Third step - the migration

Our next step was to migrate all other modules to TypeScript. Some of them were highly coupled, and we had to think of a way to quickly make them work. After some additional research and brainstorming for a couple of hours, we came up with a rather simple strategy.

First, we would rename all our files to '.ts' and watch our project burn in its compilation errors. Then we would add the missing TypeScript declarations and 'any' types. TypeScript’s “any” data type allows you to create and assigns different data types to variables without worrying about a particular type. Such as we declare a variable which can hold the various data types at the same time, such is a string, number, boolean, etc.

After two days fighting with types and declarations, we had our first full TypeScript compilation. But we still had much more work to do. We had to provide better checking, add proper type declarations for our external libraries and refactor our old ES5 code to ES6.

Declaration files describe the shape of a library. By using them (also called .d.ts files), you can avoid misusing libraries and get things like completions in your editor. With TypeScript 1.8, to add a new declaration file, you had to install a separate npm package globally search the declaration files for the specific library and store all library definition files in a folder called 'typings'. It was not the most elegant solution, to be honest. TypeScript 2.0 introduced a new way of managing library declarations - using the npm registry. You just need to type:

npm install --save @types/jquery

and import it:

import * as $ from 'jquery'

and you are done. No more tedious task of managing different files and folders. External libraries - check!

Our next step was to change all 'any' declarations to proper types. It would have taken days to complete it. So, we decided to start slow and change only the part of the code we are working on. This would include refactoring the particular code to ES6 and changing its types. After a couple of weeks, we had incrementally migrated 80% of the code to ES6, with meaningful type annotations. We could move to our next milestone - testing.

Testing with Typescript

We had a close to perfect development environment and were ready to move to the next big step for the project - E2E testing. We decided to stick with JavaScript. Our arsenal - Mocha, Chai, Selenium and Selenium's WebDriver. We isolated the testing code - our Selenium framework, the tests, and all dependencies by proving it with a separate tsconfig.json.

The presence of a tsconfig.json file in a directory indicates that the directory is the root of a TypeScript project. The tsconfig.json file specifies the root files and the compiler options required to compile the project.

We had to create one tsconfig for the project itself, and refactor the Grunt task to target the different configuration files correctly. Now we were ready to start working on our Selenium framework.

The structure of the framework consists of 7 major modules and five helpers modules:

  • A driver handler, which starts, stops and timeouts Selenium.
  • A selector module, consisting all elements we use in the tests
  • Main actions file, which is responsible for providing API for targeting different parts of the editor and performing interactions like clicking, selecting, writing, etc.
  • A module for exporting and deleting assets
  • A logger
  • A screenshot module, which would take a screenshot after every failed test
  • A module for mocking our backend API

The helper modules are separated by a test category. We have a JavaScript execution module, properties panel module, timeline panel module, and so on. They all have methods for making specific tasks.

We've also added a reported - mochawesome, which outputs a complete report of all tests after their execution.

We were also able to leverage the power of async/await using TypeScript, and as a result we ended up with cleaner and better-organized code.

Future plans

There is much more ahead of us. Migrating to TypeScript was only a drop in the ocean. Some of our plans include:

  • Refactor the whole architecture of the project
  • Add various features like the 3D selection and bounding box, visual data bind connections between elements and much more.
  • Improve the total execution time of our tests, which is currently around 30 min for 440 tests.

We are also experimenting with virtual DOM libraries like React and InfernoJS. They both have strong TypeScript support, and our project can significantly benefit from their performance boost over regular DOM manipulation techniques.

Apart from that, we are carefully looking at how TypeScript is moving. The latest releases introduce features like mapped types, Object spread and rest, downlevel async functions for ES3 and ES5 and their soon to be released features like generator support for ES3/ES5, generic defaults, and asynchronous iterators.

Conclusion

The decision to use TypeScript for our project has given us numerous benefits - improved productivity, better control over the code, and ES6 features to name a few. I was a bit skeptical initially about TypeScript, but after using it I can say it's what I had been missing for such a long time.

Although I strongly suggest trying and playing with it, you should consider some of its tradeoffs such as the time you need to invest in migrating the codebase, the project scope and future refactoring factors and most importantly your team's vision/opinion about TypeScript.

That being said, I still believe that TypeScript has its place in the JavaScript world and its influence will lead to better things for all of us.

Feel free to let me know what you think in the comments below!

1