On the benefits of type-checking…

duck-typing

I’ve been writing a lot of code, both front-end and back-end, for my current project, and lately I’ve had to do a lot of data-munging—processing legacy data from a badly designed PostgreSQL database into a—better designed, I hope—Google FireStore noSQL database with lots of attendant issues.

The front-end code is 100% Typescript, the back-end code is 100% Javascript. So I’ve had a lot of opportunity to see the pros and cons of the. two “languages” side-by-side.

  1. The single biggest thing that catches errors is probably Google Firestore throwing exceptions if you try to store an object with any property that is undefined. Seriously. We actually looked at setting the option to have undefined properties simply be ignored but no, it’s better to have it. It’s a great feature.
  2. Typescript definitely helps catch some errors and especially when the debug-loop is necessarily long (right now we’re having issues with both the Firebase Emulator and deploying Google Cloud Functions which means that debugging a Cloud Function has a 5-10 minute debug loop—when you aren’t getting quick turnaround, having a persnickety language is definitely a win).
  3. Typescript is virtually useless in handling errors involving objects from services. If a service gives you something that’s not right, Typescript just eats shit. It made me wish I was coding in Javascript using type-by-example.
    E.g. Google Firestore uses string ids for everything. These are long random base36 numbers if you let Firestore pick them for you, and we use Javascript’s crypto functions to generate our own. Our legacy data has sequential integer ids. If you try to load or store a record in Firestore with an integer id, it throws an exception. But a foreignKey can be a number, no problem. So we ended up with a bewildering array of type mismatches that Typescript just didn’t react to at all. This led to annoying stuff like having to convert foreign keys to strings in case they came from a botched migration…

This reminds me of why I wrote type-by-example in the first place. We were working on a project at Uber ATG (their former dysfunctional self-driving car group) and we had this pretty complex app—100% Typescript—that was throwing incomprehensible errors. Thing is, we had lots of type declarations, but Typescript is very bad at being a “single source of truth” for types.

  • Typescript doesn’t exist at runtime.
  • It’s easy to end up with multiple parallel declarations that rely on the fact that it’s all just duck-typing.
  • Typescript is only as fine-grained as Javascript. We were using Google’s protobuf as our transport layer which meant the slightest type errors blew up everything, and some of those errors involved being more specific about numeric types than Typescript is capable of being.
const Point2DType = {x: 0, y: 0} 
const ArrayOfStringsAndNumbersType = ['hello', 17]

A type-by-example “type” is—95% of the time—just an object like the thing you want. So one value can be both a type and a mock, and be available at runtime.

More complex and specific types are represented by strings:

const GeolocationType = {
  lat: '#number [-90,90],              // between -90 and 90 inclusive
  long: '#number [-180,180],           // between -180 and 180 inclusive
  'altitude?': 0                       // (optional) number
}

// a wall is an array of posts and ads...
const FacebookWallType = [
  {
    type: 'post',
    'videoUrl?': 'https://fb.com/video.mp4',
    'imageUrl?': 'https://fb.com/happysnap.jpg',
    message: 'string',
    likes: [{name: 'bob'}]
  },
  {
    type: 'ad',
    imgUrl: 'https://fb.com/ad.png',
    callToAction: 'Greedy? Stupid? Click here before it’s too late!',
    href: 'https://fb.com/landing-page'
  }
]

const ConfigType = {
  '#is[A-Z]\\w+': true,                 // isFoo is boolean
  '#\\w+Id': '#regexp [0-9a-z]{14,20}', // foreign keys are 14-20 base 36
  '#\\w+Count': '#int [0',              // fooCount is a non-negative int
  '#\\w+': '#any',                      // other props are ¯\_(ツ)_/¯
}

I wrote a precursor to type-by-example (based on the byExample library I had written for b8rjs) so that we could assert what we thought we should be getting at the point-of-failure and get human-readable failure messages. It took a few hours to write the code, and it found the bug instantly. Not only is type-by-example pure Javascript, its core types are serializable and thus can allow services to be self-documenting, it is complemented by filter-shapes which allows you to use the same “type” objects as “shape filters” to reduce data to a shape (e.g. the way GraphQL does, but with far less implementation overhead), and its core types include lots of special number types (which is super helpful with protobuf which has a cornucopia of numeric types, like uint8, which Typescript simply cannot comprehend, but with is simply ‘#int [0,256)’ in type-by-example).

Anyway, I can’t help thinking that Typescript would be saving me a little bit of time chasing down bugs in the back-end code, and I think thanks to working entirely in TypeScript for my hobby projects over the last year, it’s no longer the productivity hole on the front-end that it once was (this in no small part due to near instantaneous transpilation—typically 50-500ms) but that if I’d been using type-by-example I’d just be ahead across the board.