D&D5e, Web-components, & Performance

tl;dr — web-components (edit: with shadowRoots) are kind of slow.

The graph shows the average of five trials (post warmup) in Chrome on my Macbook Pro (it’s a maxed-out 2015 model). I saw nothing to suggest that a larger number of trials would significantly change the results.

Over the long weekend I was hoping to play some D&D with my kids (it never ended up happening). One of the things I find pretty frustrating with the new (well, 2014) Players Handbook is that the spells are all mixed together in strict alphabetical order. (It’s not like earlier editions were better organized. It’s also not clear how they could really fix this, since any arrangement will either require repeating spell descriptions or some kind of tedium.) So to figure out what spells are available to, say, a first level Druid, and what they do, you need to do a lot of page-flipping and cross-referencing.

As an aside, in the late-lamented DragonQuest (SPI, 1980), magic is divided up into “colleges” with a given character having access to the spells of only one college. Each college, along with its spells, is described in one place, with the only page-flipping being required for a relatively small number of spells that are common between colleges. A typical college has access to 20-30 different spells which are thematically unified (e.g. Adepts of the College of Fire have fire-related spells).

This kind of organization is not possible for D&D because the overlap between classes is so great. Sorcerors essentially get a subset of Wizard spells, as do Bards. And so on.

So it struck me that some kind of easily filterable list of spells which let you decide how you wanted them sorted and what you wanted shown would be handy, and this seemed like a nice test for b8r’s web-component system.

Now, b8r is the third rewrite of a library I originally developed with former colleagues for USPTO. Initially we had a highly convoluted “data-table” (not an original name, of course) that essentially let you point a descriptive data-structure at an array and get a table with all kinds of useful bells and whistles. This quickly became unmanageable, and was particularly horrible when people started using it to create quite simple tables (e.g. essentially short lists of key/value pairs) and then add fancy behaviors (such as drag-reordering) to them.

So, I developed a much simpler binding system (“bind-o-matic”) that allowed you to do fine-grained bindings but didn’t try to do bells and whistles, and it quickly proved itself better even for the fancy tables. b8r‘s data-binding is the direct descendant of bind-o-matic.

Anyway, all-purpose tables are hard. So I looked around for something off-the-shelf that would just work. (I’m trying to purge myself of “not invented here” syndrome, and one of the joys of web-components is that they’re supposed to be completely interoperable, right?) Anyway, the closest thing I could find (and damn if I could find a compelling online demo of any of them) had been abandoned, while the next closest thing was “inspired by” the abandoned thing. Not encouraging.

So I decided to write a minimal conceptual set of custom elements — in essence placeholder replacements for <table> <tr> <td> and <th> — which would have zero functionality (they’d just be divs or spans with no behavior) that I could later add CSS and other functionality to in order to get desired results (e.g. the ability to switch a view from a virtual grid of “cards” to a virtual table with a non-scrolling header and resizable columns — i.e. “the dream” of data-table implementations, right?

Now, I have implemented a function named makeWebComponent that will make you a custom element with one line of code, e.g. makeWebComponent('foo-bar', {}). So I made a bunch of components exactly that way and was horrified to see that my test page, which had been loading in ~1000ms was suddenly taking a lot longer. (And, at the time it was loading b8r from github and loading the spell list from dnd5eapi.co and then making an extra call for each of the 300-odd spells in the list.)

So, I ended up implementing my interactive D&D spell list using old school b8r code and got on with my life.

Hunting down the problem

I’m pretty enthusiastic about web-components, so this performance issue was a bit of a shock. So I went to the bindinator benchmark page (which lets me compare the performance of b8r to special-purpose vanilla javascript code doing the same thing with foreknowledge of the task.

In other words, how fast is b8r compared with javascript hand-coded to do the exact same thing in a completely specific way? (An example of how unfair the comparison can be is swapping two items in a large array and then updating the DOM vs. simply moving the two items.)

So the graph at the top of the page compares creating a table of 10,000 rows with a bunch of cells in each row using vanilla js to b8r, and then the other three columns show b8r with two simple <a> tags replaced with <simple-span>, where <simple-span> is defined using makeWebComponent vs. hand-coded (but still with a style node) vs. hand-coded (with nothing except a slot to contain its children).

class SimpleSpan extends HTMLElement {
  constructor() {
    super();

    // const style = document.createElement('style');
    // style.textContent = ':host { font-style: italic; }';
    const shadow = this.attachShadow({mode: 'open'});
    const slot = document.createElement('slot');
    // shadow.appendChild(style);
    shadow.appendChild(slot);
  }
}
window.customElements.define('simple-span', SimpleSpan);

Above is the hand-coded <simple-span> so you can see exactly what I’m doing and second-guess me. The commented-out lines are the difference between the fourth and fifth columns.

I should add that I tried rewriting the above code to use cloneNode(true) instead of creating the new nodes in the constructor, but performance was no different in this case. If the nodes being created were more complex there would likely be an advantage. (I make extensive use of cloneNode(true) in b8r since past experimentation showed a benefit, and makeWebComponent makes use of it.)

I should add that even the simplest component made with makeWebComponent has a style node because Google’s best practices suggest that all custom elements should support the hidden attribute, and doing this with styles is by far the simplest (and I would hope the most performant) way to do this.

It also occurred to me that having the <style> contain :host { font-style: italic; } might be more expensive that :host([hidden]) { display: none; } but, again, that was a wash. Similarly, it occurred to me that initializing the shadowRoot as {mode: 'closed'} might help. It didn’t.

So, this appears to show that the overhead for just two trivial custom elements in each of 10,000 rows is comparable to the overhead for b8r in its entirety for the entire table row. Now, b8r is creating a given table row by cloning, ad-hoc, the hierarchy it’s using as a template, and then querying that hierarchy to find bound elements and then inserting values as appropriate.

When you consider the kinds of uses to which browsers put custom elements internally (e.g. <video> and <input type=”data”>) these are generally not something you’re going to have 20,000 of in the DOM at once. It’s easy, however, to imagine wanting every item in a pretty huge list to include a custom checkbox or popup menu. It’s worth knowing that even simple custom elements have a non-trivial overhead (and we’re talking 1ms for 20 of them on a fast, high-end laptop).

Addendum

I didn’t realize that you could create a web-component with no shadowRoot. (This isn’t exactly something anyone makes obvious — I’ve found literally zero examples of this in tutorials, etc.)

If you do this, the performance issue mostly goes away.

Now, you do lose all the “benefits” of the shadow DOM but you gain performance and still can attach style rules to custom (unique!) tagNames, which makes managing the resulting CSS easier. This is particularly nice given my desire to replace b8r components with custom elements, since there needn’t be any performance overhead (you can “seal” the styling in a container with just one shadowRoot rather than ten thousand instances).

As the Wwworm Turns

Microsoft’s recent announcement that it is, in effect, abandoning the unloved and unlamented Edge browser stack in favor of Chromium is, well, both hilarious and dripping in irony.

Consider at first blush the history of the web in the barest terms:

  • 1991 — http, html, etc. invented using NeXT computers
  • 1992 — Early browsers (Mosaic, NetScape, etc.) implement and extend the standard, notably NetScape adds Javascript and tries to make frames and layers a thing. Also, the <blink> tag.
  • 1995 — Microsoft “embraces and extends” standards with Internet Explorer and eventually achieves a 95% stranglehold on the browser market.
  • 1997 — As NetScape self-destructs and Apple’s own OpenDoc-based browser “Cyberdog” fails to gain any users (mostly due to being OpenDoc-based), Apple begs Microsoft for a slightly-less-crummy version of IE5 to remain even vaguely relevant/useful in an era where most web stuff is only developed for whatever version of IE (for Windows) the web developer is using.
  • 2002 — FireFox rises from the ashes of NetScape. (It is essentially a cross-platform browser based on Camino, a similar Mac-only browser that was put together by developers frustrated by the lack of a decent Mac browser.)
  • 2003 — Stuck with an increasingly buggy and incompatible IE port, Apple develops its own browser based on KHTML after rejecting Netscape’s Gecko engine. The new browser is called “Safari”, and Apple’s customized version of KHTML is open-sourced as Webkit.
  • As a scrappy underdog, Apple starts a bunch of small PR wars to show that its browser is more standards-compliant and runs javascript faster than its peers.
  • Owing to bugginess, neglect, and all-round arrogance, gradually Microsoft loses a significant portion of market share to FireFox (and, on the Mac, Safari — which is at least as IE-compatible as the aging version of IE that runs on Macs). Google quietly funds FireFox via ad-revenue-sharing since it is in Google’s interest to break Microsoft’s strangehold on the web.
  • 2007 — Safari having slowly become more relevant to consumers as the best browser on the Mac (at least competitive with Firefox functionally and much faster and more power efficient than any competitor) is suddenly the only browser on the iPhone. Suddenly, making your stuff run on Safari matters.
  • 2008 — Google starts experimenting with making its own web browser. It looks around for the best open source web engine, rejects Gecko, and picks Webkit!
  • Flooded with ad revenue from Google, divorced from any sense of user accountability FireFox slowly becomes bloated and arrogant, developing an email client and new languages and mobile platforms rather than fixing or adding features to the only product it produces that anyone cares about. As Firefox grows bloated and Webkit improves, Google Chrome benefits as, essentially, Safari for Windows. (Especially since Apple’s official Safari for Windows is burdened with a faux-macOS-“metal”, UI and users are tricked into installing it with QuickTime.) When Google decides to turn Android from a Sidekick clone into an iPhone clone, it uses its Safari clone as the standard browser. When Android becomes a success, suddenly Webkit compatibility matters a whole lot more.
  • 2013 — Google is frustrated by Apple’s focus on end-users (versus developers). E.g. is the increase in size and power consumption justified by some kind of end-user benefit? If “no” then Apple simply won’t implement it. Since Google is trying to become the new Microsoft (“developers, developers, developers”) it forks Webkit so it can stop caring about users and just add features developers think they want at an insane pace. It also decides to completely undermine the decades-old conventions of software numbering and make new major releases at an insane pace.
  • Developers LOOOOVE Chrome (for the same reason they loved IE). It lets them reach lots of devices, it has lots of nooks and crannies, it provides functionality that lets developers outsource wasteful tasks to clients, if they whine about some bleeding edge feature Google will add it, whether or not it makes sense for anyone. Also it randomly changes APIs and adds bugs fast enough that you can earn a living by memorizing trivia (like the good old days of AUTOEXEC.BAT) allowing a whole new generation of mediocrities to find gainful employment. Chrome also overtakes Firefox as having the best debug tools (in large part because Firefox engages in a two year masturbatory rewrite of all its debugging tools which succeeds mainly in confusing developers and driving them away).
  • 2018 — Microsoft, having seen itself slide from utter domination (IE6) to laughingstock (IE11/Edge), does the thing-that-has-been-obvious-for-five-years and decides to embrace and extend Google’s Webkit fork (aptly named “Blink”).

A Brief Foray into Random Name Generation

I got a bee in my bonnet about name generation this morning, so here’s a simple Javascript module for randomly generating names:

/*
# NameGenerator

Usage:

  const starNameGenerator = new NameGenerator(['Ceti Alpha', 'Scorpio', 'Draconis'...]);
  starNameGenerator.generate(); // -> random name

Works better with a decent sized (hundreds) of thematically similar examples to work from.
*/
/* global module */

function pick(array) {
  return array[Math.floor(Math.random() * array.length)];
}

class NameGenerator {
// data is a map from character-pairs to observed successors,
// consider the examples "how", "now", "brown", "cow"
// the pair "ow" would have the following successors
// [undefined, undefined, "n", undefined] (undefined -> end of word)

  constructor(examples) {
    const data = {'': []};
    examples.
    map(s => s.toLowerCase()).
    forEach(example => {
      let pair = '';
      data[pair].push(example[0]);

      for(let i = 0; i < example.length; i++) {
        pair = pair.substr(-1) + example[i];
        if (! data[pair]) data[pair] = [];
        data[pair].push(example[i + 1]);
      }
    });

    console.log(data);
    this.data = data;
  }

  generate() {
    let s = pick(this.data['']);
    let next = pick(this.data[s]);
    while(next){
      s += next;
      next = pick(this.data[s.substr(-2)]);
    }
    return s;
  }
}

module.exports = NameGenerator;

I wrote a much more convoluted (but simple-minded) star name generator for my galaxy generator several years ago.  The approach I took was to take a collection of star names and break them up into syllables (starting, middle, and ending) and then given a range of syllable lengths, assemble a name out of random pieces.

Today it occurred to me that I’ve never explicitly implemented anything using Markov chains before, so how about I build something that way and see how it compares? If you follow the link, you’ll see examples of star names generated randomly the old way (names with “bad words” in them are rejected).

I took a list of proper star names from Wikipedia and cleaned it up (e.g. the Greek letter prefix of a star name simply indicates its brightness relative to other stars in the same constellation, while a Roman Numeral is simply an indicator of a star being part of a binary or trinary system). This gave me a bit over 600 star names with which to seed the generator, and the results are pretty nice. (Again, no need for bad world filtering.)

The major disadvantage of the new generator is that it can generate really long names pretty easily because of the way it terminates. If I implemented a more sophisticated generator that looked at things like overall length and length of current word in weighing the probabilities it would probably help here, but that might be overthinking it (it’s pretty easy just to reject overly long names).

In general, I think the new generator produces more pronounceable names than my earlier attempt, and some of the new names it generates seem like they should be real names, which is exactly what I’m hoping for.

Algorithm

The algorithm is very simple. The constructor looks at which characters follow a given pair of characters in the input data, so if you start with “how”, “now”, “brown”, “cow” you get the following data for the pair “ow”: [undefined, undefined, ‘n’, undefined]. So, using this data to generate names, 75% of names will end immediately after an ‘ow’ and 25% will have an ‘n’.

In this system, the first character of a name is following the empty string, while the second character of a name is following the first letter. It follows that using [“how”, “now”, “brown”, “cow”] all names will begin with ‘h’, ‘n’, ‘b’, or ‘c’. and most will end in ‘w’ and the rest will end in ‘n’. Not super interesting.

Let’s try slightly more interesting seed data: [‘Frodo’, ‘Peregrin’, ‘Meriadoc’, ‘Bilbo’, ‘Adalgrim’, ‘Bandobras’, ‘Celandine’, ‘Doderic’, ‘Erin’, ‘Griffo’]. This doesn’t seem like it will be enormously fruitful at first glance, but it immediately generates lots of new names that, to me, sound authentic: Adalgrin, Adobris, Grine, Bandine, Froderim.

And here’s a link to a jsfiddle to see it in action (with a bigger set of names from Middle Earth as the seed). One of the really nice things about it is that you don’t need to filter out bad words, because they pretty much don’t get created if they’re not in the source data.

It occurs to me that a lot of the random content generation stuff I’ve done in the past was, in effect, recreating Markov chains naively, and understanding what I’ve done in those terms is powerful and clarifying.

And with that, I leave you with a random Jabberwockish word generator. Don’t bewortle! And have a borpallith day.

b8r 1.0

I decided to make the current revision of b8r “1.0” (it’s still marked “prerelease”) based on the fact it’s proven solid and usable during a year of constant use and improvement. It has been at least as robust and easy to work with as the jQuery-dependent library we developed at USPTO. I’ve just updated bindinator.com and my galaxy demo.

Recently, I made the first deliberately breaking changes and the difficulty of adapting various codebases that use b8r was pretty minor. So, I’m pretty confident that b8r is in good shape.

b8r's "fiddle" component in action
b8r’s “fiddle” component in action — b8r’s fiddle.component.js currently weighs in at 272 loc including markup, css, and comments.

I also improved the appearance of the inline fiddle and test components, and added prism.js code-rendering to all the various inline code examples to make the documentation pages look snazzier. A nice change to the test library makes sure that async test results are consistently ordered, and added a visible “pending” state so you can see tests that somehow failed to complete.

b8r has some pretty nice stuff. (Although much of this nice stuff needs to be documented.) You can download b8r, put nwjs in your Applications directory and/or npm install electron and double-click a .command file to see the b8r documentation inside a desktop app. Or you can install nodejs and double-click a .command file and serve it locally via https (I also provide a .command file will generate local ssl keys). (The .command stuff is currently Mac only, for which I apologize. I imagine it would be very easy to do it for Linux and Windows, but I haven’t tried.)

"electron-file-browser" component (running inside nwjs)
“electron-file-browser” component (running inside nwjs)

There’s a cute feature if you load the b8r documentation in nodejs or electron and command-click on a component — the component is loaded in a new window. I’m planning on leveraging this functionality to let the documentation app function as an IDE.

I’m currently working on convenience methods for multi-window desktop applications (it would be super cool if you could transparently bind across windows and browser tabs). I’ve also written a new version of foldermark that uses a very simple PHP back end (nodejs servers are still a much bigger pain to deal with than PHP) combined with b8r on the client-side.

The biggest shortcoming of b8r remains the fact that my team is the only group really using it. Because we’re developing a desktop app using Electron, we aren’t constantly testing on Edge, Firefox, Safari, etc.. I know for a fact that it has problems in IE and Edge, and that some of the example components aren’t touch-friendly, and we’re definitely doing more stuff for Electron than for nwjs (nwjs is much simpler to work with, but it’s becoming increasingly irrelevant, I fear). But if you’re working in reasonably recent releases of Chrome or Chromium, b8r should be very solid.

So, that’s the way it is: b8r 1.0.

BBEdit 12

BBEdit 12 (dark theme) in action BBEdit 12 is out. You can nearly make it look better with a dark theme now (although the circular “close” buttons indicating an open file are ugly) although it seems like there’s a bug in the theme customization right now. The Ulysses folks might consider this: cost of BBEdit 12 upgrade: $30. Cost of Ulysses for existing owners: $30. BBEdit 11 was released in 2014. BBEdit 12 has more new features than Ulysses has features. BBEdit is targeted at a smaller audience than Ulysses, so it’s not like it makes up for its low pricing on volume. That said, Ulysses is definitely prettier than BBEdit.