Must all data-tables kind of suck?

b8r's data-table in action
b8r now has a data-table component. It’s implemented in a little over 300loc, supports custom header and fixed headers, a correctly-positioned scrollbar, content cells, sorting, resizing, showing/hiding of columns.

Bindinator, up until a few weeks ago, did not have a data-table component. It has some table examples, but they’ve not seen much use in production because b8r is a rewrite of a rewrite of a library named bindomatic that I wrote for the USPTO that was implemented to do all the data-binding outside of data-tables in a large, complex project and then replaced all the data-tables.

It turns out that almost any complex table ends up being horrifically special-case and bindomatic, by being simpler, made it easier to build complex things bespoke. (I should note that I also wrote the original data-table component that bindomatic replaced.)

Ever since I first wrote a simple file-browser using b8r-native I’ve been wishing to make the list work as well as Finder’s windows. It would also benefit my RAW file manager if that ever turns into a real product.

The result is b8r’s data-table component, which is intended to scratch all my itches concerning data-tables (and tables in general) — e.g. fixed headers, resizable columns, showing/hiding columns, sorting, affording customization, etc. — while not being complex or bloated.

At a little over 300 loc I think I’ve avoided complexity and bloat, at any rate.

As it is, it can probably meet 80% of needs with trivial configuration (which columns do you want to display?), another 15% with complex customization. The remaining 5% — write your own!

Implementation Details

The cutest trick in the implementation is using a precisely scoped css-variable to control a css-grid to conform all the column contents to desired size. In order to change the entire table column layout, exactly one property of one DOM element gets changed. When you resize a column, it rewrites the css variable (if the number of rows is below a configurable threshold, it live updates the whole table, otherwise it only updates the header row until you’re done).

Another cute trick is to display the column borders (and the resizing affordance) using a pseudo-element that’s 100vh tall and clipped to the table component. (A similar trick could be used to hilite columns on mouse-over.)

Handling the actual drag-resizing of columns would be tricky, but I wrote b8r’s track-drag library some time ago to manage dragging once-and-for-all (it also deals with touch interfaces).

Next Steps…

There’s a to-do list in the documentation. None of it’s really a priority, although allowing columns to be reordered should be super easy and implementing virtualization should also be a breeze (I scratched that itch some time back).

The selection column shown in the animation is actually implemented using a custom headerCell and custom contentCell (there’s no selection support built in to data-table). I’ll probably improve the example and provide it as an helper library that provides a “custom column” for data-table and also serves as an example of adding custom functionality to the table.

I’d like to try out the new component with some hard-core use-cases, e.g. the example RAW file manager (which can easily generate lists of over 10,000 items), and the file-browser (I have an idea for a Spotlight-centric file browser), and — especially — my galaxy generator (since I have ideas for a web-based MMOG that will leverage the galaxy generator and use Google Firebase as its backend).

snake_case, camelCase, and other Misadventures in Coding Standards

some_programmers_like_snake_case while othersPreferCamelCase (and dont-talk-to-me-about-css-or-html) but life really becomes entertaining when different programmers — sorry “software engineers” — enforce their preferences locally.

Consider a service, implemented in python (where snake_case_is_popular) that passes data to a web application, i.e. javascript (whereCamelCaseIsPopular). You might end up with a data structure like:

{
    my_id: 17,
    my_name: 'seventeen'
}

In modern Javascript, you could write something like:

const { my_id, my_name } = await getSomeData()

And get on with your life.

But it won’t lint, because someone has enforced camelCase in your project lint rules, so now you write:

const data = await getSomeData()
const myId = data.my_id
const myName = data.my_name

And now your code lints.

No problems!

One day, you discover a problem with the data in myName and it stymies you for a bit until you find the assignment statement that renamed the value and you’re reminded that it gets populated from a service with the property my_name and you figure it out.

No problems, right? Chalk it up to experience.

After a lot of this, you kind of wish the problem would just disappear, so you do something like write snakeToCamel which converts an object with snake_case properties into an object with camelCase properties, and now you can write:

const { myId, myName } = snakeToCamel(await getSomeData());

And your code is nearly as clean as if you, say, had turned off the camelCase lint rule in the first place. But hey, snake_case is nauseating.

Later, you modify the helper library that builds async data methods (after all, snakeToCamel called on a conforming object is a no-op, isn’t it?!) so that snakeToCamel always gets called, and now you can finally write:

const { myId, myName } = await getSomeData();

You can now write a new lint rule that flags redundant inline calls to snakeToCamel so that every time a file gets touched, the redundant calls are flagged, or flex your awesomeness and write a code-transform to remove them all in one glorious diff.

Now, your code is beautiful, it’s all in camelCase, and it’s sleek and elegant and oh-so-2017…

Except that there’s no clue in the code for someone hunting down a reference to myName that it might be called my_name deeper in the pipe (e.g. in the Network tab), and now you have a bunch of folks bikeshedding about how things like my_uuid or inner_html should be treated. Because innerHTML does not work well in the other direction, and the python folks have their own lint rules…

Also, deep in your service layer, some dodgy regex is rewriting every property name you ever see, and it’s probably completely clueless about unicode (quick quiz — which unicode characters are OK in javascript variable names?! I looked up the answer and you don’t really want to know.) So, that’s definitely never coming back to bite you.

But hey, at least you don’t have to look at a mixture of snake_case and camelCase in your source code, because the cost of that is… zero?

Blender 2.8 is coming

Unbiased rendering? Check. Realtime rendering? Check. Unified shader model? Check. Class-leading user interface. Check. Free, open source, small? Check. Blender 2.8 offers everything you want in a 3d package and nothing you don’t (dongles, copy protection, ridiculous prices, massive hardware requirements).

There aren’t many pieces of open source software that have been under continuous active development that haven’t gone through a single “major version change” in twenty years. When I started using Blender 2.8 in the early 2000s, it was version 2.3-something. In the last year it’s been progressing from 2.79 to 2.8 (I think technically the current “release” version is 2.79b, b as in the third 2.79 release not beta).

What brought me to blender was a programming contract for an updated application which, in my opinion, needed an icon. I modeled a forklift for the icon in Silo 3D (which introduced me to “box-modeling”) but needed a renderer, and none of my very expensive 3d software (I owned licenses for 3ds max, ElectricImage, and Strata StudioPro among other thins) on my then current hardware. Blender’s renderer even supported motion blur (kind of).

The blender I started using had a capable renderer that was comparatively slow and hard to configure, deep but incomprehensible functionality, and a user interface that was so bad I ended up ranting about it on the blender forums and got so much hatred in response that I gave up being part of the community. I’ve also blogged pretty extensively about my issues with blender’s user interface over the years. Below is a sampling…

Blender now features not one, not two, but three renderers. (And it supports the addition of more renderers via a plugin architecture.) The original renderer (a ray-tracing engine now referred to as Workbench) is still there, somewhat refined, but it is now accompanied by a real-time game-engine style shader based renderer (Eevee) and a GPU-accelerated unbiased (physically-based) renderer (Cycles). All three are fully integrated into the editor view, meaning you can see the effects of lighting and procedural material changes interactively.

The PBR revolution has slowly brought us to a reasonably uniform conceptualization of what a 3d “shader” should look like. Blender manages to encapsulate all of this into one, extremely versatile shader (although it may not be the most efficient option, especially for realtime applications).

Eevee and Cycles also share the same shader architecture (Workbench does not) meaning that you can use the exact same shaders for both realtime purposes (such as games) and “hero renders”.

Blender 2.8 takes blender from — as of say Blender 2.4 — having one of the worst user interfaces of any general-purpose 3D suite, to having arguably the best.

The most obvious changes in Blender 2.8 are in the user-interface. The simplification, reorganization, and decluttering that has been underway for the last five or so years has culminated in a user interface that is bordering on elegant — e.g. providing a collection of reasonable simple views that are task-focused but yet not modal — while still having the ability to instantly find any tool by searching (now command-F for find instead of space by default; I kind of miss space). Left-click to select is now the default and is a first class citizen in the user interface (complaining about Blender’s right-click to select, left click to move the “cursor” and screw yourself is this literally got me chased off Blender’s forums in 2005).

Blender still uses custom file-requesters that are simply worse in every possible way than the ones the host OS provides. Similarly, but less annoyingly, Blender uses a custom-in-window-menubar that means it’s simply wasting a lot of screen real estate when not used in full screen mode.

OK so the “globe” means “world” and the other “globe” means “shader”…

Blender relies a lot on icons to reduce the space required for the — still — enormous numbers of tabs and options, and it’s pretty hard to figure out what is supposed to mean what (e.g. the “globe with a couple of dots” icon refers to scene settings while the nearly identical “globe” icon refers to materials — um, what?). The instant search tool is great but doesn’t have any support for obvious synonyms, so you need to know that it’s a “sphere” and not a “ball” and a “cube” and not a “box” but while you “snap” the cursor you “align” objects and cameras.

Finally, Blender can still be cluttered and confusing. Some parts of the UI are visually unstable (i.e. things disappear or appear based on settings picked elsewhere, and it may not be obvious why). Some of the tools have funky workflows (e.g. several common tools only spawn a helpful floating dialog AFTER you’ve done something with the mouse that you probably didn’t want to do) and a lot of keyboard shortcuts seem to be designed for Linux users (ctrl used where command would make more sense).

The blender 2.8 documentation is pretty good but also incomplete. E.g. I couldn’t find any documentation of particle systems in the new 2.8 documentation. There’s plenty of websites with documentation or tutorials on blender’s particle systems but which variant of the user interface they’ll pertain to is pretty much luck-of-the-draw (and blender’s UI is in constant evolution).

Expecting a 3D program with 20 years of development history and a ludicrously wide-and-deep set of functionality to be learnable by clicking around is pretty unreasonable. That said, blender 2.8 comes close, generally having excellent tooltips everywhere. “Find” will quickly find you the tool you want — most of the time — and tell you its keyboard shortcut — if any — but won’t tell you where to find it in the UI. I am pretty unreasonable, but even compared to Cheetah 3D, Silo, or 3ds max (the most usable 3D programs I have previously used) I now think Blender more than holds its own in terms of learnability and ease-of-use relative to functionality.

Performance-wise, Cycles produces pretty snappy previews despite, at least for the moment, not being able to utilize the Nvidia GPU on my MBP. If you use Cycles in previews expect your laptop to run pretty damn hot. (I can’t remember which if any versions of Blender did, and I haven’t tried it out on either the 2013 Mac Pro/D500 or the 2012 Mac Pro/1070 we have lying around the house because that would involve sitting at a desk…)

Cranked up, Eevee is able to render well-beyond the requirements for broadcast animated TV shows. This frame was rendered on my laptop at 1080p in about 15s. Literally no effort has been made to make the scene efficient (there’s a big box of volumetric fog containing the whole scene with a spotlight illuminating a bunch of high polygon models with subsurface scattering and screenspace reflections.

Perhaps the most delightful feature of blender 2.8 though is Eevee, the new OpenGL-based renderer, which spans the gamut from nearly-fast-enough-for-games to definitely-good-enough-for-Netflix TV show rendering, all in either real time or near realtime. Not only does it use the same shader model as Cycles (the PBR renderer) but, to my eye, for most purposes it produces nicer results and it does so much, much faster than Cycles does.

Blender 2.8, now in late beta, is a masterpiece. If you have any interest in 3d software, even or especially if you’ve tried blender in the past and hated it, you owe it to yourself to give it another chance. Blender has somehow gone from having a user interface that only someone with Stockholm Syndrome could love to an arguably class-leading user interface. The fact that it’s an open source project, largely built by volunteers, and competing in a field of competitors with, generally, poor or at best quirky user interfaces, makes this something of a software miracle.

Epic Fail

Atul Gawande describes the Epic software system being rolled out in America’s hospitals.

It reads like a  potpourri of everything bad about enterprise IT. Standardize on endpoints instead of interoperability, Big Bang changes instead of incremental improvements, and failure to adhere to the simplest principles of usability.

The sad thing is that the litany of horrors in this article are all solved problems. Unfortunately, it seems that in the process of “professionalizing” usability, the discipline has lost its way.

Reading through the article, you can just tally up the violations of my proposed Usability Heuristics, and there’s very few issues described in the article that would not be eliminated by applying one of them.

The others would fall to simple principles like using battle-tested standards (ISO timestamps anyone?) and picking the right level of database normalization (it should be difficult or impossible to enter different variations of the same problem in “problem lists”, and easier to elaborate on existing problems).

There was a column of thirteen tabs on the left side of my screen, crowded with nearly identical terms: “chart review,” “results review,” “review flowsheet.”

I’m sure the tabs LOOKED nice, though. (Hint: maximize generality, minimize steps, progressive disclosure, viability.)

“Ordering a mammogram used to be one click,” she said. “Now I spend three extra clicks to put in a diagnosis. When I do a Pap smear, I have eleven clicks. It’s ‘Oh, who did it?’ Why not, by default, think that I did it?” She was almost shouting now. “I’m the one putting the order in. Why is it asking me what date, if the patient is in the office today? When do you think this actually happened? It is incredible!”

Sensible defaults can be helpful?! Who knew? (Hint: sensible defaults, minimize steps.)

This is probably my favorite (even though it’s not usability-related):

Last fall, the night before daylight-saving time ended, an all-user e-mail alert went out. The system did not have a way to record information when the hour from 1 a.m. to 1:59 a.m. repeated in the night. This was, for the system, a surprise event.

Face meet palm.

Date-and-time is a fundamental issue with all software and the layers of stupidity that must have signed off on a system that couldn’t cope with Daylight Savings boggles my mind.

A former colleague of mine linked to US Web Design System as if this were some kind of intrinsically Good Thing. Hilariously, the site itself does not appear to have been designed for accessibility or even decent semantic web, and blocks robots.

Even if the site itself were perfect, the bigger problems are that (a) there are plenty of similar open source projects, they could have just blessed one; (b) it’s a cosmetic standard, and (c) there’s pretty much no emphasis on the conceptual side of usability. So, at best it helps make government websites look nice and consistent.

(To be continued…)

Please enter date of birth…

I’d like every person who has implemented a date picker control to enter the birthdays of their living relatives one-hundred times using their own date-pickers. Then try some ancestors.

Date pickers are perhaps one of the worst controls one deals with on a daily basis. They’re pretty terrible even when used for their expected purpose (entering dates for appointments). Perhaps the simplest criterion by which to judge them is:

Is your date picker easier to use than a keyboard?

A followup question would be:

If you want to use your keyboard anyway, will you find it harder than if your date picker weren’t there?

The primary reason date pickers exist is not for the user’s convenience. Entering a date using a keyboard requires, typically, 4 keystrokes for a nearby date (e.g. “4/1” or “12/25”) and at most 10 keystrokes for a date with specific year. Now, if you’re picking a date within a predictable and small period of time, you can provide graphical calendars to reduce date entry to a single click, and date-range-entry to a single click-and-drag. So, in a certain restricted set of use-cases, date pickers can be convenient.

The iOS Date Picker
The iOS Date Picker makes me furious every time I use it. The absolute best case is that you want to set something in the next few days. Anything else and you need to remember where you started and then scroll. Now what if you want to schedule your next teeth cleaning in six months? OK, what if you were entering a date of birth? Good. Fucking. Luck. And there’s no option to simply type in the damn date.

So, if I’m trying to enter a new appointment, and the date picker defaults to, say, showing today’s date and time, we’re in pretty good shape. Mostly.

Similarly, if I want to move an appointment from today to tomorrow or from Tuesday to a week from Tuesday we’re again in pretty good shape.

But if I want to select the time I started working at Andersen Consulting (October 1993?) things get a lot less pleasant fast. If I want to select the time I spent living in Canberra (March 1983 to September 1993… ish) it’s getting seriously unpleasant. If I need to enter birthdates for my family… blech. Luckily with America’s healthcare system I probably only need to do that a few dozen times a year.

One reason date pickers exist is because parsing typed or written dates is difficult. To begin with, the US has chosen to use a date format that is different from that used in every other part of the world — month / day / year. Since a lot of dates are ambiguous (if the first two numbers are both 12 or less then you can’t know for sure which is which) a date picker allows the UI designer to make the user’s choice clear to the software.

The simplest option in most cases would be to provide a calendar with a text field. If you type a date, it gets parsed and displayed on the calendar. If you click on the calendar, it replaces the typed date. In most cases, the calendar can be something that’s only visible when the text field is in focus.

Chrome's date picker
Chrome’s date picker is actually one of the best I’ve seen. That said, it has “clever” touches that can be confusing (e.g. if you type “4” as month, it automatically advances to the next field before you type “/” or start typing the day) which in turn means that it won’t let me enter the date in non-US order and flip it, let alone enter the date in non-numeric format (e.g. March 15), and it doesn’t deal well with anything less than a four-digit year.

Another reason date pickers exist is that you often want to display dates, and date pickers allow dates to be rendered in a consistent form, often with useful context. E.g. today vs. the date selected. Also, if you’re going to display something, then it’s nice if you can also directly manipulate it.

But, directly manipulating dates doesn’t really make sense (unless maybe you’re from Gallifrey). What you want to manipulate is events embedded within calendars. In the end, the best solution is probably to figure out how to avoid date pickers altogether.

Mac OS X Calendar
Mac OS X Calendar doesn’t use a date picker. You simply type in your appointment in shorthand (and look, it figured out I had entered date/month automagically). This also happens to have been a feature of the Newton (which would actually figure out that what you had entered was an appointment).