Blender 2.8 is coming

Unbiased rendering? Check. Realtime rendering? Check. Unified shader model? Check. Class-leading user interface. Check. Free, open source, small? Check. Blender 2.8 offers everything you want in a 3d package and nothing you don’t (dongles, copy protection, ridiculous prices, massive hardware requirements).

There aren’t many pieces of open source software that have been under continuous active development that haven’t gone through a single “major version change” in twenty years. When I started using Blender 2.8 in the early 2000s, it was version 2.3-something. In the last year it’s been progressing from 2.79 to 2.8 (I think technically the current “release” version is 2.79b, b as in the third 2.79 release not beta).

What brought me to blender was a programming contract for an updated application which, in my opinion, needed an icon. I modeled a forklift for the icon in Silo 3D (which introduced me to “box-modeling”) but needed a renderer, and none of my very expensive 3d software (I owned licenses for 3ds max, ElectricImage, and Strata StudioPro among other thins) on my then current hardware. Blender’s renderer even supported motion blur (kind of).

The blender I started using had a capable renderer that was comparatively slow and hard to configure, deep but incomprehensible functionality, and a user interface that was so bad I ended up ranting about it on the blender forums and got so much hatred in response that I gave up being part of the community. I’ve also blogged pretty extensively about my issues with blender’s user interface over the years. Below is a sampling…

Blender now features not one, not two, but three renderers. (And it supports the addition of more renderers via a plugin architecture.) The original renderer (a ray-tracing engine now referred to as Workbench) is still there, somewhat refined, but it is now accompanied by a real-time game-engine style shader based renderer (Eevee) and a GPU-accelerated unbiased (physically-based) renderer (Cycles). All three are fully integrated into the editor view, meaning you can see the effects of lighting and procedural material changes interactively.

The PBR revolution has slowly brought us to a reasonably uniform conceptualization of what a 3d “shader” should look like. Blender manages to encapsulate all of this into one, extremely versatile shader (although it may not be the most efficient option, especially for realtime applications).

Eevee and Cycles also share the same shader architecture (Workbench does not) meaning that you can use the exact same shaders for both realtime purposes (such as games) and “hero renders”.

Blender 2.8 takes blender from — as of say Blender 2.4 — having one of the worst user interfaces of any general-purpose 3D suite, to having arguably the best.

The most obvious changes in Blender 2.8 are in the user-interface. The simplification, reorganization, and decluttering that has been underway for the last five or so years has culminated in a user interface that is bordering on elegant — e.g. providing a collection of reasonable simple views that are task-focused but yet not modal — while still having the ability to instantly find any tool by searching (now command-F for find instead of space by default; I kind of miss space). Left-click to select is now the default and is a first class citizen in the user interface (complaining about Blender’s right-click to select, left click to move the “cursor” and screw yourself is this literally got me chased off Blender’s forums in 2005).

Blender still uses custom file-requesters that are simply worse in every possible way than the ones the host OS provides. Similarly, but less annoyingly, Blender uses a custom-in-window-menubar that means it’s simply wasting a lot of screen real estate when not used in full screen mode.

OK so the “globe” means “world” and the other “globe” means “shader”…

Blender relies a lot on icons to reduce the space required for the — still — enormous numbers of tabs and options, and it’s pretty hard to figure out what is supposed to mean what (e.g. the “globe with a couple of dots” icon refers to scene settings while the nearly identical “globe” icon refers to materials — um, what?). The instant search tool is great but doesn’t have any support for obvious synonyms, so you need to know that it’s a “sphere” and not a “ball” and a “cube” and not a “box” but while you “snap” the cursor you “align” objects and cameras.

Finally, Blender can still be cluttered and confusing. Some parts of the UI are visually unstable (i.e. things disappear or appear based on settings picked elsewhere, and it may not be obvious why). Some of the tools have funky workflows (e.g. several common tools only spawn a helpful floating dialog AFTER you’ve done something with the mouse that you probably didn’t want to do) and a lot of keyboard shortcuts seem to be designed for Linux users (ctrl used where command would make more sense).

The blender 2.8 documentation is pretty good but also incomplete. E.g. I couldn’t find any documentation of particle systems in the new 2.8 documentation. There’s plenty of websites with documentation or tutorials on blender’s particle systems but which variant of the user interface they’ll pertain to is pretty much luck-of-the-draw (and blender’s UI is in constant evolution).

Expecting a 3D program with 20 years of development history and a ludicrously wide-and-deep set of functionality to be learnable by clicking around is pretty unreasonable. That said, blender 2.8 comes close, generally having excellent tooltips everywhere. “Find” will quickly find you the tool you want — most of the time — and tell you its keyboard shortcut — if any — but won’t tell you where to find it in the UI. I am pretty unreasonable, but even compared to Cheetah 3D, Silo, or 3ds max (the most usable 3D programs I have previously used) I now think Blender more than holds its own in terms of learnability and ease-of-use relative to functionality.

Performance-wise, Cycles produces pretty snappy previews despite, at least for the moment, not being able to utilize the Nvidia GPU on my MBP. If you use Cycles in previews expect your laptop to run pretty damn hot. (I can’t remember which if any versions of Blender did, and I haven’t tried it out on either the 2013 Mac Pro/D500 or the 2012 Mac Pro/1070 we have lying around the house because that would involve sitting at a desk…)

Cranked up, Eevee is able to render well-beyond the requirements for broadcast animated TV shows. This frame was rendered on my laptop at 1080p in about 15s. Literally no effort has been made to make the scene efficient (there’s a big box of volumetric fog containing the whole scene with a spotlight illuminating a bunch of high polygon models with subsurface scattering and screenspace reflections.

Perhaps the most delightful feature of blender 2.8 though is Eevee, the new OpenGL-based renderer, which spans the gamut from nearly-fast-enough-for-games to definitely-good-enough-for-Netflix TV show rendering, all in either real time or near realtime. Not only does it use the same shader model as Cycles (the PBR renderer) but, to my eye, for most purposes it produces nicer results and it does so much, much faster than Cycles does.

Blender 2.8, now in late beta, is a masterpiece. If you have any interest in 3d software, even or especially if you’ve tried blender in the past and hated it, you owe it to yourself to give it another chance. Blender has somehow gone from having a user interface that only someone with Stockholm Syndrome could love to an arguably class-leading user interface. The fact that it’s an open source project, largely built by volunteers, and competing in a field of competitors with, generally, poor or at best quirky user interfaces, makes this something of a software miracle.

As the Wwworm Turns

Microsoft’s recent announcement that it is, in effect, abandoning the unloved and unlamented Edge browser stack in favor of Chromium is, well, both hilarious and dripping in irony.

Consider at first blush the history of the web in the barest terms:

  • 1991 — http, html, etc. invented using NeXT computers
  • 1992 — Early browsers (Mosaic, NetScape, etc.) implement and extend the standard, notably NetScape adds Javascript and tries to make frames and layers a thing. Also, the <blink> tag.
  • 1995 — Microsoft “embraces and extends” standards with Internet Explorer and eventually achieves a 95% stranglehold on the browser market.
  • 1997 — As NetScape self-destructs and Apple’s own OpenDoc-based browser “Cyberdog” fails to gain any users (mostly due to being OpenDoc-based), Apple begs Microsoft for a slightly-less-crummy version of IE5 to remain even vaguely relevant/useful in an era where most web stuff is only developed for whatever version of IE (for Windows) the web developer is using.
  • 2002 — FireFox rises from the ashes of NetScape. (It is essentially a cross-platform browser based on Camino, a similar Mac-only browser that was put together by developers frustrated by the lack of a decent Mac browser.)
  • 2003 — Stuck with an increasingly buggy and incompatible IE port, Apple develops its own browser based on KHTML after rejecting Netscape’s Gecko engine. The new browser is called “Safari”, and Apple’s customized version of KHTML is open-sourced as Webkit.
  • As a scrappy underdog, Apple starts a bunch of small PR wars to show that its browser is more standards-compliant and runs javascript faster than its peers.
  • Owing to bugginess, neglect, and all-round arrogance, gradually Microsoft loses a significant portion of market share to FireFox (and, on the Mac, Safari — which is at least as IE-compatible as the aging version of IE that runs on Macs). Google quietly funds FireFox via ad-revenue-sharing since it is in Google’s interest to break Microsoft’s strangehold on the web.
  • 2007 — Safari having slowly become more relevant to consumers as the best browser on the Mac (at least competitive with Firefox functionally and much faster and more power efficient than any competitor) is suddenly the only browser on the iPhone. Suddenly, making your stuff run on Safari matters.
  • 2008 — Google starts experimenting with making its own web browser. It looks around for the best open source web engine, rejects Gecko, and picks Webkit!
  • Flooded with ad revenue from Google, divorced from any sense of user accountability FireFox slowly becomes bloated and arrogant, developing an email client and new languages and mobile platforms rather than fixing or adding features to the only product it produces that anyone cares about. As Firefox grows bloated and Webkit improves, Google Chrome benefits as, essentially, Safari for Windows. (Especially since Apple’s official Safari for Windows is burdened with a faux-macOS-“metal”, UI and users are tricked into installing it with QuickTime.) When Google decides to turn Android from a Sidekick clone into an iPhone clone, it uses its Safari clone as the standard browser. When Android becomes a success, suddenly Webkit compatibility matters a whole lot more.
  • 2013 — Google is frustrated by Apple’s focus on end-users (versus developers). E.g. is the increase in size and power consumption justified by some kind of end-user benefit? If “no” then Apple simply won’t implement it. Since Google is trying to become the new Microsoft (“developers, developers, developers”) it forks Webkit so it can stop caring about users and just add features developers think they want at an insane pace. It also decides to completely undermine the decades-old conventions of software numbering and make new major releases at an insane pace.
  • Developers LOOOOVE Chrome (for the same reason they loved IE). It lets them reach lots of devices, it has lots of nooks and crannies, it provides functionality that lets developers outsource wasteful tasks to clients, if they whine about some bleeding edge feature Google will add it, whether or not it makes sense for anyone. Also it randomly changes APIs and adds bugs fast enough that you can earn a living by memorizing trivia (like the good old days of AUTOEXEC.BAT) allowing a whole new generation of mediocrities to find gainful employment. Chrome also overtakes Firefox as having the best debug tools (in large part because Firefox engages in a two year masturbatory rewrite of all its debugging tools which succeeds mainly in confusing developers and driving them away).
  • 2018 — Microsoft, having seen itself slide from utter domination (IE6) to laughingstock (IE11/Edge), does the thing-that-has-been-obvious-for-five-years and decides to embrace and extend Google’s Webkit fork (aptly named “Blink”).

RAW Power 2.0

RAW Power 2.0 in action — the new version features nice tools for generating black-and-white images.

My favorite tool for quickly making adjustments to RAW photos just turned 2.0. It’s a free upgrade but the price for new users has increased to (I believe) $25. While the original program was great for quickly adjusting a single image, the new program allows you to browse directories full of images quickly and easily, to some extent replacing browsing apps like FastRAWViewer.

The major new features — and there are a lot of them — are batch processing, copying and pasting adjustments between images, multiple window and tab support, hot/cold pixel overlays (very nicely done), depth effect (the ability to manipulate depth data from dual camera iPhones), perspective correction and chromatic aberration support.

The browsing functionality is pretty minimal. It’s useful enough for selecting images for batch-processing, but it doesn’t offer filtering ability (beyond the ability to show only RAW images) or the ability quickly modify image metadata (e.g. rate images), so FastRAWViewer is still the app to beat for managing large directories full of images.

While the hot/cold pixel feature is lovely, the ability to show in-focus areas (another great FastRAWViewer feature) is also missing.

As before, RAW Power both functions as a standalone application and a plugin for Apple’s Photos app (providing enhanced adjustment functionality).

Highly recommended!

Lost Productivity

A couple of months ago I listened to a Planet Money podcast discussing the mysterious slowdown in US productivity growth (the link is to one of several podcasts on this topic). Like most NPR content, the story got recycled through a number of different programs, such as Morning Edition.

The upshot was, that productivity — which is essentially GDP/work — has stalled since the — I dunno — 90s, and it doesn’t make sense given the apparent revolutions in technology — faster computers, better networks, etc.

Anyway, the upshot — and I’m basing this on memory because I can’t find the exact transcript — is that there’s a mysterious hole in productivity growth which, if it were filled, would add up to several trillion dollars worth of lost value added.

Well, I think it’s there to be found, because Free Open Source Software on its own adds up to several trillion dollars worth of stuff that hasn’t been measured by GDP.

Consider the dominant tech platforms of our time — Android and iOS. Both are fundamentally built on Open Source. If it weren’t for Open Source, iOS at minimum would have been significantly altered (let’s assume NeXTStep would have had a fully proprietary, but still fundamentally POSIX base) and Android could not have existed at all. Whatever was in their place would have had to pay trillions in licenses.

On a micro level, having worked through a series of tech booms from 1990 to the present — in the 90s, to do my job my employer or I had to spend about $2000-5000 for software licenses every year just to keep up-to-date with Photoshop, Director, Illustrator, Acrobat, Strata 3d, 3dsmax, Form*Z, and so on and so forth. By the mid aughts it was maybe $1000 per year and the software was So. Much. Better. Today, it’s probably down to less than $500.

And, in this same period, the number of people doing the kind of work I do, is done by far more people.

That’s just software. This phenomenon is also affecting hardware.

The big problem with this “lost productivity” is that the benefits are chiefly being reaped by billionaires.

Photo-backup, Hubic, and Other Stories

So, a year ago I started backing up to HubiC after my previous backup service decided to stop servicing retail customers. At the time, HubiC seemed on paper like a great option — potentially offering the simplicity and utility of DropBox with effectively unlimited storage capacity.

In practice, HubiC is useless. I’ve had a 2012 Mac Pro constantly connected to HubiC via a fast cable connection for 12 months and managed to back up only about 1/4 of the files I’ve pointed at it. The damn thing breaks down constantly. Every time I log in it wants me to change passwords. They just billed me for renewal, but I’m two days late and I can’t cancel my account without paying for another year. Good luck with that, guys.

Now, I haven’t been sitting on my hands while I watch HubiC fail to deliver on any of its promises. Most of my stuff continues to be backed up locally via Time Machine. The stuff I work on is stored in the cloud (iCloud, GitHub, Google Drive, and/or DropBox).

My big problem is photographs and video.

The disaster that is my Flickr account
Flickr’s auto-uploader turned my Flickr account into an unfixable mess

Now, when Flickr raised its “free” tier to offer 1TB of cloud storage for JPEGs, I jumped on that. It may not be RAW storage, but it’s better than nothing, and 1TB is enough space for a huge number of JPEGs. The big problem, Flickr’s (since discontinued) auto-uploader was so stupidly designed that it successfully rendered my Flickr account borderline useless (it created an album for every folder it found an image in, and it uploaded every image it found, including things like UI images inside applications and development trees, so I have “albums” comprising sprites from sample game development projects and logos for PHP templates) and Flickr’s account management tools look and work like something an intern abandoned in 2005, so just deleting stuff is an exercise in frustration. It looks to me like Flickr’s abandonware API isn’t really up to the task of even supporting a third party application to untangle the mess.

And of course, since Yahoo changed hands and various security scandals unfolded just logging into Yahoo accounts is a pain, and you need to navigate ads to even get into your account. Yahoo is the GeoCities of 2018.

Google photos in "action"
When I scroll to some arbitrary point on my timeline, this is what I see for some random (long) amount of time…

Recently, Google raised the “free” tier of photos.google.com to unlimited storage of photos where RAW files are JPEGs are processed into high-but-not-full-quality JPEGs on-the-fly. I’ve tried it and it’s pretty damn good. The uploader is smart enough to skip files that are clearly not important photos (e.g. too small, wrong format) and ignore obvious duplicates. The problems are (i) that the uploader application periodically just hangs and needs to be manually killed and restarted (ii) the web app seems to be weirdly slow and unreliable (I can log on with two machines side-by-side and they’ll see different subsets of my photos), (iii) no Apple TV support, and (iv) online photo editor seems to need one or two extra clicks to accomplish anything (but it’s a lot better than nothing). I’m pretty confident that my stuff is there, just not in my ability to see a given photo from a given machine on a given day. It’s certainly the most complete, easy-to-navigate, and shareable archive I’ve ever managed to create of my photographs. And if I can find a photo there, I can locate the original RAW image pretty easily.

Now, the absolute best system for dealing with my photographs thus far is iCloud. If I could simply rent 10TB from iCloud for a reasonable price (let’s say, $25/month) and get my Mac to automatically sync multiple volumes to iCloud, my problem would be solved. Obviously, I’m a happy Apple customer. If I were a more-than-casual Windows or Linux user then this would not be a useful option to me, and I’m not sure what I’d do, because I’m pretty sure there’s no equivalently seamless option for people who don’t want to pay the “Apple Tax”. Google Drive isn’t even a tolerable substitute for DropBox (although I think it has Sharepoint beaten).

Here’s where iCloud beats all other options:

  • I don’t need to think about it or do anything. (Well, on a desktop device, I need to NOT avoid storing my data in iCloud) If I take a photo, then it ends up in the cloud pretty quickly (basically, when the device gets recharged while on a LAN, if not sooner).
  • By default, full-resolution images are not propagated to all my devices (as would be the case for DropBox, or Hubic if it actually worked). Instead, as with everything in iCloud it’s available on-demand. (Indeed, it’s a bit reminiscent of the way iTunes deals with movies… superficially less convenient than pure streaming, but a lot more flexible and useful in practice).
  • If I ingest a RAW photo from a camera onto a device, then it’s in the cloud and available from any device on-demand (but it’s not wasting space on all my devices).
  • If I want to work on a photo, I can use the best native tools that are available on the device I’m using — seamlessly (although I’m inclined to actively avoid Adobe applications because Adobe’s workflow involves use of Adobe’s barely functional Cloud ecosystem).

The big problem — of course there has to be one — is that Apple’s highest storage tier is 2TB. I’m currently on 200GB which is plenty for the stuff I need that isn’t photos and videos, but hopelessly inadequate for photos and videos. 2TB (the next tier up, and it’s competitively priced) would be sufficient for my photos and videos if I were to curate them, but I don’t want to curate shit. I want to dump it in the cloud and not think about it.

Missing in Action

All of this adds up to a bunch of pretty disappointing non-solutions. Even though Apple provides a file sync system that works pretty well for personal photographs, it wouldn’t work for say a small photography business. (I guess you could use some kind of “family plan” but I’m pretty sure that would run you into weirdness pretty fast.) And it’s not like we’re talking advanced workflow support here — I just want my photos backed up and available.

Where is a tool that automatically detects blurred, underexposed, or overexposed photos and flags them as less worthy of backup? (Google’s photos app does a pretty good job of automatically correcting exposure, I wonder if it’s smart enough to task the uploader with going back to the RAW and reprocessing and re-uploading the photo?)

Where is the tool that remembers which photos have been opened or zoomed in and flags them as more interesting or worthy of backup?

Where is the tool that correlates the GPS location data of your iPhone photos and tentatively applies them to your corresponding camera photos?

Aperture used to collect photos from bursts into a single set and represent them with what it guessed was the best one. Where did this idea disappear to?

There’s a ton of low-hanging fruit here. Someone, please do something. I’m busy.