Google and the <video> tag

Though H.264 plays an important role in video, as our goal is to enable open innovation, support for the codec will be removed and our resources directed towards completely open codec technologies.

From HTML Video Codec Support in Chrome

Well that sucks.

Gruber asks a few “simple questions” here. Aside from the question of hypocrisy w.r.t. Flash bundling, I think his points are more than neutralized by these ten questions:

You are a proponent of Apple using its influence to diminish the importance of Flash for the web. Yet, when Google makes similar moves to rid the web of a similarly closed and patented, albeit different type of technology, you do not support them. Why is Apple promoting an open web a good thing, but Google promoting an open web a bad thing?

I think that if Google pulled Flash support from Chrome there would be no question that Google were on the side of the angels (although it would still be a dumb thing to do), but since there’s no hint of this it seems purely like a cynical move to hurt Apple’s anti-Flash campaign which will damage HTML5 <video> adoption. I think you can make the argument that HTML5 <video> adoption with H264 as the defacto standard codec is a Bad Thing.

Anyway it’s a bigger mess now than it was before Google decided to do this. Ultimately it will come down to “what will let the most people see the most porn using the most devices?”

Postscript: “standards”

One of the arguments made in favor of WebM/VP8 is that it can be part of the W3C standard, unlike H264, because it’s not encumbered by license fees. The problem here is that WebM/VP8 almost certainly is encumbered (as was GIF in earlier days), it just hasn’t been sued yet because no-one uses it. But this is beside the point — the CSS font-family property supports any font, and almost all the fonts that anyone cares about are encumbered (i.e. subject to royalties, copyright, and so on). Just as CSS font-family can specify a non-free non-open-source font, there’s no reason why a video tag can’t point to an arbitrarily encoded video.

To put it another way:

There’s no conflict between the HTML specification being open and royalty-free and H264 video playback being supported in HTML5 video tags as long as the codec doesn’t need to be implemented by the browser. Just as a slab of text with font-family “Verdana” won’t necessarily display on every browser correctly (if the font is not installed) it would follow that not every video will play back in every browser.

As a practical matter, it would be nice if serving a page with video were as simple an affair as possible. E.g. figuring out which video to serve didn’t involve sniffing the browser, operating system, and so forth; better yet, if one video format worked everywhere. As a practical matter right now H264 is the best candidate. VP8/WebM will never be the best candidate because by the time there’s a critical mass of hardware support out there it will be obsolete. This is a stupid, stupid fight.

And yet one more thing:

It’s interesting that the companies still in favor of h264 (Apple, Microsoft) are precisely those companies who do not implement the codec in the browser. Apple and Microsoft both implement h264 as a plugin architecture at OS level rather than a plugin at browser level (a much worse thing — see this excellent piece that daringfireball brought to my attention).

Is a it a font or an image file format?

The flipside of my argument that H264 should be considered analogous to a font is that, generally speaking, text is still legible when presented in the wrong font. By that argument H264 is more like an image format (JPG, PNG, etc.). If we accept this argument — which I’d say is the most h264-hostile stance (within reason) to take with respect to video codecs — then consider that most browsers simply let you display pretty much any image that’s convenient inside an <img> tag (sometimes badly, as per Internet Explorer’s notorious mishandling of PNG files over the years), generally by using the underlying OS’s APIs for handling images, which is exactly what I’d suggest the idealistic and pragmatic approach for video ought to be.

Would it be great if there were one codec out there that worked everywhere that web developers could target? Sure. But that doesn’t mean not supporting video codecs that happen to be around anyway, just as I can click on a PSD or TIFF in Safari and see it in the browser.

Ultimately, Google’s stance would have web browsers simply refuse to play back content with non-standards-based content (unless it’s Flash). What kind of “principled” or “non-evil” position is that? Again, if Google were to drop Flash support and make the argument that HTML5 is “the platform”, then it could make some kind of argument about consistency, but that’s not it. Google is making Flash part of “the platform” but not H264.

Chrome Frame, IE Woes, Canvases

Google’s latest salvo in the War Against Terror Microsoft is Chrome Frame, a plugin for IE that replaces IE’s HTML and JavaScript engines with Chrome’s, i.e. Webkit with slightly inferior text rendering and slightly different JavaScript performance.

I’ve spent almost exactly the last six months working on a project called Acumen. My previous job was web advertising, and with web advertising the most important target is the lowest common denominator. In large part this is because no-one wants to see what you have to offer (no-one is going to download a plugin just to see your flashing banner ad). The other side of this is that not only does IE6 represent the lowest common denominator in terms of computer hardware and software, but it’s a pretty handy indicator of the lowest common denominator in pretty much everything else.

(Who is going to click on a “lose weight and get rich by clicking here” banner ad that is pretending to be a Windows error dialog that looks really convincing, except for the fact it jiggles and has a Windows ME frame?)

So it’s refreshing to work on a project that (a) people might deliberately want to use, that (b) is attempting in some small way to make the world a better place (by providing web access to the UA Library system’s special collections, and — ultimately we hope — special collections everywhere) and (c) values standards compliance over the lowest common denominator. The sad thing is that while the front end works beautifully in Safari and IE8, and quite well in Firefox (yes, as of writing Firefox has bugs that Safari and IE8 do not), it explodes in a screaming heap even on IE7.

Acumen's Asset Viewer is an unholy collection of nested divs implemented in JavaScript
Acumen's Asset Viewer is an unholy collection of nested divs implemented in JavaScript It supports two views of a given image -- this is the smaller size.

The main problem for me is the “asset viewer” component — about 400 lines of JavaScript that, when loaded on top of a metadata file (the digital equivalent of a catalog index card) performs a quick query of the server for assets belonging to the metadata file (in other words, if this is a metadata file for the papers of S. D. Cabanis, do we have any of the actual papers available on line?) and then presents them in a vaguely useful way in the page.

The first cut of this component uses jQuery and CSS to assemble an unholy collection of nested divs to produce a fairly straightforward browser UI. You get a horizontal scrolling list of asset thumbnails and, if you click on one, you get to see it up close in a window below. This is very similar to the UI of any number of image browsers such as Picasa or iPhoto. As I said, it works quite nicely on any current reasonably standards-compliant browser.

Even IE8.

But there are all kinds of issues. To begin with, displaying images inside an <img> tag has all kinds of downsides, not least of which is that you can’t wrap a draggable interface around <img> tags because they don’t receive the events. (I blame Safari! Safari — back in version 1.0 — introduced the ability to drag images from Safari into any other desktop app rather than right-click to “Save Image As…”. This is incredibly useful when browsing typical web pages — e.g. it probably reduces the time it takes to download a whole bunch of porn significantly — but for web app developers it means that images don’t get left clicks and drag-type events because the browser is using them.)

Now, I’d love to use <img> tags for the simple reason that they afford you a way of scaling images. If you want to scale an image to 73% of its actual size you (a) load it into an Image object in JavaScript, (b) when it’s loaded and you know its size, you create an <img> tag and sets its height and width to 73% of its native dimensions. But then you can’t scroll the image around by dragging. Also, browsers tend to do an absolutely crappy job of scaling images this way. (If it weren’t for this last problem, I’d probably work around the <img> not receiving events issue using a floating <div>.)

So what I originally did (and if you click the Acumen link you can see it in action) was store two sizes of image — 512×512 and 2048×2048 — and load the images into a <div> tag (which can receive all mouse events). It works pretty well (except in IE6 and IE7 which just can’t deal, and Firefox 3.5 which occasionally loses the plot and thinks you’re clicking on an <img> tag — I’ve reported the bug) but you can only scale to “too small” and “too big”, and generating huge numbers of differently scaled images server-side seems like a Bad Idea to me.

There’s another bad aspect to this asset viewer, and that’s the architecture is ugly. Unless I’m willing to build my own UI library or use an off-the-shelf one I actually like (when will Atlas finally ship?) there’s a huge disconnect between the way you build up a UI inside a browser via the DOM and the way it actually works. E.g. drawing a scrollbar requires the programmer to be acutely aware of the fact that it’s a div (the scrollbar) wrapping another div (the thumb) that has dimensions equal to the sum of its size and border widths. It’s very hard to simply write a routine that knows how to draw scrollbars and another routine that knows how to handle events relating to scrollbars. It’s just ugly. This makes the logic that handles events and refreshes very unpleasant indeed.

Google to the Rescue

The ideal option would be to use the HTML5 canvas for everything. This means you can handle all your events in once place (the canvas, or the element containing the canvas, receives all the events) and you can handle all your drawing — more or less — in one place (everything gets drawn in the canvas). It also lets you do things like scale images to arbitrary sizes on the client (reducing the amount of crap the server needs to store and/or the amount of work the server needs to do — e.g. resizing images or generating giant lists of available images).

But no version of IE, even IE8, supports the canvas.

Then Google did something very clever. It implemented a version of the HTML5 canvas in JavaScript by faking most of the functionality of a Canvas using an unholy combination of VML (Microsoft’s not-quite-equivalent of SVG, I think) and nested <div> tags. All you do is import excanvas.js and initialize any canvas objects you create on the fly with a one-line call and you’re golden.

Well, almost.

Acumen Asset Viewer 2 is implemented using canvas (excuse the debug crap in the top-left corner).
Acumen Asset Viewer 2 is implemented using canvas. Note that arbitrary zooming is supported, so the image has been "scaled to fit".

I’ve got everything working quite beautifully in IE8, Firefox, and Safari. Not quite so well in Opera (but, really, who the frack cares?). In IE7 it works just well enough to convince me that I can probably get it working either by (a) fooling IE7 into going into the right “mode” or (b) making specific allowances for IE7 (basically IE seems to be mismatching metrics for bitmaps and vectors so all the bitmaps end up in the wrong places). This falls into the “lots of extra work to support a broken platform” category, and I’d rather avoid it. Me and pretty much everyone who isn’t a Microtard.

Now, Google has done something both very clever and — perhaps more importantly — cunning. Chrome Frame promises to let us develop standards-compliant websites and just stop caring about IE, which is simply huge. And since it’s completely open source that pretty much means IE can be dead as far as anyone — except web advertisers — is concerned. If I want my site to render nicely in IE then I just add the Chrome Frame tags and forget about it.

Google Wave

I’m in bed with some kind of virus and was thus sufficiently bored to actually watch the entire Google Wave Demo. It’s a very impressive demo and a very nice piece of technology, and it’s being implemented in an idealistic fashion.

The big problem with it is that it isn’t backwards compatible with email. I would have thought that being able to interoperate with email would have been a major priority, but in fact the only mention of it was a half-mumbled remark towards the end about how it would be nice if someone wrote an extension for this. It’s especially odd that this issue didn’t get dealt with more explicitly, since Wave is explicitly intended to replace email.

It’s interesting to imagine what a conversation between a Wave user and a conventional email user might look like — either the email user is going to lose all context each time he/she gets a reply, or the Wave user is going to get a distinctly un-Wave-like mess of pointlessly quoted crap.

The New (Green?) Economy

A very good friend of mine once worked as a Transport Economist. He tells a story about explaining to his boss’s boss about projections on Australia’s fuel consumption going into the next two decades. The results were pretty shocking and the response was, “what about the impact of the new economy” (meaning the eCommerce etc.). The response was “this is the effect of the new economy”. Or to put it another way — delivering large piles of goods to stores is a lot more energy efficient than delivering lots of small parcels to individual households, even if a lot of the stuff delivered to stores is wasted.

Ars Technica brought this blog post (from Google) to my attention. The thrust of it is that a typical query on Google … wait for this … only uses 1kJ of energy and generates about 0.2g of CO2.


1kJ is 1000W for 1s, or 1W for 1000s. It’s as much power as one of the new LED flashlights consumes in a quarter of an hour. Or as much power as a really fast PC or Mac would use in several seconds (ignoring graphics card and monitor, since servers don’t have either).

Or, to put it another way, it’s more energy than you get from eating a Krispy Kreme regular donut. (1 kJ is roughly 240 calories, a regular glazed Krispy Kreme donut is 200 calories.)

Google Chrome

Google has officially announced it will be releasing its own browser, named Chrome. The web comic explaining their thinking is well worth a look, but the key points are:

Each browser tab is a separate process. This means that one web page hanging or script stalling only affects that page. Your browser doesn’t crash or hang — individual tabs crash or hang and can be killed. It’s so obvious it’s brilliant. I imagine Firefox, Camino, and Safari will all follow suit as soon as they can.

Google has been working on a new JavaScript engine, named V8, that’s roughly twice as fast as the latest versions of Firefox or Safari nightly build. Remember that these engines were in turn a lot faster than Internet Explorer or anything else we had a year ago. We’re looking at something like a 10x improvement in overall JavaScript performance over the last two years.

It’s based on Webkit.

It’s entirely open source.