A Little Rhyme

ClickToFlash in action
ClickToFlash in action

Well, there we were.
We were browsing like that.
When what should we see
But that stupid Flash crap?

“Uh-oh,” said my iPad
“I won’t load any Flash
It’s a slow buggy plugin
That drains my battery flat.

That plugin’s a bad one
Don’t you let it appear.
You know what it did
The last time it was here.”

(With apologies to Dr. Seuss.)

What’s really annoying is reading a NY Times article with an accompanying animated diagram implemented — for no good reason — in Flash, and being intrigued enough to try loading it on my laptop, only to discover it was incredibly lame and a waste of time.

In the end, I would love for Flash to be supported under Safari on iOS, but only if it were wrapped in something exactly like ClickToFlash and had the ability to bypass Flash altogether when playing H264 video. I suspect Adobe would balk at both because it would (accurately) portray Flash as a horrible piece of legacy crap that sensible people only use in desperation.

Given a choice of no Flash at all, or Flash everywhere by default, I am happy enough with no Flash at all.

How to use HTML5 <video> tags everywhere and have them Just Work™

Chrome's HTML5 video player interface
Chrome's HTML5 video player interface (just a screenshot, it doesn't actually work)

I’ve been coding raw html since 1994 (when I learned just enough to create a home page for Prince of Destruction — check out the all the <BR>s). I can remember all kinds of random CSS properties and half the jQuery API, but when I need to embed video in a web page I still need to google a bunch of references to get anything that works halfway decently.

Until Google pulled the rug out from under H264 there was some hope that embedding video inside a web page might soon become as simple as embedding images has been since the days of NCSA Mosaic. It’s hard to imagine a non-trivial piece of code simpler than a basic image tag. (If you have any doubts as to whether Google really wants the video tag to succeed, take a look up at how horrible the video player UI in Chrome still is as of version 5. I might add that it can hardly play video without skipping and flickering.)

Whether or not Chrome (and Android) support H264, we’re going to be stuck with a ton of legacy users (Firefox, IE6/7) for a long time, and the only viable solution if we want to encode video in as few formats as possible is Flash.

Well, Flash supports H264 and Webkit supports H264, so what I want is to:

  • encode my video once, in some flavor of H264 that “just works” — ideally I want to be able to open a movie in QuickTime, Save As… and I’m done.
  • embed my video using the simplest, easiest-to-remember html possible, i.e. <video src="foo.m4v" width="800" height="450" controls="true">
  • and have my video play back everywhere

So, here’s my solution. Use HTML5 video tags, and use a tiny bit of JavaScript to automagically convert the video tag into a Flash embed where necessary using the information already in the web page (which can be found in the video tag’s attributes).

I’ve built a simple prototype here. It uses video and h264 detection code from Dive into HTML5 (which means that for now Chrome plays back the video natively) and jQuery for cross-browser DOM manipulation, but hand-coding the relevant bits wouldn’t be that difficult if I wanted to make the code self-contained.

In essence, all I need is: $( function(){ $('video').replaceWith( flash_embed_crap ) } );

Most of the code is simply regurgitating boilerplate html for Flash embedding. (Incidentally, the code will have issues in IE7 thanks to the “Eolas bug”, but if I were to put the script in an external JavaScript file that issue would disappear. Any weird IE behavior should be gone.)

The Flash video player I’m using is very simple — it pretty much uses Flash’s built-in video components and has some simple glue code allowing the embedding web page to talk to it once it’s embedded (not that I’m taking advantage of this here).

It works.

P.S. As per the comments in my source code, I plan to extend this code to provide a number of useful convenience functions and support <audio> tags the same way, and finally to provide a simple but robust open source Flash media player. Aside from anything else, this will help me implement cross-platform media playback in Acumen.

P.P.S. My initial attempts to do for <audio> tags what I’ve just done for <video> tags have not met with much success.

My big problem is that Flash’s FLVPlayer component doesn’t seem to want to play MP3s for me. I’m not sure what’s going on there; it may be some odd combination of Flash security settings hosing locally hosted playback and my ISP’s server configuration hosing remote playback, or FLVPlayer may just be fragile. I’d like the audio and video player UIs to be as identical as possible and for the player to be as simple as possible — having two completely different code paths for audio and video seems like hard work (mostly from a UI point of view).

So the upshot is I can’t change this article’s title to include <audio> tags (yet). Sigh.

Google and the <video> tag

Though H.264 plays an important role in video, as our goal is to enable open innovation, support for the codec will be removed and our resources directed towards completely open codec technologies.

From HTML Video Codec Support in Chrome

Well that sucks.

Gruber asks a few “simple questions” here. Aside from the question of hypocrisy w.r.t. Flash bundling, I think his points are more than neutralized by these ten questions:

You are a proponent of Apple using its influence to diminish the importance of Flash for the web. Yet, when Google makes similar moves to rid the web of a similarly closed and patented, albeit different type of technology, you do not support them. Why is Apple promoting an open web a good thing, but Google promoting an open web a bad thing?

I think that if Google pulled Flash support from Chrome there would be no question that Google were on the side of the angels (although it would still be a dumb thing to do), but since there’s no hint of this it seems purely like a cynical move to hurt Apple’s anti-Flash campaign which will damage HTML5 <video> adoption. I think you can make the argument that HTML5 <video> adoption with H264 as the defacto standard codec is a Bad Thing.

Anyway it’s a bigger mess now than it was before Google decided to do this. Ultimately it will come down to “what will let the most people see the most porn using the most devices?”

Postscript: “standards”

One of the arguments made in favor of WebM/VP8 is that it can be part of the W3C standard, unlike H264, because it’s not encumbered by license fees. The problem here is that WebM/VP8 almost certainly is encumbered (as was GIF in earlier days), it just hasn’t been sued yet because no-one uses it. But this is beside the point — the CSS font-family property supports any font, and almost all the fonts that anyone cares about are encumbered (i.e. subject to royalties, copyright, and so on). Just as CSS font-family can specify a non-free non-open-source font, there’s no reason why a video tag can’t point to an arbitrarily encoded video.

To put it another way:

There’s no conflict between the HTML specification being open and royalty-free and H264 video playback being supported in HTML5 video tags as long as the codec doesn’t need to be implemented by the browser. Just as a slab of text with font-family “Verdana” won’t necessarily display on every browser correctly (if the font is not installed) it would follow that not every video will play back in every browser.

As a practical matter, it would be nice if serving a page with video were as simple an affair as possible. E.g. figuring out which video to serve didn’t involve sniffing the browser, operating system, and so forth; better yet, if one video format worked everywhere. As a practical matter right now H264 is the best candidate. VP8/WebM will never be the best candidate because by the time there’s a critical mass of hardware support out there it will be obsolete. This is a stupid, stupid fight.

And yet one more thing:

It’s interesting that the companies still in favor of h264 (Apple, Microsoft) are precisely those companies who do not implement the codec in the browser. Apple and Microsoft both implement h264 as a plugin architecture at OS level rather than a plugin at browser level (a much worse thing — see this excellent piece that daringfireball brought to my attention).

Is a it a font or an image file format?

The flipside of my argument that H264 should be considered analogous to a font is that, generally speaking, text is still legible when presented in the wrong font. By that argument H264 is more like an image format (JPG, PNG, etc.). If we accept this argument — which I’d say is the most h264-hostile stance (within reason) to take with respect to video codecs — then consider that most browsers simply let you display pretty much any image that’s convenient inside an <img> tag (sometimes badly, as per Internet Explorer’s notorious mishandling of PNG files over the years), generally by using the underlying OS’s APIs for handling images, which is exactly what I’d suggest the idealistic and pragmatic approach for video ought to be.

Would it be great if there were one codec out there that worked everywhere that web developers could target? Sure. But that doesn’t mean not supporting video codecs that happen to be around anyway, just as I can click on a PSD or TIFF in Safari and see it in the browser.

Ultimately, Google’s stance would have web browsers simply refuse to play back content with non-standards-based content (unless it’s Flash). What kind of “principled” or “non-evil” position is that? Again, if Google were to drop Flash support and make the argument that HTML5 is “the platform”, then it could make some kind of argument about consistency, but that’s not it. Google is making Flash part of “the platform” but not H264.

Affordable CS5 Alternatives

The best affordable Adobe CS5 alternative is the academic version — e.g. CS5 Web Premium costs $1799 retail, $599 to upgrade from CS4 (more from earlier versions), and $549 brand new at Academic prices. Design Standard is $1499 vs $449. (There are even cheaper “Student & Teacher” options, but they can’t be upgraded and shouldn’t be used for commercial work.)

To understand how good a deal this is — Web Premium includes Dreamweaver, Fireworks, Photoshop Extended, Illustrator, and Flash. Coda, Photoline, and Intaglio would cost $280, and this doesn’t give you a Fireworks or Flash replacement. And to say that Photoline and Intaglio aren’t full replacements for Photoshop and Illustrator is a huge understatement.

If you aren’t eligible for an academic discoun, then things get more complicated.

The core components of Adobe CS5 are:

Photoshop: there’s no real alternative to Photoshop if you use Photoshop’s high-end capabilities. If you don’t use any of them then you can get away with a much cheaper program, such as Photoshop Elements, Photoline, Paintshop Pro, Acorn, Pixelmator, or even The GIMP. If you want natural media style painting abilities, Photoshop is actually not the best choice, and you should look at Painter or Art Rage (or Sketchbook Pro if you’re using a Tablet PC).

Illustrator: there’s a very good free alternative to Illustrator if you’re willing to live with an X-11 app, and that’s InkScape. Despite being X-11, it actually is very usable (more usable in some respects than Illustrator). Otherwise, there’s Intaglio, Lineform, Zeusdraw, VectorDesigner, and EasyDraw among others. (Sorry, but I’m not familiar with Windows alternatives, and Googling didn’t really help — ah Macs, so starved for software.)

Flash: while there are third-party Flash development tools (SWF is a pretty well-documented format), most are jokes or one-trick ponies. If you want to do serious work with SWF, you’ll need Flash, and you’ll probably need Photoshop and Illustrator since no other graphics tools are well-integrated with Flash.

After Effects: there are lots of alternatives to After Effects, but they’re all in the same price ballpark. If you’re a Final Cut Studio user, Final Cut Pro and Motion pretty much match After Effects’s feature set and are well-integrated. Similarly, if you’re an Autodesk/Avid user then why are you reading this paragraph?

Premiere: there are no good free video editing packages that I know of, although if you don’t need Premiere’s capabilities there’s iMovie and Final Cut Express (on the Mac).

Soundbooth: many, many alternatives, including the free and quite good Audacity.

On Location: there was a really awesome program called On Location, way back, that was essentially like Spotlight only over ten years earlier. This isn’t it. This is essentially a utility for capturing, adding metadata to, and otherwise managing digital footage when on a shoot, and I don’t know of any obvious alternatives (certainly no cheap ones).

Dreamweaver: if you’re not a hardcore Dreamweaver user (or, for example, a Cold Fusion developer), chances are you don’t need it at all. There are plenty of free and cheap web development tools out there. I’d recommend Coda for technically-savvy Dreamweaver users who are looking for an alternative and would like to give up their training wheels.

Contribute: I haven’t used Contribute, but from what I understand it’s a simplified web content editor aimed at people who essentially want to populate templates created by someone else. I don’t really like any of the desktop web development tools aimed at non-technical types, but if forced to pick one I’d probably pick Rapidweaver. Personally, I think this kind of thing is best handled using web-based content management tools such as WordPress.

Fireworks: Fireworks is a strange product. On the one hand most of its functionality is superfluous and it has a truly terrible user interface. On the other hand it has a number of — for UI designers — almost indispensable functions, notably pixel-centric vector drawing tools. If you need it, you need it.

InDesign: the only real alternative to InDesign is XPress (which, if anything, is more expensive and comes from a far more obnoxious vendor), unless you’re not using InDesign’s higher end capabilities, in which case your word processor (notably Pages) is a great alternative that you already know how to use.

Acrobat: again, if you need it you need it. Otherwise, not. Personally, I can’t stand Acrobat and don’t even want Acrobat Reader on my computers; for Mac users there’s Preview and the Print-to-PDF function both of which are far more pleasant to use than Acrobat (until you get technical). There are many better alternatives to Acrobat Reader, until you need to do something complicated, like fill in forms.

iPad: Netflix App, Musings, & Wishes

I read that Netflix is coming to the iPad. Given that Adobe already has a widget for turning Flash applets into iPhone apps, there’s no reason not to expect Hulu et al to be able to provide iPad apps for their content as well.

So, given that we’ll almost certainly have Netflix, and most likely Hulu working on the iPad, and that many newspaper sites will simply switch to HTML5 / H264 — what’s missing? Nothing of note, it seems.

The interesting thing is that by allowing Adobe Flash apps in via the App Store but not via the web, Apple makes it easier for developers — of casual games for example — to monetize their products while improving the user experience. Unless you like being deluged with butt ugly ads while playing online games.

It looks like the iPad may be a very complete media consumption device pretty much on day one. What I’d really like to see for the iPad are:

  1. A clamshell case that effectively turns the iPad into a laptop with keyboard. The ideal design would probably have a pass-through dock connector, stereo speakers, USB slot, SD slot, and extra battery power.
  2. A paint program — I know Brushes was demoed on stage at the launch (I own the iPhone version), but despite its high-profile user, I find it pretty poorly designed and would love to see major improvements in its UI or some real competition.
  3. A image editor (i.e. Photoshop-like program). The obvious candidate here would be a port of Acorn since the program is already 100% cocoa, and the UI is almost iPad ready. (Then again, the Pixelmator team has been really quiet for a long time, notwithstanding vacations.)
  4. A 3d sketch modeler (like Teddy or Curvy). The iPad would be perfect for this kind of 3d “sketching”. (I’m actually tempted to have a shot at writing such a tool.)
  5. Coda or something similar — i.e. a combination programmers’ editor and FTP client (ideally with some kind of simple image editing functionality so you don’t need to perform acrobatics to crop or scale images, etc.) — but then projects like Bespin would allow you to integrate basic editing tools into your websites.