A few weeks ago I started down the road of implementing some JavaScript that would scrap the html5 video and audio tags from a page and swap in Flash embeds on an as-needed basis. This would allow simple, clean HTML5 web pages to Just Work where the browser was capable, and Just Work (via Flash) otherwise.
Oddly enough, while I got all the video stuff done in an hour or two, with a few more hours of tweaking, audio proved to be surprisingly intractable.
To begin with, the audio tag itself is not terribly well implemented. E.g. the audio tag is supposed to support playlists, but as far as I can tell it isn’t implemented on any browser. So, scratch that functionality (for now, at least).
Next, unlike in QuickTime, audio doesn’t work like video with a “zero-size” visual component. So, you can’t dimension the audio tag’s control UI in any browser by setting the width. (In my ideal world, the audio, video, and img tags would be interchangeable and Just Work. But then I’ve been spoiled by QuickTime which has taken this view of media since version 1.0.)
Third, Flash, also unlike QuickTime, uses completely different and incompatible APIs to handle audio and video. (In Flash, as far as I can tell, external MP3s can only be played back in code using the Sound and SoundChannel APIs which lack rudimentary functionality, such as telling you whether or not something is actually playing.)
(It’s pretty depressing how all the folks who came after QuickTime and basically copied it did such a piss poor job.)
With Flash it gets quite a bit worse. To begin with the FLVPlayback component is designed to work best with special controls that work in some bizarre and undocumented way. My initial thought was “why invent the wheel?” I built a bog standard FLVPlayback wrapper and then tried to write glue code that would let the FLVPlayback controls work with the Sound and SoundChannel components (which are just totally annoying — but that’s a separate piece). Big mistake — I even tried paring the problem I was having down to an absolute minimum and posting it to Stack Overflow with no luck. Oh well.
Every time I’ve built a video player in Flash before I’ve ended up rolling my own controls from scratch. Now I knew why! So back to the drawing board. I designed a very minimal UI in Illustrator and then quickly got it working in Flash and voila.
Now, as a side note, let me just rant about how stupid the audio side of Flash is. With video you get the FLVPlayback component (actually two different ones) with a huge bunch of very easy-to-use (if ugly) controls that you can customize to any extent (you can wire just the controls you need, or simply pick from a huge array of prebuilt control panels and it all Just Works). At API level there are oodles of obvious convenience functions which will tell you where the playhead is, whether the video is playing, what its duration is, and so on. Exactly what you’d want and expect.
With sound there’s a Sound component and a SoundChannel component. To play a sound you create a SoundChannel (something like var s:SoundChannel = MySound.play( fromStartingPosition ) ). To stop a Sound, or even pause it, you stop the SoundChannel then destroy it. (And keep as much state as you’d like to track yourself, such as the playhead position, because it’s GONE). There are precious few useful methods or properties (e.g. no way of actually telling if a sound is playing other than seeing if the SoundChannel still exists). The whole thing looks like an ugly afterthought that hasn’t received any love for years. (And I figured out how all this works by looking at the top-ranked open source Flash MP3 player around, so smarter people than I haven’t found any better option.)
Anyhoo, I’ve managed to create a reasonably elegant solution that Just Works for video and audio (and the edge case where you accidentally embed an mp3 in a video tag will also work, maybe). In essence, you can play H264 video and MP3s pretty much anywhere with this code, and all you have to do is use video and audio tags and link in the javascript file.
It’s not quite perfectly cleaned up yet, but it’s pretty nice. I need to do a bit of extra work to have the Flash audio player figure out when a track has actually ended, and (for my own purposes) I’ll need to extend the Flash player’s API and then write an abstraction layer for handling both HTML5 and Flash media transparently, and then I’ll implement track-list support on top of that (in essence, I need to do all this to meet functional requirements for Acumen).
Post Script
It’s amazing just how bad Flash is at its bread-and-butter functionality. If you look very closely at the screen grab at the top you’ll notice that there’s a pixel missing from the bottom-right of the progress rectangle. My immediate assumption was that I had either (a) inadvertently deleted one pixel of the outline somehow (it was an actual Flash “stroke”), or (b) there was a slight error in the dimensioning or positioning of the object (e.g. it wasn’t precisely located at 0,0 or its dimensions weren’t quite right.
I’ve been constantly amazed that a program chiefly aimed at and used by artists has problems rendering bitmaps correctly (until recently, any alpha-channelled bitmap was virtually guaranteed to be displayed incorrectly in Flash — it was such a common issue that artists would routinely add extra rows of pixels to their images in an attempt to forestall problems). It seems that Flash has now extended this to vector art. It turns out my rectangle was perfect in all respects, showed up correctly in the editing view, but mysteriously lost a pixel when “built”. So I deleted the rectangle and replaced the outline with a solid rectangle with another rectangle “subtracted from it”. And now it works.