Kindle vs. the Apple \w+

A DIY bookscanner from diybookscanner.org
A DIY bookscanner from diybookscanner.org

One of the interesting things about Apple’s foray into mass media is that it has successfully become a mass media giant without making media a major profit center. If you read analysis of Apple’s earnings reports you won’t see much dwelling on how many music tracks Apple sold.

So, today I read this article in Wired about a DIY book scanner built for $300 in parts. The thing that really caught my eye in the article though was that apparently you can buy a commercial book scanner for around $5000.

If we assume Apple’s business model for books will be to sell them at break even with as little copy-protection as possible (i.e. it’s movie and music model) then this means when and if Apple’s \w+ finally comes out it will have a major advantage competing against Amazon which makes money on razor blades, not handles. (Forgetting any other differences, such as the Apple product’s almost inevitably better design.)

It wouldn’t be too hard for Apple to offer, or encourage third parties to offer, a book scanner for iTunes (if it’s still called iTunes) to allow users to rip their existing libraries. It’s a killer application — I for one would rather rip my library than move it. Heck, I’d probably rather build a scanner and rip my library than move it.

Interesting times.

iPodRip

A quick word on the iPodRip saga. The iPod has some simple (and not incredibly effective) protections against users copying music from an iPod to a Mac or PC. It’s one of those things that “only stops honest people”. The reason that iPods do this is that Apple was very anxious not to make the iPod appear to be (and actually be) a vehicle for piracy. Imagine how much more often music would be pirated if iPods let you sync music in both directions. Imagine how hard this would be for Apple to implement.

This is not about technology, or even legality — it’s about Apple’s relationship with music publishers.

iPodRip  is one of many programs that circumvents this restriction and lets you copy music back off your iPod. There are, of course, legitimate uses for it — imagine if you’re the proud owner of a large library of music bought from the App Store or painstakingly ripped from CDs, you copy everything onto your iPod, and then your hard disk dies. More plausibly, you might simply want to copy all your music from your home computer to your iPod and then to your office computer or laptop. I suspect, however, that the vast majority of iPodRip users are not doing this — more likely they’re just college students who want a dump of everything on their friends’ iPods.

So here’s the point: iPodRip is simply a program written by someone with no respect for trademark or copyright laws who is whining because Apple is doing the absolute minimum to protect its brand name and relationship with music publishers from a parasite. iPodRip wouldn’t be successful without the iPod’s success, and the iPod wouldn’t be successful if Apple hadn’t been able to cut deals with music publishers, and Apple wouldn’t have been able to cut deals with music publishers without demonstrating that it was reasonably serious about not turning iPods into a huge piracy channel. Apple can’t stop programs like iPodRip from existing or — easily — prevent them from working, but it can stop them using their trademarks.

The Eolas Bug

Back in 2005 I was working at Fastclick.com (now ValueclickMedia) and we found out that Microsoft had just settled with a company called Eolas that claimed to have invented the browser plugin, and that the net effect of the settlement was that Microsoft would patch Internet Explorer to require a click to activate content rendered by a plugin if — and only if — the embedded html code was staticly included in the web page or created by code included in the web page (e.g. produced via JavaScript document.write() calls).

Did you get that?

Let me try to make it clearer:
<html> ... <embed ... > ... </html> fails, the embed code is in the page itself.

<html> ... <script> document.write('<embed ... >');</script> ... </html> also fails, the embed code is created by javascript in the page itself.

<html> ... <script src="write_embed.js"></script> ... </html> works!

And by “fails” I mean that instead of your plugin stuff appearing you get a dotted frame and a “click to activate” message. (Note that ads are delivered as linked JavaScript (no problem), server-side JavaScript (problem), and HTML to iframes (problem). Also note that Adobe has since modified the way it publishes Flash to html — adding a separate .js file to do the embedding.)

Now web advertising, as it was conceived in the late 90s (which is largely the technology we’re still stuck with) is a primitive beast. I won’t go into details but it’s sufficiently primitive that most flash ads are stored in ad network databases as HTML or JavaScript that writes HTML at page load (one of the most annoying things about coding for browsers). And most of this code was written by people with no clue what they’re doing using random versions of Dreamweaver, Flash, or Notepad. The point is that you can’t just fix your single “embed flash.js” file, but you need to spelunk tens of thousands of pieces of adcode and rewrite them all, or do something clever.

I ended up doing something clever, which was a simple script that trolled the page it was in for Flash ads, ripped them out, and then put them back in again (but now, because they’d been inserted by JavaScript not directly in the page, no Eolas Bug). Now, implementing the Eolas Bug was presumably no mean feat — I guess it works off the same metadata JavaScript sandboxing has to use to decide which script can see and talk to whom. Anyway, moving right along…

Now Eolas is suing a whole bunch of people over the same patent.

OpenDoc may turn out to be useful for something after all.
OpenDoc may turn out to be useful for something after all.

I hate software patents. I don’t even like them when they’re being used to screw Microsoft. But I particularly hate it when someone buys a patent (this patent was originally awarded to the University of California), sits on it for years, and then starts suing people over it — having done nothing useful with it or anything else in the meantime. But let’s take a look at this patent on its merits:

A system allowing a user of a browser program on a computer connected to an open distributed hypermedia system to access and execute an embedded program object. The program object is embedded into a hypermedia document much like data objects (1). The user may select the program object from the screen (2). Once selected the program object executes on the user’s (client) computer or may execute on a remote server or additional remote computers in a distributed processing arrangement (3). After launching the program object, the user is able to interact with the object as the invention provides for ongoing interprocess communication between the application object (4) (program) and the browser program. One application of the embedded program object allows a user to view large and complex multi-dimensional objects from within the browser’s window. The user can manipulate a control panel to change the viewpoint used to view the image. The invention allows a program to execute on a remote server or other computers to calculate the viewing transformations and send frame data to the client computer thus providing the user of the client computer with interactive features and allowing the user to have access to greater computing power than may be available at the user’s client computer (5).

I’ve labeled what seem to me to be the main claims in the abstract. (I will get to the specific claims — er claim — later, and IANAL let alone a patent lawyer). Note that a lot of text — none of it bold — is devoted to implementing something like VRML but rendered on the server — I don’t think they’ll be suing anyone over that nugget.

  1. My initial reaction to this is that it’s not how browsers work at all, but on rereading perhaps what they mean by “embedded” is “linked” (after all, both data objects and plugins are linked to web pages, not actually embedded in them). The devil may be in the details here.
  2. Nope, the user does no such thing.
  3. This kind of describes networked computing for which there is ample prior art.
  4. This is important — browsers do pass event information to plugins, and plugins do tell browsers how to draw pages etc.
  5. This is exactly the opposite of what plugins do. Plugins let you do stuff on the client specifically to avoid wasting CPU time on the server and/or bandwidth.

So, based on the abstract, one claim is utterly trivial and covered by ridiculous amounts of prior art, item 1 is possibly interesting, item 4 is right on the mark and the others are utterly irrelevant. But then from what I can see patent lawsuits, especially software patent lawsuits, often seem to hinge on some tiny nook or cranny of a patent that seems to have no real relevance to the thing described in total.

Let’s look at item 4, communication between the browser and the plugin. This seems like it might have some legs. Did anyone before 1994 have a neat mechanism for automagically embedding arbitrary data with arbitrary software to handle it inside a container program? This sounds a lot like SGML conceptually, and it also sounds a lot like a really exciting and innovative type of technology that turned out to be a giant stinking turd: OpenDoc.

While no-one remembers OpenDoc particularly fondly, it was a shipping piece of software, it ran — often for minutes — without crashing, and it did some pretty remarkable things. You could open a document comprising a collection of pieces of data for which you might or might not have specific editor or viewer components, and the OpenDoc framework would just deal with it. It would allow the components (“editors” or “viewers”) to tell it which kinds of data they could handle, and events were passed back and forth (to allow editing, obviously). What’s more, conceptually, OpenDoc was very much inspired by other shipping programs such as PageMaker and Quark XPress which had their own plugin architectures which did similar (but less general) things. Since OpenDoc shipped in 1992, and was in development for years beforehand, I think it’s safe to say that Apple has prior art up the wazoo. So, good-bye item 4.

And of course OpenDoc worked just fine across a network. In fact, Apple shipped a web browser component for OpenDoc (Cyberdog) which was perhaps the most well-loved feature of OpenDoc and the only thing most users missed when Apple finally scrapped it.

OK, so back to item 1. Let’s drill down into the blow-by-blow claims in the patent’s body. Actually the two meatiest and most incomprehensible chunks of text pertain to the first point, so let’s take a look at first and slightly more comprehensible chunk:

…  a browser application, that parses a first distributed hypermedia document to identify text formats included in said distributed hypermedia document and for responding to predetermined text formats to initiate processing specified by said text formats; utilizing said browser to display, on said client workstation, at least a portion of a first hypermedia document received over said network from said server, wherein the portion of said first hypermedia document is displayed within a first browser-controlled window on said client workstation, wherein said first distributed hypermedia document includes an embed text format, located at a first location in said first distributed hypermedia document, that specifies the location of at least a portion of an object external to the first distributed hypermedia document, wherein said object has type information associated with it utilized by said browser to identify and locate an executable application external to the first distributed hypermedia document, and wherein said embed text format is parsed by said browser to automatically invoke said executable application to execute on said client workstation in order to display said object and enable interactive processing of said object within a display area created at said first location within the portion of said first distributed hypermedia document being displayed in said first browser-controlled window.

In other words, the “embedded” data object has “type information” used to pick the external plugin that displays and allows interaction with its data in a display area in the browser window.

This appears to be the single thing on which they are basing their lawsuit. It’s no small thing, but it does seem to me to be obvious. After all, how do browsers handle links to non-textual embedded data that doesn’t require a plugin. The only major example in 1995 was images, e.g. gifs and jpegs. These were, and are, embedded using <img> tags which contain a URL (here’s where the actual image data is) and dimensioning information (here’s the width and height). <img> tags don’t contain any real metadata beyond themselves (I’m an <img> tag, which connotes I refer to an image) and — arguably — the “filename extension” at the end of their src URL (which is often garbage).

Now, browsers don’t render images themselves, as a rule, they call on external libraries or the operating system to do it. So an <img> tag is basically saying “here’s a link to some image data, and this is how big I’d like you to render it” and the browser gets this information and calls external software to render it. Images aren’t interactive, so that’s not covered — but OpenDoc and PageMaker and so forth have interprocess event handling covered in spades.

So let’s suppose you’re a programmer and you know about <img> tags and it occurs to you that there’s no way to embed QuickTime movies (say) in your web page. And no way to handle Illustrator files. And no way to display 3D models. What to do? Any decent programmer is going to immediately consider coming up with a new kind of tag for each kind of document, and then replace the code that handles <img> tags with a huge switch statement somewhere and decide this sounds like hard work and bad architecture.

Abandoning the totally obvious — but obviously bad — solution, the next most obvious solution is to invent a generic embed tag and move the information connoted by the tag’s name (“img”) into the attributes of the tag itself (e.g. type=”image”). And, again obviously, expecting the browser to be modified and recompiled every time we pull a new media type out of our asses is stupid, so let’s generalize the “get the OS to render it” side of things by mapping types to specific pieces of software (“plugins”). Oh, and the data might be interactive (e.g. movies need a play button and they need to be refreshed more often than images) so we’ll need to pass events to and fro. OMG call up the patent attorney.

The only thing missing from OpenDoc was the ability to embed a url for downloading the “part editor” you need if you don’t already have it. OK, that may be genuinely novel. But how is it also not obvious given this is web browsers we’re talking about. I didn’t use OpenDoc for much serious work, so I don’t remember what OpenDoc did when you opened a document containing parts for which you lacked editors — I know it coped, but I can’t remember exactly how much information it provided you with. If it told you something about the component needed to deal with the part and from whom it might be obtained, then Apple even has prior art for this.

It’s easy to claim something is obvious in hindsight. I think I’ve demonstrated that — at the very least — OpenDoc represents prior art for almost everything in the patent (even the minutiae). If OpenDoc documents were web pages, and OpenDoc data embedding automatically included a way to obtain (download) the software required to handle it (and — arguably — it did, just not URLs since OpenDoc wasn’t a web browser) then this patent would be completely meaningless. (And, frankly, if web pages were OpenDoc documents it would be pretty awesome — if OpenDoc were still around and didn’t suck.)

Corrections / Amendments

After looking over more documents, it looks like the correct chronology is more like Publish and Subscribe (part of System 7.x) and OLE (Windows 3.x) were 1990-1991, OLE II (Windows 95) and OpenDoc (System 7.5) were being specced in 1992, both shipping in the 1994-1995 timeframe. (But again, in-house stuff will predate this significantly.) And OpenDoc was (eventually) part of a huge (unwieldy) anti-Microsoft alliance, so all of its concepts were very widely and openly documented long before it shipped.

Acknowledgements

I got the screenshot from this article on MacTech — I hope they don’t mind. It sure beats reinstalling OpenDoc on an old Mac. Ugh.

I also found this really interesting paper written by people on the OpenDoc team. It’s a sobering reminder of how brilliant, idealistic, thoughtful people can produce really terrible software. It also failed to address the question of how OpenDoc handled missing editors. Maybe I will have to find an old Mac and reinstall it.

ODF vs. OOXML

This is kind of outside my usual bailiwick — or not, I guess — but I read an article on Groklaw concerning the recent brouhaha over Stallman allegedly calling Miguel de Icaza (the guy behind Mono) a traitor — allegedly — for developing Mono. (One line summary: Stallman didn’t call him a traitor over Mono, he called him a traitor for acting as a schill for Microsoft and in particular OOXML.) If you’re interested in open standards, digital archaeology, how standards processes work, and the whole philosophy of free software, the entire — lengthy — article is very much worth reading. This email (to Miguel) in particular struck a nerve:

Re:OOXML. (Score:5, Insightful)
by stilborne (85590) on Monday September 10 2007, @10:41PM (#20548069) Homepage

> but we have to support them both *anyways*, so its not like its a big deal.

Holy mackerel.

First: I really don’t care to get into a pissing match about the deficiencies of OOXML as a possible standard (they are legion and often fundamental; and whether or not you understand that and/or choose to minimize the severity of these things changes nothing). I will say that I’m very happy to finally see at least *some* open documentation for the new Microsoft Office format; that has to make things easier for the people implementing filters. As such I am completely unsurprised that those people are happier than they were a couple years ago. In fact, I’d be surprised if they weren’t. That part is probably something you and I agree on =)

However the quote above is utterly shocking. Let me explain what I mean:

You are right that we have to support both OOXML and ODF out of practicality. But you know what? That sucks. It would be best for everyone if there was only one format to support. Nobody would lose in that scenario, except perhaps the owners of companies with business models that depend on format variance to sell their product.

In the case of document format storage, a standard is truly important because formats (poor or not) that eventually lose implementations over time carve out blank spaces in our history once we can’t read them properly. These same formats are also the source of certain information inequalities in society (e.g. those who can’t obtain an implementation for financial, social or political reasons). This may not matter so much for Acme Inc’s quarterly reports but it sure does for government, health and other socially vital information. Remember when some hurricane Katrina victims couldn’t use the FEMA website because they had slightly older computers? This isn’t a made up boogyman, this is stuff that bites us as a society fairly regularly. Now imagine a hundred years from now when we can still read the constitutions of our countries, research papers, poetry and other examples of human kind’s great literary works that are hundreds or even thousands of years old … but can’t read the documents we’re creating at the start of the 21st century. How will we learn from our history if we can’t study it fully?

Getting proprietary formats out of the way as soon as possible so that we do not extend this mess any further than necessary is absolutely the responsible thing to do in light of our (hopeful) future.

By allowing OOXML to pass from “specification” to “international standard” would be doing exactly that: extending the problem as it will give years if not decades more life to the format. If OOXML was rationally implementable and properly documented, it wouldn’t be as big of an issue. It would be, as you put it, simply suboptimal. The fact of the matter is that OOXML is not rationally implementable and not properly documented. That’s why it lost the recent vote; it wasn’t because of lobbying (and trying to imply that when Microsoft got its hand caught in the cookie jar is pretty ballsy, by the way). Are some interests acting out of concerns for their business models or pet projects when they rally for ODF and against OOXML? I’m sure they are; but that alone isn’t reason to dismiss the fact that OOXML is problematic and that we don’t need two standards (any more than it is to dismiss OOXML just because it comes from Microsoft).

So please, admire OOXML for what it is: a step forward in documenting what historically has been one of the more pernicious sets of file formats we’ve had to deal with; but don’t mistake that for being a reason to make it an international standard which will only prolong the issues that are part and parcel of the Microsoft Office formats, even in this current version of the specification.

I know that having a bunch of people shit on you in public sucks major donkey nuts and certainly would put most rational people into a rather ungracious mood, but please think above that noise and consider with your intelligent mind exactly what you are promoting here by saying “it’d be fine as an ISO standard”.

ODF is currently incomplete (formulas, blah blah blah) but has exactly the right foundations, the right momentum for support across the industry, and the missing parts are being filled in very nicely. Properly, I might add. Those are the attributes that people who care about Freedom should appreciate, respect and support. In this case, that support means being willing to reject a competing specification that is not well suited for such international ratification. And that, in a nutshell, is why this is precisely a “big deal”.

(As an aside, this also shows how much more intelligent the level of debate on Slashdot often gets.)

CNN on Hulu: Argh, the Stupidity… It Burns!

US TV Network Death Watch, Revisited

There’s been some talk about hulu delivering content via the iPhone lately, and I’ve been pretty much ignoring it, but my boss sent me a link to this piece on CNN, which is almost spectacular in its lack of accurate content. Just so you don’t need to read it, here are some key points of from the article.

  • Hulu has deals with all the major networks except CBS (wrong, it doesn’t have a deal with ABC) OK my stupidity burns too. The whole starting point for the article is that Disney and hulu cut a deal, abc.com’s content will go on hulu and Disney owns a chunk of hulu. Oops, my bad.
  • Hulu is hurting for traffic (wrong, it’s way ahead of ABC and CBS, for instance, on traffic; not ESPN though)
  • Hulu would need Apple’s approval to distribute video on the iPhone (wrong, it doesn’t need Apple’s approval to deliver content via the web, only via an app)

This is exactly why we need the mainstream media to report the news. Obviously bloggers aren’t professionals who can research facts and chase up key details. Obviously a mere blogger can’t log on to hulu and attempt to watch an ABC show to see if it works, or visit Alexa.com to find out what Hulu’s traffic is like, or visit developer.apple.com to figure out whether it’s possible to deliver content to the iPhone without needing Apple’s approval. It takes a media insider with contacts and professional journalism skills.

Aside: hulu’s Alexa ranking is particularly impressive when you consider that it’s not a site designed for maximizing page counts. A typical user will go there, look up a show, and then watch that show for 50 minutes. That’s maybe three or four page views in an hour compared with sites that break up a two page article into four page views.

The Real Issues

There are two definite issues Hulu would face in delivering content to the iPhone and two probable.

First, Hulu suffers from a really embarrassing technical problem: often, when you watch one of its shows, it gets to the place where an ad would go, and — miracle of miracles — it actually has an ad to show you, but, for some utterly unbelievable reason, it doesn’t show you the ad, but instead shows you 15s or 30s of black. This is “dead air” and is a fiasco. Getting this stuff right should not only be a top priority, it should be easy to do. Speaking as someone who put together a video advertising system for Valueclick Media in about three months with the help of two other people, it isn’t rocket science, and this is where they make their money. It’s like trying to create McDonalds and forgetting to make sure the cash registers work.

Second, Hulu doesn’t have any advertisers. (Well, it has so few it’s embarrassing.) If you watch hulu much at all, you’ve probably seen a ridiculous number of PSAs (public service announcements). These are ads produced by the Ad Council to be used as filler when no paying ad is available or to fulfill a public service requirement (e.g. TV stations are required to show a certain amount of stuff for the public good, which is how you occasionally see an ad against teen drunk driving in prime time). In short, Hulu’s problem isn’t traffic (page hits) but fill (ads to put in page hits). If Hulu were a magazine it would be thin and have very few ads in it, and there would be subscription cards offering lifetime subscriptions for only the cost of three issues at the news stand.

This leads to the first probable problem. Clearly, hulu has terrible cash flow (i.e. it’s bleeding money), and spending its cash reserves to get more traffic when it actually needs more fill only makes sense if its investors take a long view. Maybe they do, maybe they don’t, maybe they’re on the fence. But convincing them is going to be an issue. (It’s possible that hulu’s investors are simply insane, since they don’t seem to be too incensed about the first issue mentioned.)

And the final probable problem is that hulu almost certainly has a contractual restriction against serving video content to a range of IP addresses which the IP owners (NBC and Fox) can change at whim. It’s quite possible that serving video to cell phones (Apple’s or anyone else’s) might require renegotiating their licensing deals. And we all know how reasonable and sane the networks are.

Note that the only technical issue here is (I would hope) pretty simple and actually applies to hulu’s existing business — delivering ads (when there are ads to deliver) should be the absolute top technical priority for a company whose bottom line is entirely dependent on doing so. Hulu needs to fix that whether or not it targets cell phones. Note that the second problem (convincing investors it’s worth doing) isn’t a consequence of it being hard to do — it’s perfectly possible to be sitting in a high tech company with money sitting in a giant pile “on the table” and have a perfectly good, inexpensive or free no-risk plan to get that pile of cash, and be unable to convince anyone that the company should do it. Believe me, I’ve been there. (“We wouldn’t want to cannibalize our other [non-existent] revenue streams.”)

So will hulu do it? No clue. But there’s no technical issue, and the only third-party dependency are hulu’s network partners. But we all know how reasonable and sane they are.

Technical Aside: isn’t Hulu flash-based?

To deliver video on iPhone without requiring Apple’s approval requires using a codec supported by the iPhone. Luckily, the iPhone supports the best video codec available (H264) and so does Flash. Now, I don’t know how Hulu’s video is currently encoded, so they might well have to re-encode everything for the iPhone (et al), but doing so is technically trivial and — really — you want lower video bandwitch (and resolution) for small devices anyway. It wouldn’t hurt hulu to be able to deliver a lower-bandwidth version of its content to non-mobile devices and laptops anyway, since a lot of folks (like me) are bandwidth constrained by wireless rather than their internet connection, and mobile users always are.

Having re-encoded video in an iPhone (or whatever) friendly format, the only remaining issue is inserting ads and providing a seamless experience. Luckily this is easily done using either Quicktime’s interactivity features (i.e. building a QuickTime shell around streamed content) or fairly simple JavaScript. If this is seriously an issue for hulu (or anyone else) I can knock something up for them in a week or so (from scratch so you don’t get sued by my former employer). Seriously, I’m cheap!

Post Script: Disney Deal

While hulu’s inability to place ads where they go is pitiful, abc.com’s was worse. Not only does — or should I say did? — abc.com require you to download a special “player” to watch their content (presumably Adobe AIR, which is to say Flash in a web page pretending to be an application), but the player would drop out of full-screen mode to play ads, occasionally hang on ads, and — like hulu — often show dead air instead of ads which, combined with hanging and disabling the UI during ads, was truly annoying. So if anyone had a good reason to go the hulu route it was Disney.

Now, given Jobs is Disney’s single largest shareholder, and Disney now owns a chunk of hulu, maybe hulu will get fixed.