Back in 2005 I was working at Fastclick.com (now ValueclickMedia) and we found out that Microsoft had just settled with a company called Eolas that claimed to have invented the browser plugin, and that the net effect of the settlement was that Microsoft would patch Internet Explorer to require a click to activate content rendered by a plugin if — and only if — the embedded html code was staticly included in the web page or created by code included in the web page (e.g. produced via JavaScript document.write() calls).
Did you get that?
Let me try to make it clearer:
<html> ... <embed ... > ... </html> fails, the embed code is in the page itself.
<html> ... <script> document.write('<embed ... >');</script> ... </html> also fails, the embed code is created by javascript in the page itself.
<html> ... <script src="write_embed.js"></script> ... </html> works!
And by “fails” I mean that instead of your plugin stuff appearing you get a dotted frame and a “click to activate” message. (Note that ads are delivered as linked JavaScript (no problem), server-side JavaScript (problem), and HTML to iframes (problem). Also note that Adobe has since modified the way it publishes Flash to html — adding a separate .js file to do the embedding.)
Now web advertising, as it was conceived in the late 90s (which is largely the technology we’re still stuck with) is a primitive beast. I won’t go into details but it’s sufficiently primitive that most flash ads are stored in ad network databases as HTML or JavaScript that writes HTML at page load (one of the most annoying things about coding for browsers). And most of this code was written by people with no clue what they’re doing using random versions of Dreamweaver, Flash, or Notepad. The point is that you can’t just fix your single “embed flash.js” file, but you need to spelunk tens of thousands of pieces of adcode and rewrite them all, or do something clever.
I ended up doing something clever, which was a simple script that trolled the page it was in for Flash ads, ripped them out, and then put them back in again (but now, because they’d been inserted by JavaScript not directly in the page, no Eolas Bug). Now, implementing the Eolas Bug was presumably no mean feat — I guess it works off the same metadata JavaScript sandboxing has to use to decide which script can see and talk to whom. Anyway, moving right along…
Now Eolas is suing a whole bunch of people over the same patent.
I hate software patents. I don’t even like them when they’re being used to screw Microsoft. But I particularly hate it when someone buys a patent (this patent was originally awarded to the University of California), sits on it for years, and then starts suing people over it — having done nothing useful with it or anything else in the meantime. But let’s take a look at this patent on its merits:
A system allowing a user of a browser program on a computer connected to an open distributed hypermedia system to access and execute an embedded program object. The program object is embedded into a hypermedia document much like data objects (1). The user may select the program object from the screen (2). Once selected the program object executes on the user’s (client) computer or may execute on a remote server or additional remote computers in a distributed processing arrangement (3). After launching the program object, the user is able to interact with the object as the invention provides for ongoing interprocess communication between the application object (4) (program) and the browser program. One application of the embedded program object allows a user to view large and complex multi-dimensional objects from within the browser’s window. The user can manipulate a control panel to change the viewpoint used to view the image. The invention allows a program to execute on a remote server or other computers to calculate the viewing transformations and send frame data to the client computer thus providing the user of the client computer with interactive features and allowing the user to have access to greater computing power than may be available at the user’s client computer (5).
I’ve labeled what seem to me to be the main claims in the abstract. (I will get to the specific claims — er claim — later, and IANAL let alone a patent lawyer). Note that a lot of text — none of it bold — is devoted to implementing something like VRML but rendered on the server — I don’t think they’ll be suing anyone over that nugget.
- My initial reaction to this is that it’s not how browsers work at all, but on rereading perhaps what they mean by “embedded” is “linked” (after all, both data objects and plugins are linked to web pages, not actually embedded in them). The devil may be in the details here.
- Nope, the user does no such thing.
- This kind of describes networked computing for which there is ample prior art.
- This is important — browsers do pass event information to plugins, and plugins do tell browsers how to draw pages etc.
- This is exactly the opposite of what plugins do. Plugins let you do stuff on the client specifically to avoid wasting CPU time on the server and/or bandwidth.
So, based on the abstract, one claim is utterly trivial and covered by ridiculous amounts of prior art, item 1 is possibly interesting, item 4 is right on the mark and the others are utterly irrelevant. But then from what I can see patent lawsuits, especially software patent lawsuits, often seem to hinge on some tiny nook or cranny of a patent that seems to have no real relevance to the thing described in total.
Let’s look at item 4, communication between the browser and the plugin. This seems like it might have some legs. Did anyone before 1994 have a neat mechanism for automagically embedding arbitrary data with arbitrary software to handle it inside a container program? This sounds a lot like SGML conceptually, and it also sounds a lot like a really exciting and innovative type of technology that turned out to be a giant stinking turd: OpenDoc.
While no-one remembers OpenDoc particularly fondly, it was a shipping piece of software, it ran — often for minutes — without crashing, and it did some pretty remarkable things. You could open a document comprising a collection of pieces of data for which you might or might not have specific editor or viewer components, and the OpenDoc framework would just deal with it. It would allow the components (“editors” or “viewers”) to tell it which kinds of data they could handle, and events were passed back and forth (to allow editing, obviously). What’s more, conceptually, OpenDoc was very much inspired by other shipping programs such as PageMaker and Quark XPress which had their own plugin architectures which did similar (but less general) things. Since OpenDoc shipped in 1992, and was in development for years beforehand, I think it’s safe to say that Apple has prior art up the wazoo. So, good-bye item 4.
And of course OpenDoc worked just fine across a network. In fact, Apple shipped a web browser component for OpenDoc (Cyberdog) which was perhaps the most well-loved feature of OpenDoc and the only thing most users missed when Apple finally scrapped it.
OK, so back to item 1. Let’s drill down into the blow-by-blow claims in the patent’s body. Actually the two meatiest and most incomprehensible chunks of text pertain to the first point, so let’s take a look at first and slightly more comprehensible chunk:
… a browser application, that parses a first distributed hypermedia document to identify text formats included in said distributed hypermedia document and for responding to predetermined text formats to initiate processing specified by said text formats; utilizing said browser to display, on said client workstation, at least a portion of a first hypermedia document received over said network from said server, wherein the portion of said first hypermedia document is displayed within a first browser-controlled window on said client workstation, wherein said first distributed hypermedia document includes an embed text format, located at a first location in said first distributed hypermedia document, that specifies the location of at least a portion of an object external to the first distributed hypermedia document, wherein said object has type information associated with it utilized by said browser to identify and locate an executable application external to the first distributed hypermedia document, and wherein said embed text format is parsed by said browser to automatically invoke said executable application to execute on said client workstation in order to display said object and enable interactive processing of said object within a display area created at said first location within the portion of said first distributed hypermedia document being displayed in said first browser-controlled window.
In other words, the “embedded” data object has “type information” used to pick the external plugin that displays and allows interaction with its data in a display area in the browser window.
This appears to be the single thing on which they are basing their lawsuit. It’s no small thing, but it does seem to me to be obvious. After all, how do browsers handle links to non-textual embedded data that doesn’t require a plugin. The only major example in 1995 was images, e.g. gifs and jpegs. These were, and are, embedded using <img> tags which contain a URL (here’s where the actual image data is) and dimensioning information (here’s the width and height). <img> tags don’t contain any real metadata beyond themselves (I’m an <img> tag, which connotes I refer to an image) and — arguably — the “filename extension” at the end of their src URL (which is often garbage).
Now, browsers don’t render images themselves, as a rule, they call on external libraries or the operating system to do it. So an <img> tag is basically saying “here’s a link to some image data, and this is how big I’d like you to render it” and the browser gets this information and calls external software to render it. Images aren’t interactive, so that’s not covered — but OpenDoc and PageMaker and so forth have interprocess event handling covered in spades.
So let’s suppose you’re a programmer and you know about <img> tags and it occurs to you that there’s no way to embed QuickTime movies (say) in your web page. And no way to handle Illustrator files. And no way to display 3D models. What to do? Any decent programmer is going to immediately consider coming up with a new kind of tag for each kind of document, and then replace the code that handles <img> tags with a huge switch statement somewhere and decide this sounds like hard work and bad architecture.
Abandoning the totally obvious — but obviously bad — solution, the next most obvious solution is to invent a generic embed tag and move the information connoted by the tag’s name (“img”) into the attributes of the tag itself (e.g. type=”image”). And, again obviously, expecting the browser to be modified and recompiled every time we pull a new media type out of our asses is stupid, so let’s generalize the “get the OS to render it” side of things by mapping types to specific pieces of software (“plugins”). Oh, and the data might be interactive (e.g. movies need a play button and they need to be refreshed more often than images) so we’ll need to pass events to and fro. OMG call up the patent attorney.
The only thing missing from OpenDoc was the ability to embed a url for downloading the “part editor” you need if you don’t already have it. OK, that may be genuinely novel. But how is it also not obvious given this is web browsers we’re talking about. I didn’t use OpenDoc for much serious work, so I don’t remember what OpenDoc did when you opened a document containing parts for which you lacked editors — I know it coped, but I can’t remember exactly how much information it provided you with. If it told you something about the component needed to deal with the part and from whom it might be obtained, then Apple even has prior art for this.
It’s easy to claim something is obvious in hindsight. I think I’ve demonstrated that — at the very least — OpenDoc represents prior art for almost everything in the patent (even the minutiae). If OpenDoc documents were web pages, and OpenDoc data embedding automatically included a way to obtain (download) the software required to handle it (and — arguably — it did, just not URLs since OpenDoc wasn’t a web browser) then this patent would be completely meaningless. (And, frankly, if web pages were OpenDoc documents it would be pretty awesome — if OpenDoc were still around and didn’t suck.)
Corrections / Amendments
After looking over more documents, it looks like the correct chronology is more like Publish and Subscribe (part of System 7.x) and OLE (Windows 3.x) were 1990-1991, OLE II (Windows 95) and OpenDoc (System 7.5) were being specced in 1992, both shipping in the 1994-1995 timeframe. (But again, in-house stuff will predate this significantly.) And OpenDoc was (eventually) part of a huge (unwieldy) anti-Microsoft alliance, so all of its concepts were very widely and openly documented long before it shipped.
Acknowledgements
I got the screenshot from this article on MacTech — I hope they don’t mind. It sure beats reinstalling OpenDoc on an old Mac. Ugh.
I also found this really interesting paper written by people on the OpenDoc team. It’s a sobering reminder of how brilliant, idealistic, thoughtful people can produce really terrible software. It also failed to address the question of how OpenDoc handled missing editors. Maybe I will have to find an old Mac and reinstall it.