Javascript Dick Waving FTW

Ars Technica’s rather gushy coverage of Squirrelfish Extreme (and the somewhat critical comments on that coverage) prompted me to do some of my own testing. Certainly, Squirrelfish Extreme keeps Webkit (and, ultimately, Safari) among the contenders for best endowed web browser, but my results were a little less definitive.

Google’s V8 benchmark suite is here. The Sunspider benchmark suite is here. Domaeo (the Mozilla test suite) is here. You might like to check them out in a few browsers. I just tried the current Safari release, the current Firefox release, and the Webkit Nightly Build. As I understand it, the Webkit speed claims are based on testing the JavaScript engines in isolation from the command line. Of course, very few folks run JavaScript this way…

The following are my very slipshod results obtained by firing up browsers and trying each benchmark once or twice. (I wasn’t careful about shutting stuff down, minimizing background activity, closing extra tabs, etc.)

Google’s V8 Benchmark Suite

This set of benchmarks takes less time to run and appears to be less thorough. (It certainly produces less immediately intelligible results… you just have to accept that big numbers are a good thing.)

Firefox 3.0.1 (OS X 10.5.5)

  • Score: 213
  • Richards: 226
  • DeltaBlue: 238
  • Crypto: 205
  • RayTrace: 162
  • EarleyBoyer: 245

Safari 3.1.2 (OS X 10.5.5)

  • Score: 218
  • Richards: 124
  • DeltaBlue: 173
  • Crypto: 187
  • RayTrace: 309
  • EarleyBoyer: 401

WebKit r36685 (OS X 10.5.5)

  • Score: 1312
  • Richards: 1317
  • DeltaBlue: 990
  • Crypto: 2553
  • RayTrace: 575
  • EarleyBoyer: 2035

This is just wonderful. (I tried to download a Firefox Nightly to check out Tracemonkey for myself but figuring out how took more than ten seconds, so I gave up. Here are some stats from Windows (note that these are running under VMWare Fusion, so it’s not fair to compare to the Mac figures directly).

Firefox 3.0.1 (Windows XP Professional/VMWare Fusion)

  • Score: 208
  • Richards: 207
  • DeltaBlue: 264
  • Crypto: 157
  • RayTrace: 180
  • EarleyBoyer: 251

Google Chrome (Windows XP Professional/VMWare Fusion)

  • Score: 2117
  • Richards: 2593
  • DeltaBlue: 2482
  • Crypto: 1692
  • RayTrace: 1456
  • EarleyBoyer: 2685

Internet Explorer 8.0 Beta 2

Note: IE8 interrupts each run with a “script is running slowly” dialog. I tried to click this off as quickly as I could but it definitely hurt the benchmark slightly. I ran it three times, dismissing the dialog as quickly as possible and took the best score.

  • Score: 58
  • Richards: 47
  • DeltaBlue: 52
  • Crypto: 50
  • RayTrace: 69
  • EarleyBoyer: 75

Sunspider

Sunspider’s results are really detailed, and I’m not going to post the whole things here, but I got the following bottom line results from a single run of each. In the Webkit case, this was with half a dozen tabs open (including this blog entry), while Chrome had two tabs open, one of which was displaying the results of its earlier benchmark (not very taxing, I expect).

Webkit r36685: 1614.6ms +/- 7.5% (running native under OS X)

Google Chrome: 1458.2ms +/- 0.4% (running under XP/VMWare; yes, faster is better)

Dromaeo Results

Again, the results are very detailed and also seem very real world (they use code from a lot of widely used JavaScript libraries such as ExtJS, Dojo, and Yahoo Widgets as part of the suite). You can see them for yourself here (Webkit) and here (Chrome). During the Chrome run, Chrome kind of seized up, whichwas probably Windows XP’s fault. So the overall result may be garbage, but if you look at the details you’ll find that V8 is clearly faster at some things and Squirrelfish at others. What this says to me is that both have quite a bit of headroom even yet.

Webkit r36685: 6652.60ms (running native under OSX)

Google Chrome: 10540.80ms (running under XP/VMWare)

So on both Google’s and Apple’s preferred benchmarks, I’m seeing better performance out of Google Chrome, but not by much. Dromaeo, which is a very DOM-intensive suite and also appears the most comprehensive, favors Webkit. (And, yes, IE8 is pitiful. No, I am not going to try Sunspider or Dromaeo on IE8.)

The really annoying thing is that Apple isn’t very aggressive about pushing advances in webkit out to the masses. Or, to put it another way, they use new versions of Safari to sell new versions of Mac OS X*. Chances are Squirrelfish Extreme‘s performance will be a key selling point of Snow Leopard when it ships, instead of a free upgrade to Safari, which is what it ought to be. In the mean time, we’ll probably need to use some other Webkit-based browser or Webkit Nightly, or simply use Firefox to enjoy better performance.

Note: as a comment — rather rudely — points out, Apple doesn’t charge for Safari upgrades. But it does hold new features back so it can announce them as though they were new OS features. (Edit: and, actually, it doesn’t support Safari on versions of Mac OS X before 10.4.x, so technically I was completely correct and the aptly named “bs” was full of bs.) A case in point is the debugging functionality added to Webkit over a year before it became available in Safari as part of Leopard. (And yes, you can get it for Tiger, but the marketing would have you believe it was a Leopard feature, and you simply can’t get it for Panther or earlier.)

And the really puzzling thing is that Microsoft seems to be happy to allow JavaScript performance to suck dog’s balls under IE8.

So, Javascript is fast now?

This site is the most interesting attempt I’ve found to seriously compare execution speed across languages. It should obviously be taken with more than a grain of salt since, apart from all the usual issues with benchmarks (artificiality, etc.), the quality of the code written in each language may vary greatly, as may the actual algorithms, and different algorithms may favor different languages, and different languages are used for different things and, obviously, benchmark comparisons try to force everything to do the same thing. Still, it is interesting.

Based on some quick spelunking, JavaScript (Spidermonkey) is comparable in speed to PHP or Ruby, perhaps a third as fast as Python, a fifth as fast as Lua, about a hundred times slower than Java, C# Mono, or Lisp, and a hundred and fifty or so times slower than C++.

What that says to me is that JavaScript should easily be able to get two or three times faster than Spidermonkey (because it has a lot in common with Python), but then it will get tough. It also speaks rather well of Lisp (which manages to be truly dynamic and still competitive with the big guns). Since it looks like Squirrelfish Extreme and V8 are both around twice as fast as Spidermonkey, and that if you took the best bits out of each from the detailed benchmarks and combined them you’d maybe get 50% faster again, it looks like we’re nearly out of low-hanging fruit in the JavaScript engine race.

The Best Programming Language Ever?

I was having a conversation at work today about what people’s tastes in programming languages were and there was some discussion about human readability, which led me to mention HyperTalk. HyperTalk was the programming language used by HyperCard, it had the virtue of being incredibly readable and (unlike the similarly readable AppleScript) also very easy to write. I mentioned that “whenever you see — used to denote a comment, there lies the ghost of HyperTalk” to which someone replied, “like in HTML?”.

An HTML comment is denoted by “<!–” and “–>” (note that WordPress is screwing this up, I really do know what HTML comments look like) which seems like a non-coincidence. Tim Berners Lee invented HTTP and HTML using NeXT computers, which didn’t run HyperCard, but which were spiritually related to the Mac, and it’s hard to imagine that HyperCard wasn’t a major influence on the design of the web.

HyperCard has the distinction of being one of the most influential programs ever written, perhaps the most. (Bill Atkinson wrote HyperCard and MacPaint, two of the most influential programs ever written. Photoshop is, essentially, MacPaint running on modern hardware.) Paul Allen spent a fortune cloning it (Toolbook) and it was clearly a huge influence on Visual Basic. Director’s Lingo started out as a straight clone of HyperTalk. Even 3D Studio Max’s MaxScript bears its mark. If we assume that the web was directly inspired by HyperCard — a not unreasonable assumption — that makes it perhaps the most influential  program ever written.

HyperCard was amazingly nice to code in. HyperCard 2 had a modern debugger, there was a third party compiler (written in HyperTalk that could compile to assembler), and it virtually never crashed. By default, all state was preserved (in fact, if you wanted to write a conventional application you had to jump through hoops to create “empty” documents). The main failing of HyperTalk was its lack of support for OO and functional programming (it was internally an object-oriented message-passing architecture, but you couldn’t build your own objects). It would be pretty simple to bolt that kind of functionality onto it, but no-one has, and HyperTalk has largely passed into obscurity.

Surprisingly, there are three modern HyperCard clones still being maintained. Toolbook has focused on authoring CBT, while Runtime Revolution tries to be all things to all people, and SuperCard is Mac only. Unfortunately, all seem pretty clunky by today’s standards. (And, frankly, for all its virtues, HyperCard was a bit clunky even when it was new. It never allowed you to build proper user interfaces.)

It would be interesting to see HyperTalk updated to support modern language features OR something like HyperCard built on top of a modern language like Python or Ruby. The obvious thing to do would be to build a web-based IDE on top of Django or Rails.