node-webkit

C3D Buddy 2.0 in action — built in an evening using node-webkit

C3D Buddy 2.0 in action — built in an evening using node-webkit

I don’t do very much desktop software development these days, and I’ve become increasingly frustrated with Realbasic (now Xojo), so the prospect of forking out $500 for an upgrade that may or may not be usable did not fill me with glee. Furthermore, as the years roll on, there’s more and more functionality tied to the web that desktop-centric development tools struggle to work with.

I was dimly interested in a thread on Hacker News this morning concerning a workflow for developing desktop applications with node. Turns out that the workflow was mostly to do with things like angular and jade, which I’m not especially keen on, and not so much with the actual desktop stuff — the heavy lifting for which is all done by node-webkit. So I started looking into node-webkit.

I like a tool which has you up-and-running in about three paragraphs.

It goes something like this:

Download an appropriate node-webkit binary. (Disappointingly, the Mac version is restricted to 32-bit.)

  • Write an index.html file. (The sample “hello world” app calls node inline.)
  • Write a package.json file. (The documentation suggests it’s like a “manifest” file, but really it’s more like a configuration file.)
  • Zip this stuff into a file into an archive whose name ends in .nw.
  • Double-click the archive.
  • Voila, your app works.

After working on this for about fifteen minutes, I knocked up a bash script (which I turned into a .command file, allowing me to double-click it in Finder) which does the archiving and then launches the file in one step. (It does leave dead terminal windows lying around, and it doesn’t quit the app if it’s still running. I think I may build an app for building apps if I do much more of this.)

And to build a distributable application you can simply put the archive in the application bundle (on the Mac) renaming it “app.nw” or name the file package.nw and put it in the same directory as the application (for those benighted platforms that don’t have bundles). One minor annoyance is that it’s not clear exactly what needs to be changed in the Info.plist file to fully customize your application.

So what is node-webkit? It’s essentially a version of Chromium that puts node.js alongside the DOM in your web pages, along with a few custom HTML and DOM extensions that let you do things like handle platform-native file requesters).

I should note that it’s a potential security nightmare as it stands. There’s no sandboxing (that’s kind of the point), so deploying an app that loads your banking website and then watches you press keys is totally doable. That said, it’s a desktop app so it can also delete all your files or encrypt them and hold them hostage.

Anyway, I had a lot of fun this evening rewriting a utility application I originally wrote in Realbasic a few years ago. It probably took me about twice as long to write it as it took me to do it in Realbasic. Part of that is I know (or knew) Realbasic very well, and I don’t know node very well. Another part of it is that Realbasic is almost always synchronous while node is almost always asynchronous. Writing code that spelunks a directory tree asynchronously is a lot more complex both conceptually and in practice than doing so in linear fashion, but the payoff is quite impressive — despite running an “interpreted” language, a task that took significant time in Realbasic (loading and displaying hundreds of thumbnails) happens in the blink of an eye.

One of the things that’s especially intriguing about node-webkit is that it gives you control over the browser you normally don’t have (e.g. window placement, file system access) — these are a constant usability sore-point in the project I am currently working on (which is a web app that is replacing a suite of desktop apps). It would be so much easier to get a lot of the things we want to do working smoothly using a hybrid desktop application built on top of something like node-webkit — it’s kind of a lemma of Alan Kay’s observation that anyone who wants to write really good software needs to build hardware — anyone who wants to write really good web apps really needs to build the browser.

The github repo for my project is here. It’s not a very general-purpose application; if you don’t use Cheetah 3D it’s completely useless.

Cloudflare

 

I found out about Cloudflare from a link on Hacker News. Assuming it works (and if you’re reading this 24-48h after I post it, then it presumably is) it’s just insanely wonderful.

Basic idea — sit in between your website and users (including robots and would-be attackers) and massage what the users see (e.g. replace harvestable email addresses with scriptlets) and the way they are able to access your content (e.g. cache static content, insert analytics code, block cross-links and war dialers) while acting as a CDN. And do it for free (at least for now).

Setup is wonderfully handled if not quite completely painless — although it’s close. You need to enter your DNS records one-by-one (at least, you’re supposed to — I’m pretty sure the automatic detection it does would probably be sufficient in most cases) which took me a while for loewald.com since it has a whole lot of bizarre legacy subdomains. (And since a denial of service attack doesn’t care how popular a subdomain is, you shouldn’t ignore them.)

So, I’ll see how it goes, and add any updates here.

First update: it works as advertised. I’ve set it up for loewald.com, but not for my other sites. I’ve since switched on some additional features including Javascript minification. Check out this link.

Second update: I added riddlemethis.net to Cloudflare. Unlike loewald.com (which has all kinds of whacky DNS records because of all the different prototypes and experiments I’ve set up on it over the years) this took about two minutes and didn’t involve any data entry.

Tentative Conclusion

If Cloudflare cost $20/month/site it wouldn’t be worth it for my little personal site. It probably wouldn’t even be worth it for RiddleMeThis. But it’s free. For any kind of serious site, the $20/month seems like a no-brainer.

The joy of Cloudflare is that it frees you up to create websites in a more sensible manner. Instead of working with normal JavaScript files then minifying them at the last moment you can simply code with sensible files (and debug with them) and then simply flip a switch and everything is minified.

Dropbox: Deduplication with Privacy

There’s been a bit of a scare regarding Dropbox related to the possible use of deduplication to determine who has copies of “illegal” files and then the use of warrants to identify infringing Dropbox users and basically hose them.

The problem

When you store a file on Dropbox it will be hashed (more-or-less uniquely identified by scanning its content) and then the hash and the file’s size will be used to determine if the file already exists on Dropbox’s server (i.e. if your ripped copy of Avatar matches someone else’s it will have the same hash value and the exact same file size). If so, rather than uploading the file your account will simply get a new file entry pointing at the existing file. “Upload” is instant, Dropbox saves money on storage, everybody wins.

But, suppose James Cameron uploads a ripped copy of Avatar to Dropbox and notices that this 3GB MP4 file uploaded instantly. He now knows someone else has such a file on Dropbox which is reasonable cause to suspect that piracy is happening and, in theory, he can require Dropbox to tell him everyone who has a copy of that file in their account.

Hence the scare.

The obvious solution to this problem is to not knowingly store illegally duplicated files in your Dropbox account or to encrypt them using your own unique key if you do.

But it’s quite possible that any of us might accidentally put an illegal file — or perhaps a file normal people consider “fair use” but the MPAA (say) might not consider legal — in your Dropbox account. E.g. I might rip Avatar using Handbrake so I can watch it on my iPhone, and this might create an identical file to your handbraked copy of Avatar, and according to the MPAA we might both be horrible criminals who deserve the gas chamber and given that Congress only cares about people who provide large campaign donations…

A possible solution

I’ve proposed this solution on both HackerNews and DropBox’s forums. It’s not perfect — maybe someone can refine it.

I imagine Dropbox has a list of files with unique ids, sizes, and hash values, and every user has a list of files with their own personal path (where they think it is and what they think it’s called) along with the unique id of the actual underlying file. This is the heart of the problem.

Instead of storing the unique id of the underlying file in the user’s file table, Dropbox needs to store a number offset by a hash value generated client-size from the user’s password and the user’s name for the file (i.e. something that will be different for each user and each file and not replicable with data stored in Dropbox’s own database).

Note that if the user’s password is changed then every file id will need to be changed accordingly, which is definitely a downside. (And if you forget your password then your files cannot be recovered.)

Also note that presumably someone like the MPAA could simply obtain a warrant and wait for people to access an “illegal” file, but this is surely going to be a much slower and more difficult process than simply doing a query on the entire database and sending out threatening letters to everyone in the result list.

Thing is, this isn’t technically complex  to implement and could be a user preference. Would you prefer privacy with the risk of losing all your files if your password is lost? Given that you will probably have multiple backups of all your Dropbox files, it’s actually not a big problem. (In fact, if you consider the case where you are forced to reset your Dropbox password and thus Dropbox forgets you own all your files — re-uploading them from one of your computers will be instantaneous for all the files you previously had uploaded owing to deduplication.)

Edit: another problem with my proposed solution is that you can lose track of files (e.g. you can’t maintain an accurate reference count). This is probably not as big an issue as it might seem since Dropbox already retains files for a month after a non-paying user deletes them and forever for paying users. Presumably it retains copies of files left by users who stop using the service.

Final Note: I have no affiliation with Dropbox (although I do use the product) and have no stake in it. If you’d like to try Dropbox and give me more space to store potentially illegal files, please use this link.

Inconvenience without Security

Apparently there’s news of an exploit that completely hoses Vista’s security and which probably can’t be fixed. Before the Microsoft-haters all start celebrating, let me make a couple of observations.

  • It’s not clear whether the general approach taken might not be equally effective against other operating systems.
  • The people discussing this exploit seem entirely too gleeful. Remember, you’re supposed to be good guys looking for security holes so we can fix them before bad guys take advantage of them.

“… the genius of this is that it’s completely reusable. They have attacks that let them load chosen content to a chosen location with chosen permissions. That’s completely game over.”

That just doesn’t sound like a dispassionate researcher reporting significant findings that may be of concern to us all. It sounds more like someone relishing Microsoft’s discomfort, or maybe Hudson — that guy in Aliens who totally loses it.

Here’s a link to the actual paper.

Game over man.

Note: the actual researchers are quite reasonable and their paper is entirely aimed at helping Microsoft and other vendors improve their platforms’ security. The guy I was quoting was “popular security researcher” Dino Dai Zovi. I think he’s popular because he says insane crap like that.