Tech Free Saturdays with the Kids

Romilly with her iPad
Romilly with her iPad

Like many parents, Rosanna and I are concerned about our kids’ obsession with “technology”, so we tried “tech free Saturdays”, and it worked for about an hour (I — more than slightly ironically — spent that hour with the girls playing with an Elenco electronics set — something far better than the “150-in-one” electronics kit I dreamed of when I was a kid), and then gave up. Short of getting exercise outdoors — which while almost certainly a Good Thing To Do is hardly something of why I am an examplar — what was there to do without “technology”?

I don’t pretend to know what stuff is going to be important to the success of my kids. A lot of the stuff I learned in school has turned out to be useful, or at least makes for interesting conversation (apparently, most people forget almost everything they learned in school, and — having had no interest in it when they were 14, find it intriguing as adults). But the most useful stuff I learned as a kid is the stuff society — i.e. teachers and parents — made me feel guilty about spending time on. And I don’t think this is rare. I think it’s the people who were obsessed with computer games, or Science Fiction, or Dungeons & Dragons back in 1982 who are created the world we live in today.

We won (or, at least, we’re ahead — when we all die young from heart disease and diabetes because we never get any exercise, the jocks from high school whose knees still work can gloat).

And having won by willfully ignoring society’s ideas of what a “healthy” obsession was when we were kids, who are we to impose our ideas of what a “healthy” obsession is on our kids? Well, we’re parents, of course, and “a foolish consistency is the hobgoblin of small minds”. Perhaps we’re just that much smarter than our parents and teachers.

Another possibility that occurs to me is that a passion for anything — programming role-playing games, the collected works of Jack Vance — only turns into something powerful and character-building if it involves pushing against social pressure. In other words, it’s OK for us to try to stop our kids from doing what they want to do, but it’s even better if they defy us and it anyway.

In the end, I don’t mind if my kids are obsessed with Minecraft, or even Youtube. What worries me is that its too easy to feed those obsessions, and I don’t think technology is the problem. But, having said that, my father narrowly avoided the Holocaust and my mother lived through famine and the Vietnam War, whereas I had to cope with the poor selection of science fiction in local libraries and the fact that our school only had one Apple II computer.

Returning to the Adobe fold… sort of

I remain very frustrated with my Photography workflow. No-one seems to get this right and it drives me nuts. (I’m unwilling to pay Apple lots of money for a ridiculous amount of iCloud storage, which might work quite well, but it still wouldn’t have simple features like prioritizing images that I’ve rated or looked at closely over others automagically, or allow me to rate JPEGs and have the rating carried over to the RAW later.)

Anyway, Aperture is sufficiently out-of-date that I’ve actually uninstalled it and Photoshop still has some features (e.g. stitching) that its competition cannot match. So, $120 for a year of Photoshop + Lightroom… let’s see how it goes.

Lightroom

I was expecting Lightroom to be awesome what with all the prominent folks who swear by it. So far I find it unfamiliar (I did actually use LR2, and of course I am a Photoshop ninja) to the point of frustration, un-Mac-like, and ugly, ugly, ugly.

Some of my Lightroom Gripes
Some of my Lightroom Gripes

A large part of the problem is terrible use of screen Real Estate. It’s easy to hide the menubar (once you find where Adobe has hidden its non-standard full screen controls), but it’s hard (impossible) to hide the idiotic mode menu “Identity Plate”. (I found the “Identity Plate Editor” (WTF?) by right-clicking hopefully all over the place, which allowed me to shrink the stupidly large lettering but it just left the empty space behind. How can an application that was created brand new (and initially Mac-only) have managed to look like a dog’s breakfast so quickly?

But there are many little things that just suck.

  • All the menus are horrible — cluttered and full of nutty junk. Looks like design by committee.
  • The dialog box that appears when you “Copy…” the current adjustments is a crime against humanity (it has a weird set of defaults which I overrode by clicking “check none” when I only wanted to copy some very specific settings and now I can’t figure out how to restore the defaults).
  • The green button doesn’t activate full screen mode. There are multiple full screen modes and none of them are what I want.
  • Zooming with the trackpad is weird. And the “Loupe” (nothing like or as nice as Aperture’s) changes its behavior for reasons I cannot discern. (I finally figured out that the zoom in shortcut actually goes to 1:1 by default, which is useful, although it’s such a common feature I’d have assigned a “naked” keystroke to it, such as Z, which instead toggles between display modes.)
  • The main image view seizes up after an indeterminate amount of use and shortly afterwards Lightroom crashes. (This is on maxed out Macbook Pro 15″.)
  • I can’t hide the stupid top bar (with your name in it). I can’t even make it smaller by reducing the font size of the crap in it.
  • Hiding the “toolbar” hides a random thing that doesn’t seem to me to be a toolbar.
  • By default the left side of the main window is wasted space. Oh, and the stupid presets are displayed as a list of words — you need to mouse over them to get a low-fidelity preview.
A crime against humanity.
A crime against humanity.

I found Lightroom’s UI sufficiently annoying that I reinstalled Aperture for comparison. Sadly, Lightroom crushes Aperture in ways that really matter. E.g. its Shadow and Highlight tools simply work better than Aperture’s (I essentially need to go into Curves to do anything slightly difficult in Aperture), and it has recent features (such as Dehaze — I’m pretty sure inspired by a similar feature DxO was very proud of a while back). After processing a few carefully underexposed RAW images* in both programs, Lightroom gets me results that Aperture simply can’t match (it also makes it very tempting to make the kind of over-processed images you see everywhere these days with amped up colors, quasi-HDR effects, and exaggerated micro-contrast).

(* Quite a few years ago someone I respect suggested that it’s a good idea to “underexpose” most outdoor shots by one stop to keep more highlight detail. This is especially important if the sky is visible. These days, everyone seems to be on the “ISO Invariance” bandwagon which is essentially not doing anything to the signal off the sensor (boosting effective ISO) when capturing RAW, in essence, “expose to the left” automatically — the exact opposite of the “expose to the right” bandwagon these clowns were all on two years ago — here’s a discussion of doing both at the same time. Hilarious. On the bright side, ISO Invariance pretty much saves ETTR nuts from constantly blowing their highlights.)

The Photos App is far more competitive with Lightroom than Aperture
The Photos App is far more competitive with Lightroom than Aperture. And its UI is simply out of Lightroom’s league (see those filters on the right? Lightroom simply has a list of names).

Funny thing though is that the new Photos app gives Lightroom a much better run for its money (um, it’s free), has Aperture’s best UI feature (organic customization), and everything runs much faster that Lightroom. The problem with Photos is it is missing key features of Lightroom, e.g. Dehaze, Clarity, and (most curiously) Vibrance. You just can’t get as far with Photos as you can with Lightroom. (If you have Affinity Photo you can use its Dehaze more-or-less transparently from Photos. It’s a modal, but then Lightroom is utterly modal.)

On the UI level, though, Photos simply spanks Lightroom’s Develop mode. Lightroom’s organization tools, clearly with many features requested by users, are completely out of Photos’ league.

I also tried Darktable (the open source Lightroom replacement) for comparison. I think its user interface is in many ways nicer than Lightroom’s — it looks and feels better — although much of its lack of clutter is owed to a corresponding lack of features), but the sad news is that Darktable’s image-processing capabilities don’t even compete with Aperture, let alone Photos. (One thing I really like about Darktable is that it applies “orientation” (automatic horizon leveling), “sharpen”, and “base curve” automagically by default. Right now this isn’t customizable — there’s a placeholder dialog — but if it were it would be an awesome feature.)

The lack of fit and finish in Lightroom is unprofessional and embarrassing
The lack of fit and finish in Lightroom is unprofessional and embarrassing. If it’s not obvious to you, the red line shows the four different baselines used for the UI elements.
Hilarious. Lightroom's "About Box" uses utterly non-standard buttons that behave like tab selectors.
This is hilarious. Lightroom’s “About box” uses utterly non-standard buttons that behave like tab selectors. This is actually regression for Adobe, which used to really take pride in its About boxes.

At bottom, Aperture doesn’t look or feel like an application developed by or for professionals. It’s very capable, but its design is — ironically — horrible.

Photoshop

Photoshop’s capabilities are, by and large, unmatched, but its UI wasn’t good when it first came out and many of its worst features have pretty much made it through unscathed by taste, practicality, or a sense of a job well done. Take a look at this gem:

Adobe Photoshop's horrible Radial Blur dialog
Adobe Photoshop’s horrible Radial Blur dialog

This was an understandably frustrating dialog back in 1991 — in fact the attempt to provide visual cues with the lines was probably as much as you could expect, but it hasn’t changed since — every other application I use provides a GPU-accelerated live preview (in Acorn it’s non-destructive too). What’s even worse is that it looks like the dialog’s layout has been slightly tweaked to allow for too-large-and-non-standard buttons (with badly centered captions that would look worse if there were a glyph with a descender in it). At least it doesn’t waste a buttload of space on a mode menu: instead there’s a small popup lets you pick which (customizable) “workspace” you want to use, and the rest of the bar is actually useful (it shows common settings for the currently selected tool).

In the end, Photoshop at least looks reasonably nice , and its UI foibles are things I’ve grown accustomed to over twenty-five years.

I can’t wait until I get to experience Adobe’s Updater…

Free Business Idea

I’ve mentioned this idea casually many times, but I’m posting it here begging for someone to steal this idea and run with it. I think it’s technically simple to do, and there’s money in it.

The idea: time-shifting radio, i.e. TiVo for (car) radios.

Why?

  • NPR fund-raising drives
  • Commercial radio ads
  • Repeating songs (e.g. to find out their names)
  • Pausing and archiving interesting stories
  • Catching radio programs that are on at inconvenient times.

You know, like TiVo, but for radio. In your car.

Other benefits

  • Hurts commercial radio, more convenient than podcasts

Possible Downside

Hurts NPR fund-raising. But, seriously, you can pay Sirius/XM the cost of a typical NPR annual donation and NPR with get no fundraising at all.

Please, someone. Do this.

Usability on the Underside

A minimalist cocoa app
A minimalist cocoa app

I’ve always thought it ironic that Apple makes the most usable computers and the least usable APIs. I’m referring, of course, to AppleScript — just kidding. At first I thought it was a huge failing (and belies Steve Jobs’s claimed obsession with making even the stuff the user never sees beautiful; but then he likely never looked at APIs); then I excused it as Apple wanting to make dealing with its APIs a pons asinorum that less capable programmers wouldn’t be able to cross.

(Note: I’m not kidding that AppleScript is horrible. Just that it’s twenty years old and that horse has been beaten to death.)

I think I was right the first time.

But, it has gotten a lot better.

Perhaps the single most impressive piece of software Apple ever shipped was HyperCard. There was, basically, nothing wrong with HyperCard that couldn’t have been easily fixed by version 3. The chief problems with HyperCard were that:

  • Images were not a first class entity (in VB3, for example, you could load an image into an image variable much like you could load a text file into a string variable in almost any language with decent string manipulation (i.e. not C/C++ or Pascal, but every other popular language).
  • Binary blobs were not a first class entity (something that you’d probably implement for free while implementing the above).
  • The UI gadgets weren’t native UI gadgets.
  • There was no bridge to the native Toolbox short of writing a plugin in C/C++ or using the crazy-but-genius third party HyperTalk compiler.

That may seem like a lot of key flaws, but there’s nothing there that can’t be solved with a bunch of rudimentary programming. Consider what HyperCard got right:

  • First of all, it was shockingly stable — you could often work on HyperTalk projects for days with no serious crashes (let alone System reboots). This was in an era when computers (both Mac and PC) crashed like crazy.
  • You could quickly build user interfaces with drag-and-drop, including bitmap drawing tools
  • State persistent by default
  • Every application was its own database
  • By default you ran inside the development environment — you lived in the development environment
  • Shockingly fast — once HyperTalk 2 came out with its JIT compiler, performance was amazingly good (see note below).
  • Its language was amazingly accessible — both readable and writable — more readable than AppleTalk and far, far more writeable. Most people, including non-coders, could figure out how to do simple stuff within a few hours.
  • Awesome levels of introspection (which allowed metaprogramming)
  • First class debugging tools
  • Always-available REPL

Many of these virtues have yet to be matched by any programming language. VB3 essentially took a few of these virtues, replaced the incredibly easy to work with but hard-to-implement HyperTalk with the very easy to use and already-implemented BASIC, fixed all the obvious faults, and became the foundation of Microsoft’s entire approach to software development. And, say what you will about Microsoft, they do make good development tools.

Note on performance: we had an ambitious multimedia project written in VB3 that barely ran on maximally configured Windows PCs (we were using then-bleeding-edge 486 DX2/66 desktops for development, and our target platform was the then just-shipped IBM Thinkpad 750 (which cost $11,000 in Australia and had a 486DX4/75 CPU by my recollection; it was also the first name-brand MS-DOS compatible laptop with both color graphics and sound). I cloned the project on my aging Quadra 700 (68040 25MHz) using HyperCard and it ran circles around the PCs.

Usable Languages

It’s not easy to create a really approachable programming language.

Obviously.

It’s been done a few times — notably with BASIC. Javascript is a major achievement, but compared to HyperTalk it’s still very difficult to learn. Javascript essentially thrived by being ubiquitous, indispensable, and based on a simplified subset of the widely understood C-ish syntax:

if (x == y && z != q){ p = foo ? bar : baz; }

could be in any one of a dozen C-ish languages, including Javascript. HyperCard was based on a simplified subset of the even more widely understood English syntax:

get the width of button ok
get it * 2
set the width of button ok to it

Javascript was chiefly indispensable because browsers lacked obvious functionality and simply wouldn’t address it — so instead of simple ways to center stuff or make image-buttons change when the user clicked on them, we got a weird new programming language. (E.g. it took the addition of CSS for us to make it possible for something to change its appearance based on mouse events without programming, and having to code CSS is hardly an improvement over having to code in Javascript.) I’m not sure if this was a conspiracy by Netscape to make Javascript popular — never ascribe to any other cause that which is adequately explained by incompetence — but it really took a huge amount of community effort for programming in Javascript to become even vaguely tolerable.

When I finally was forced to get good at programming Javascript (around 2005), just figuring out how to get some kind of debugger working was a major pain in the neck that many developers never bothered to figure out, and writing browser-portable code was a so difficult most developers threw up their hands and just targeted IE6.

My point is that Javascript isn’t a usable language, it’s just more usable than C or Java (not saying much) and supported by a ubiquitous and indispensable environment (the browser) which has grown to have many of the virtues of HyperCard over time (e.g. browsers are pretty stable now, you get a REPL and a decent debugger) while retaining many of the faults (native UI elements are conspicuously missing, and would be amazingly useful given how much effort goes into making half-assed faux elements).

(It’s worth noting that the web was at least partially inspired by HyperCard. The syntax for comments:

<!-- comment -->

looks to me like a little ode to HyperCard, whose comment syntax was

-- comment

although I’m told they both got it from some precursor; still the influence of HyperCard on the web is pervasive.)

I’m not saying that we should have to write

set the borderLeft of the style of button "OK" to "2px solid black" 

instead of the far more economical (but hardly intuitive):

$("#ok").css({borderLeft: "2px solid black"})

but the fact all web developers find the latter programming style tolerable is something of a miracle, and relies on the rise of jQuery — the currently preferred library for papering over the manifold stupidities of web browsers.

I think it’s safe to say that something like

find("button.ok").style.borderLeft = "2px solid black"

would be reasonably intuitive, C-ish, and decently economical, but despite 25 years of progress and a huge amount of effort by some of the best programmers working at some of the richest companies in the world, the best we can come up with right now out-of-the-box is something like:

document.querySelector("button.ok").style.borderLeft = "2px solid black"

which is both far less readable and less intuitive than the HyperTalk version and yet less concise.

A tangential rant on idiotic naming and the misuse of namespaces

How is it that despite everything being namespaced to hell and back these days, “querySelector” isn’t called — I don’t know — “find”? Why not have a global function named “find” for fuck’s sake, or a global find object that supported find.byURL(), find.byCSSSelector(). What’s the point of sticking everything in its own private name space if you can’t give things people use constantly a short, meaningful, intuitive name?

End rant.

Which gets us back to Apple

The first thing in the original Inside Macintosh (after introductory stuff) was a minimalist Macintosh application (I can’t remember whether it was in Pascal or C) that launched, opened up a Window with a text editor inside it, and had a menu and a main event loop. It was something like four pages of code. Note that this application did not handle undo, did not save or load files (maybe it did), and didn’t support multiple documents, and it didn’t behave nicely with other applications (e.g. redraw its Window after being sent to the background) because MacOS was single-tasking at the time. It was barely more than “hello world” in a Window.

This was because, in the original toolbox, when you created a window you had to do things like track mouse behavior in the window’s close box yourself. The fact you didn’t need to handle every keystroke inside the text editor was actually one of the miraculously cool things about the Mac toolbox in 1984 (but the Inside Mac documentation and natively-hosted compilers were still a couple of years away in 1984).

Thanks to over thirty years of progress, and the efforts of some of the greatest programmers and the most design-and-usability-focused (and now richest) company the world has ever known we can now do the equivalent of the above in — I have no idea how much code, so let’s find out. I tried googling for an example somewhere and the best I could come up with was this pure-code Obj-C “Hello World” for iOS.

This approach isn’t necessarily the first thing you should learn when learning to develop applications for a platform, but it should probably be the second or third. You should have a pretty good idea how absolutely everything works at some level.

#import <Foundation/Foundation.h>
#import <UIKit/UIKit.h>

@interface MyDelegate : UIResponder< UIApplicationDelegate >
@end

@implementation MyDelegate
- ( BOOL ) application: ( UIApplication * ) application
           didFinishLaunchingWithOptions: ( NSDictionary * ) launchOptions {
  UIWindow *window = [ [ [ UIWindow alloc ] initWithFrame: 
    [ [ UIScreen mainScreen ] bounds ] ] autorelease ];
  window.backgroundColor = [ UIColor whiteColor ];
  UILabel *label = [ [ UILabel alloc ] init ];
  label.text = @"Hello, World!";
  label.center = CGPointMake( 100, 100 );
  [ label sizeToFit ];
  [ window addSubview: label ];
  [ window makeKeyAndVisible ];
  [ label release ];

  return YES;
}
@end

int main( int argc, char *argv[ ] )
{
  UIApplicationMain( argc, argv, nil, NSStringFromClass( [ MyDelegate class ] ) );
}

That’s actually a lot better than I would have guessed, and a gigantic improvement over writing a minimal Mac application in 1986 (but about on par with writing a MacApp 2.x application in 1993). I can’t find a similar example for Swift, and I’d prefer to implement a desktop application (as its minimal functionality is less minimal than an iPhone app which (for example) is killed by the OS rather than being quit. So I used this longer but similarly minimal Obj-C desktop example as a starting point. Note that this example doesn’t even display “Hello World”, so I added that.

One thing I really like about this second example is that it doesn’t require defining a new class or subclass — like the original Inside Mac example it’s a function, so it doesn’t seem as much like magic — create an instance of the Application class, and send it the .run() method and you’re done, but now WTF does that class do? Sure it’s dealing with objects but it’s really clear what’s going on and where you need to drill down to understand a particular thing better. Similarly, it gives you a very good idea of (for example) what loading a XIB (or NIB) does for you and where it fits into the application’s life cycle.

import Cocoa

func main() -> Int {
    // give me an application instance
    let app = NSApplication.sharedApplication()
    app.setActivationPolicy(.Regular) // no clue if this is necessary or default
                                      // behavior would be fine
    
    // build out our menu (iOS apps do not need to do any of this
    let menubar = NSMenu()
    let appMenuItem = NSMenuItem()
    menubar.addItem(appMenuItem)
    app.mainMenu = menubar
    let appMenu = NSMenu()
    let appName = NSProcessInfo.processInfo().processName
    let quitTitle = "Quit " + appName
    
    // the only thing we're making that actually DOES something is implementing 
    // the Quit menu item
    let quitMenuItem = NSMenuItem(title: quitTitle, action: "stop:", keyEquivalent: "q")
    quitMenuItem.target = app
    appMenu.addItem(quitMenuItem)
    appMenuItem.submenu = appMenu
    
    // this is where we create our window and completely define how it looks -- it 
    //could be entirely replaced by loading a XIB
    let window = NSWindow(
        contentRect: NSRect(x: 0, y: 0, width: 400, height: 400),
        styleMask: NSTitledWindowMask,
        backing: .Buffered,
        `defer`: false // I assume defer is a keyword
    )
    window.cascadeTopLeftFromPoint(NSPoint(x: 20, y: 20)) // playing nice with 
                                 // other apps -- iOS apps don't do this either
    window.title = appName // iOS apps don't have a title, and we don't really 
                           //need to set this
    
    // now we're putting "Hello, world" in nice big letters in the Window
    let label = NSText(frame: NSRect(x: 0, y: 250, width: 400, height: 40))
    label.editable = false
    label.selectable = false
    label.string = "Hello, world"
    label.font = NSFont(name: "Helvetica Neue", size: 30.0)
    label.alignment = .Center
    label.backgroundColor = NSColor.windowBackgroundColor()
    window.contentView?.addSubview(label)
    
    // Having defined the window we're good to go
    window.makeKeyAndOrderFront(nil)
    app.activateIgnoringOtherApps(true)
    app.run() // Magic happens here!
    return 0;
}

I think this is really neat. You can copy and paste this code into an XCode Swift Playground and invoke main() and voila!

This is longer than the iPhone example, but a lot of the lines could be skipped if I went for brevity. If I didn’t prettify the text or define lots of constants (per the example I copied) then it would be just as short as the iOS example while, I think, using less magic and being clearer in what it does. I’m particularly proud of figuring out that I could get the Quit item to send a message (“stop:”) to NSApp without creating a selector referring to the local context (which would have forced me down the “define a subclass and yada yada magic happens” route… I think — I tried just defining a local function and passing its name as a selector but that did not work). That said, the keyboard shortcut for the quit menu item doesn’t work (as I note in the comments).

I actually don’t think the current Apple APIs are all that bad. They’re about on par with MacApp. There is a pretty big impedance mismatch between what the base initializers for the different UI elements do and what you would want them to do if you wanted to write applications this way, which I suspect is because almost no-one does, but while defaults may not look good, they’re not dysfunctional. I could just create the label and set its string property and it would work fine — but why is the default background color not the window background color or — better — transparent? why is the default font not the standard font and size for a label or a text field but something else entirely (the default text settings for a 1989 NeXT machine perhaps)?

I think the problem is that this isn’t where learning app development starts (after perhaps a simpler, more engaging introduction — look how I can create a “Hello, world” app in one minute using XCode. OK, fine, but how the hell would I make an application from scratch if I needed to? How is all this being hooked together? How does the Window hook up to its view controller? How can I use the same view controller for different views? (By the way, this is not addressed by my little example.) Understanding how the basic wiring works also makes XCode’s ridiculously complicated UI more comprehensible — since you now know certain things exist, you know to look for them, and you also know how to simply make stuff work without guessing where it’s buried in the XCode UI.

That said, this is far better than I expected. It would be great if there were initializers that allowed a typical task to be completed in one line (e.g. create a label, position it, and set its content) and if there were better defaults (and the label’s default font settings corresponded to reasonable expectations). If you omit the call to NSWindow.cascadeTopLeftFromPoint then the window appears in a really stupid place. Similarly, why isn’t there a NSMenuItem initializer that lets you stick the new item in a menu? I’m clearly missing something with respect to event handling, and that’s because the relevant initializers aren’t making it easy to do the common, correct thing. I don’t think you can argue that the badness of Apple’s Cocoa APIs is acting as a pons asinorum. In fact it’s more like it’s creating extra work for mediocre programmers.

(Correction: ignore the stuff I say about the menu item not working and the correct event wiring not being implemented by default — by putting a capital “Q” as the parameter for the keyboard equivalent I inadvertently made the shortcut command+shift+q which was why it appeared not to work when I didn’t bother to look at my menu. So my estimation of the APIs improves by a small notch — better default behavior but more magic going on: how does the window know it belongs to app?)

Learning to write Unity Shaders

Dynamically generated infinite scrolling terrain with a custom shader
Dynamically generated infinite scrolling terrain with a custom shader (inside the Unity editor)

Unity has replaced Photoshop in terms of adding features faster than I can assimilate them. It’s been possible to write custom shaders for years, but every time I tried to do something non-trivial I would give up in frustration. Recently, however, I finally wrote some pretty nice dynamic terrain and dynamic planet code and was very frustrated with shading options.

What I wanted to do, in essence, was embed multiple tiling textures inside a single texture, and then have a shader continuously interpolate between a pair of those shaders based on altitude, biome, or whatever. This doesn’t appear to be something other people are doing, so I was not going to be able to take an existing shader and tweak a couple of things to make it work. I’d actually need to understand what I was doing.

If you look at the picture, you’ll see seamless transitions (based on altitude) between a sand dune texture (at sea level) and a forest texture (at middle levels) and further up the beginning of a transition to bare rock. I’ve got a darker blue-tinged rock below the sand, so the material I’m using looks like this:

A single texture that contains multiple tiling textures.
A single texture that contains multiple tiling textures. Most of the texture is blank, but you get the idea.

Obviously there’s room for expansion. I could do some really interesting things with this technique (even moreso if I can interpolate between three or four samples without choking the GPU). I haven’t figured out how to benchmark this stuff — I’m not seeing any hit on the GPU using Unity’s profile, but I haven’t tried running this on a mobile device — so far, this shader seems to run just as fast as (say) a standard diffuse shader.

How to Start

Writing shaders is actually pretty simple. The big problems are finding useful documentation (I couldn’t find any useful documentation on ShaderLab, but it turns out that nVidia’s cg documentation appears to do the trick) and tutorials (I couldn’t find any). It doesn’t help that Unity has made radical changes to its Shader language over time (not really their fault, the underlying GPU architectures have been in flux) which makes a lot of the tutorials you do find worse than useless.

For the record, I’m still using Unity 4.6.x so this may all be obsolete in Unity 5.x. That said, I was working off the latest Unity 5.x online documentation.

The closest thing to useful tutorials I could find is in the Unity documentation — specifically Surface Shaders Examples. Sadly, you’re going to need to infer a great deal from these examples, because I couldn’t find explanations of the simplest things (e.g. how data gets from the shader’s UI to the actual pixel shader code — there’s a lot of automagical linking going on).

Shader "Custom/DynamicTerrainShader" {
	Properties {
		_MainTex ("Base (RGB)", 2D) = "white" {}
		_Color ("Main Color", Color) = (0,0,0,1)
		_WorldScale("World Scale", Float) = 0.25
		_AltitudeScale("Altitude Scale", float) = 0.25
		_TerrainBands("Terrain Bands", Int) = 4
	}
	SubShader {
		Tags { "RenderType"="Opaque" }
		LOD 200
		
		CGPROGRAM
		#pragma surface surf Lambert

		sampler2D _MainTex;
		float _WorldScale;
		float _AltitudeScale;
		float4 _Color;
		int _TerrainBands;

		struct Input {
			float3 worldPos;
			float2 uv_MainTex;
            float2 uv_BumpMap;
		};

		void surf (Input IN, inout SurfaceOutput o) {
			float y = IN.worldPos.y / _AltitudeScale + 0.5;
			float bandWidth = 1.0 / _TerrainBands;
			float s = clamp(y * (_TerrainBands + 2) - 2, 0, _TerrainBands);
			float t = frac(s);
			t = t < 0.25 ? t * 0.5 : ( t > 0.75 ? (t - 0.75) * 0.5 + 0.875 : t * 1.5 - 0.25);
			float band = floor(s)  * bandWidth;
			float2 uv = frac(IN.uv_MainTex * _WorldScale) * (bandWidth - 0.006, bandWidth - 0.006) + (0.003, 0.003);
			uv.y = uv.y + band;
			float2 uv2 = uv;
			uv2.y = uv2.y + bandWidth;
			half4 c = tex2D(_MainTex, uv) * (1 - t) + tex2D(_MainTex, uv2) * t;
			o.Albedo = c.rgb * 0.5;
			o.Alpha = c.a;
		}
		ENDCG
	} 
	FallBack "Diffuse"
}

So here’s my explanation for what it’s worth.

To refer to properties in your code, you need to declare them (again) inside the shader body. Look at occurrences of _MainTex as an instructional example. As far as I can tell you have to figure out which parameters are available where, and which type declarations in the shader body correspond with the (different) types in the properties declaration by osmosis.

The Input block is where you declare which bits of “ambient” information your shader uses from the rendering engine. Again, what you can declare in here and what it is you simply need to figure out from examples. I figured out how worldPos worked from the example which turns the soldier into strips.

Note that the Input block declaration determines what is passed as Input (referred to as IN in the body). The way the declaration works (you declare the type rather than the variable) is a bit puzzling but it kind of makes sense. The SurfaceOutput object is essentially a set of parameters for the pixel that is about to be rendered. So the simplest shader body would simply be something like o.Albedo = (1,0,0,1) which would be the constant color red (depending on the basic shader type, lighting would or wouldn’t be applied, etc.).

Variables and calculations are all about vectors. Basically everything is a vector. A 3D point is a float3, a 2D point is a float2. You can add and multiply vectors (component by component) so (1,0.5) + (-0.5, 0.25) -> (0.5,0.75). You can mix scalars and vectors in some obvious and not-so-obvious ways (hint: it usually pays to be explicit about components).

The naming conventions are interesting. For vectors, you can use x, y, and z as shorthand for accessing specific components. I’m not sure if the fourth coordinate is w or a or something else. I’m also pretty sure that spatial coordinates are not in the order I think they are (so I do ok with foo.x, but get into trouble if I try to handle specific components via (,,) expressions. Hence lines like uv.y = uv.y + band instead of uv = uv + (0,band,0) (which doesn’t work).

You may have noticed some handy functions such as floor and frac being used and wonder what else there is. I couldn’t find any list or references on the Unity website, but eventually found this cg standard library documentation on nVidia’s website (for its shader language). Everything I tried from this list seemed to work (on my nVidia-powered Macbook Pro).

If you’re looking for control structures and the like, I haven’t found any aside from the ternary operator — condition ? value-if-condition-true : value-if-condition-false — which is fully supported, can be nested, etc.. This alone would probably have driven me away just five years ago before I learned to stop worrying and love the ternary operator.

Why no switch statements, loops and such? I’m writing a pixel shader here and I suspect it relies on every program executing the same instruction at the same time, so conditional loops are out. (Actually I may be wrong about this — see the cg language documentation. I don’t know how closely ShaderLab corresponds to cg though.)

Once you see that list of functions you’ll understand why I am using piecewise linear interpolation between the materials (it looks just fine to me).

Some final points — the shader had terrible problems with the edges of the sub-textures until I changed the bitmap sampling to point (versus linear or trilinear interpolation). I suspect this may be a wrapping issue, but (as you may find) debugging these suckers is not easy.

One final comment — even though you’re essentially writing assembler for your GPU, shader programming is pretty forgiving — I haven’t crashed my laptop once (although Unity itself seems to periodically die during compiles).