Learning to write Unity Shaders

Dynamically generated infinite scrolling terrain with a custom shader
Dynamically generated infinite scrolling terrain with a custom shader (inside the Unity editor)

Unity has replaced Photoshop in terms of adding features faster than I can assimilate them. It’s been possible to write custom shaders for years, but every time I tried to do something non-trivial I would give up in frustration. Recently, however, I finally wrote some pretty nice dynamic terrain and dynamic planet code and was very frustrated with shading options.

What I wanted to do, in essence, was embed multiple tiling textures inside a single texture, and then have a shader continuously interpolate between a pair of those shaders based on altitude, biome, or whatever. This doesn’t appear to be something other people are doing, so I was not going to be able to take an existing shader and tweak a couple of things to make it work. I’d actually need to understand what I was doing.

If you look at the picture, you’ll see seamless transitions (based on altitude) between a sand dune texture (at sea level) and a forest texture (at middle levels) and further up the beginning of a transition to bare rock. I’ve got a darker blue-tinged rock below the sand, so the material I’m using looks like this:

A single texture that contains multiple tiling textures.
A single texture that contains multiple tiling textures. Most of the texture is blank, but you get the idea.

Obviously there’s room for expansion. I could do some really interesting things with this technique (even moreso if I can interpolate between three or four samples without choking the GPU). I haven’t figured out how to benchmark this stuff — I’m not seeing any hit on the GPU using Unity’s profile, but I haven’t tried running this on a mobile device — so far, this shader seems to run just as fast as (say) a standard diffuse shader.

How to Start

Writing shaders is actually pretty simple. The big problems are finding useful documentation (I couldn’t find any useful documentation on ShaderLab, but it turns out that nVidia’s cg documentation appears to do the trick) and tutorials (I couldn’t find any). It doesn’t help that Unity has made radical changes to its Shader language over time (not really their fault, the underlying GPU architectures have been in flux) which makes a lot of the tutorials you do find worse than useless.

For the record, I’m still using Unity 4.6.x so this may all be obsolete in Unity 5.x. That said, I was working off the latest Unity 5.x online documentation.

The closest thing to useful tutorials I could find is in the Unity documentation — specifically Surface Shaders Examples. Sadly, you’re going to need to infer a great deal from these examples, because I couldn’t find explanations of the simplest things (e.g. how data gets from the shader’s UI to the actual pixel shader code — there’s a lot of automagical linking going on).

Shader "Custom/DynamicTerrainShader" {
	Properties {
		_MainTex ("Base (RGB)", 2D) = "white" {}
		_Color ("Main Color", Color) = (0,0,0,1)
		_WorldScale("World Scale", Float) = 0.25
		_AltitudeScale("Altitude Scale", float) = 0.25
		_TerrainBands("Terrain Bands", Int) = 4
	}
	SubShader {
		Tags { "RenderType"="Opaque" }
		LOD 200
		
		CGPROGRAM
		#pragma surface surf Lambert

		sampler2D _MainTex;
		float _WorldScale;
		float _AltitudeScale;
		float4 _Color;
		int _TerrainBands;

		struct Input {
			float3 worldPos;
			float2 uv_MainTex;
            float2 uv_BumpMap;
		};

		void surf (Input IN, inout SurfaceOutput o) {
			float y = IN.worldPos.y / _AltitudeScale + 0.5;
			float bandWidth = 1.0 / _TerrainBands;
			float s = clamp(y * (_TerrainBands + 2) - 2, 0, _TerrainBands);
			float t = frac(s);
			t = t < 0.25 ? t * 0.5 : ( t > 0.75 ? (t - 0.75) * 0.5 + 0.875 : t * 1.5 - 0.25);
			float band = floor(s)  * bandWidth;
			float2 uv = frac(IN.uv_MainTex * _WorldScale) * (bandWidth - 0.006, bandWidth - 0.006) + (0.003, 0.003);
			uv.y = uv.y + band;
			float2 uv2 = uv;
			uv2.y = uv2.y + bandWidth;
			half4 c = tex2D(_MainTex, uv) * (1 - t) + tex2D(_MainTex, uv2) * t;
			o.Albedo = c.rgb * 0.5;
			o.Alpha = c.a;
		}
		ENDCG
	} 
	FallBack "Diffuse"
}

So here’s my explanation for what it’s worth.

To refer to properties in your code, you need to declare them (again) inside the shader body. Look at occurrences of _MainTex as an instructional example. As far as I can tell you have to figure out which parameters are available where, and which type declarations in the shader body correspond with the (different) types in the properties declaration by osmosis.

The Input block is where you declare which bits of “ambient” information your shader uses from the rendering engine. Again, what you can declare in here and what it is you simply need to figure out from examples. I figured out how worldPos worked from the example which turns the soldier into strips.

Note that the Input block declaration determines what is passed as Input (referred to as IN in the body). The way the declaration works (you declare the type rather than the variable) is a bit puzzling but it kind of makes sense. The SurfaceOutput object is essentially a set of parameters for the pixel that is about to be rendered. So the simplest shader body would simply be something like o.Albedo = (1,0,0,1) which would be the constant color red (depending on the basic shader type, lighting would or wouldn’t be applied, etc.).

Variables and calculations are all about vectors. Basically everything is a vector. A 3D point is a float3, a 2D point is a float2. You can add and multiply vectors (component by component) so (1,0.5) + (-0.5, 0.25) -> (0.5,0.75). You can mix scalars and vectors in some obvious and not-so-obvious ways (hint: it usually pays to be explicit about components).

The naming conventions are interesting. For vectors, you can use x, y, and z as shorthand for accessing specific components. I’m not sure if the fourth coordinate is w or a or something else. I’m also pretty sure that spatial coordinates are not in the order I think they are (so I do ok with foo.x, but get into trouble if I try to handle specific components via (,,) expressions. Hence lines like uv.y = uv.y + band instead of uv = uv + (0,band,0) (which doesn’t work).

You may have noticed some handy functions such as floor and frac being used and wonder what else there is. I couldn’t find any list or references on the Unity website, but eventually found this cg standard library documentation on nVidia’s website (for its shader language). Everything I tried from this list seemed to work (on my nVidia-powered Macbook Pro).

If you’re looking for control structures and the like, I haven’t found any aside from the ternary operator — condition ? value-if-condition-true : value-if-condition-false — which is fully supported, can be nested, etc.. This alone would probably have driven me away just five years ago before I learned to stop worrying and love the ternary operator.

Why no switch statements, loops and such? I’m writing a pixel shader here and I suspect it relies on every program executing the same instruction at the same time, so conditional loops are out. (Actually I may be wrong about this — see the cg language documentation. I don’t know how closely ShaderLab corresponds to cg though.)

Once you see that list of functions you’ll understand why I am using piecewise linear interpolation between the materials (it looks just fine to me).

Some final points — the shader had terrible problems with the edges of the sub-textures until I changed the bitmap sampling to point (versus linear or trilinear interpolation). I suspect this may be a wrapping issue, but (as you may find) debugging these suckers is not easy.

One final comment — even though you’re essentially writing assembler for your GPU, shader programming is pretty forgiving — I haven’t crashed my laptop once (although Unity itself seems to periodically die during compiles).

Daikon Forge

Daikon Forge lets you edit your UI live in the scene editor and preview it in the game view.
Daikon Forge lets you edit your UI live in the scene editor and preview it in the game view.

Progress continues on Project Weasel, and last weekend I bought a license for Daikon Forge ($90 in the Unity Asset Store) after mulling over the Unity GUI situation for over two years (basically since Unity announced its new GUI system in San Francisco in 2011). Like many Unity developers, I’ve been waiting for Unity to fix its execrable GUI library for some time, and had avoided investing in nGUI, Daikon Forge, or Coherent because something better — and free — was coming out real soon now.

I reached the point where not having a functional UI library was blocking progress, so I decided I was going to buy a better UI library — the question was which one? nGUI has been the market leader, but from what I’ve seen the API is pretty terrible (and the fact that the nGUI dev is the guy building the new Unity GUI does not fill me with enthusiasm). Coherent — based on HTML5 — seems like a great idea, but according to reviews I’ve read it consumes quite a bit of CPU on mobile devices (and you’d expect it to, given that web browsers can tax mobile devices). From what I could see, Daikon Forge was the least expensive option and possibly the best (if not as widely supported as nGUI). It’s very well reviewed, including by quite a few nGUI refugees. It also struck me that the Unity GUI would probably be, in effect, an improved nGUI, so I’ll have the best of both worlds once it actually ships.

Anyway, so far I’ve found Daikon Forge is pretty nice but not without issues. But it’s worth noting that all of the weaknesses I’ve seen so far affect the developer and not the delivered product.

(Mostly) Good Stuff

I whipped up a 3d pinned information panel in less than an hour
I whipped up a 3d pinned information panel in less than an hour

To create a user interface with Daikon Forge you use a wizard to set up a “manager”. On the plus side, it works! On the minus side it’s a “wizard” — automatic fail. I’d prefer you to be able to drag a prefab into the scene, or drag a script onto a GameObject, or select a GameObject and pick a menu item. The sad thing is that there’s no reason it can’t work that way. Functionality A. Usability C.

Next, you need to assign a default skin and a default font to the UI gadget. In my opinion it should Just Work without this. (How? Easy — stick a default skin and a default font in a Resources folder and load them on the fly if needed.) This particular bit initially stymied me (the picker didn’t “see” the relevant assets), and I needed to find the relevant components manually and drag them into place. Functionality B. Usability F.

Daikon Forge makes creating and maintaining a UI atlas very easy
Daikon Forge makes creating and maintaining a UI atlas very easy
Using Sketch (this is the old version) you can whip up a UI skin in a matter of minutes
Using a tool like Sketch (this is the old version) you can whip up a UI skin in a matter of minutes

The tools for creating your own skins and fonts are almost brilliant. (Select the assets and pull a menu item — and you can easily add items to and remove items from your UI atlas any time you need to.) Functionality A. Usability B.

Next, you can right-click the UI “gizmo” in the scene editor and add controls to it directly. This also stymied me because the context menu does not work when your inspector panel is in debug mode, which mine happened to be in at the time. (Who knew?) Again, on the plus side it works, on the down side many controls come with idiotic defaults (e.g. huge or dimensionless). Functionality B. Usability C.

So I finally managed to get my Galaxy working so that I could click on a star and have a little information display pin itself to the star and show information like the name and spectral class of the star and how many planets it has orbiting it (and the display is not scaled, renders in a minimum of draw calls, and is pixel perfect, but is pinned to the projected 3d position of the star — very cool). On the downside, I found the documentation for binding values to controls to be incomprehensible, and did it all by brute force. Functionality C (so far). Usability F.

Bad Stuff

Every widget has a lot of properties, and they're not given sensible defaults
Every widget has a lot of properties, and they’re not given sensible defaults

First of all, there’s a big workflow problem. When I first imported Daikon Forge into my project it didn’t work properly (e.g. when I tried to assign a skin to one of Daikon Forge’s UI managers it wouldn’t find the relevant assets). This problem eventually “went away” (no idea why) but it was a sign of things to come.

Second, I keep my project in Dropbox and would work on it from my laptop and desktop at different times. When I switch machines, the other machine would load changed files and Just Work. But after I installed Daikon Forge this process became completely broken. Daikon Forge would become completely broken and need to be reimported whenever I switched machines. Worse, every script component assignment would become detached from its relevant script (only for Daikon Forge scripts) and have to be manually repaired. I don’t have a fix for this except to stop using DropBox, which might not be a Bad Thing.

Third, the documentation is good as a class reference, but bad as a guide/overview/cookbook. The tutorials are good but video only and somewhat lengthy. The examples are good but no substitute for actual documentation. Then there’s the website. To post on the website you need to register, and when I try to register it tells me my username or email address flags me as a spammer and that if I want to fix this I need to reply (how?) or use the contact form (where?). I tried my usual handle (podperson) and my usual email address, and my real name and my gmail address — all spammers apparently.

The WYSIWYG editing of your interface is great as far as it goes, but that’s not far. E.g. there are guides, but nothing snaps to them. There’s a grid but nothing snaps to it. There’s a tool to distribute multiple controls horizontally or vertically, but it doesn’t work properly. Worst of all, simply moving and editing things is very finicky. You can easily get into a state where the transform widget is overlapping some other control (e.g. a sizing handle) so that whenever you try to move a control you actually resize it. I ended up resorting to the keyboard for moving controls, and directly entering values for sizing them most of the time.

In a nut

Daikon Forge delivers on its promises in terms of functionality and performance. It renders user interfaces efficiently and attractively. It’s very easy to create and maintain texture atlases. And it supports scalable fonts. Furthermore, its design leverages idiomatic Unity development (e.g. you can make and reuse prefab controls in the obvious manner.) But it is far from perfect: ironically for a UI building tool, its own UI is spotty to put it charitably. Thus far I’ve found the website to be borderline useless and the documentation weak. I’m not sure if the video tutorials and examples really fill the gap (I haven’t had the time or inclination to watch all the tutorials).

C#’s Belt and Suspenders

In discussing my (fairly minor) annoyances with C# with a colleague, I finally figured out the nub of the problem. C# tries to protect you from certain bugs (overflowing bounds and buffers in particular) in so many different ways that the relevant warnings and fixes become irritations and noise, probably making code less reliable rather than more

E.g. why does assigning a double precision value to a float stop code from compiling? What’s the problem this solves?

Another example, why does moving values from unsigned to signed integer types stop compiles? The main reasons for worrying about integer overflows or getting a negative number when you expect a non-negative number is array bounds checking, but C# has array bounds checking, so why do this at all?

It seems to me that C# ought to treat all numbers with periods or exponents as double precision by default and allow assignment to float with implicit casting. This emphasizes correctness (precision) over performance by default, which is as it should be. Similarly, assigning an int to a uint should simply be allowed. (The compiler can insert code to throw an error if the value is negative if so desired.)

Now, I should say I’m using Unity’s (i.e. Mono’s) C# compiler here, so perhaps Microsoft deals with these annoyances at IDE level (e.g. by offering to automatically fix the relevant problems) in much the same way as Apple has leveraged compiler technology to provide automatic fixes, optimizations, and suggested improvements in XCode.

C# Part II

Galaxy Generated and Rendered in Unity 3D
Galaxy Generated and Rendered in Unity 3D

In the end, it took me four evenings to replicate the functionality in my Javascript prototype. I’d have to point out that I was able to do some really nice UI stuff (e.g. drag to rotate, mousewheel to zoom) in Unity that I hesitated to mess with in Javascript (the galaxy was rendered to a canvas in Javascript).

On the whole, I have some impressions.

First, C# is very strict about types, and while I can see some benefits, I think a lot of it is utterly wasted. E.g. having to constantly cast from one numeric type to another is simply a pain in the butt (I routinely expect to have to tease apart a bunch of cryptic type-related errors every time I compile a few lines of new code).

// given an array of weights, pick an index with corresponding weighted probability,
// e.g. [1,2,3] -> 1/6 chance of 0, 2/6 chance of 2, 3/6 chance of 3
public uint Pick( uint[] weights ){
    int s = 0;
    uint idx;
    foreach( uint w in weights ){
        s += (int)w;
    }
    s = Range (1, s); // returns a random int from 1 to s, inclusive
    for( idx = 0; idx < weights.Length; idx++ ){
        s -= (int)weights[idx];
        if(s <= 0){
            break;
        }
    }
    return idx;
}

And all of this rigor didn’t actually prevent or even help debug an error I ran into with overflowing a uint (I have a utility that picks weighted random numbers, but I overflowed the weights leading to an out-of-bounds error. (A simple fix is to change idx < weights.Length to idx < weights.Length – 1.) On the whole it would be nice if you could simply live in a world of “numbers” (as in Javascript) and only convert explicitly to a specific representation when doing something low level.

Second, there’s this weird restriction on naming classes within a namespace such that the namespace and class sometimes may not match and the class you want is often not in the namespace you expect. (E.g. I wanted to create the namespace LNM.PRNG and define the class PRNG in it, but this confuses the compiler, so I ended up calling the namespace LNM.Random — so code referring to this is “using LNM.Random” which mysteriously causes a class called PRNG to become available.) I don’t see why namespaces and class names can’t be the same.

Oddly enough in some cases I am allowed to name the class the same as the namespace, and I don’t know why. So LNM.Astrophysics implements the Astrophysics class, but I had to rename Badwords to BadwordFilter at some point because the compiler started complaining.

I’ve been using Monodevelop, which is an editor produced by the Mono folk and lightly customized to work with Unity. It’s a bit of a slug (if it’s developed using Mono, then it’s not a terrific advertisement for Mono). In particular, its autocomplete is great when it works, but utterly frustrating far more often. It fails to match obvious words (e.g. local variable names) and often makes it impossible to type something short (which matches something longer) on the first attempt. The autocomplete is darn useful, or I’d simply switch back to Sublime.

Flying my untextured spaceship around an undecorated, partially implemented solar system
Flying my untextured spaceship around an undecorated, partially implemented solar system

So the current reckoning is that I ended up producing the following:

  • Astrophysics.cs — defines the LNM.Astrophysics namespace and implements the Astrophysics utility class, some useful enumerations, and the Galaxy, Star, and Planet classes.
  • PRNG.cs — defines the LNM.Random namespace and implements the PRNG class which provides convenience functions for the Meisui.Random Mersenne Twister implementation I’m using.
  • Badwords.cs — defines the LNM.Badwords namespace and implements the BadwordFilter class which checks to see if any of a pretty comprehensive list of nasty words is present in a given string. (Used to prevent obscene star names from being generated.)
  • Billboard.cs — a Monobehavior that keeps a sprite facing the camera. It’s a little cute in that it tries to save CPU time by only updating a given star every 10 physics updates. There’s probably a better way.
  • FixedCamera.cs — a Monobehavior that implements a mouse/touch controlled camera that keeps a fixed view of a specified target unless explicitly moved. I’m planning on using this as my main view control throughout the game.
  • NewtonianScoutship.cs — a Monobehavior implementing an Asteroids-style player ship. I’ve also experimented with a “Delta Vee” style abstracted Newtonian ship controller but good old rotate and accelerate just feels better, and makes movement a pleasant challenge in and of itself. (It also makes becoming “stationary” in space almost impossible, which I think is a Good Thing.)
  • GalaxyGenerator.cs — a Monobehavior that instantiates a galaxy and renders it with sprites. (Right now every single star is a Draw Call, so I’m going to need to do some optimization at some point.)
  • Starsystem.cs — a Monobehavior that instantiates a Star (and its planets) and renders them, a navigation grid (to provide a sense of movement when passing through empty space) and orbits using a bunch of Prefabs.

So, while I continue to find C# quite tedious to develop with, I am making significant progress for the first time in over a year, and while C# feels much less productive than real Javascript, I do think it’s very competitive with Unityscript, and it’s been quite easy to get over the hump.

C#

Screen shot of my galaxy generator in action
Screen shot of my galaxy generator in action

I’ve been developing stuff with Unity in my spare time for something like eight years (I started at around v1.5). I was initially drawn in by its alleged Javascript support. Indeed, I like Unityscript so much, I defended it vocally against charges that using C# is better, and wrote this article to help others avoid some of my early stumbles. I also contributed significant improvements to JSONParse — although the fact that you need a JSON module for Unityscript tells you something about just how unlike Javascript it really is.

I’m a pretty hardcore Javascript coder these days. I’ve learned several frameworks, written ad unit code that runs pretty much flawlessly on billions of computers every day (or used to — I don’t know what’s going on with my former employers), created a simple framework from scratch, helped develop a second framework from scratch (sorry, can’t link to it yet), built services using node.js and phantom.js, built workflow automation tools in javascript for Adobe Creative Suite and Cheetah 3D, and even written a desktop application with node-webkit.

The problem with Unityscript is that it’s not really Javascript, and the more I use it, the more the differences irk me.

Anyway, one evening in 2012 I wrote a procedural galaxy generator using barebones Javascript. When I say barebones, I mean that I didn’t use any third-party libraries (I’d had a conversation with my esteemed colleague Josiah Ulfers about the awfulness of jQuery’s iterators some time earlier and so I went off on a tangent and implemented my own iteration library for fun that same evening).

Now, this isn’t about the content or knowledge that goes into the star system generator itself. It’s basic physics and astrophysics, a bit of googling for things like the mathematics of log spirals, and finding Knuth’s algorithm for generating gaussian random distributions. Bear in mind that some of this stuff I know by heart, some of it I had googled in an idle moment some time earlier, and some of it I simply looked up on the spot. I’m talking about the time it takes to turn an algorithm into working code.

So the benchmark is: coding the whole thing, from scratch, in one long evening, using Javascript.

Now, the time it took to port it into Unityscript — NaN. Abandoned after two evenings.

I’m about halfway through porting this stuff to C# (in Unity), and so far I’ve devoted part of an afternoon and part of an evening. Now bear in mind that with C# I am using the Mono project’s buggy auto-completing editor, which is probably a slight productivity win versus using a solid text editor with no autocomplete for Javascript (and Unityscript). Also note that I am far from fluent as a C# programmer.

So far here are my impressions of C# versus Javascript.

C#’s data structures and types are a huge pain. Consider this method in my PNRG class (which wraps a MersenneTwister implementation I found somewhere in a far more convenient API):

// return a value in [min,max]
public float RealRange( double min, double max ){
    return (float)(mt.genrand_real1 () * (max - min) + min);
}

I need to cast the double (that results from mt.genrand_real1 ()). What I’d really like to do is pick a floating point format and just use it everywhere, but it’s impossible. Some things talk floats, others talk double, and of course there are uints and ints, which must also be cast to and fro. Now I’m sure there are bugs caused by, for example, passing signed integers into code that expects unsigned, but seriously. It doesn’t help that the Mono compiler generates cryptic error messages (not even telling you, for example, what it is that’s not the right type).

How about some simple data declarations:

Javascript:

var stellarTypes = {
    "O": {
        luminosity: 5E+5,
        color = 'rgb(192,128,255)',
        planets = [0,3]
    },
    ...
};

C#:

public static Dictionary<string, StellarType> stellarTypes = new Dictionary<string, StellarType> {
    {"O", new StellarType(){
        luminosity = 50000F,
        color = new Color(0.75F,0.5F,1.0F),
        minPlanets = 0, 
        maxPlanets = 3
    }},
    ...
};

Off-topic, here’s a handy mnemonic — Oh Be A Fine Girl Kiss Me (Right Now Smack). Although I think that R and N are now referred to as C-R and C-N and have been joined by C-H and C-J so we probably need a replacement.

Note that the C# version requires the StellarType class to be defined appropriately (I could have simply used a dictionary of dictionaries or something, but the declaration gets uglier fast, and it’s pretty damn ugly as it is. I also need use the System.Collections.Generic namespace (that took me a while to figure out — I thought that by using System.Collections I would get System.Collections.Generic for free).

Now I don’t want to pile on C#. I actually like it a lot as a language (although I prefer Objective-C so far). It’s a shame it doesn’t have some obvious syntax sugar (e.g. public static auto or something to avoid typing the same damn type twice) and that its literal notation is so damn ugly.

Another especially annoying declaration pattern is public int foo { get; private set; } — note the lack of terminal semicolon, and the fact that it’s public/private. And note that this should probably be the single most common declaration pattern in C#, so it really should be the easiest one to write. Why not public int foo { get; }? (You shouldn’t need set at all — you have direct internal access to the member.)

I’m also a tad puzzled as to why I can’t declare static variables inside methods (I thought I might be doing it wrong, but this explanation argues it’s a design choice — but I don’t see how a static method variable would or should be different from an instance variable, only scoped to the method. So, instead I’m using private member variables which need to be carefully commented. How is this better?

So in a nutshell, I need to port the following code from Javascript to C#:

  • astrophysics.js — done
  • badwords.js — done; simple code to identify randomly generated names containing bad words and eliminate them
  • iter.js — C# has pretty good built-in iterators (and I don’t need most of the iterators I wrote) so I can likely skip this
  • mersenne_twister — done; replaced this with a different MT implementation in C#; tests written
  • planet.js — I’ve refactored part of this into the Astrophysics module; the rest will be in the Star System generator
  • pnrg.js — done; tests written; actually works better and simpler in C# than Javascript (aside from an hour spent banging my head against weird casting issues)
  • star.js — this is the galaxy generator (it’s actually quite simple) — it basically produces a random collection of stars offset from a log spiral using a gaussian distribution.
  • utils.js — random stuff like a string capitalizer, roman numeral generator, and object-to-HTML renderer; will probably go into Astrophysics or be skipped

Once I’ve gotten the darn thing working, I’ll package up a web demo. (When Unity 5 Pro ships I should be able to put up a pure HTML version, which will bring us full circle.) Eventually it will serve as the content foundation for Project Weasel and possibly a new version of Manta.