Getting an Nvidia 1070 (or similar) GPU working on a Mac Pro 5, 1

Victory!

I’ve been using a chipped Radeon 7950 in my 2012 Mac Pro for several years (it was a serious upgrade to my original 5770 or whatever it was that came with it) but eventually my Dell 2715Q (a 4K display) stopped working reliably with it at full resolution and I had to drop down to 1080p. Then it stopped working in 1080p.

I was pretty sure the problem was with the display (which also wasn’t working properly with my Macbook Pros), but the GPU had always been twitchy (sometimes not working on boot, and not driving all its display ports) so when Nvidia announced drivers for its latest GPUs, I figured what the heck?

Anyway, here’s the correct process along with gotchas from not doing it this way, since I found zero reliable guides online to help me.

Warning: if anything goes wrong you’ll need to screenshare into your Mac Pro from another Mac to see what’s going on, so make sure your Mac’s network connection is robust and you can screenshare into it before you do anything you’re going to regret. Luckily for me (since I fucked all this stuff up multiple times) our Macs can all “see” each other (mainly so I can get at parental controls on other Macs easily).

  1. Update your Mac to 10.12.4 (or whatever is current).
  2. Go to Nvidia’s website and download their out-of-date Mac OS X drivers, install them, and then update them in the control panel. I don’t know when you’re reading this but you want your drivers as up-to-date as possible.
  3. You may also want to install CUDA drivers, but that’s not critical.
  4. Shut down, unplug, power off, remove the Mac’s cover.
  5. I got a 1070 bundled with Mass Effect Andromeda. (Don’t care about the bundle, since I’ve got it for PS4 and hate Windows, but it was $50 cheaper than the same card without Mass Effect Andromeda. I don’t think much of Mass Effect Andromeda, but it’s definitely worth more than -$50.)
  6. The 1070 is physically a total pain to get into the Mac Pro (the 7950 seems to have been just as bad, but I have cheerfully lost all memory of it). Be careful to remove all the rubbery covers so that they don’t fall off on top of the PCI slot and cause you enormous consternation.
  7. The Mac Pro comes with two 6-pin power cables for graphics cards. The Nvidia 1070 takes one 8-pin cable, but there should be a 2x 6-pin to 8-pin adapter cable in the box. You’ll need that. Sadly it creates a lot of slack in your cables that will be snaked inside your otherwise tidy (if horribly dusty) Mac Pro.
  8. Make sure everything is securely hooked up. Close the box, plug it in, plug in displays, and boot. (If you’re using a wired network, make sure that’s plugged in.)
  9. Power on, wait for the chime, and hopefully you will be golden.

Troubleshooting

Here are the ways I fucked this up.

First, I didn’t realize the current version of Mac OS was 10.12.4, so I had 10.12.3 and installed the (January) version of the Nvidia drivers which then claimed to be up-to-date.

After I installed the card my Mac wouldn’t display jack shit from any port at any resolution. After trying two different displays and four different ports, I screen-shared into it and verified (a) it was working properly, (b) it could see the video card and recognize the vendor but couldn’t do anything with it, and (c) that the Nvidia panel could see the video card but not do anything with it.

I then found a post showing someone had successfully installed a 1080 on their Hackintosh with 10.12.4. Whoops! I installed 10.12.4 and rebooted. No dice. I went into the Nvidia panel and found it no longer claimed to be up-to-date, so I installed a new version, rebooted, and my Dell monitor came to life at a resolution I’d never seen it in before. (Easily fixed. I am now looking at my Mac Pro’s desktop in glorious 1440p, as God Steve intended.)

Returning to the Adobe fold… sort of

I remain very frustrated with my Photography workflow. No-one seems to get this right and it drives me nuts. (I’m unwilling to pay Apple lots of money for a ridiculous amount of iCloud storage, which might work quite well, but it still wouldn’t have simple features like prioritizing images that I’ve rated or looked at closely over others automagically, or allow me to rate JPEGs and have the rating carried over to the RAW later.)

Anyway, Aperture is sufficiently out-of-date that I’ve actually uninstalled it and Photoshop still has some features (e.g. stitching) that its competition cannot match. So, $120 for a year of Photoshop + Lightroom… let’s see how it goes.

Lightroom

I was expecting Lightroom to be awesome what with all the prominent folks who swear by it. So far I find it unfamiliar (I did actually use LR2, and of course I am a Photoshop ninja) to the point of frustration, un-Mac-like, and ugly, ugly, ugly.

Some of my Lightroom Gripes
Some of my Lightroom Gripes

A large part of the problem is terrible use of screen Real Estate. It’s easy to hide the menubar (once you find where Adobe has hidden its non-standard full screen controls), but it’s hard (impossible) to hide the idiotic mode menu “Identity Plate”. (I found the “Identity Plate Editor” (WTF?) by right-clicking hopefully all over the place, which allowed me to shrink the stupidly large lettering but it just left the empty space behind. How can an application that was created brand new (and initially Mac-only) have managed to look like a dog’s breakfast so quickly?

But there are many little things that just suck.

  • All the menus are horrible — cluttered and full of nutty junk. Looks like design by committee.
  • The dialog box that appears when you “Copy…” the current adjustments is a crime against humanity (it has a weird set of defaults which I overrode by clicking “check none” when I only wanted to copy some very specific settings and now I can’t figure out how to restore the defaults).
  • The green button doesn’t activate full screen mode. There are multiple full screen modes and none of them are what I want.
  • Zooming with the trackpad is weird. And the “Loupe” (nothing like or as nice as Aperture’s) changes its behavior for reasons I cannot discern. (I finally figured out that the zoom in shortcut actually goes to 1:1 by default, which is useful, although it’s such a common feature I’d have assigned a “naked” keystroke to it, such as Z, which instead toggles between display modes.)
  • The main image view seizes up after an indeterminate amount of use and shortly afterwards Lightroom crashes. (This is on maxed out Macbook Pro 15″.)
  • I can’t hide the stupid top bar (with your name in it). I can’t even make it smaller by reducing the font size of the crap in it.
  • Hiding the “toolbar” hides a random thing that doesn’t seem to me to be a toolbar.
  • By default the left side of the main window is wasted space. Oh, and the stupid presets are displayed as a list of words — you need to mouse over them to get a low-fidelity preview.
A crime against humanity.
A crime against humanity.

I found Lightroom’s UI sufficiently annoying that I reinstalled Aperture for comparison. Sadly, Lightroom crushes Aperture in ways that really matter. E.g. its Shadow and Highlight tools simply work better than Aperture’s (I essentially need to go into Curves to do anything slightly difficult in Aperture), and it has recent features (such as Dehaze — I’m pretty sure inspired by a similar feature DxO was very proud of a while back). After processing a few carefully underexposed RAW images* in both programs, Lightroom gets me results that Aperture simply can’t match (it also makes it very tempting to make the kind of over-processed images you see everywhere these days with amped up colors, quasi-HDR effects, and exaggerated micro-contrast).

(* Quite a few years ago someone I respect suggested that it’s a good idea to “underexpose” most outdoor shots by one stop to keep more highlight detail. This is especially important if the sky is visible. These days, everyone seems to be on the “ISO Invariance” bandwagon which is essentially not doing anything to the signal off the sensor (boosting effective ISO) when capturing RAW, in essence, “expose to the left” automatically — the exact opposite of the “expose to the right” bandwagon these clowns were all on two years ago — here’s a discussion of doing both at the same time. Hilarious. On the bright side, ISO Invariance pretty much saves ETTR nuts from constantly blowing their highlights.)

The Photos App is far more competitive with Lightroom than Aperture
The Photos App is far more competitive with Lightroom than Aperture. And its UI is simply out of Lightroom’s league (see those filters on the right? Lightroom simply has a list of names).

Funny thing though is that the new Photos app gives Lightroom a much better run for its money (um, it’s free), has Aperture’s best UI feature (organic customization), and everything runs much faster that Lightroom. The problem with Photos is it is missing key features of Lightroom, e.g. Dehaze, Clarity, and (most curiously) Vibrance. You just can’t get as far with Photos as you can with Lightroom. (If you have Affinity Photo you can use its Dehaze more-or-less transparently from Photos. It’s a modal, but then Lightroom is utterly modal.)

On the UI level, though, Photos simply spanks Lightroom’s Develop mode. Lightroom’s organization tools, clearly with many features requested by users, are completely out of Photos’ league.

I also tried Darktable (the open source Lightroom replacement) for comparison. I think its user interface is in many ways nicer than Lightroom’s — it looks and feels better — although much of its lack of clutter is owed to a corresponding lack of features), but the sad news is that Darktable’s image-processing capabilities don’t even compete with Aperture, let alone Photos. (One thing I really like about Darktable is that it applies “orientation” (automatic horizon leveling), “sharpen”, and “base curve” automagically by default. Right now this isn’t customizable — there’s a placeholder dialog — but if it were it would be an awesome feature.)

The lack of fit and finish in Lightroom is unprofessional and embarrassing
The lack of fit and finish in Lightroom is unprofessional and embarrassing. If it’s not obvious to you, the red line shows the four different baselines used for the UI elements.
Hilarious. Lightroom's "About Box" uses utterly non-standard buttons that behave like tab selectors.
This is hilarious. Lightroom’s “About box” uses utterly non-standard buttons that behave like tab selectors. This is actually regression for Adobe, which used to really take pride in its About boxes.

At bottom, Aperture doesn’t look or feel like an application developed by or for professionals. It’s very capable, but its design is — ironically — horrible.

Photoshop

Photoshop’s capabilities are, by and large, unmatched, but its UI wasn’t good when it first came out and many of its worst features have pretty much made it through unscathed by taste, practicality, or a sense of a job well done. Take a look at this gem:

Adobe Photoshop's horrible Radial Blur dialog
Adobe Photoshop’s horrible Radial Blur dialog

This was an understandably frustrating dialog back in 1991 — in fact the attempt to provide visual cues with the lines was probably as much as you could expect, but it hasn’t changed since — every other application I use provides a GPU-accelerated live preview (in Acorn it’s non-destructive too). What’s even worse is that it looks like the dialog’s layout has been slightly tweaked to allow for too-large-and-non-standard buttons (with badly centered captions that would look worse if there were a glyph with a descender in it). At least it doesn’t waste a buttload of space on a mode menu: instead there’s a small popup lets you pick which (customizable) “workspace” you want to use, and the rest of the bar is actually useful (it shows common settings for the currently selected tool).

In the end, Photoshop at least looks reasonably nice , and its UI foibles are things I’ve grown accustomed to over twenty-five years.

I can’t wait until I get to experience Adobe’s Updater…

Learning to write Unity Shaders

Dynamically generated infinite scrolling terrain with a custom shader
Dynamically generated infinite scrolling terrain with a custom shader (inside the Unity editor)

Unity has replaced Photoshop in terms of adding features faster than I can assimilate them. It’s been possible to write custom shaders for years, but every time I tried to do something non-trivial I would give up in frustration. Recently, however, I finally wrote some pretty nice dynamic terrain and dynamic planet code and was very frustrated with shading options.

What I wanted to do, in essence, was embed multiple tiling textures inside a single texture, and then have a shader continuously interpolate between a pair of those shaders based on altitude, biome, or whatever. This doesn’t appear to be something other people are doing, so I was not going to be able to take an existing shader and tweak a couple of things to make it work. I’d actually need to understand what I was doing.

If you look at the picture, you’ll see seamless transitions (based on altitude) between a sand dune texture (at sea level) and a forest texture (at middle levels) and further up the beginning of a transition to bare rock. I’ve got a darker blue-tinged rock below the sand, so the material I’m using looks like this:

A single texture that contains multiple tiling textures.
A single texture that contains multiple tiling textures. Most of the texture is blank, but you get the idea.

Obviously there’s room for expansion. I could do some really interesting things with this technique (even moreso if I can interpolate between three or four samples without choking the GPU). I haven’t figured out how to benchmark this stuff — I’m not seeing any hit on the GPU using Unity’s profile, but I haven’t tried running this on a mobile device — so far, this shader seems to run just as fast as (say) a standard diffuse shader.

How to Start

Writing shaders is actually pretty simple. The big problems are finding useful documentation (I couldn’t find any useful documentation on ShaderLab, but it turns out that nVidia’s cg documentation appears to do the trick) and tutorials (I couldn’t find any). It doesn’t help that Unity has made radical changes to its Shader language over time (not really their fault, the underlying GPU architectures have been in flux) which makes a lot of the tutorials you do find worse than useless.

For the record, I’m still using Unity 4.6.x so this may all be obsolete in Unity 5.x. That said, I was working off the latest Unity 5.x online documentation.

The closest thing to useful tutorials I could find is in the Unity documentation — specifically Surface Shaders Examples. Sadly, you’re going to need to infer a great deal from these examples, because I couldn’t find explanations of the simplest things (e.g. how data gets from the shader’s UI to the actual pixel shader code — there’s a lot of automagical linking going on).

Shader "Custom/DynamicTerrainShader" {
	Properties {
		_MainTex ("Base (RGB)", 2D) = "white" {}
		_Color ("Main Color", Color) = (0,0,0,1)
		_WorldScale("World Scale", Float) = 0.25
		_AltitudeScale("Altitude Scale", float) = 0.25
		_TerrainBands("Terrain Bands", Int) = 4
	}
	SubShader {
		Tags { "RenderType"="Opaque" }
		LOD 200
		
		CGPROGRAM
		#pragma surface surf Lambert

		sampler2D _MainTex;
		float _WorldScale;
		float _AltitudeScale;
		float4 _Color;
		int _TerrainBands;

		struct Input {
			float3 worldPos;
			float2 uv_MainTex;
            float2 uv_BumpMap;
		};

		void surf (Input IN, inout SurfaceOutput o) {
			float y = IN.worldPos.y / _AltitudeScale + 0.5;
			float bandWidth = 1.0 / _TerrainBands;
			float s = clamp(y * (_TerrainBands + 2) - 2, 0, _TerrainBands);
			float t = frac(s);
			t = t < 0.25 ? t * 0.5 : ( t > 0.75 ? (t - 0.75) * 0.5 + 0.875 : t * 1.5 - 0.25);
			float band = floor(s)  * bandWidth;
			float2 uv = frac(IN.uv_MainTex * _WorldScale) * (bandWidth - 0.006, bandWidth - 0.006) + (0.003, 0.003);
			uv.y = uv.y + band;
			float2 uv2 = uv;
			uv2.y = uv2.y + bandWidth;
			half4 c = tex2D(_MainTex, uv) * (1 - t) + tex2D(_MainTex, uv2) * t;
			o.Albedo = c.rgb * 0.5;
			o.Alpha = c.a;
		}
		ENDCG
	} 
	FallBack "Diffuse"
}

So here’s my explanation for what it’s worth.

To refer to properties in your code, you need to declare them (again) inside the shader body. Look at occurrences of _MainTex as an instructional example. As far as I can tell you have to figure out which parameters are available where, and which type declarations in the shader body correspond with the (different) types in the properties declaration by osmosis.

The Input block is where you declare which bits of “ambient” information your shader uses from the rendering engine. Again, what you can declare in here and what it is you simply need to figure out from examples. I figured out how worldPos worked from the example which turns the soldier into strips.

Note that the Input block declaration determines what is passed as Input (referred to as IN in the body). The way the declaration works (you declare the type rather than the variable) is a bit puzzling but it kind of makes sense. The SurfaceOutput object is essentially a set of parameters for the pixel that is about to be rendered. So the simplest shader body would simply be something like o.Albedo = (1,0,0,1) which would be the constant color red (depending on the basic shader type, lighting would or wouldn’t be applied, etc.).

Variables and calculations are all about vectors. Basically everything is a vector. A 3D point is a float3, a 2D point is a float2. You can add and multiply vectors (component by component) so (1,0.5) + (-0.5, 0.25) -> (0.5,0.75). You can mix scalars and vectors in some obvious and not-so-obvious ways (hint: it usually pays to be explicit about components).

The naming conventions are interesting. For vectors, you can use x, y, and z as shorthand for accessing specific components. I’m not sure if the fourth coordinate is w or a or something else. I’m also pretty sure that spatial coordinates are not in the order I think they are (so I do ok with foo.x, but get into trouble if I try to handle specific components via (,,) expressions. Hence lines like uv.y = uv.y + band instead of uv = uv + (0,band,0) (which doesn’t work).

You may have noticed some handy functions such as floor and frac being used and wonder what else there is. I couldn’t find any list or references on the Unity website, but eventually found this cg standard library documentation on nVidia’s website (for its shader language). Everything I tried from this list seemed to work (on my nVidia-powered Macbook Pro).

If you’re looking for control structures and the like, I haven’t found any aside from the ternary operator — condition ? value-if-condition-true : value-if-condition-false — which is fully supported, can be nested, etc.. This alone would probably have driven me away just five years ago before I learned to stop worrying and love the ternary operator.

Why no switch statements, loops and such? I’m writing a pixel shader here and I suspect it relies on every program executing the same instruction at the same time, so conditional loops are out. (Actually I may be wrong about this — see the cg language documentation. I don’t know how closely ShaderLab corresponds to cg though.)

Once you see that list of functions you’ll understand why I am using piecewise linear interpolation between the materials (it looks just fine to me).

Some final points — the shader had terrible problems with the edges of the sub-textures until I changed the bitmap sampling to point (versus linear or trilinear interpolation). I suspect this may be a wrapping issue, but (as you may find) debugging these suckers is not easy.

One final comment — even though you’re essentially writing assembler for your GPU, shader programming is pretty forgiving — I haven’t crashed my laptop once (although Unity itself seems to periodically die during compiles).

Affinity Photo — First Impressions

Affinity Photo in action

Note: if you’re interested in using Affinity Photo for processing RAW photos (i.e. its “non-destructive workflow”) you’re probably going to be horribly disappointed. See my followup article.

Affinity Photo has just come out of beta and is being sold for a discounted price of $40 (its regular price will be $50). As with Affinity Designer, it’s well-presented, with an attractive icon and a dark interface that is reminiscent of late model Adobe Creative Cloud and Apple Pro software. So, where does it fit in the pantheon of would-be Photoshop alternatives?

In terms of core functionality, it appears to fit in above Acorn and below Photoline. In particular, Photoline supports HDR as well as 16-bit and LAB color, while Affinity Photo lacks support for HDR editing. Unless you work with HDR (and clearly not many people do) then Affinity Designer is both less expensive than Photoline, and far more polished in terms of the features it does support.

Affinity Designer supports non-destructive import of RAW files. When you open a RAW file you enter “Develop” mode where you can perform adjustments to exposure, curves, noise, and so forth on the RAW data before it gets converted to 8- or 16-bit RGB. Once you leave Develop mode, you can return and second-guess your adjustments (on a layer-by-layer basis). This alone is worth the price of admission, and leaves Acorn, Pixelmator, and Photoline in the dust.

In essence you get the non-destructive workflow of Lightroom and the pixel-manipulation capabilities of Photoshop in a single package, with the ability to move from one to the other at any point in your workflow. Let me repeat that — you can “develop” your raw, go mess with pixels in the resulting image, then go back and second-guess your “develop” settings (while retaining your pixel-level manipulations) and so on.

This feature isn’t quite perfect. E.g. you can’t go back and second-guess a crop, and vector layer operations, such as text overlays, get reduced to a “pixel” layer if you go back to develop mode. But it’s a big step in the right direction and for a lot of purposes it’s just dandy.

This is just my first impressions, but there are some things that could be better.

Affinity Photo provides adjustment layers, live filter layers, filters, and layer effects — in many cases providing multiple versions of the same filter in different places. Aside from having functionality scattered and in arbitrary buckets, you get several different user interfaces. This is a mess, and it is a direct result of copying Photoshop’s crazy UI (accumulated over decades of accumulated functionality) rather than having a consolidated, unified approach the way Acorn does.

At first I thought Affinity Photo didn’t support layer styles, but it does. Unfortunately you can’t simply copy and paste layer styles (the way you can in Photoshop and Acorn), so the workflow is a bit more convoluted (you need to create a style from a selection and then apply it elsewhere — often you just want to copy a style from A to B without creating a reusable (or linked) style so this is a bit unfortunate).

I really like the fact that the RGB histogram gives a quick “approximate” view but shows a little warning symbol on it. When you click it, it does a per-pixel histogram (quite quickly, at least on my 24MP images).

I don’t see any support for stitching images, so if that’s important to you (and it’s certainly very important to landscape photographers) then you’ll need to stick with Adobe, or specialized plugins or software.

It also seems to lack smart resize and smart delete or Photoshop’s new motion blur removal functions. (Photoline also does smart delete and smart resize.)

Anyway, it’s a great first release, and definitely fulfills the promise of the public betas. It seems to me that it’s a more solid overall effort than Affinity Designer was when first released, and I’m probably a more demanding user of Photoshop-like programs than I am of Illustrator-like programs. I can understand the desire to provide a user interface familiar to Adobe products even at the cost of making them unnecessarily confusing and poorly organized, but I hope that sanity prevails in the long run.

Bottom line: a more complete and attractive package than either Photoline or Acorn (its most credible competitors) and better in some ways than Photoshop.

Dell P2715Q Display

About this Mac — Displays

So, I came across this Dell 4K display while visiting the new Nebraska Furniture that has opened close to us. (It’s quite an amazing place — retail’s revenge on Amazon.com — and it sells a lot more than furniture.) Anyway, I got it home, plugged it into my Macbook Pro 15″ (2014) and it just works (at 60Hz). That’s about all I can say about it.