Back in the early days of the Internet you’d frequently find sensationalist stories along the lines of “teenager learns how to make bomb from internet” in the media. Similarly, we’ve seen headlines along the lines of “mass murderer played violent video games“, or “rapist had huge collection of porn” (no links because my search results are dominated by stories about pedophiles, which is a whole different topic), or — one of my favorites — “teenager who played D&D commits suicide“. Another very common example these days is “woman causes fatal car accident while texting“. All of these headlines tend to have some things in common:
- The basic facts are true, e.g. the teenager who committed suicide did in fact play D&D.
- The implied causal factor is something novel to, or stigmatized by, society, or both.
- There is no actual attempt to determine if there is even a correlation, let alone causation (e.g. are D&D players more or less likely to commit suicide than non-D&D players?) — usually the stories don’t bother to even cite relevant statistics.
It’s also worth noting that you almost never see the opposite kind of story. E.g. “teenager who plays D&D volunteers at suicide help line,” even though there are doubtless plenty of examples, because they don’t fit the desired narrative of “thing you already don’t like and are kind of suspicious of is actually morally bad and here’s why”.
Back in the 90s, whenever I heard relatives quoting insane alarmist stories about the internet to me (and I was “the guy” in my extended family who “knew about computers” so I got a lot of this) I would say “replace the word internet in the story with books or letters and see whether it is equally applicable”. After all, there are plenty of stories about women being seduced by serial killers by mail, or people learning to do terrible things in a library. In a recent discussion with my very smart wife and some of her very smart colleagues, the question of electronic vs. printed books came up. All of the colleagues were young (well, younger than us), technically savvy, and had a background in either psychology or communications or both. All claimed there was a body of research showing the retention or comprehension rates are lower when you read electronic vs. printed material.
(You know, I remember when some programmers liked to print hard copies of their code for debugging because reading code on crappy CRTs sucked. But you don’t see that much any more.)
Later, there was discussion of how, for example, “things that girls do” tend to be trivialized and stigmatized by society (e.g. “texting” is something generally ascribed more to girls than boys), and the term “moral panic” came up. This is the general term for when “society” (e.g. the mass media, or whatever) encounters some new or marginalized phenomenon and — finding cases where it appears to be linked to some other bad thing (e.g. a fatal car accident) — loses its shit. The current vogue example is selfies (it was hard to find the linked story since most such stories are about people accidentally or deliberately shooting themselves while taking selfies — hooray for gun owners — but let’s not get ahead of ourselves).
I pointed out that the anti-eBook stance they were all taking was an example of moral panic.
First, I tried to find the research that shows lower comprehension rates when reading eBooks. Here’s an example. Let me quote the article:
“When you read on paper you can sense with your fingers a pile of pages on the left growing, and shrinking on the right,” said Mangen. “You have the tactile sense of progress, in addition to the visual … [The differences for Kindle readers] might have something to do with the fact that the fixity of a text on paper, and this very gradual unfolding of paper as you progress through a story, is some kind of sensory offload, supporting the visual sense of progress when you’re reading. Perhaps this somehow aids the reader, providing more fixity and solidity to the reader’s sense of unfolding and progress of the text, and hence the story”
If you read the article, they have two studies, one of 50 people and another of 72. In the first case the researcher mostly got null results (i.e. the readers of both media had equal success in answering comprehension questions) but discovered that the readers of eBooks had more difficulty placing events in order. In the second case the researcher found “students who read texts in print scored significantly better on the reading comprehension test than students who read the texts digitally”. Of course both tests were badly flawed. In the first test only two subjects were familiar with the software and hardware being used (everyone, of course, is familiar with printed material). In the second case the “digital” version was a PDF not, say, an ePub which is designed to be read on screen. Imagine if I did such a test comparing reading a web page with a printed version of that web page in whatever revolting format the browser elected to print it in.
So you could equally draw the conclusion that: “For most intents and purposes, even inexperienced eBook readers using terrible hardware and software have the same comprehension rates as readers of paper books.” Let me also suggest that based on the elaborate and completely unfounded theory that one of the researchers has for why these rather shaky results were obtained, I suspect we’re looking at confirmation bias. She had a theory that eBooks were somehow worse and then went looking for data.
Mechanism of Action
My biggest methodological problem with this research is that it is horribly artificial. Let’s suppose you have an actual task — e.g. you watched Ken Burns’s The Civil War and were incredibly impressed by Shelby Foote and want to read his magnum opus — The Civil War, A Narrative. Or you have to write a paper discussing a bunch of papers for your Art History class. How quickly and easily are you able to accomplish the task you set out to complete? How happy are you with the outcome? And how did this figure into a bigger picture? E.g. how much did you actually learn (not just about the text in question but in general?)
E.g. one of the things I treasure about reading electronically is how quickly I can skip over crap. E.g. if I read something interesting but wonder if it covers a specific point, I can quickly search for it and skip to the next article if not (or decide whether what the article has to say on that point prompts me to read the entire article more carefully). If you were testing my comprehension of the article, I would fare worse for an article that didn’t interest me — but I would have spent far less time deciding I didn’t need to read it; time I could spend reading something else.
I pick both of these examples from personal experience. In the former case I read The Civil War in paperback. I borrowed the three enormous books from a friend; it was absolutely riveting; and I enjoyed it very much. But, if I were to read it today I would far prefer to read it in electronic form and (a) I would be able to highlight or annotate anything I thought was interesting or which I didn’t understand without fear of damaging my friend’s books; (b) I would be able to instantly look up any word I didn’t understand or whose meaning I was unsure of without losing my spot; (c) I could jump to-and-from end-notes without losing my spot; and (d) I would be saved a great deal of back and neck pain caused by trying to read three trade-sized 1000+ page books in bed.
That is not to say that e-books are in all ways superior to printed books. E.g. the reproduction of images is often terrible; e-books often have typographical and layout errors that would be considered egregious in a printed book; and e-readers are simply not up to the task of displaying highly-visual books. But these aren’t intrinsic flaws of e-books, they are implementation flaws. It’s like complaining about printed books in a land of incompetent printers. Conversely, I won’t claim that fact that videos play badly on paper books is a point for e-books, even though that’s true and it’s unlikely to get better.
In the second case I had digital copies of all the papers I had to read and discuss. When I found things I didn’t understand I could instantly look them up, capture and download them when appropriate, find related or contrary pieces and download those, and quickly skip around to relevant pieces or find things I had read using text search (for the papers that were text PDFs rather than scanned documents, at least). I remember doing this kind of thing with paper manuscripts and it was horrible. Heck, the readings for Anthropology A01 were heavier than a laptop and far less useful.
When you’re trying to explain why something is better or worse you don’t just need data, you also need a mechanism of action. Data tells you something is going on, but mechanics tell you what to look for. (Just as the researcher who likes how books feel and smell goes looking for evidence that this somehow improves reading comprehension, but I prefer some kind of more direct mechanism of action than that.) The “mechanics” the researchers claim make printed books better sound like cargo cultism or fetishism to me. E.g. the idea that the number of pages in your left hand versus your right somehow gives you cues to help keep track of the temporal order of events does not stand up to inspection. What if I am reading a short story in a large anthology? What if I am reading a temporally confusing book (such as Iain M Banks’s Use of Weapons) where chronological order does not in any way match page order? What about constantly having to shift posture to cope with reading left-vs-right pages in a heavy book? This is not going to show up in a study of a 28 page short story, but it’s a real consideration for actual books. I can show some real, tangible advantages to reading electronically, and it would be easy to devise experiments to test whether these advantages are real (although getting subjects to read 3000 page trilogies might make getting volunteer test subjects difficult).
How Wrong Can You Be?
Now, let’s pause and balance this out by considering the idea that the liberal consensus in favor of gun control might equally be a moral panic. This is similar to some of the arguments people who I somehow seem to know on Facebook have made against gun control. “Don’t blame the guns.” Let’s ignore the vapidity of that argument and assume that the more cogent argument (that this is a moral panic) was made.
First, let’s look at the degrees to which an assumed causation can be wrong. I hope you already know that “correlation does not equal causation”. E.g. if places with fewer murders happen to have fewer guns all you have is a correlation. It may be that in places where people hate each other they are more likely to buy weapons. The example my Stats professor liked to use was the birthrate in Stockholm is strongly correlated to the stork population.
When a correlation turns out not to reflect causation, the technical term used is that there are mediating factors — in this case season. In extreme cases like this, you’re likely to find that the mediating factor completely eliminates the original correlation (or perhaps even makes it slightly negative); e.g. we can probably find outliers where for some reason there were fewer storks but the birthrate did not dip and vice versa, so taking season into account the correlation between storks and babies becomes much lower and possibly negative.
So, assuming that correlation equals causation is a fairly common trap. You can be wrong, but you’ll find yourself in good company.
People who read text written in sans serif fonts have poorer comprehension than people who read text written in serif fonts. Therefore sans-serif typefaces cause poorer reading comprehension.
This “fact” was well-known when I was starting out in the usability business. (The suggested “mechanism of action” was that the serifs helped readers visually line up the letters or something — given that serifs were invented to prevent rain from eroding letters carved in stone, and then copied by typesetters to lend printed text the authority of Roman carvings, this idea seems fanciful.) In the nineties a lot of contrary evidence was showing up because very low resolution computer displays and not-especially-high-resolution laser printers both sucked at reproducing serif fonts. But it wasn’t long before the result stopped replicating even for documents printed at very high resolution. It turns out it was an example of this kind of trap. What seems to have been happening was that people who were brought up reading serif fonts (in the 1970s this was almost everyone) were better at reading serif fonts.
People who read text on an e-reader have poorer comprehension than people who read text on paper. Therefore e-readers cause poorer reading comprehension.
We’ve just seen that in the cases above, a specific e-reader was used and the users weren’t used to it. We also don’t know whether, for example, good typefaces were picked on the e-reader (e.g. Apple, Microsoft, and Google have each done a lot of work to maximize legibility of screen text; Amazon, not so much.) And, “comprehension” was only worse in one specific way (and the theorized reason for it could easily be addressed by improving the e-reader software, or using a better e-reader).
But again, moral panic is frequently guilty of assuming correlation itself.
Assuming correlation and then jumping to causation is a way of being even more wrong than actually finding a correlation and assuming causation.
And then, worst of all, there’s the availability heuristic: we hear lots of news about X therefore X happens a lot. This is a major reason why people are more afraid of terrorists than staircases and flying than driving. “Swimmer eaten by shark.” “D&D player commits suicide.” “Illegal immigrant is a serial killer.” “black guy kills cop.” The fact there is more news about something doesn’t mean there is more something. Sharks kill very few people. D&D players are less suicidal than non-D&D players. Illegal immigrants are more law abiding than pretty much anyone else. Violence against cops is at historic lows.
So the hierarchy of wrong goes something like this:
- Incredibly wrong. We hear about it a lot, therefore it must be common. (Availability Heuristic.)
- Wrong. We hear about A and B together a lot, therefore A must be related to B. (Assuming correlation.) Therefore A causes B. (And then jumping to causation.)
- Wrong but there’s probably something there. We’ve found that A seems to occur with B. Therefore A causes B. (Assuming causation.)
Moral panics are usually more wrong than merely assuming causation. E.g. it turns out that D&D players are less likely to commit suicide than non-D&D players. Similarly, the evidence linking porn to rape is pretty weak (if anything, the ready availability of sexual outlets such as porn and prostitution appears to reduce rape).
Spot the Moral Panic
Can we apply the same arguments to the gun debate? Or texting while driving?
I’ll pick these two examples because I am in favor of gun control, but I think concerns about texting while driving are overblown.
Evidence in favor of gun control:
- Countries with more gun control have fewer gun deaths. (Factual correlation.)
- Mass shootings disappeared in Australia after ban on semi-automatic weapons. (Factual correlation.)
- US States with stronger gun control laws have lower gun violence when you correct for economic factors, etc. etc. (Factual correlation that’s hard to explain to people, especially those who are disinclined to believe in Math or evidence in general.)
- Black market prices for weapons soared in Australia after ban on semi-automatic weapons. (Factual correlation.)
- It’s hard to shoot people when you don’t have a gun. (Mechanism of Action.)
- Many mass shooters buy guns legally. (Mechanism of Action.)
- Studies show that people with guns do not help (in fact make things worse) in active shooter situations. (Mechanism of Action.)
- Switzerland. (Lots of guns, but very low gun violence. That said, Swiss gun control laws — which are national, standardized, and consistently enforced — would more than satisfy the most liberal gun control advocate in the US, and gun ownership in Switzerland is roughly half that in the US and declining.)
No, this isn’t a moral panic, it’s a very solid case. The one “exception” looks, upon examination, to be more evidence in favor of the case. This is probably why the NRA has done its best to block any scientific investigation of the subject.
Let’s try texting while driving:
- States that passed strong laws against texting while driving saw reductions (of about 7%) in hospitalizations from car accidents. (Factual correlation.)
- States that allow police to stop people on suspicion of texting while driving had bigger reductions than those which did not. (Factual correlation.)
- Tests have found that people who are texting are approximately as impaired as those who are inebriated. (Mechanism of Action.)
- After the initial reduction in accidents, things went back to normal. (Factual non-correlation. This doesn’t actually show that texting while driving isn’t a bad thing, just that laws against it aren’t terribly effective.)
- You can stop texting if you need to. You can’t stop being drunk. (Common sense.)
This case is weaker, but there’s still something there. It’s not that texting while driving isn’t bad, it’s just that it’s not clear whether laws against it are much use. (And we might just as well have laws against eating, listening to engaging music (or NPR), or having conversations while driving.)
Back to Dead Trees
So, let’s circle around to e-readers vs. printed books:
Evidence in favor of printed books:
- If you get 50 people to read a short story in printed form vs. a Kindle then the people who read on the Kindle are not as good at placing a list of events in the story in the correct order. (Factual correlation.)
- If you get 72 Norwegian 10th-graders to read a text in printed form vs. a PDF then the students who read the PDF scored “significantly better in comprehension tests”. (Factual correlation.)
I should point out that this is an actively researched field, but based on my quick searches on Google Scholar, the more nuanced research is producing much more equivocal results. But the underlying problem remains: the population is still predominantly more familiar with printed books (remember serif typefaces) and the software is still primitive. The points of comparison are flawed and the population samples are biased. And, even then, the pro-book results are pretty weak.
Evidence/Arguments against fall into some basic categories:
- The media being tested aren’t equally familiar. Everyone is familiar with printed books. Even those people familiar with e-readers probably aren’t using their own e-reader with their own favorite settings; it’s very hard to run a fair test. (Correlations are based on bad experiments.)
- In each case, the test is unnecessarily compromised. E.g. a PDF is essentially an electronic representation of a paper document. An ePub is a digital document designed for reading on a screen. It is possible to create PDFs that are not horrible to read on screen, but I doubt this was done, and even the best designed PDF won’t allow the reader to choose their favorite typeface and font size and contrast. A blind person won’t get an accessible copy. (Advantages of electronic media are methodically ignored.)
- The person is being asked to read a specific document, not complete a [realistic/representative] task. Even if the task is “read such and such a book” then, assuming it’s available as an e-book, in the real world the person with the electronic device is already hours or days ahead because they’ll have the book more-or-less instantly. If the task is “find the answer to X” then again the reader with the device will almost certainly find more information, be able to chase up more references, and will likely find more specifically applicable information. (The wrong questions are being asked.)
In the end, most of the arguments against one sees for or against electronic books are nothing to do with the relative quality of the media, e.g.:
“I am still a total paper book lover. It’s just satisfying curling up with a book, the smell of the pages, the heft of the book. And there is the classic “Three B test” – bath, bus, bed.”
If you’ve ever read a big, thick book in bed I doubt you’d consider paper books to have any real advantage over electronic. I personally end up turning over every other page to avoid keeping three pounds of paper suspended in the air. And by the same token it’s much easier to read an electronic book in a bus. As for the bath — get a waterproof case and your point is?
“I was Switzerland in this discussion, but the ebook I was reading told me I was 84% finished with the book when the book ended. The remaining 16% was excerpts from the author’s other books, an author interview, and a discussion guide. Paper books are far superior when it comes to letting you know your place in a book, and that’s why I prefer them.”
That is annoying. Of course, I first encountered this kind of crap in printed books, so I don’t really see how that’s a strike against digital. And that’s without garbage put in by the author him or her-self, e.g. the story part of Return of the King ends something like 75% of the way through; the next 25% of the book is appendices, most of which few would want to read.
It is refreshing to see someone talk about stuff other than whether they prefer the smell of paper to that of a kindle, e.g.:
“For me, it depends on the book—how visual it is (graphic novels I like in paper format), whether I’m more likely to race through it (a good novel) or linger and bounce around (poetry), how big it is (I wish the gigantic Robert Moses book was in eBook form), and how well the text was translated to Kindle (I heard bad things about the Game of Thrones digital versions, so went with paper for that).”
At last, some rationality. I personally would rather buy a graphic novel or a fine art book as paper because ebooks generally have very poor image quality (and even if they don’t it’s hard to tell before you buy them whether this book is the exception) and even if they had great image quality, there’s not really any good way to consume them (until the iPad Pro comes out, perhaps). But, if I knew that the images were going to be in great shape, and I had — say — an iPad that could project books at full resolution onto a 4K display, I would pick electronic books there as well.
It’s also worth noting that in many cases Kindle and other electronic editions will be perversely more expensive than their dead tree counterparts. (That’s why I went with dead trees for Game of Thrones, and it turned out to be a mistake because I lost the damn things.)