How Not to Be Misled by Data

Numbers can deceive just as surely as words—so here’s a guide to avoid being led astray.

By Jordan Ellenberg, WSJ
June 26, 2015 10:16 a.m. ET

A number has a way of ending an argument. What can you say to it? There’s no nuance, no room for interpretation—it is what it is.

Unfortunately, numbers turn out to be a lot like words: powerful and illuminating but capable of being deployed to bad ends. Here’s a little manual of some of the most common ways that data, for all its precision, can take you down a wrong path.

Failure to compare. Last fall, New York Gov. Andrew Cuomo crowed, “The news that our unemployment rate has dropped to its lowest since 2008 is proof that New York is on the move.” What Mr. Cuomo said about the unemployment rate was true: Only 6.2% of New Yorkers were unemployed in September 2014. But he didn’t mention another number: the overall U.S. unemployment rate, which stood at 5.9%—also the lowest since 2008. If New York is on the move, it is moving at the same speed as the country as a whole. A number by itself is often meaningless; it is the comparison between numbers that carries the force. (Gov. Cuomo’s office didn’t respond to a request to comment.)

My favorite example of this lapse came from the blogger Vani Hari (aka “Food Babe”), who warned her air-traveling readers in 2011, “The air that is pumped in [to an airplane cabin] isn’t pure oxygen either, it’s mixed with nitrogen, sometimes almost at 50%.” Almost 50% adulteration sounds terrible—until you remember that the natural proportion of nitrogen in Earth’s atmosphere is 78%. (Yvette d’Entremont wrote about the mistake on Gawker; Ms. Hari has pulled down the offending post.)

(More here.)