Book Review: Damned Lies and Statistics

Title: Damned Lies and Statistics

Subtitle: Untangling Numbers from the Media, Politicians, and Activists

Author: Joel Best

Date of Publication: 2001

Quote…

This book offers some guidelines for thinking critically about social statistics. It identifies some common problems with social statistics and illustrates them with specific examples. It is often easier to understand a particular example than to understand and recognize the general problem or principle that the example illustrates. Still, I hope that, having read this book, you [will] become more familiar with some of the most common flaws that bedevil social statistics; that you can ask some basic questions about a statistic's origins (definition, measurement, sampling, and the other issues covered in chapter 2); that you are familiar with some of the ways that statistics can be mangled (chapter 3); that you understand the risks of inappropriate comparisons (chapter 4); and that you can do more than simply throw up your hands when confronted with a debate featuring competing statistics (chapter 5). (Pp. 161-162)

…Unquote

Joel Best is a sociologist and, as a result, this is not a book about the mathematics of statistics, but about its sociology. That is, it's about the ways in which bad statistics are generated and spread through society.

People tend to accept statistics as facts, but all statistics are created by people, and many of those people have agendas. Social statistics―statistics about social problems, such as prostitution or suicide―are often produced by activists who are concerned about the problem, and may exaggerate it. When not produced by activists, statistics are often a product of government, which may be motivated in the opposite direction of the activists, namely, to play down a problem.

Given that statistics are created by people, there are three questions that should be asked about them, according to Best:

1. Who created the statistic? (P. 27)
2. Why was the statistic created? (P. 28)
3. How was the statistic created?

Answering questions 1 and 2 may give reasons to doubt the statistic's accuracy, if the source has a motive for exaggerating or downplaying it (1), or if it was created to advance a particular cause or to sell a product (2). Also, if you can answer the first two questions, you may be able to allow for the bias of a statistic. In contrast, learning that the source was unbiased should increase your confidence in the statistic.

However, most of the book concerns question 3, that is, the way in which flawed statistics are produced. There are four main ways that people come up with bad statistics, according to Best (Ch. 2):

1. Guessing (p. 32): Guessing about social statistics is more common than you might think. Social problems that have been ignored or are hard to count―such as rape, prostitution, homosexuality, AIDS, etc.―have a "dark figure", that is, a number of cases that are not counted for various reasons, including embarrassment or fear of arrest. So, when activists or bureaucrats are called upon by reporters to estimate the size of a problem, they may have to guess.
2. Bad Definitions (p. 39): Most social problems are vague, for instance, child abuse: non-abusive treatment of children gradually shades over into abusive beatings or neglect. In order to count instances of child abuse, decisions must be made about whether to include or exclude borderline cases. Those who wish to call attention to the problem are motivated to include borderline cases, while those who want to ignore it will exclude those same cases. Both sides may claim to be counting incidents of "child abuse", though they come up with different numbers.

   Also, activists tend to pick extreme examples to publicize a problem (p. 56). Such examples are selected because they are not typical of the problem, yet they may mislead people into committing the Anecdotal Fallacy. For example, in order to call attention to the problem of child abuse, activists may point to a case of murder, though neglect is a more typical form of abuse.

3. Flawed Measurement (pp. 45-52): Flawed measurement can come about through badly-worded survey questions, for instance. However, if I have one criticism of this book, it is that Best's main illustration of the problem of flawed measurement is "poverty". While there are doubtlessly measurement problems involved in counting the poor, Best's discussion centers on a definition problem, and thus belongs under the previous heading. "Poverty" is a vague concept, which leads to all of the problems mentioned above of counting the number of cases of vague concepts, and of contrary definitions used by those with competing interests, so well explained by Best in the rest of the book.
4. Weak Sampling (p. 52): Poor sampling takes two forms:
   1. Samples that are too small: See the entry for Hasty Generalization.
   2. Unrepresentative Samples: See the entry of the same name.

   Another all too common source of weak samples is what's known as "convenience sampling" (p. 55): a convenience sample is a nonrandom, "unscientific" sample that is drawn in whatever way is most convenient. Such samples are cheap and easy, which is what makes them "convenient", and also what makes them attractive to the media or interest groups. However, the most convenient sample is likely to be unrepresentative of the larger population. When you read that a poll is "scientific" that means that it was conducted using random sampling, as opposed to convenience sampling.

Even when all of these pitfalls have been avoided and good statistics produced, there are still what Best calls "mutant statistics". These are numbers that have been reported and passed from person to person, mutating in the process, as in the game of "telephone". Such mutations can come about through incorrect generalizations, misinterpretation, or misunderstanding. For example, an estimate that 150,000 American women had anorexia mutated into the claim that that many women died of anorexia each year (pp. 63-64).

Finally, there is the "apples to oranges" comparison. One of the most common such comparisons is the comparing of prices at different times. Because of inflation, money tends to lose value over time, which means that comparing the price of, say, a gallon of gasoline today with one ten years ago is an "apples to oranges" comparison. To compare apples to apples or oranges to oranges, monetary comparisons need to take inflation into account.

"Stat wars" occur when competing interests advance contrary statistics for a problem and attack each others numbers. It's hard to untangle the statistics bandied about in the press by activists and politicians during such wars, because the media often just report the different numbers without attempting to figure out which are right, leaving most of their readers in the dark (p. 137). Of course, if different sides are using contrary definitions of a problem, then each side's numbers may be right; but this won't be obvious if the different definitions are not reported, as is often the case. Luckily, in the years since Best's book was published, there are a number of new media outlets that look critically at statistics, such as The Wall Street Journal's "Numbers Guy", and Annenberg Political Fact Check.

The final chapter (ch. 6) discusses four mindsets towards social statistics:

1. Awestruck (p. 162): Treating statistics as unchallengable facts.
2. Naive (p. 162): This is where most of us are.
3. Cynical (p. 164): The cynical treat all statistics as "damned lies" and reject them without due consideration. Also, from the ranks of the cynics come some of the worst statistical abuses.
4. Critical (p. 166): A critical mindset does not swallow all statistics whole―as the awestruck do―nor does it reject them all without thought―as the cynics do.

These same mindsets apply more generally to argumentation. One of the dangers of a book such as this is that learning about misleading statistics may lead to a cynical dismissal of all statistics. Similarly, learning about logical fallacies may lead to the rejection of all argumentation as untrustworthy. Like Best, I hope that my readers will not turn into cynics but into critics.

The title of this book is, of course, taken from the cynical phrase "lies, damned lies, and statistics", which is usually interpreted as grouping statistics with lies. Perhaps a more apt, more critical title would come from the phrase: "figures don't lie, but liars can figure". Despite its cynical title, Best's book is one of the best ways to learn how to cease being awestruck by statistics and to start critically evaluating them.

Book Review: The Secret

Title: The Secret

Author: Rhonda Byrne

Publisher: Atria Books

Date of Publication: 2006

Quote…

I never studied science or physics at school, and yet when I read complex books on quantum physics I understood them perfectly because I wanted to understand them. The study of quantum physics helped me to have a deeper understanding of The Secret, on an energetic level. (P. 156)

…Unquote

This book is the latest inductee to the Fallacy Files Shelf of Shame, which is a collection of books based entirely on logical fallacies. Other volumes on the shelf: The Abortion Holocaust, Comet of Nostradamus: August, 2004―Impact!, and Hitler: Neither Vegetarian nor Animal Lover.

So, what is the big secret? Actually, even the book itself as much as admits that it's no secret, since the idea has been around for a long time. So that you don't have to waste your time reading the book to find out, here's the so-called secret:

The Universe obeys what's called the "Law of Attraction". No, it's not gravity, nor is it magnetism, though the book likes to use these forces as metaphors and blurs the distinction between them. Rather the "Law of Attraction" is the notion that good thoughts attract good things and bad thoughts attract bad things. So, if you want a ham sandwich, all that you have to do is think that you have a ham sandwich, and the Universe will so arrange itself that you will have a ham sandwich, without having to do anything tiresome like go to the fridge. In other words, the Law of Attraction is a pseudoscientific euphemism for wishful thinking.

The Law of Attraction is just a version of the old idea of "positive thinking". Think happy thoughts and happy things will happen to you; think sad thoughts and the Universe will give you something to be sad about. However, there's one catch to the Law: you have to be careful what you wish for because the Universe is an idiot. The Universe will grant your every whim, but it doesn't understand negation. So, if you wish: "I don't want basal cell carcinoma", the stupid Universe understands you to say: "I want basal cell carcinoma", and gives it to you. So, the proper way to engage in wishful thinking is never to use a negative. "I want clear, healthy skin", is what you should wish for.

Why doesn't the Universe understand negation? The Law of Attraction is that positive thoughts attract positive things and negative thoughts attract negative things, but the word "negative" is ambiguous because it can mean either of two things:

1. Bad or harmful. Antonym: Positive
2. Having to do with logical negation. Antonym: Affirmative

The Universe, according to Byrne, can't understand the difference between these two ideas, though I suspect that the real reason is that Byrne herself doesn't understand it. Of course, if it weren't for this claim, the Law of Attraction would be even more obviously false than it already is. Since there's no evidence given in the book to believe that the Universe is so stupid as not to understand negation, the only explanation I can think of for this strange claim is that it's an ad hoc hypothesis adopted to save the theory from refutation. After all, anyone who isn't delusional will look around them and immediately realize that the world does not automatically conform itself to our whims; that the Universe misunderstands our wishes and keeps sending us things that we don't want helps explain wars, famines, and pestilences.

Another odd thing about The Secret is how concerned it is with making money. If you could get things just by wishing for them, what would you need money for? Here's Byrne's startling explanation of the spiritual significance of wealth:

If you have been brought up to believe that being wealthy is not spiritual, then I highly recommend you read The Millionaires of the Bible Series by Catherine Ponder. In these glorious books you will discover that Abraham, Isaac, Jacob, Joseph, Moses, and Jesus were not only prosperity teachers, but also millionaires themselves, with more affluent lifestyles than many present-day millionaires could conceive of. (P. 109)

I didn't make that up. I have my doubts about Jesus, but I have no doubt that Rhonda Byrne has managed to make herself a millionaire with The Secret.

What evidence does the book give for its dubious claims? Not much, but what there is falls into two categories:

1. Appeals to authority: The book lists various great men of history who supposedly knew The Secret:

   The greatest teachers who have ever lived have told us that the law of attraction is the most powerful law in the Universe. … Great thinkers including Socrates, Plato, Ralph Waldo Emerson, Pythagoras, Sir Francis Bacon, Sir Isaac Newton, Johann Wolfgang von Goethe, and Victor Hugo shared it in their writings and teachings. (P. 4)

   The only actual evidence given that any of these men wrote about or taught The Secret is a supposed Emerson quote: "The secret is the answer to all that has been, all that is, and all that will be." (P. 183) What secret? Assuming that this is a genuine quote, how do we know that he was talking about The Secret? The book gives no citation, so it's virtually impossible to actually check whether Emerson ever said this. I did a Google book search, but couldn't find it. Since I'm unwilling to give Byrne the benefit of the doubt, I consider the quote bogus until proven genuine.

   I'm not an expert on Emerson, but I know quite a lot about Plato, and there's nothing in his philosophy that bears the remotest resemblance to The Secret. Maybe he knew of it but kept it secret! How, then, did Rhonda Byrne find that out?

   Other authorities appealed to are the 24 "co-authors" who are extensively quoted in the book and interviewed in the companion video. Why should we give them any credence? Among them is a chiropractor, a feng shui master, and a "life adventurer". A few claim to be philosophers, but I've never heard of any of them before, and none of them list any academic credentials in philosophy in the biographies in the back of the book. So, they appear to be "philosophers" only in the sense in which anyone can make that claim.

   One of the best known contributors to the book is John Gray, the author of Men are from Mars, Women are from Venus. Gray has a Ph.D. in psychology, but it's from the unaccredited Columbia Pacific University which was shut down by the state of California. So, he has a doctorate in psychology in about the same sense that the Scarecrow had a doctorate in Thinkology. He's also a signatory of the so-called "9/11 Truth Statement", which tells you something about both Gray and the statement.

   The main thing that these people have in common is that most of them have books, programs, or seminars that they want to sell you. Now that you know The Secret I suggest that you just wish for them rather than forking over any cash.

   So, the appeal to authority is fallacious because the so-called "authorities" appealed to are not experts on anything related to the subject matter, or anything at all in many cases.

2. Quantum Quackery: The only other source of evidence for The Secret is the claim that the Law of Attraction is somehow based on quantum mechanics, and is therefore scientific. A couple of the contributors to the book are actual physicists. The best known one is John Hagelin, who got very few votes running three times for President of the United States on the Natural Law Party ticket, and currently is associated with the Maharishi University of Management. Both of these ventures are offshoots of the Maharishi Mahesh Yogi's transcendental meditation movement. In other words, Hagelin is an eccentric, to put it mildly.

   Quantum Mechanics (QM) is a highly mathematical theory which is extremely difficult to understand and, therefore, is a favorite of pseudoscientists. All that they have to do is claim that QM supports their snake oil, in that way making the snake oil sound scientific, and scarcely anyone in their audience will be able to tell otherwise.

   The Secret is simply the latest attempt to use QM to sell new age snake oil. I'm not a physicist, so I will simply let you consult the two articles on quantum quackery listed in the Sources below.

I've indulged in a lot of negative thoughts about The Secret, but I'll leave you with a couple of positive ones. It's beautifully designed and printed. Also, it's probably the most unintentionally funny book I've ever read.

Sources:

• Michael Shermer, "Quantum Quackery", Scientific American, 1/2005
• Victor J. Stenger, "Quantum Quackery", Skeptical Inquirer, January/February 2007

Reader Response: Lloyd Herring writes to raise three questions about this review, so I will address each one separately:

1. I have some questions about the review of The Secret. Is there a fallacy during the discussion on the two views of negation? You said something to the effect that the real reason that the Universe didn't understand the two views of negation was that Byrne herself didn't understand the two views. This may be true, but I am wondering if this is an attack against Byrne herself, in other word, an ad hominem.

   It is an ad hominem―or, in this case, ad feminam―attack, but not a fallacious one. Not every criticism of a person commits a logical fallacy, only those that are logically irrelevant. In this case, Byrne's claim is obviously wrong to anyone who is logically literate. I'm not arguing against her silly views on the grounds that she is ignorant; rather, I am speculating that she is ignorant, because ignorance would explain why she adopts such silly views. Another explanation is that she does understand the difference between logical negation and "negativity", but thinks that her readers will not and wishes to mislead them. Ignorance and ignominy are the only explanations that I can think of for this absurd idea.

2. I would also like to ask if there is a fallacy where you try to explain why the author thought the Universe is stupid. You say: "Since there's no evidence given in the book to believe that the Universe is so stupid as not to understand negation, the only explanation I can think of for this strange claim is that it's an ad hoc hypothesis adopted to save the theory from refutation." I am wondering if this is the fallacy of ignorance. I don't mean that you're ignorant. Instead, could this be a fallacy in the sense that just because you can't think of an explanation doesn't mean an explanation doesn't exist. However, I'm not sure if I am applying the fallacy correctly.

   It's possible that there is some reason for Byrne's claim that isn't given in the book or accompanying video, but the burden is on her to provide the evidence. In this case, given her failure to provide such evidence it's reasonable to conclude that she lacks any. This is a nonfallacious type of appeal to ignorance, such as the conclusion that there is no Loch Ness monster given the lack of good evidence for its existence. See the Exposure section of the entry for the appeal to ignorance for more explanation and examples.

3. I am also wondering if there is a fallacy in the manner of your characterization of John Hagelin. You say Hagelin is associated with the Maharishi Mahesh Yogi of Transcendental Meditation fame through the Maharishi University of Management. You say that this shows that Hagelin is "an eccentric." Is this an example of guilt by association? Again, I am not sure if I am applying the fallacy correctly.

   This is another nonfallacious personal attack. In the book and video, Byrne trots out Hagelin as part of an appeal to authority. Since Hagelin is a physicist we are supposed to be convinced that there is something scientific about The Secret. As I explain in the Exposition section of the entry for the appeal to misleading authority, it's almost always possible to find eccentric scientists who take any position you like. For instance, there's an eccentric physicist who claims that the government is covering up a crashed flying saucer. An appeal to such authorities is worthless as evidence: it's not any different than going from doctor to doctor until you find one who will tell you what you want to hear. My point about Hagelin's eccentricity is that he represents a minority opinion among physicists, and that most would tell you that The Secret is pseudoscientific nonsense. Specifically, its references to QM are quantum quackery.

Please don't get me wrong about all these questions about the criticism of the Byrne book. I am not a fan of this book or its type. I just want to make sure and get the criticism right so supporters of this book have no ammunition for their responding criticisms.

Fair enough.