WEBLOG
March 4th, 2014 (Permalink)
New Book: The Improbability Principle
Statistician David Hand has a new book out called The Improbability Principle: Why Coincidences, Miracles, and Rare Events Happen Every Day. Coincidences are important because they are the stuff of conspiracy theories and contribute to people's belief in psychic powers. People usually greatly underestimate the likelihood of such events―for an example, see Hand's discussion of the birthday paradox in Source 3, below. As a result, we often jump to causal conclusions based on chance occurrences. An example that I've discussed previously―see the Resources, below―is the argument that the so-called "fine-tuning" of the physical constants of the universe requires a causal explanation. I've no idea how Hand comes down on this issue, but I see from the index of the book―which you can see with Amazon's "Look Inside!" feature―that he discusses it, and I'm curious what he thinks.
Sources:
- David J. Hand, "It’s Not Actually a Miracle", Slate, 2/12/2014
- David J. Hand, The Improbability Principle: Why Coincidences, Miracles, and Rare Events Happen Every Day (2014)
- David J. Hand, "Math Explains Likely Long Shots, Miracles and Winning the Lottery [Excerpt]", Scientific American, 2/1/2014
Resources:
- The Arguments that Failed, 3: The "Fine-Tuning" Argument, 1/20/2009
- New Book: The Fallacy of Fine-Tuning, 6/19/2011
- Fine-Tuning the Fine-Tuning Argument, 6/27/2011
- One Universe is Enough!, 8/7/2011
March 1st, 2014 (Permalink)
Bad News About the Good News
There's some good news and some bad news. The good news is found in the following headline from The New York Times:
Obesity Rate for Young Children Plummets 43% in a Decade
Unfortunately, as too often happens, the bad news is that the good news is wrong. The headline is from a report on a CDC study of obesity rates―see Source 4, below. The bad news comes from an article by Razib Khan in Slate―see Source 2, below, and read the whole thing, it's good!
There are two main problems with the headline and its accompanying article:
- Absolute vs. Relative Measures: Here's the relevant passage from the Times report:
Federal health authorities on Tuesday reported a 43 percent drop in the obesity rate among 2- to 5-year-old children over the past decade, the first broad decline in an epidemic that often leads to lifelong struggles with weight and higher risks for cancer, heart disease and stroke. … About 8 percent of 2- to 5-year-olds were obese in 2012, down from 14 percent in 2004.
This is a confusing issue because we're dealing with percentages of percentages as well as with percentage points, which are actually an absolute measure of the difference between percentages. That is, the "plummet" from 2004 to 2012 was six percentage points, and six is about 43% of fourteen, so the drop of six percentage points is also a drop of 43% of 14%. Thus, six percentage points and 43% are ways of measuring the same change. If you're still confused, see Source 3, below. In any case, the reporter chose to emphasize the larger, more impressive number.
- Multiple Comparisons: The more important problem is that the researchers were not just looking at the obesity rate for 2-to-5-year-olds, but for different age groups―how many I do not know. As explained in the entry for the multiple comparisons fallacy, when many comparisons are made it is possible that a statistically significant result may be found just by chance. As a result, it's necessary to use a lower threshold for statistical significance in order to avoid spurious results. According to Khan, the researchers reported in the study itself not making any allowances for multiple comparisons, but the Times reporter fails to mention this. So, the six percentage point drop may well have been nothing but a chance result.
Sources:
- "Kids Eat Free Live Clown", Funny Signs
- Razib Khan, "The Obesity Rate for Children Has Not Plummeted", Slate, 2/28/2014
- Doctor Peterson, "Clarifying Percentages vs. Percentage Points", The Math Forum at Drexel, 11/21/2003
- Sabrina Tavernise, "Obesity Rate for Young Children Plummets 43% in a Decade", The New York Times, 2/25/2014
Resource: Check it Out, 3/31/2012
Fallacy: The Multiple Comparisons Fallacy
February 19th, 2014 (Permalink)
Another Doublespeak Dictionary Puzzle
What does "orthographically impaired" mean, as in the sentence: "The headline writers at Slate are orthographically impaired"?
Source: Hugh Rawson, Dictionary of Euphemisms and Other Doubletalk, Revised Edition (1995)
February 17th, 2014 (Permalink)
What is a P-value?
I have some comments on a recent article in Nature by Regina Nuzzo concerning statistical problems in scientific research. I've discussed many of these problems here in recent years, and they concern some of the probabilistic fallacies in the files, which I will mention by name later. The article is well worth reading and neither very long nor very technical―see the Source, below―so I recommend reading it before my comments.
I want to add some additional explanation to the article, as well as to tie it in explicitly with the fallacies and problems I've discussed here before. Here's how the article explains what a P-value is:
…[W]hen UK statistician Ronald Fisher introduced the P value in the 1920s, he did not mean it to be a definitive test. He intended it simply as an informal way to judge whether evidence was significant in the old-fashioned sense: worthy of a second look. The idea was to run an experiment, then see if the results were consistent with what random chance might produce. … This probability was the P value. … For all the P value's apparent precision, Fisher intended it to be just one part of a fluid, non-numerical process that blended data and background knowledge to lead to scientific conclusions.
In other words, the P-value―"P" for "probability"―just tells you the probability that the result of a study is due to chance. Nuzzo continues:
…[There] is an abundance of confusion about what the P value means. … Most scientists would look at … [a] P value of 0.01 and say that there was just a 1% chance of … [the] result being a false alarm. But they would be wrong. The P value cannot say this.
That is, the P-value is not the probability that the experimental hypothesis is false. If a study has a low P-value, that means its results are unlikely to be due to chance. However, the P-value does not tell you what other than chance may be responsible for those results. Typically, scientists begin research with an hypothesis that they wish to test, and if the test has a low P-value they may conclude that the hypothesis is true. However, the P-value is not itself a measure of the unlikelihood of the hypothesis, nor is the probability that the hypothesis is true simply 1 minus the P-value.
[The P value] cannot work backwards and make statements about the underlying reality. That requires another piece of information: the odds that a real effect was there in the first place.
What Nuzzo is alluding to here is the so-called "prior probability" of the experimental hypothesis, which is one of the values one needs in order to use Bayes' theorem to determine the "posterior probability" of that hypothesis, which is what we want to know―for Bayes' theorem, see the entry for Probabilistic Fallacy.
To ignore this would be like waking up with a headache and concluding that you have a rare brain tumour―possible, but so unlikely that it requires a lot more evidence to supersede an everyday explanation such as an allergic reaction.
This is an example of the base rate fallacy, that is, the mistake of neglecting to take into consideration the prior probability of a hypothesis. The "base rate" of brain tumors is much lower―they're "rare"―than the prior probability of allergic reactions, so that the headache is more likely to be due to an allergy than to a tumor.
The more implausible the hypothesis―telepathy, aliens, homeopathy―the greater the chance that an exciting finding is a false alarm, no matter what the P value is.
This is related to Carl Sagan's slogan that "extraordinary claims require extraordinary evidence", and both are consequences of Bayes' theorem: the lower the prior probability of a hypothesis, the stronger the evidence needed to raise the posterior probability to a level worth taking seriously.
Critics also bemoan the way that P values can encourage muddled thinking. A prime example is their tendency to deflect attention from the actual size of an effect. … To pounce on tiny P values and ignore the larger question is to fall prey to the “seductive certainty of significance”, says Geoff Cumming, an emeritus psychologist at La Trobe University in Melbourne, Australia. But significance is no indicator of practical relevance, he says: “We should be asking, 'How much of an effect is there?', not 'Is there an effect?'”
This is a different type of problem than those discussed previously by Nuzzo, each of which was related to understanding probability, especially in relationship to Bayes' theorem. She is referring to the danger of equivocating on the meaning of the word "significant" in the phrase "statistically significant".
To say that the result of a scientific study is "statistically significant" simply means that the P-value is less than or equal to a somewhat arbitrary "level of significance", which is usually set at .05. So, a statistically significant result is just one that is unlikely to have happened by chance; specifically, it usually means that there is less than or equal to one chance in twenty. This says nothing at all about whether the result is "important" or "significant" in any other sense.
However, this is not a "black or white" issue: There's no reason why we can't ask and answer both of Cumming's questions, and attend to both the statistical "significance" and the practical importance of scientific results.
Source: Regina Nuzzo, "Scientific Method: Statistical Errors", Nature, 2/12/2014.
Resource: Steven Novella, "P-Hacking and Other Statistical Sins", NeuroLogica, 2/13/2014.
Fallacies:
February 14th, 2014 (Permalink)
Charts & Graphs:
A Not-So-Scary Parallel
If you followed along in the "Charts & Graphs" series from last year, you ought to be able to see what's wrong with the chart above. If you didn't, what're you waiting for?―see the Lessons, below.
The chart shows two lines representing the Dow Jones Industrial Average (DJIA): the black line showing the DJIA from 1928-1929―that is, the period leading up to and including the stock market crash of 1929; and the red line representing the DJIA in the last year and a half. As suggested by the chart's scary title "Scary Parallel", you're supposed to notice the similarity between the two lines up to the end of the red line, and then extrapolate the red line into the future following the lead of the black line. In other words, the chart represents a visual argument that we're currently heading for a stock market crash similar to the one shown by the black line's Black Tuesday.
Can you explain why you shouldn't be so scared by this scary parallel? If you think you can, click on the link below.
Why is the parallel not so scary?
Lessons in Charts & Graphs: 1, 2, 3, 4, 5, 6, 7, 8, 9
February 12th, 2014 (Permalink)
New Book: The Critical Thinker's Dictionary
Philosopher Robert Todd Carroll of Skeptic's Dictionary fame has now produced a Critical Thinker's Dictionary. Judging from the Table of Contents, it covers many of the classic fallacies together with a few less familiar ones, such as the regressive and Texas sharpshooter fallacies. There are also entries for most of the cognitive biases and illusions discovered by psychologists in recent decades, which is unusual for a book on critical thinking but a welcome development. Unfortunately, it appears to be available only as a Kindle book, but the price is certainly right!
Source: Robert Todd Carroll, The Critical Thinker's Dictionary: Biases, Fallacies, and Illusions and What You Can Do About Them (2013)
February 11th, 2014 (Permalink)
Name That Fallacy!
Your argument must be coherent if I'm to understand it. Because I can't understand it, it must be incoherent.
Can you name the fallacy committed in the above passage? If you think you can, click the link below.
Source: "Ham Sandwich Town", Really Funny Pictures
February 9th, 2014 (Permalink)
Headlines
The Gayest Olympics Ever
Decoding the Anti-Gay Messages in the Olympics Opening Ceremony
Next on Slate: Have your cake. Eat it, too!
Sources:
- Simon Doonan, "The Gayest Olympics Ever", Slate, 2/7/2014
- June Thomas, "Decoding the Anti-Gay Messages in the Olympics Opening Ceremony", Slate, 2/7/2014