August 30th, 2014 (Permalink)
New Book: Standard Deviations
Michael Shermer's latest "Skeptic" column in Scientific American―see Source 2, below―is not a review exactly, but is at least based on a new book by Gary Smith, Standard Deviations: Flawed Assumptions, Tortured Data, and Other Ways to Lie with Statistics.
In the column, Shermer discusses "survivor bias", which I think is the same thing as "survivorship bias", though I'm not certain for a reason I'll explain later. This caught my attention because the first story told in Jordan Ellenberg's new book, How Not to be Wrong, is an illustration of survivorship bias. I've mentioned Ellenberg's book here previously―see Resource 2, below― and you can read the first part via the "Look inside" feature at Amazon, if you're interested.
After the first two paragraphs, Shermer's article is a description of the same statistical phenomenon that Ellenberg discusses. I'll leave it to you to read either Shermer's short account or the longer one given by Ellenberg. However, I'm puzzled by Shermer's first two paragraphs where he tells two similar anecdotes, of which I'll quote the first:
When I purchased my latest vehicle, I was astonished to get the license plate 6NWL485. What are the chances that I would get that particular configuration? Before I received it, the odds were one in 175,760,000. (The number of letters in the alphabet to the power of the number of letters on the plate times the number of digits from 1 to 10 to the power of the number of digits on the plate: 263 × 104.) After the fact, however, the probability is one. This is what Pomona College economist Gary Smith calls the “survivor bias,” which he highlights as one of many statistically related cognitive biases in his deeply insightful book Standard Deviations….
By the way, I've fixed two typos that are in the online version of this article―they're not in the print version―namely, the exponents are not shown in superscript, so that it appears that 263 × 104 = 175,760,000. Not even close.
Anyway, here's what puzzles me: both of the little stories that Shermer tells seem to be related to Stephen Law's "lottery fallacy", which I've discussed in the context of the fine-tuning argument and the multiverse theory―see Resource 1, below. In other words, Shermer claims to have been "astonished" at the license plate he received because it was so unlikely that he would receive that particular one. Of course, he wasn't really astonished, because it would have been just as unlikely for him to have received any other plate. I won't recapitulate the whole argument here, which you can read in the Resource and other sources and resources linked to it.
However, what do these anecdotes have to do with survivorship bias? I suppose that one can stretch the notion of "survivors" to include the numbers that come up in a lottery, but the point of survivorship bias is that we either forget, ignore, or sometimes aren't even aware of those that don't survive. In the case of the lottery fallacy, it's because we're acutely aware of just how many other numbers were not picked that we're "astonished" at our luck.
So, these don't seem to me to be related phenomena, but perhaps Smith explains what's going on here in his book more clearly than Shermer does in his short column.
- Jordan Ellenberg, How Not to be Wrong: The Power of Mathematical Thinking (2014), pp. 3-9
- Michael Shermer, "How the Survivor Bias Distorts Reality", Scientific American, 8/19/2014
August 27th, 2014 (Permalink)
New Edition: Logic Gallery (Third Edition, Enlarged)
David Marans' Logic Gallery is now available in an actual paper edition―imagine that: a whole book printed on paper! Not only that, but your purchase of the paperback edition will benefit the charitable organization Doctors Without Borders. Of course, there's still a free electronic version available. See the Sources, below, for links to both versions.
The Logic Gallery is a chronological collection of pictures of famous logicians, together with short biographical information, and interesting quotes. In fact, I intend to mine it for quotes about logic and fallacies.
Sources: David Marans, Logic Gallery: Ancient Greece to the 21st Century, Third Edition, Enlarged (2014):
Resource: New Book, 8/26/2010
August 22nd, 2014 (Permalink)
A "STUNNING" and "BRILLIANT" Sequel
Here's something I've never noticed before. A newspaper ad for the new movie Sin City: A Dame to Kill For―a sequel to Sin City, of course―has a total of ten blurbs. The ad seems to be trying to impress the reader with the sheer number of quotes.
Currently, the movie has a "rotten" 43% rating on Rotten Tomatoes' "Tomatometer", an average of many reviews. Metacritic's similar "Metascore" is "44", which is described as "Mixed or Average Reviews", but is only a few points above the "Generally Unfavorable Reviews" category. As a result, there were probably few favorable reviews to quote, so where did the ad writer find so many good blurbs?
Three of them are attributed to Debbie Lynn Elias of the Culver City Observer, but I haven't been able to find a review on the newspaper's website, so I can't check them. However, two are credited to Joel Amos of the Movie Fanatic website: "STUNNING." and "JOSH BROLIN IS BRILLIANT." So, the ad writer found so many quotes partly by drawing multiple ones from the same critic. In fact, there are only six different sources for the ten blurbs.
Surprisingly, if you check Amos' review of the movie―see Source 3, below―you'll find that Amos gave the movie only two stars out of a possible five, corresponding to a 40% rating, which is in line with the Rotten Tomatoes and Metacritic averages. Not only that, but the words "stunning" and "brilliant" are nowhere to be found in the review. So, where did those words come from?
I can only speculate, but Amos did use the word "stunning" in a different article―see Source 2, below―which was a short introduction to a "featurette" on the movie, of which he writes:
The visual style is what made Sin City such a marvel and judging by what we’ve seen so far of the second film, it is even more mesmerizing on the eyes. Watching this behind-the-scenes featurette, viewers get a fantastic look at how Rodriguez and Miller [the directors] achieved this stunning look…
Presumably, this was written prior to Amos actually seeing the entire movie, which must have turned out to be not so stunning. But what about the word "brilliant" as applied to Josh Brolin?
I found an interview with Brolin conducted by Amos two years ago for Men in Black 3, the title of which refers to Brolin's "brilliance"―see Source 1, below. Could this really be what the ad is referring to? If so, this is a stunning and brilliant display of blurbing prowess!
- Ad for Sin City: A Dame to Kill For, The New York Times, 8/22/2014, p. C7
- "Movie Marquee Generator"
- Joel D. Amos, Movie Fanatic,
July 31st, 2014 (Permalink)
Check it Out, Too
I have a review in Notre Dame Philosophical Reviews of Emotive Language in Argumentation, a book by philosopher Douglas Walton and Fabrizio Macagno.
Source: Gary Curtis, "Review of Emotive Language in Argumentation", Notre Dame Philosophical Reviews, 7/31/2014
July 30th, 2014 (Permalink)
Fact Check it Out
Mike Rothschild of Skeptoid has fact-checked claims linking Monsanto, flouridation, aspartame, and microwave ovens to the Nazis―see the Source, below. Check it out. I would just add that in addition to not checking out factually, such claims don't check out logically, as Rothschild briefly alludes to at the start of the article:
The lazy shorthand for calling something or someone evil is to compare them to the Nazis. It’s a cheap and easily-understood way of demonizing something you personally don’t like. Call it guilt by association….
Or, call it "the Hitler card", more specifically―see the Fallacy, below. And now I'm going to heat up some flouridated water in my microwave oven and sweeten it with aspartame―after steeping a tea bag in it, of course.
Source: Mike Rothschild, "Stop Comparing Everything You Don’t Like to the Nazis", Skeptoid, 7/29/2014
Fallacy: The Hitler Card
July 26th 2014 (Permalink)
A Prize Puzzle for a Rainy Day
It looks to be a rainy day in Centerville. Dark clouds block the sun and distant thunder rolls. The weather girl on TV gives a 90% chance of precipitation. The following facts are true of those hardy Centervillians who decide to brave the weather:
- 20% of Centervillians who go outside do not wear a hat.
- The same percentage of Centervillians wear only a hat―of course, they wear other clothes, but nothing specifically to ward off rain―as the percentage of those who carry an umbrella and wear galoshes but no hat.
- 25% of Centervillians wear both a hat and galoshes but do not carry an umbrella.
- 35% of Centervillians do not wear galoshes outside.
- The percentage of Centervillians who wear only a hat outside is twice that of the percentage who wear only galoshes.
- 40% of Centervillians who venture outside do not carry an umbrella.
Update (8/23/2014): I should have stated above that no Centervillian is so foolish as to venture outside on a day with such a high chance of precipitation without at least one of the three items of rain gear. Thanks to Louis for pointing out this omission.
What percentage of Centervillians are extremely careful and wear a hat and galoshes as well as carry an umbrella?
If you can answer the above question, please email me the solution. The first person to submit the correct solution will receive a coupon for a free copy of cognitive scientist Luc P. Beaudoin's new book Cognitive Productivity: The Art and Science of Using Knowledge to Become Profoundly Effective, which is an e-book that you can download in a number of formats for different types of reader. Any runners-up will receive an honorable mention on this honorable weblog.
Update (7/27/2014): Congratulations to Cameron Hunter, who was the first person to submit the correct answer!
Acknowledgment: Thanks, again, to Luc P. Beaudoin for supplying the prize for this contest.
July 21st 2014 (Permalink)
Wikipediocracy has a good article on the unreliability of Wikipedia―see the Source, below. There are two points of particular importance made by the article, both of which I've mentioned in previous watches―see the list of watches, below:
- An important concern about Wikipedia is the danger of circular citation―what the article calls "citogenesis":
The process whereby spurious information added to Wikipedia is blindly copied by other publications that are then added to the Wikipedia article as sources, cementing the spurious information in place (after all, it now has a footnote!)….
Theoretically, claims are not supposed to be added to Wikipedia entries without a citation to an external source, but of course in reality it happens. Subsequently, many external sources will copy the unsourced claim from Wikipedia. Now, the originally unsourced claim can be cited to an external source!
- I've argued previously that Wikipedia should not be used as an encyclopedia, but as a research guide similar to a search engine. For this reason, you should never cite Wikipedia as a source for information for the same reason that you would not cite Bing or Google. Wikipedia is often a better way to find information on a topic than a search engine since it's organized by people rather than an algorithm, but it lacks the reliability of a proper encyclopedia:
This is Wikipedia in a nutshell: genuine research mixed with completely unreliable information in such a way that looking at any Wikipedia article the reader never knows what is correct and what is made up.
The article also mentions Charles Seife's new book―see the previous entry. Check it out.
Source: Andreas Kolbe, "How pranks, hoaxes and manipulation undermine the reliability of Wikipedia", Wikipediocracy, 7/20/2014
July 19th 2014 (Permalink)
New Book: Virtual Unreality
Charles Seife, author of Proofiness, has a new book entitled Virtual Unreality: Just Because the Internet Told You, How Do You Know It’s True?. I guess the subtitle pretty much tells you what it's about.
Source: Clara Moskowitz, "Book Review: Virtual Unreality", Scientific American, 6/1/2014. Added 7/25/2014: I originally forgot to include a link to this short book review, which was my source for the entry.
July 3rd 2014 (Permalink)
Strike Three…You're Out!
The final strike against the Slate article discussed in the two previous entries is its tabloid-style headline:
Do Americans Think Corporations Have the Right to Religious Freedom?
We did a survey, and the answer is no.
As explained in the previous entries, this was not shown by the survey. Putting aside the problems with how the poll's sample was selected and its size, the most it showed is that Americans think that the right to religious freedom of individuals is more important than that of corporations, not that the latter have no such right at all. The article itself, while occasionally exaggerating the poll's results, was more accurately written―see the quote in the first entry for this month, below. The language used in the body of the article is carefully and consistently comparative, since the most that the poll shows is greater support for the religious freedom of individuals than of groups.
Many people are likely to read such a headline and not bother with the article itself, thus getting a false impression about what it shows. Even those who read the entire article may be misled into misinterpreting it along the lines suggested by the headline. Perhaps the headline writer misunderstood the poll results, or the editors at Slate are willing to write false but provocative headlines in order to trick people into reading their articles.
Update (7/4/2014): I've revised the second paragraph to remove a silly, unintended implication.
July 2nd 2014 (Permalink)
The same Slate article discussed in yesterday's entry also provides a good test of your understanding of the pitfalls of charts and graphs. Look at the chart below, which accompanies the article and reports the poll results. Using the terminology given in the lessons on charts and graphs from last year―see the links, below―can you identify what type of chart this is? See below the chart for the answer.
This is a gee-whiz line graph―see Lesson 1, below. The y-axis doesn't start at 1, which is the lowest number in the scale, thus visually exaggerating the differences between the results for people and businesses. Worse, the graph does not have a broken y-axis near the x-axis, which is standard practice in chartmaking when cutting off the bottom of a graph. One might excuse the truncation, but not failing to include a visual sign of it, for that is charting malpractice.
As I pointed out in the previous entry, it's not at all clear what the numbers are supposed to mean in this survey. Someone just glancing at the graph might get the false impression that, because the line for groups is close to the bottom of the chart, that must mean that most people do not believe that groups deserve "religious liberty rights". However, all of the results but one are at or above 4 on a scale of 1 to 7, which might just as well indicate differing degrees of support for such rights.
July 1st 2014 (Permalink)
A current article in Slate is a good test of your understanding of the pitfalls of polling and its reporting. See if you can figure out what, if anything, is wrong with this poll before I discuss it. Here's a brief synopsis of the article, though I suggest reading the whole thing, which isn't long―see the Source linked below:
The Supreme Court ruled 5–4 on Monday that Hobby Lobby, a family-owned hardware retail chain, has a right to religious freedom. … What does the American public think about religious rights for corporations? Over the past few months, we’ve conducted a series of surveys, delivered online to more than 300 people across the country between the ages of 18 and 76. … To understand how people think about corporate rights, we asked how important they thought the religious liberty right of a company’s owner, its employees, and the company itself, on a scale of 1 to 7. … Our respondents consistently agree that religious liberty is important for both individual employees and owners. In contrast, respondents were significantly and consistently less willing to grant the same scope of protection to all the for-profit companies we presented them. …[P]eople did distinguish between for-profit and non-profit organizations, deeming the latter to be more deserving of religious liberty rights. But even when the religious liberty of a church is at stake, people viewed it as much less deserving of religious rights than its employees and officers.
There are several problems with this poll or the article that reports it:
- This was an online poll, and most such polls use self-selected samples, that is, they are not "scientific" polls that use random samples―for more on the distinction between self-selected and scientific surveys, see the Resource below. The article itself doesn't explain how the sample was taken, nor can I find a link to a longer report on the poll that would do so. In the absence of some assurance that the sample was taken randomly, I think we have to assume that it was not.
- There is no mention in the article of a margin of error (MoE) for the poll, which violates the usual American journalistic standard to report the MoE. Since MoEs can only be calculated for surveys based on random samples, this is further evidence that the sample was probably self-selected. However, even if the sample was taken randomly, 300 respondents is a small sample for a public opinion poll, which typically will have one more than three times as large. Assuming a random sample and a confidence level of 95%, the MoE would be around five and a half percentage points.
- Unfortunately, even assuming a random sample, it's impossible to use the MoE to evaluate the poll since none of the results are reported as percentages. Instead, the results are numbers on a scale from 1 to 7, which are presumably averages of the answers given by respondents. However, as far as we can tell, it may be that none of the differences reported by the poll are statistically significant. See the Resource, below, for more on MoE errors.
- Finally, the poll asked respondents to rate the religious rights of people and companies on a numerical scale, but what do the numbers mean? Presumably, a "1" would mean that respondents think that the person or company has no rights, and a "7" would mean that they do, but what does a "4" or "5" mean? Did the survey itself explain the meanings of the numbers? If so, then the article is poorly reported.
In sum, there's no reason to believe the sample used in this survey is representative of the American public, and no evidence that the differences in results are statistically significant. So, as far as we can tell from this article, the poll is worthless.
Source: Moran Cerf, Aziz Huq & Avital Mentovich, "Do Americans Think Corporations Have the Right to Religious Freedom?", Slate, 7/1/2014
Resource: How to Read a Poll
Online dissertation services such as MastersThesisWriting may help with your thesis or dissertation.