Punked by Mother Nature

By: Aaron Datesman

While I have no doubt that the Fukushima disaster will result in the deaths of Americans (read what Arnie Gunderson has to say about "hot particles" here), I'm glad to say that the Sherman and Mangano report of excess infant deaths in the Pacific Northwest appears to be incorrect. If you like, you can read refutations of the claim from Scientific American here and from the group Nuclear Power? Yes Please! here. (My thanks to a commenter on this post last week for pointing out this controversy, of which I was not aware.)

I think credit is due to Sherman and Mangano for poking around in a corner where nobody else bothers to look, under the assumption that there is nothing there worth finding. What's more, while their result is (probably) incorrect, Sherman and Mangano actually stumbled across something rather interesting. At least, I find it rather interesting, since it's a good illustration of how nature can be a right cuss, and doesn't care one whit about our assumptions.

In my opinion, it's also correct to take a slap at the critics, Michael Moyer at Scientific American, and especially Dr. Mattias Lantz of NP?YP!, who should know better. It's one thing to perform a piece of original research which turns out to be incorrect because Mother Nature is a punk, which you'll see is what happened to Sherman and Mangano. It's another thing to critique original research done by somebody else, with all of the heavy lifting of creative thinking and data identification already completed, and fail absolutely to improve the piece of science under examination.

In short, while Moyer and Lantz most likely got the correct answer, and there has so far been no increased infant mortality in the U.S. due to Fukushima, the analysis each of them uses to arrive at that conclusion is no stronger than the analysis they criticize. Moyer's article especially led me to wonder, "Was this written by an eighth grader?" On the other hand, having been a physics teacher, I'm highly touchy about idiotic uses of the Excel "Add Trendline" function.

To explain how Sherman and Mangano were tricked by Mother Nature, I want to return to the table of infant mortality statistics in the area surrounding Three Mile Island I examined in this post. Look first at the rightmost column, "State Average", which shows that the infant mortality rate in Pennsylvania in the late 1970's rose and fell seasonally with a variation of 13% (from a low of 12.3 during the summer to a high of 14.1 in winter and spring). I would characterize this data set as being well-behaved. It turns out that the expectation that similar data sets from other times and other locations will not behave like statistical demon spawn is the root of a rather interesting problem.

The data in the next column, "Excluding Harrisburg", shows much more variation compared between years in one season; however, again averaging the three years within each season, summer once more exhibits the lowest level of mortality. In this case, the variation is 24% (9.3 to 12.3) rather than 13%. Looking at the three-year averages, this set of data is also reasonably well-behaved, but certainly less so than the state-level data. The smaller sample size (infant mortality among a population of tens of thousands rather than millions) no doubt contributes to the variability to some extent.

Until I read the critiques of Sherman and Mangano's article and began to work on the Pacific Northwest data myself, the chart above was pretty much all I knew about temporal (in time) variations in infant mortality. Intuitively, it seems very reasonable that infant mortality should vary seasonally; however, I would find it rather surprising if the rate of infant mortality varied dramatically over the course of a year.

If this is true - if the infant mortality rate (measured across a population of millions) varies only within 10% or so - then Sherman and Mangano's analysis method was appropriate, and their result should be considered correct. Therefore, I set about to see whether there are significant temporal trends in the infant mortality data for which they (and certainly, their critics) should have tested. I will flesh this out further in a subsequent post, but here is the answer:

It varies by more than a factor of two. Mother Nature can really be a punk sometimes.

— Aaron Datesman