New ESP Study

The New York Times has a story about a new study by Daryl J. Bem of Cornell University that makes a case for ESP. Of course one needs to consider it carefully–peer review, replication, and all that–before jumping in, but it seems likely that The Journal of Personality and Social Psychology will be in for some awkward attention over the next few months.

Most ESP studies get shot down on their methodology–anyone who’s read Martin Gardner and the Amazing Randi knows that. If the participants don’t cheat, they’re influenced by the examiners. Assuming that this study’s controls are as airtight as its author says they are, I suspect its Achilles heel will be in its statistical modeling. I scanned the piece and realized I don’t know anywhere near enough about math to venture any kind of informed opinion, but having just read Jonah Lehrer’s New Yorker piece on “the decline effect,” I can imagine the kinds of objections that might emerge. Lehrer’s piece considers the troubling phenomena by which experimental results become less replicable over time.

“Scientific research,” Lehrer writes, “will always be shadowed by a force that can’t be curbed, only contained: sheer randomness….

Although many scientific ideas generate conflicting results and suffer from falling effect sizes, they continue to get cited in the textbooks and drive standard medical practice. Why? Because these ideas seem true. Because they make sense. Because we can’t bear to let them go. And this is why the decline effect is so troubling. Not because it reveals the human fallibility of science, in which data are tweaked and beliefs shape perceptions. (Such shortcomings aren’t surprising, at least for scientists.) And not because it reveals that many of our most exciting theories are fleeting fads and will soon be rejected. (That idea has been around since Thomas Kuhn.) The decline effect is troubling because it reminds us how difficult it is to prove anything. We like to pretend that our experiments define the truth for us. But that’s often not the case. Just because an idea is true doesn’t mean it can be proved. And just because an idea can be proved doesn’t mean it’s true. When the experiments are done, we still have to choose what to believe.

8 thoughts on “New ESP Study

  1. Wow. I just read the NYT article. As someone who does research at a major university I can debate about this topic for hours, I won’t though I promise.

    I will say one thing though. As long as Bem is telling the truth, his claims (no matter how outrageous) deserve to be be published. We social scientists use pretty sophisticated statistics to ensure one thing, that our results (the differences we find between groups) is there because of our manipulation and not because of random chance. If one is statistically (95%) certain that his results are NOT due to random chance, then he or she deserves to be published.

    Whether these results are caused by ESP or some other less controversial phenomenon is fully open to debate.

  2. You’re confusing science and mathematics. Mathematics is deductive and deals in proofs; science is inductive and does not. Put simply, scientists don’t seek to prove anything; they put forward hypotheses and try to disprove them. If, after a sufficient amount of time and effort, an hypothesis has not been disproved, it is advanced to the level of theory. Hope this helps.

    1. My p0int wasn’t that any claim for the paranormal is bogus until it’s proven to the level of certainty of a mathematical theorem, it was that the history of psi experiments has been very fraught, going back to Rhine.

      And then, even if all of Bem’s protocols turn out to be sound, there is the decline effect that Lehrer described, the phenomenon by which some scientific studies–on the effectiveness of pharmaceuticals, for example–seem to become less and less replicable over time.

      Bem’s test subjects did only slightly better than 50% overall–good enough to be statistically significant, but not so much better than chance. It’s possible that with enough repetition, the results will fall back into the realm of randomness.

  3. I’m interested in how these off-kilter studies interact with peer review. The norms of peer review are good for spotting standard failings in experimental design and statistical and conceptual flaws. But the process seems to be vulnerable to two rare types of research: that which is conducted in bad faith and capable of deliberately hiding it’s flaws (like Andrew Wakefield’s autism studies, or Hwang Woo-suk’s stem cell work), or that which deals with a domain of knowledge novel enough that no peer reviewers can understand the ways in which the concepts under study should circumscribe the design and statistics.

    I haven’t read the paper or the response papers, but off hand this looks like the latter is operating. Most veteran researchers are aware of the sticky details of experimental design and statistical methodology necessary to really nail their field of research. But no one (or very few) know how to deal with ESP. So the standard of review may be necessarily lower for ESP research, but not necessarily because anyone (including the author) wants it to be that way.

    Maybe they actually should have asked James Randi to be a reviewer. Like for real.

    1. Really interesting comment, Hugh, as is your longer piece at your own blog (interested readers click here). You say it all, and much better than I could!

  4. Thanks Arthur! I often come here for context when there’s something paranormal or conspiracy related in the news, and it was your post that got me thinking enough to write mine.

  5. Why should science be afraid to try something? They make it VERY clear that ESP is taboo in the science field. But what an amazing world we live in. There are so many mysteries to discover.

    Our own construction is amazing in itself. How well do we know the human capibilities?

    So what if science wants to discover the human more fully. What is wrong with that?

    When did science get so mudane? What happened to the adventure of science knowing no boundries?

    What kind of a world would we live in today if people didn’t use science to try crazy things?

    1. But the whole point of this post is that this scientist and this scientific journal ARE trying. ESP isn’t taboo, but it’s proven very hard to test. Skepticism comes with the territory.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s