Easing the pain of preregistration: Data Unboxing Parties!

Hi all. It’s been a while, for sure – lots of stuff been going on over the past months, including giving a TEDx talk, getting a new puppy, getting an annoying diagnosis, and me presenting at the Parapsychological Association meeting in Boulder. Yes, I’ve turned to the dark side. Sort of. In Boulder, I met several wonderful people, and one of them gave me a brilliant idea. As you know, in parapsychology people happily borrow concepts from physics, often with disastrous/hilarious results (see here). However, I think this idea I got in Boulder from a physicist will appeal to many.

It is about preregistration. Yet another epic blow to my Introduction to Psychology lecture slides: smiling does not make you feel better! Well thanks, Eric-Jan, now I have to disappoint another 450 students. What’s next, terror management theory not being true, so I can throw my joke in which I remind students about their mortality right before the weekend out of the window (oh… cr@p)? Anyway, all this has led to another revival of the preregistration debate. Should we preregister our studies?

I am not going re-iterate what has been already said about the topic. The answer is unequivocally YES. Really, there is absolutely no sound argument against preregistration. It does not take away creativity, it does not take away ‘academic freedom’, all it does is MAKE YOUR SCIENCE BETTER. However, many people do fear preregistration is at best unnecessary, and, at worst, a severe limitation to academic freedom.

In all seriousness – I think we need to be a bit less stressed out about preregistration. Basically, it’s a very simple procedure in which you state your a priori information and beliefs about the outcomes of your manipulation. Together with the actual data and results, this gives a far more complete record of what an empirical observation (i.e., the outcomes of a study) actually tells us. That’s it. Nothing more. The preregistration is simply an extension of the data, telling us the beliefs and expectations of the researcher, allowing for better interpretation of the data. And yes, this is what an introduction section of a paper is for, but simply think of your preregistration as a verifiable account of that piece of data, just as your uploaded/shared data are a verifiable account of your observations.

This also means that if you have *not* preregistered your study or analysis, it’s still a valuable observation. But less so than a preregistered one, for the simple reason that we lack a verifiable account of the a priori information, and need to trust a researcher on her/his blue eyes – similar to researchers who refuse to share empirical data for no good reason.

All this does not preclude exploratory analyses – you can still do them. However, it’s up to the reader to decide upon the interpretation of such outcomes. A preregistration (or lack thereof) will make this process easier and more transparent.

Now, how to implement all this in good lab practice and make it less of a pain?

A physicist I met in Boulder told me a very interesting thing about his work (a.o. at LIGO): for any experiment, first, they develop the data analysis protocols. In this stage, they allow themselves all degrees of freedom in torturing pilot datasets. Once the team has settled on the analysis, the protocols are registerd, and data collection begins. All data is stored in a ‘big black box’. No one is allowed to look at the data, touch the data, manipulate the data, or think about the data (I think I made that last one up). Then, once the data is in, the team gathers, with several crates of beer/bottles of wine/spirit/etc., and unboxes the data by running the preregistered script. The alcohol has two main uses: either, to have a big party if the data confirm the hypothesis, or to drown the misery if the data does not confirm the hypothesis.

I found this such a great idea I’m implementing this in my lab as well. We’re going to have data unboxing parties!

Ideally, we’ll do stuff like this from now on:

[1] go crazy on experimental design
[2] run pilot *), get some data in (for most of my stuff, typically N=5)
[3] write analysis script to get desired DVs. If necessary, go back to [1]
[4] preregister study at aspredicted.org, upload analysis software and stimulus software to OSF
[5] collect data, store data immediately on secure server
[6] get lab together with beer, run analysis script
[7] sleep off hangover, write paper regardless of outcome

So – who’s in on this?!

*) the pilot as mentioned here is a full run of the procedure. This is not to get an estimate of an effect size, or to see ‘if the manipulation works’, but rather a check to see if the experimental software is running properly, if the participants understand what they need to do, if they do not come up with alternative strategies to do a task, etc. The data from these sessions is used to fine tune my analyses – often, I look for e.g. EEG components that need to be present in the data. My ‘signature paradigm’ for example evoked a strong 10 Hz oscillation. If I cannot extract that from a single dataset, I know something is wrong. So that’s what the pilot is for.

On difficult surnames, reputation traps and a loose cable

Leonid Schneider asked me for my thoughts on his post on Frontiers in Paranormal Activities, in response to my sharing of Sam Schwarzkopf’s annoyance with people getting his last name wrong. I’ve got a difficult surname as well – it’s pronounced ‘yolay’, should that be of interest; ‘ij’ is a diphtong in Dutch, Jolij is the Dutchified version of my French ancestors’ name Joly – hence, the understanding. I had read Leonid’s post before, actually, when I saw it in relation to the Bial drug trial tragedy. At that time I did not respond, although I certainly did have a thought or two on the matter, but now that Leonid is asking, here we go.

What is the deal? Early 2014, a special issue in Frontiers in Psychology (or better, a Research Topic) was hosted by Etzel Cardeña and Enrico Facco on ‘Non-ordinary Mental Expressions‘. Some of the papers included in this topic are actually fairly ‘main stream’ (effects of psychedelics on neural activity, for example), but other papers were slightly more radical, including a paper on retro-priming and Cardeña’s editorial calling for an open view to the study of consciousness. These topics are, to say the least, controversial, and I do not think I have to elaborate on why that is so. This entire issue resurfaced this week when Etzel Cardeña published an ‘uncensored’ version of his editorial, and pointed out that research into the paranormal is typically ridiculed, researchers in the field are not taken seriously, and the ideas are basically dismissed without any consideration of data and or theory. There is a reputation trap: once you get associated with ‘weird stuff’, people will not talk with you anymore. Huw Price wrote a very worthwhile piece on this.

As said, Leonid Schneider wrote a long post on the special issue on ‘NOMEs’ in Frontiers, basically asking himself whether this is not one big practical joke on Editor-in-Chief Hauke van Heekeren. Because, you know, paranormal stuff?

The snark is strong in Leonid’s post. It’s quite clear that he does not take the study of psi as serious business. As I have indicated earlier, it does annoy me that skeptics all too easily ridicule researchers who are engaged in this type of research. This sentiment is very clear in Schneider’s piece, and it is also the reason for me not to comment earlier. I simply do not like the tone. What settled it for me, though, is the final addendum in which psi research is linked to the Bial drug trial tragedy. But more on that later.

I have argued before in several posts that I do believe psi can be a valid and relevant topic of study. Given that I am getting more and more involved with this debate, this may be a good occasion to give full disclosure on how and why I arrived at this position and show my true colours to friend and foe. Decide for yourself whether you want to group me with psi opponents, proponents, skeptics, or wafflers (though I am curious to hear from you with whom you would group me!)

First, there is a clear sentiment in Leonid’s post that psi research is not real science. I disagree. The sentiment seems to be based on the idea that psi cannot exist, and therefore researchers studying this topic cannot be taken seriously, and are probably running psychic hotlines next to their day-jobs, or are gullible fools who believe in fairies, Martians, and the Illuminati. More on that later.

With regard to what is science, I think science is not a belief system, but rather a structured method to increase knowledge about the world. As long as you stick to the rules of the game, there should be no taboo research areas. Of course, there may be research areas that make more or less sense than others, but as long as you stick to the scientific method, you’re doing science. In that respect, I do understand that Van Heekeren had no problems with a special issue on non-ordinary mental expressions in Frontiers. People do have weird experiences, after all. Regardless of what is actually going on, people do report out-of-body experiences, near-death experiences, and so on. These experiences are empirical fact (as in: people report having them). Therefore they are fair game for further study. I mean, if we could not study crazy experiences, psychiatrists and clinical psychologists would be out of a job, right? That said, let me be the first to admit that there is A LOT of god-awful (a.o. self-published) psi research and theories. Sturgeon’s Law (90% of everything is crap) applies to psi research more than to any other field I know of.

But shouldn’t Sturgeon’s Law for Psi Research not read ‘100% of everything is crap’, because a) psi cannot exist, b) psi researchers are idiots, and b) there is no theoretical framework in psi? No. First, it’s an easy straw man to craft a story about how psi researchers study clairvoyants (or may be clairvoyant themselves), run around with EM-meters to study haunted houses, and commune with the spirits to channel their research results. Admittedly, there are people doing that kind of stuff. And, no, I do not think we should take them very seriously.

However, as a science, experimental parapsychology has known higher methodological standards than many other areas in psychology. Preregistration, Bayesian statistics, publication of negative results – parapsychologists did all that stuff in the 1980s already, way before some main stream psychologists realized such methodological rigor is a must for any serious science. In that respect, if you think parapsychology is not a science, you should be fair and extend that opinion to all areas of psychology, and quite some more fields.

Anyway, there are quite some people who have found odd effects in carefully set up experiments that call for further investigation. Contrary to popular belief, there are some models/hypotheses out there for these lab-induced phenomena that are not completely at odds with our present understanding of physics. I say that as someone who studied physics for a couple of years (although I am the first to admit that the fact I got a degree in experimental psychology in the end is telling about my qualities as a physicist). Although these models rely on a rather specific interpretation of in particular the metaphysical status of consciousness, this is not a reason to dismiss them out of hand. I would like to remind the audience that the mainstream physicalist position on consciousness (i.e., consciousness is a brain process) itself is a metaphysical assumption about the nature of consciousness, and a position that is even slowly eroding.

This is where things got interesting for me. My research focuses on consciousness, and in particular on the mind-body problem. Psi phenomena, should they exist, would shine an entire new light on the metaphysical assumptions we make about consciousness. Just for that reason I think it’s worthwhile to have at least a look into the matter. My research interest in this area goes back to my early years a psychology undergraduate when professor Dick Bierman used to be my academic mentor. We talked a lot about this line of work, but I lost my touch with the area when I started doing my PhD with Victor Lamme, with a very strict materialist agenda. Dick and I got back in touch a couple of years ago when I got back in the Netherlands.

As I briefly mentioned in an earlier post, things got really interesting when anomalous effects started popping up in my own data. For example, using a single-trial EEG classifier I was able to decode the identity of an upcoming stimulus in a visual detection task, on basis of the baseline alone… Upon closer inspection of the data, it turned out there was a randomization problem. Ergo, I thought I had cracked the problem of all these alleged precognitive effects (improper randomization), fixed that by using a combination of hardware RNGs (if you see these odd photos of green glassware from my lab – that’s my hardware RNG ;-), and present that at the conference. Except the precognitive effect was still there. I triple-checked everything – stimulus script, analysis protocol, filter settings, hardware filters in the EEG amplifier: nothing. Yet, the effect is huge (d = 1.44).

So, what do you do? I decided to be honest and report what I found: I could decode the future, which I submitted as an abstract for the bi-annual winter meeting of the Dutch Psychonomic Society. Sure, I could filedrawer this weird effect, but why would or should I do that? I had a hypothesis, tested it, and failed. I was wrong – ‘precognitive’ effects are not caused by improper randomization. For those of you interested: we are going to replicate this study in a multi-lab setup. Drop me a line if you want to have a look at the data and code.

Anyway, to cut a long story short – over past five years I have done several quite large replication experiments on controversial areas (social priming, psi). The bottom line is that all my attempts at replication social priming effects failed, but the psi ones did not… So, hell yeah, I’m fascinated, and all of you would be if you got three whopping psi replications in a row. As a matter of fact, Dick and I are now getting some people together to work on a large-scale, multi-site adversarial collaboration project to run a number of high-powered replication studies to figure out if there really is such thing as a replicable psi effect. The only way to do this is by maintaining the highest methodological standards. Adversarial collaboration, preregistration, high power, open data, open materials, and proper experimental design are essential, otherwise you might as well not do it.

Now, to get back to Leonid Schneider’s post – I already mentioned I did not like the tone. I’ve read some of his other work, and fortunately, this piece is not representative of his qualities as a science journalist. Beacuse, as a science journalism piece, it fails, in my opinion. Audi alterem partem is what is missing here. That is disappointing. Moreover, you don’t have to agree with someone to still show at least some respect (or at least pretend you do). I’m totally fine with people thinking psi research is nonsense. I am also fine with people thinking my research is nonsense (I call such people “reviewers”). I am not fine with making fun of people and bullying them, which is what happens in this post.

Anyway, making fun of psi researchers is one thing, and I guess most are used to it. However, I think Schneider crosses a line when he suggests a link between the psi research and Bial trial incident. Bial is a Portugese pharmaceutical company that got into the news recently because of a clinical trial gone horribly wrong: healthy participants showed a severe adverse effect to a new drug, resulting in the death of one volunteer, and serious brain damage in several others. Schneider flat out suggests a relation between this tragedy and the fact that the Bial Foundation, a foundation sponsored by the founder of the company, funds psi research.

This suggestion is nothing short of slander. First of all, there is no relation between the activities of the company and the foundation, other than that the foundation annually gets a big bag of money from the guy who owns the company. Second, even if there would have been a direct link between the activities of the foundation and the company: as I mentioned earlier, the research standards in experimental parapsychology are at least comparable to those in ‘normal’ psychology. Third, clinical trials are legally regulated and closely monitored by medical ethics committees which assess the protocol and guard participant safety. Even in the case Bial did ask a psychic to develop the protocol for this trial using a crystal ball, or had a necromancer come up with the drug formula, the French authorities would/should have stopped this. All in all, the fact that Schneider uses this tragedy to make a point about parapsychological research is a really, really low blow.

In June, I attended the TSC 2015 conference, which also has quite a large number of talks on anomalous phenomena, and I had the pleasure to meet the kind of the people who are at the receiving end of Schneider’s snarky comments. They turned out to be fairly normal scientists, working at universities, about as knowledgeable or even more knowledgeable about research methods than the average psychologist. Most did not believe in fairies, they did not hold seances during their talks, not a single one brought a crystal ball, and there were no nightly shamanic sessions involving druidic dancing around monoliths (or at least, I was not invited to such happenings). The main difference is that these people work on effects most scientists find very, very implausable.

I think that we should measure psi researchers (or any researcher for that matter) not by their topic of study, but by the way they study their topic. Any researcher who holds up to high methodological standards, and is open to constructive criticism deserves to be taken seriously, regardless of what kind of effect she or he is working on. Period.

However. The fact that some skeptics cannot resist the urge to ridicule is no reason for the self-styled martyrdom some psi researchers engage in. Yes, psi researchers are being bullied, ridiculed, and even silenced. Schneider’s post is an excellent example. There is a reputation trap. That reputation trap, though, is often of one’s own making. Too often, psi researchers engage in wild, unfalsifiable speculation. Quantum teleportation, entaglement telepathy, that kind of stuff. Modesty should prevail: there is no convincing evidence psi effects exist, otherwise we would not have this discussion. Therefore, it is best to stay away from wild theoretical speculations that often involve misrepresented physics until there is at least some consensus between skeptics and proponents on whether anomalous effects are anything more than statistical noise. We’re not there. Yet.

Similar to most psi researchers not being fairy-worshipping druids, most skeptics are not narrow minded, sour critics. Most are actually very willing to discuss anomalous phenomena. But as data. Based on my personal interactions with them, I’d say both EJ Wagenmakers and Sam Schwarzkopf are perfectly willing to discuss experiments and datasets, but not if you come rushing in LOOK OMG HERE I FOUND PSI IN MY DATA LOOKATIT YOU WERE WRONG QUANTUM FTW! No, you found an interesting anomaly that begs for further exploration/explanation, but first we need to make sure your pattern of results is not the result of something trivial or just a random accident. Neither EJ or Sam laughed in my face when I told them about my data containing anomalies. Rather, the reply was “Interesting, what could be going on here?”

The thing is – it’s all about framing. In my mind, the present situation is very much like the faster-than-light neutrino anomaly of 2011. Researchers found evidence of particles moving faster than light, which according to special relativity is impossible. Rather than the entire field going bonkers, skeptics at CERN calling their colleagues at OPERA spirit-channeling fairy lovers, and OPERA researchers starting an anti-oppression movement because they were not allowed to share their result, the general response was “Hey, that’s interesting, let’s figure out what caused this result!” And that is the only reasonable response – if indeed particles can travel faster than light, it means we need to completely re-examine our ideas physics. Awesome! Work for generations of physicists to come!

Why can’t we do the same in psychology? There are people who seem to consistently find weird results. What’s going on? Clearly, we have not settled this matter – there is no conclusive evidence in favour of psi, but oppositely, clearly the psi proponents are also not convinced by the skeptics’ arguments and replication attempts. Skeptics should accept that there are consistent anomalies being found by intelligent, reasonable people all over the world that call for a deeper explanation than “it’s just statistical noise” or “it’s just publication bias” – I mean, weird results are popping up in my lab, FFS! Psi researchers should accept their case for the existence of psi is not strong enough, and that only with adversarial collaboration we can figure out what’s going on.

Oh, and those neutrinos? Turned out to be a loose cable in the Italian setup…

(note to self: check lab cables after the weekend)

Call for suggestions!

Hi all,

Shortly we will be running a pretty cool EEG experiment on perceptual decision making in romantically involved couples. Basically, a couple (let’s call them Alice and Bob) will come into the lab, each be assigned their own computer and then take turns in a perceptual decision making task (see Jolij and Meurs, 2011, for more details on the task itself). So, first Alice will get to see a trial and give a response; then Bob will see Alice her response (as a cue) and will do the trial, and to conclude, both will see each other’s answers. During the experiment, we’ll be measuring EEG (NB: only 8 channels). Before the experiment, both partners fill out a series of questionnaires on relationship duration, quality, etc.

In the spirit of open science, I though it might be useful to ask you all what would make this dataset useful for you. I mean, we are going to test these participants anyway, in a rather non-typical setup (two EEG measurements simultaneously, meaning you can look at all kinds of interpersonal processes, EEG synchronization, ect.), so if there is anything I could add that does not take too much time so this could be an interesting dataset for you, let me know. Think of maybe a block of eyes-closed EEG data during a breathing exercise to study interpersonal synchrony, a particular questionnaire, additional markers, whatever.

As long as it does not add too much time to the experimental protocol, or takes up too much time programming, I am happy to include stuff. Please do get in touch if you want to know more: j.jolij@rug.nl.

A quick comment to my previous post

Well, my previous blogpost was poorly timed – I did not expect such an explosion on my Twitter timeline. I cannot reply to all (I am sorting Legos with my son on my day off, quite an important and pressing matter), and replying tomorrow during my staff meeting would be a bit rude, so let me clarify on a couple of things I have seen in the message centre of my iPad.

An apology to Daniel Lakens

First, I feel I have to briefly apolize to Daniel Lakens – the first two paragraphs of my post I made little fun of him. From my part in good jest, but I admit that I may have let some of my sentiment of annoyance towards Daniel’s occasionally moralizing tone shine through a bit too much. I’m sorry, Daniel, if I offended you – if anything, know I deeply appreciate the good work you’re doing.

Can you please derive psi from GTR?

Not sure if this was directed to me, but I’ve seen this briefly in a tweet. Er, no, I cannot derive psi from the general theory of relativity. But neither can the Stroop effect be derived from the GTR. So, it’s a silly question. If the question is “can you derive psi from known physics”, then it’s a different matter. Physical laws give the boundary conditions for normal biological functioning. Psi, according to many, cannot exist because these boundary conditions forbid it. I have argued that that is not necessarily true. That does not mean that psi exists, though – the existence of pink elephants and uranium-powered dragons is also not prohibited by physics, and their existence remains also unproven by many studies (my son and I prefer uranium-powered dragons over Russell’s Teapot, but essentially it’s the same argument).

By the way – I do suspect that the asker knows that GTR is indeed potentially problematic for psi. Most physics-based psi theories are based on the concept of quantum non-locality. The non-locality aspect of quantum theory is incompatible with GTR, but yet we know both are correct – it’s one of the great problems in physics. There is presently one theory that seems to integrate both successfully, based on the very speculative concept of spontaneous collapse of the wave function. If this theory is correct, this would rule out pretty much all non-local psi theories that assign a special role to consciousness.

There is no evidence for psi, can we please stop this non-sense?

True. There is no conclusive experimental evidence for the existence of psi. If such evidence existed, we would not have this debate. However, does this mean psi does not exist? No, of course not. But, “Russell’s Teapot!”, I hear you think. Sure, it would be, if psi would be confined to a (non-existent) lab-related phenomenon. However, paranormal experiences have been reported throughout history, by all cultures. Of course, the vast majority of these phenomena can be explained nowadays by normal psychological or biological processes (including fraud). Psi research, however, started with the aim to reliably recreate such phenomena in the lab. Which does not seem to work for many phenomena. Although that’s of course not very promising, we have to acknowledge that Reality is not confined to our labs. An inability to recreate something in a lab does not mean it does not exist, of course.

You are a@#$@@#$ psi-believer!

Well, not really. As stated above, I do not agree with people like Bem and Tressoldi that there is convincing evidence for psi. However, I also do not agree with people like Daniel Lakens and EJ Wagenmakers that psi research is nonsense. I do believe psi is a very worthwhile topic of study, if done properly, because a convincing demonstration of psi would be a breakthrough for consciousness research. Given that there is a continuous stream of experiments that do seem to show effects, and that I have been getting some odd (and replicable) results in a my own lab, I am inclined to keep a close eye on this line of research and give it the benefit of doubt. However, I do not expect others to jump on the bandwagon or make it priority research (yet).

What genuinely annoys me, though, is the patronizing, scoffing, ridiculing, and accusations of QRPs or outright fraud towards psi researchers by self-proclaimed skeptics. There’s a lot a chaff between the wheat, that’s absolutely true, but I would say it’s not really necessary to make fun of intelligent people who are really trying to do serious research.

Any more?

Not for now – but if you’ve got comments/questions re: this topic, please do engage.


Why a meta-analysis of 90 studies does not tell that much about psi, or why academic papers should not be reduced to their data

Social psychologist-turned-statistics-and-publication-ethics crusader Daniel Lakens has recently published his review of a meta-analysis of 90 studies by Bem and colleagues that allegedly shows that there is strong evidence for precognition. Lakens rips apart the meta-analysis in his review, in particular because of the poor control for publication bias. According to Lakens, who recently converted to PET-PEESE as the best way to control for publication bias, there is a huge publication bias in the literature on psi, and if one, contrary to the meta-analysis’ authors, properly controls for that, the actual effect size is not different from zero. Moreover, Lakens suggests in his post that doing experiments without a theoretical framework is like running around like a headless chicken – every now and then you bump into something, but it’s not as if you were actually aiming.

I cannot comment on Daniel’s statistical points. I have not spent the last two years freshing up my stats, as Daniel so thoroughly has done, so I have to assume that he knows to some extent what he’s talking about. However, it may be worthwhile noting that the notion of what is an effect, and how to determine its existence has become somewhat fluid over the past five years. An important part of the debate we’re presently having in psychology is no longer about interpretations of what we have observed, but increasingly about the question whether we have observed anything at all. Daniel’s critical review of Bem et al.’s meta-analysis is an excellent example.

However, I do think Daniel’s post shows something interesting about the role of theory and methods in meta-analyses as well, that in my opinion stretches beyond the present topic. After reading Daniel’s post, and going through some of the original studies included in the meta-analysis it struck me that there might be going something wrong here. And with ‘here’ I mean reducing experiments to datasets and effect sizes. We all know that in order to truly appreciate an experiment and its outcomes, it does not suffice to look at the results section, or to have access to the data. You also need to carefully study the methods section to verify that an author has actually carried out the experiment in such a way that is measured what the author claims has been measured. And this is where many studies go wrong. I will illustrate this with a (in)famous example; Maier et al.’s 2014 paper ‘Feeling the Future Again’.

To give you some more background: Daniel claims that psi lacks a theoretical framework. This statement is incorrect. In fact, there are quite some theoretical models that potentially explain psi effects. Most of these make use or abuse concepts from (quantum) physics, and a as a result many psychologists either do not understand the models, or do not bother to try understand them, and simply draw the ‘quantum waffling’ card. Often this is the appropriate response, but it’s nothing more than a heuristic.

Maier et al. (2014) did not start running experiments like headless chickens hoping to find a weird effect. In fact, they quite carefully crafted a hypothesis about what can be expected from precognitive effects. Precognition is problematic from a physics point of view, not because it’s impossible (it isn’t), but because it creates the possibility for grandfather paradoxes. In typical precognition/presentiment experiments, an observer shows an anomalous response to an event that will take place in the near future, let’s say a chandelier falling down from the ceiling. However, if the observer is aware of his precognitive response, (s)he can act in order to prevent the future event (fixing new screws to the chandelier). However, now said event will not occur anymore, so how can it affect the past? Similarly, you cannot win roulette using your precognitive powers – any attempt to use a signal from the future to intentionally alter your behaviour leads to time paradoxes.

In order to avoid this loophole, Maier et al. suggest that precognition may only work unconsciously; that is, if there are precognitive effects they may only work in a probabilistic way, and only affect unconsciously initiated behaviour. Very superficially, this line of reasoning resembles Deutsch’s closed timelike curves proposal for time-travel of quantum information, but that’s besides the point here. The critical issue is that Maier et al. set up a series of experiments in which they manipulated consciousness of stimuli and actions that were believed to induce or be influenced by precognitive signals.

And that is where things go wrong in their paper.

Maier et al. used stimuli from the IAPS to evoke emotional responses. Basically, the methodology is this: participants had to press two buttons, left and right. Immediately after, two images would appear in on the screen, one of which would have negative emotional content. The images were masked in order to avoid them from entering conscious awareness. The idea is that participants would respond slower pressing the button at the same side as where the negative image would appear (ie., they would precognitively avoid the negative image). However, since this would be a strictly unconscious effect, it would avoid time paradoxes (although one could argue about that one).

What Maier et al. failed to do, though, is properly checking whether their masking manipulation worked. Properly masking stimuli is deceivingly difficult, and reading through their method sections, I am actually very skeptical whether they could have been successful at all. The presentation times of the masked stimuli were 1 video-frame, which would be necessary to properly mask the stimuli, but the presentation software used (E-Prime) is quite notorious for its timing errors, especially under Windows 7 or higher, with video cards low on VRAM. The authors, however, do not provide any details on what operating system they used, or what graphics board they used. To add insult to injury, they did not ask participants on a trial-by-trial basis whether the masked image was seen or not (and even that may not be the best way to check for awareness). Therefore, I have little faith the authors actually succeeded in successfully masking their emotional images in the lab. Their important, super-high-powered N=1221 study, which is often cited, has been carried out on-line. It’s very dubious whether masking was successful in this case at all.

If we follow the reasoning of Maier et al., conscious awareness of stimuli is important in getting precognitive effects (or not). Suppose that E-Prime’s timing messed up in 1 out of 4 trials, and the stimuli were visible – what does that mean for the results? Should these trials have been excluded? Can’t it be the case that such trials diluted the effect, so we end up with an underestimation? And, can’t it be that the inclusion of non-masked trials in the online experiment has affected the outcomes? Measuring ‘unconscious’ behaviour, as in blindsight-like behaviour, in normal individuals is extremely difficult and sensitive to general context – could this have played a role?

In sum, if you do not carefully check your critical manipulations you’re left with a high-powered study that may or may not tell us something about precognition. However, it matters when you include it in your meta-analysis – a study with such a high N will appear very informative because of its (potential) power, but if the methodology is not sound, it is not informative at all.

On a separate note, Maier et al.’s study is not the only one where consciousness is sloppily manipulated – the average ‘social priming’ or ‘unconscious thinking’ study is far worse – make sure you read Tom Stafford’s excellent commentary on this matter!

So, how it this relevant to Bem’s meta-analysis? Quite simply put: what studies you put in matters. You cannot reduce an experiment to its data if you are not absolutely sure the experiment has been carried out properly. And in particular with sensitive techniques like visual masking, or manipulations of consciousness, having some expertise matters. To some extent, Neuroskeptic’s Harry Potter Theory makes perfect sense – there are effects and manipulations which require specific expertise and technical knowledge to replicate (ironically, Neuroskeptic came up with HPT to state the opposite). In order to evaluate an experiment you not only need to have access to the data, but also to the methods used. Given that this information seems to be lacking, it is unclear what this meta-analysis actually tells us.

Now, one problem that you will run in to a whole series ‘No True Scotsman’ arguments (“we should leave Maiers’s paper out of our discussions of psi, because they did not really measure psi”), but some extent that is inevitable. The data of an experiment with clear methodological problems is worthless, even if it is preregistered, open, and high-powered. Open data is not necessarily good data, more data does not necessarily mean better data, and a replication of a bad experiment will not result in better data. The present focus in the ‘replication debate’ draws attention away from this – Tom Postmes referred to this ‘data fetishism’ in a recent post, and he is right.

So how do we solve this problem? The answer is not just “more power”. The answer is “better methods”. And a better a priori theoretical justification of our experiments and analysis. What do we measure, how do we measure it, and why? Preferably, such a justification has to be peer-reviewed, and ideally a paper should be accepted on basis of such a proposal rather than on basis of the results. Hmm, this sounds somewhat familiar

Recording of my Quantum Mind lecture (12 Nov)

Last week, I have given a lecture for the VIP, our student association, on my not-at-all controversial work and ideas on consciousness, quantum physics, and psi. Thanks to Jeffrey Harris, who made an audio recording, you can now listen to my waffling, whilst looking at the pdf with slides!

Slides: Quantum Mind


Part 1: Philosophy of Consciousness and the Hard Problem

Part 2: Introduction to Quantum Mechanics, in particular the Measurement Problem

Part 3: The Quantum Mind?

Is psi truly impossible?

Over the last weeks, several interesting posts in which psi prominently featured were posted on the web: one by Alex Holcombe, in which he argues that as long as meta-analyses find evidence for psi, our scientific publication system is broken; one by Sam Schwarzkopf in which he replies to a paper by Bem et al. finding new evidence for psi (that will be published in some time in the future; the study has not been conducted yet); and a satirical poster by Arina Bones, in which she argues that Bem’s presentiment effects were actually rather weak compared to the unlikely results tyically published in JPSP…

Now, to be fair, the mentioned blog posts primarily address methodological issues in (mainstream) science, rather than going into the theory of psi, but the posts have one thing in common: psi is implicitly or explicitly assumed to be impossible, because physics! But is psi indeed incompatible with physics?

Some interesting facts

Before going into that question, I’d like to share two interesting pieces of information. Yes, science is broken, and our publication process is largely to blame for it. The fact that we can find evidence for a phenomenon that is allegedly impossible is, according to Holcombe, a tell-tale sign of this. Two important suggestions to improve our practices are preregistration and using Bayesian rather than frequentist statistics. Interestingly, though, psi researchers seem to have a head start in improving their practices. The European Journal of Parapsychology, for example, has offered the option to preregister research, and have a paper accepted on basis of preregistered methods, independent of the actual outcome  since the late 1970s. It was big news in the psychology community when the journal Cortex started offering this service – only last year…

Moreover, the use of Bayesian statistics is presently advocated strongly by ‘professional skeptics’ to stop the surge of spurious research results (including, but not limited to psi). Interestingly, the idea to use Bayesian statistics in psi is not new. The Journal of Scientific Exploration, a journal notorious for “advocating pseudoscience”, published a critical review of psychokinesis research that explicitly called for the use of Bayesian statistics in the analysis of psi data, because, and I quote, “… a small p-value may not provide credible evidence that an anomalous phenomenon exists”. This paper was published in 1990, almost 25 years ago…

What I’d like to state with this: parapsychology as a research field is controversial, and that its topics, theories and results may be outrageous, but that some of its practices may be exemplary for mainstream psychology.

Psi and physics

Anyway, enough about meta-methodology. Time for some physics. Is psi indeed incompatible with physics? Well, yes and no. Psi effects apparently violate basic laws of physics, either because they involve a reversal of the flow of time (e.g. precognition/presentiment) or a direct influence of one body on another without a mediating force (psychokinesis). But which laws are violated?


First, can information travel back into time (often referred to as retro-causality)? There are two often-heard arguments against this idea: first, the Second Law of Thermodynamics gives time a specific direction or arrow; second, the Special Theory of Relativity (STR) states that nothing can go faster than the speed of light, which would be required if one would want to send information back into time.

I’ve argued before that the Second Law of Thermodynamics does not necessarily imply retro-causality is impossible – only that it is impossibly unlikely. So, what about this idea that nothing can go faster than light? Well, that’s not entirely correct. The only thing that Einstein proposed is that c (the speed of light) is invariant in all frames of reference of space-time. What follows from the STR is that in order to accelerate matter moving at sub-lightspeed beyond c, one would require an infinite amount of energy. Likewise, to decelerate particles moving faster than light (‘tachyons’) to sub-lightspeed  would also require an infinite amount of energy. So yes, for all practical purposes, nothing can go faster than light, but STR does not strictly forbid v > c.

Moreover, c is only constant within frames of reference. Between frames of reference, speeds exceeding c are permitted. However, this does not mean that retro-causality exists. In particular the ‘causality’ in retro-causality is problematic: in STR, causal relations can only exist between an object and objects in its past light cone (i.e., the points in space-time from which you can potentially see a photon emitted by the object). Even though velocities larger than c are permitted, this does make retro-causality in STR a no-no.

However – there may be a loophole. Or several, actually. Physically speaking, the direction of time is rather arbitrary. Without going into too much detail, there are several interpretations of electrodynamics (in particular the Wheeler-Feynman Absorber Theory) and quantum mechanics (two-state vector formalism and Cramer’s transactional interpretation) that explicitly assume time symmetry. Wheeler-Feynman’s absorber theory, for example, deals with the question why radiation seems to be emitted rather than absorbed. Maxwell’s equations for electromagnetic fields always yield two solutions: one going forward in time, and one going backward in time. We typically discard this latter solution, but for no good theoretical reason. Wheeler and Feynman have tackled this problem by assuming that the waves going backward in time are cancelled out (‘absorbed’) at a macroscopic scale. Dick Bierman has proposed that some retrocausal effects may be explained by assuming that the ‘cancelling out’ of advanced waves (‘information from the future’) has gone astray in some case, in particular when consciousness is involved (paper to be found here).

Speculative? Sure. Impossible? No.

Quantum waffling?

So, we don’t even need quantum physics to save the day, but let’s have a closer look. Quantum physics, with all its strange features, is often invoked when trying to explain more mysterious aspects of psychology, in particular consciousness and paranormal phenomena. An often-heard argument against this practice is that it is simply explaining one thing we do not understand (eg., consciousness) with another (quantum physics). Moreover, quantum physics and mysterious quantum phenomena such as entanglement and quantum superpositions are believed only to be relevant at a microscopic scale, and are completely washed out in the ‘hot and noisy’ environment that is our brain.

Surely, skepticism towards the all-too-liberal use of quantum physical concepts when explaining is warranted. In fringe and pseudo-science, quantum physics is often distorted and misunderstood. However, the same thing sometimes goes for debunking attempts from skeptics. Let’s look at some (mis)understandings.

(Mis)conception #1: Quantum physics is not relevant in biological systems such as the brain, and therefore irrelevant in the study of psi.

Can we use quantum mechanical concepts in order to explain weird mental phenomena, such as psi, or even consciousness itself? No, the skeptic argues – the brain is too hot and noisy to allow for any quantum physical phenomenon, such as entanglement or superposition, to occur. Quantum physics is only relevant when studying tiny particles in isolated systems!

Both these assumptions are not entirely correct. To start with the former, empirical evidence is accumulating that quantum processes (in particular quantum coherence) do play a role in living systems, for example in photosynthesis. Direct evidence of quantum processes in the brain still lacks, though. However, it may not be completely inconceivable that quantum processes do play a role in brain functioning. Does that open up the possibility for all kinds of spooky actions at a distance in human cognition? Well, that’s a really, really long stretch. For starters, the fact that quantum effects have been observed in plants does not mean plants are conscious entities that can do telepathy, precognition, and psychokinesis. Second, and more importantly, our understanding of the relation between brain processes at the microscopic scale and cognition is still quite limited. It is very difficult to imagine how quantum processes that play at the scale of single cells or even individual receptor sites would influence cognition and behaviour. However, simply stating that quantum physics by definition has no role in human brain function is not appropriate, I’d say.

Second, is quantum physics only relevant for studying microscopic isolated systems? No. Quantum physics is the best description of reality we have. There is no theoretical limit that states why we cannot describe macroscopic objects in the ‘language’ of quantum mechanics, Actually, we could consider classical Newtonian physics as a special case of quantum mechanics. The discrepancy between the quantum and classical world is rather an empirical observation we do not yet completely understand- there is not a theoretically prescribed boundary of at what scale we should start to use either classical or quantum mechanics to describe Nature. This brings us to a philosophical problem in physics: the measurement problem. According to some (controversial) interpretations of quantum physics (in particular the Von Neumann-Wigner interpretation), consciousness has a special statues in physics, because it ‘collapses the wave function’. I will not go into all specifics here, as this is a blog post, not an article (yet), but this is rather significant: if consciousness is instrumental to a basic physical process, this means it cannot be reduced to physical processes, or more specifically, brain activity. From this point of view, whether or not quantum physical processes play a role in brain processes is a completely irrelevant issue.

As I argued earlier, I think the most pressing reason to study psi is that demonstration of psi would falsify physicalism – the assumption that we can reduce consciousness to physical processes. We all too easily make this assumption when talking about consciousness, even to such an extent that we forget that it is an assumption. Back to psi – many psi effects could in theory be explained if we assume that ‘mind’ or ‘consciousness’ can be directly entered as a system in equations. The calculations would still work – however, the interpretation of the calculations would be rather outrageous… And whilst we’re on that page…

(Mis)conception #2: Quantum physics allows for free will, telepathy, psychokinesis, absolute idealism,  Chopra-ism, etc.!

So, quantum physics, in some interpretations at least, gives us a ‘loophole’ to escape Newtonian determinism – we can have a free will, and if the Von Neumann-Wigner interpretation is correct, we can even ‘create’ reality with our minds!

Er… no.

For starters, quantum physics is probabilistic in nature. Even if consciousness does collapse the wave function, the outcome of that collapse (e.g., finding a particle at a given location or not) will be completely random and unpredictable. As I argued in an earlier blogpost, that is not a very desirable feature of free will, or reality shaping in general. So, I would say free will and creation of reality are safely out of the scope of quantum mechanics.

So, what about psi? For the sake of argument, let’s assume quantum effects are somehow related to psi. Suppose that psi effects are the result of quantum entanglement: you, the observer, become entangled with a physical system far away in space and/or time, and for some reason something happens to the distant system. This means it would affect you instantaneously, because you and the distant system are entangled. However, quantum weirdness can never be used to communicate information at supraluminal speeds, which would be needed for causal psi. In quantum information theory, there is such a thing as the no-communication theorem. In order to make sense of the non-local interaction that happened (or even be aware that something happened at all), you need to correlate your state with the state of the distant system. This correlation can only be done via ‘classical’, sub-luminal communication.

This rules out any kind of ‘causal’ psi. With this I mean that quantum-based psi cannot be used to change your behaviour to avoid a given future, for example (i.e., the Final Destination scenario). What may exist in this framework, though, are correlational effects, such as proposed and reported by Von Lucadou and Walach.

Does psi violate physics?

Well, as it seems, not necessarily. There are interpretations of physics that do allow for psi-like effects to occur. However, these interpretations are not your main-stream ones, and do take some background in physics to fully comprehend. As physics laymen, we as psychologists/neuroscientists are often tempted to dismiss any interpretation that falls outside the ordinary stuff we learnt in high school or undergraduate physics courses. Perhaps rightfully so. However, stating that psi is impossible because physics? Hm, I’d be very careful with making that statement. Best not to calibrate our statistical methods using a phenomenon that actually might exist after all…

Why no one will win the Randi or Chopra Challenges

Whoa! Deepak Chopra is offering 1 million dollars to anyone able to present a falsifiable theory of consciousness, in response to James Randi’s $1 Million Dollar challenge to show paranormal (psi) effects exist! Of course, Twitter and Facebook are going bonkers over this. And I have been going a little bit bonkers over all the responses, to be frank. Just to blow off some steam, here are my thoughts on Chopra’s challenge, and the responses to it.

First of all, many people responded to Chopra’s call with sarcasm and cynicism, and made fun of Chopra’s lack of understanding of science.

It struck me how many of these people lack any understanding of science themselves, but I guess that’s Twitter for you. I’d like to say to these people who ‘fucking love science’: proclaiming yourself an atheist or tweeting ‘WOO WOO’ to @jref does not make you a scientist any more than making a coherent sentence with the words “quantum”, “universal”, and “spirit”.

 So, what’s this all about? Years ago, James Randi, a professional stage magician, and renowned skeptic put out a 1 million dollar prize to any individual able to show true ‘paranormal’ ability. Anyone who would be able to read the future, do telekinesis, or make money as Ghostbuster, Randi would pay one million dollars. To date the prize remains unclaimed.

Deepak Chopra, on the other hand, is an Indian MD who writes books on consciousness and quantum mysticism using the Deepak Chopra Quote Generator, and apparently makes enough money to throw a million dollars at anyone coming up with a falsifiable theory of consciousness.

Neither of these challenges make sense.

Randi’s challenge does not make sense because it operates on a straw man argument: it makes a caricature of psi and then shoots at it. No, there are no such things as seeing in the future, telekinesis, or mind reading. No matter how sad it makes me to admit this, Professor X and Jean Grey DO NOT EXIST (come on, you all at least fantasized about being able to read minds and get the remote and/or your beer and pizza without having to leave your couch!) Period. Does not work, cannot work – not according to the laws of physics, not according to present theories on psi. What might exist, though, are weak, anomalous effects that if they exist, may only be detected in high-powered studies involving a large number of subjects set up in a very specific manner, that need to be pre-registered, and replicated, and replicated again before we can even start drawing conclusions about the existence of psi. So, no individual will ever be able to show paranormal ability, and thus claim the Randi prize. Safe bet, Mr. Randi.

Chopra’s challenge makes no sense because it is horribly ill-defined. Coming up with a falsifiable scientific theory of consciousness is not possible without properly defining ‘falsifiable’ and ‘consciousness’. What Chopra means to say is he will give a million dollars to anyone who can come up with a falsifiable materialist theory of conscious experience, that is, a theory of the subject of consciousness – the experience itself; not (necessarily) its contents. And that is an impossible challenge, because it is a contradiction in terms. You cannot come up with a falsifiable materialist theory of consciousness, and claim the Chopra prize. Safe bet, Mr. Chopra.

But how does mind relate to matter, then? Why can’t we have a falsifiable theory of consciousness?

I am not going to repeat Introduction to Philosophy of Mind here, but roughly we have four classes of mind/matter-theories:

  1. Only matters exists, mind is an illusion
  2. Mind exists, independent of matter
  3. Mind is dependent on matter (or vice versa)
  4. Only mind exists, matter is an illusion

Now, let’s be good scientists, and shoot at these propositions to falsify them, shall we? Classes 1 and 2 are fairly easy to shoot at, so I’ll use proper bullets 😉 Classes 3 and 4 are somewhat more challenging, though.

Let’s start with 1, which you may call orthodox materialism. It’s easy to debunk (with one caveat, though).

  • Cogito ergo sum. I have conscious experiences. Even if these experiences (including the feeling of being the subject of conscious expriences) are illusions, I am still experiencing these illusions. Therefore consciousness exists – even if all other apparent conscious beings in the universe would be philosophical zombies (that is, beings that act rational, but lack conscious experience).
  • If consciousness exists, there is ‘mind’. This rules out orthodox materialist monism (the notion that there is only matter, and that mind is an illusion).
  • Caveat: I can only falsify this for myself, because I cannot with certainty claim anyone else has conscious experiences. Vice versa, you cannot verify my conscious experiences, so you should not believe my claim, but base your evaluation on your own conscious experience (or lack thereof).

Number 2 is good old Cartesian substance dualism. Let’s shoot!

  • In order to move a body, the mind needs a way to operate it
  • Operating a body requires brain cells to fire
  • In order to make a brain cell fire, energy is required – the mind needs to add energy to the brain in order to make this work
  • Physics (ie our understanding of matter) does not allow the creation of energy within a closed system. How can mind get energy into the brain?
  • The probabilistic nature of quantum mechanics will not save you here, Church of the Quantum Spirit. Quantum mechanics describes physical reality at the finest grained level, and contrary to classical mechanics, which is deterministic in nature, quantum mechanics is probabilistic. In other words: the x(t) = v*t gives the position of a moving object at time (t) with absolute certainty; the Schroedinger equation (or better, a transformation thereof) only gives the probability that a particle will be at a given position at a given time. A typical ‘quantum woo’-argument is that the probabilistic nature of QM potentially allows for a mechanism via which mind can influence matter. However, QM is probabilistic – the outcome of a quantum measurement is inherently unpredictable. That may sound very convenient if you want to believe in free will, but in fact it is a terrible property for a cognitive system, or social beings like us. Our entire social network, and our own mental sanity flourish by the mere fact that we are (in general) quite predictable in our actions and thoughts.  Let’s please not introduce fundamental randomness in there, I’d say…

Classes three and four are more difficult to shoot down. Since WordPress does not allow me to use mortar grenade points, but only bullet points I’ll switch back to full text.

Number 3 is the class of what I call ‘weak monism’. We accept mind and matter exist. However, the one substance is dependent on the other (or: one substance can be reduced to the other). This is the category in which we will find main stream theories of consciousness. Weak monist theories come in two flavours. Materialist (or physicalist) theories propose that mind is the result of physical processes, and can be described as such. The Orthodox Skeptics are adherents of these theories, as are most main stream scientists. Idealist theories state that mind is supreme, and that matter is created by the mind. Chopra’s Church of the Quantum Spirit is of this denomination.

Weak monism, though, suffers from the dreaded Hard Problem. How does a change in one substance result in changes in the other? This goes for both materialism and idealism. Materialists need to explain how a change in matter (brain cells) translates into consciousness, and why some physical processes (action potentials) result in consciousness in some circumstances, whereas the same physical process do not in other circumstances. However, also if you’re from the Church of the Quantum Spirit, you have a hard problem. If matter is a result of mind, how come not all mental activity results in changes in matter?

According to many materialists, including Dan Dennett, the Hard Problem is not really a problem at all. Consciousness simply is the sum of all brain activity, period. In slightly more subtle wording: consciousness is believed to be an emergent phenomenon, resulting from the complexity of the neural networks of our brain. This is called supervenience – reality can be described at different levels, and higher levels of description (consciousness, mind) are dependent on features of lower levels of description (brain, neurons). Or, as Kalat has put it in Biological Psychology for generations of psychology students: you can look at the Mona Lisa as a painting of La Giaconda, and talk about in the sense of her mysterious smile, or you can give a detailed descripton of the canvas and pigments used. Same thing, different levels of description. Similarly, mind is the same thing as brain activity, but simply described in different terms. Obviously, we can easily swap around the words ‘mind’ and ‘matter’ to fix the Hard Problem for idealism.

Now, I hate to bring this news to the Orthodox Skeptics, but this is Woo in its purest form. You cannot call any theory that only says ‘if you make something complex enough it becomes conscious’ a serious theory! How complex does a system have to be in order to become conscious? At what level of description does consciousness emerge? Does the physical system need to be a brain, or would any physical system do? In other words – calling consciousness an emergent property of brain activity and leave it at that is hardly any more scientific than declaring universal quantum love and spirit (or insert your favourite Deepak Chopra quantumism here).

There are several problems with the emergence/supervenience theories of consciousness, but I personally think John Searle brought up the best argument against supervenience theories. Let me paraphrase it in terms of the Chinese Room thought experiment: in this thought experiment, we lock up a man who only speaks English in a room. Via a slot in the door he is given sheets of paper with Chinese characters. Using a manual in the room, he is able to look up an appropriate response in English. He writes the reply on another sheet of paper, which he returns via the slot. From the outside, it looks like the man knows Chinese! In reality, of course, he does not. Searle used this to argue that true artificial intelligence does not exist – for example, if you are training a system to respond to a user in natural language, what you’re doing is giving an artificial system a manual. The system does not understand language in the sense we understand language.

The Chinese Room can also serve as a thought experiment on consciousness. Take a system (a body), and pop a computational unit in there  that can map inputs to outputs it (let’s call this magic device a ‘brain’). The brain or parts of it are not conscious in any sense – they simply map inputs to outputs. However, looking at the system, operating in the world, it is conscious, or at least, bears all signs of it. This is pretty much in the line of Alva Noë’s ideas of how consciousness depends on embodiment.

In his book “Intuition Pumps and Other Tools for Thinking”, Dan Dennett defuses Searle’s argument by stating that the thought experiment is flawed. It does not matter if the ‘guy inside’ understands Chinese or not – the system (that is, the room) does. Digging deeper for ‘understanding’ or ‘consciousness’ makes no sense. There is no ‘Hard Problem’ – conscious experience is just what a system is doing at a particular level of description.

Now, I would like to very explicitly state here that Dan Dennett is probably one of the greatest minds alive, and I am nowhere in his league. I am a great fan of his work, and I feel that it should be compulsory reading for any undergraduate in psychology. However, I think he is wrong here. The reason for that? He plays a trick on us in defusing the Chinese Room.

The trick is this: he smuggles in an external observer. The Chinese Room understands Chinese only if observed by and interacting with an external observer. The ‘understanding’ of Chinese by the room only exists in the mind of the observer! Otherwise, the actions of the Chinese Room are meaningless. Likewise, the brain-in-a-body-operating-in-the-world is only conscious if observed in an appropriate context. Following Searle, I do find this problematic. Consciousness is a first-person perspective. I know I am conscious, because I am both subject and object of my experiences. Who or what is then describing the activity of my brain-body in such a way it enables my first person consciousness? It cannot be me because I am the result of this observation, and unless we allow paradoxical cause-effect relations (which I doubt any materialist would be very keen of), we are left with a very urgent question: in whose mind do I exist?

In sum, I see pretty big problems with materialist theories of consciousness. However, converting to idealism does not solve the problem. As argued earlier, idealism also suffers from the Hard Problem, and the above analysis applies as well. The Hard Problem is deviously difficult to defuse if you accept that mind and matter exist .

One possible solution is to give consciousness a ‘fundamental’ status. Consciousness is a fundamental property of the universe, like the universal forces. Hameroff and Penrose’s Orch OR model rests on this assumption, but also Giulio Tononi’s highly fashionable and critically acclaimed IIT 3.0 model of consciousness puts as its ‘zeroth’ postulate ‘consciousness exists‘. In a recent online article, Christof Koch even explicitly explored panpsychism (the idea that everything is conscious) as a solution to the mind-body problem. However, this does not explain why consciousness exists. And given that physicists are not satisfied with merely stating that ‘gravity exists’, we as psychologists should not be satisfied with stating that ‘consciousness exists’.

Anyway, in a rather large nutshell, this is why the Chopra Challenge makes no sense. Apart from the fact it is poorly defined, we are nowhere near an empirically verifiable (or falsifiable) theory of consciousness. All we’re doing now and have been doing since the era of brain scanning is looking for neural correlates of consciousness, which is a very useful enterprise because it provides boundary conditions for consciousness, but we have not been cracking on the Hard Problem at all. The Hard Problem is probably fundamentally unsolvable within a weak monist framework. In itself this does not prove Chopra right, of course.

On a separate note: what would advance our understanding is a potential falsification of  the idea that mind can be reduced to matter. This is actually the reason I started doing psi research, apart from my lifelong wish of being a Ghostbuster when I finally grow up. If we can convincingly demonstrate that certain aspects of mental functioning cannot be reduced to physical processes, we would have a strong case to either revise our physical models, or falsify materialism. Given the potentially huge impact of psi research, and the fact that the present corpus of data does not allow for clear falsification of psi, I think it is a very worthwhile area of research. But that’s my 2c.

Oh, we have one class of theories left, don’t we? Absolute idealism or monist idealism states that only mind exists, and that matter is an illusion. Well, to quote Sherlock Holmes, “when you have eliminated the impossible, whatever remains, however improbable, must be the truth.” 😉

Woo woo…


PS regarding my last point, I can recommend reading Schroedinger’s “What is Life?” It is a short book, which you can read in a couple of hours, but will stay with you for a lifetime. Yes, I know I stole that quote from the reviews of the book, but it’s very true.

Social priming and psi

Update: before being accused of being a bully because I compare social priming researchers to parapsychologists, a) I consider myself to be a psi researcher, so if anyone takes offense to being compared to myself, I am very sorry, and b) the bottom line of this post is: if you take social priming seriously based on the empirical record, you should take psi seriously, too.

Now, there you have two controversial terms in one blog post title! No, I am not going to claim that psi may be involved in social priming or vice versa. No, I won’t make any claims here about paranormal phenomena playing a role in social priming (although…), but something struck me when going over my Twitter feed this weekend.

So, what happened? Well – sh*t really hit the fan after publication of Social Psychology’s replication issue, edited by Brian Nosek and Daniel Lakens, to be found here. A lot of things have already been written and blogged on the entire issue of replications and replicating research, and the debate has turned quite ugly from time to time. But I’ll not be addressing that here – many other did a better job at that than me.

No, what struck me is that there seem to be some interesting parallels between parapsychology and social priming research I’d like to share with you. Disclaimer beforehand: I am an active researcher in both social priming and psi. I may be prejudiced with regard to both topics – please keep that in mind when reading 😉

First of all: both fields make bold claims about the nature of the human mind and – if correct – have far-stretching consequences for our understanding of who we are. If psi exists, it would mean the mind does not answer to laws of physics, and may therefore not be reduced to brain activity. Or the laws of physics are wrong, that’s also an option. At least, the confirmed existence of psi would change our view of the world. Your mind is not what your brain does – but something more! That’s an idea many people would find attractive.

Social priming research, if true, shows that our environment has an extremely large impact on our behaviour, both overt and covert – a simple prime may make us walk slower, make us buy stuff we normally would not, or even make us more or less prone to show criminal behaviour. Taken to the extreme (a point once defended by Diederik Stapel in an interview with the Dutch ‘Academische Boekengids’, if you read Dutch, you can find it here), it means that you, or your ‘self’ – the agent that decides what you body will do next – is nothing more than a series of tendencies primed by your environment. Consciousness and free will have little to do with behaviour and are just illusions. Maybe not a pleasant idea, but very tangible – it means that human behaviour is rational, and can be completely explained and understood. Again, an idea many people would find attractive.

So, it’s clear both fields have a large appeal: they challenge our native and naive ideas about who we are. It’s therefore not surprising that both parapsychology and social priming are ‘hot topics’ in the main stream popular media.

Both social priming and parapsychology have a serious problem, though: after a series of spectacular claims and promising results (for parapsychology in the 1930s, for social priming starting in the 1990s), problems arose. Key findings turned out to be difficult to replicate. In parapsychology, there is even a term for this: the decline effect, and it’s even become a topic of study. After some initial successes to demonstrate telepathy, precognition or clairvoyance, effect sizes decreased, to disappear completely after repeated replication attempts. In social priming, we see that the large effects reported by original studies quite often turn out to be far smaller or even non-existent in subsequent replications ran with larger populations. As a result, both fields are struggling to show that the effects they study even exist. Overall, meta-analyses do show there is ‘something going on’, both for psi and social priming, but the actual effects are elusive.

The emphasis on showing effects has drawn attention away from what a mature field should do: come up with theories and test those. Both parapsychology and social priming are traditionally characterized by lack of theories that explain the phenomena that are being studied. And with ‘theories’ I mean a general, and plausible framework that can produce falsifiable claims – not post-hoc explanations for effects. In social priming, for example, I once read a nice metaphor about how behaviour is akin to a piano on a sheet of ice, subject to all kind of external forces – see here) Although this sounds very reasonable, this theory cannot be falsified – if a finding does not replicate, you can always conjure up a ‘moderating’ variable that has extinguished the effect. Another reason that in particular cognitive (neuro)scientists are very critical about social priming research is that the explanations for the effects are very implausible with regard to their (neuro)cognitive implementation.

My greatest concernt, though, is the elusiveness of the effects. I do accept that the effects may exist. I doubt, though, how relevant the effects are in everyday life. In a blogpost, Simone Schnall mentioned an online replication attempt of her (in)famous finding that washing your hands makes you behave more morally. The replication failed. Schnall was not surprised – she explicitly stated that the priming procedure would only work in the lab, where subjects can be closely monitored. This is a pretty strong blow for ecolgical validity – if an effect does not replicate outside the lab, then what does it really tell you about human behaviour?

Parapsychology, though, seems to have matured a bit more than social priming over the last years. There are several falsifiable theories out there that do predict when psi phenomena will occur, and under what circumstances, for example Von Lucadou’s model of Pragmatic Information, and Bierman’s CIRTS-model. Both these models are inspired by physics, and do make sense. Most importantly, they are falsifiable: both MPI and CIRTS make very explicit predictions about psi-effects. According to the MPI, for example, psi-phenomena can be explained as non-local correlations, analogous to quantum entanglement. As in quantum theory, MPI postulates that such non-local correlations can never be used to transmit information – if that were possible, they would allow for faster-than-light communication, and thus for nasty paradoxes. This yields some weird predictions: most importantly, as soon as an effect becomes ‘informative’ it has to disappear. For example, you may be able to find presentiment in one study. However, in the next study you now know that you may expect presentiment, and thus build a presentiment-meter (see my previous post). According to MPI, you’re not allowed to – and, poof, your effect is gone.

So, how to demonstrate psi if it disappears if you’re looking for it? Von Lucadou proposed an elegant solution: don’t look for it specifically. Von Lucadou and co-workers have published several experiments in which they show that in interactions between an observer and a quantum random number generator, the output of the qRNG will correlate with aspects of the observer. However, which aspects cannot be known beforehand. So, the one time, there is a correlation between the qRNG and the observer’s intentions (which would be the classical psychokinesis-case – it looks like you’re influencing a physical system with your mind), the next time it’s a correlation between the qRNG and the observer’s shoe size. The only solid thing is that if you measure, let’s say, 100 correlations, you will always find more than you’d expect on basis of chance alone.

So, to summarize – psi and social priming are both controversial fields, where there is good reason to assume something’s going on – but we don’t know what. Both fields have come up with theories, and parapsychology seems to be doing an even better job than social priming. However, in the end, it’s very well possible both fields are chasing ghosts. Well, if that’s the case, at least the parapsychologists can say it’s their job.

Is psi a legitimate topic of study?

Weird stuff happens from time to time – it’s a fact of life. When I was applying for a position as assistant professor in Exeter, the weeks before the interview the word ‘Exeter’ started to pop up in weird places in my life – newspaper articles, names of conference rooms I happened to present in, stuff like that. Or that time that my then four-year old, right before leaving home, started to talk about the street in front of his school being changed into a river – when we arrived fifteen minutes later we found the street to be flooded by a bursted water pipe.

Coincidence? Probably. Nevertheless, most people will have experienced such weird coincidences, and sometimes people interpret these experiences as ‘paranormal’. Paranormal experiences include clairvoyance, telepathy, precognition, and psychokinesis. Not surprisingly, these phenomena, collectively labelled as ‘psi’, are met with great interest by the general audience, but traditionally, psychologists have been very interested in them as well. William James, the founding father of psychology, studied paranormal phenomena, and the name-giver of our research intstitute, Gerard Heymans, credited as the first Dutch experimental psychologist, was also the founder of the Dutch Society for Psychical Researcy, for example.

However, psi research (or parapsychology) was met with increasing skepticism. The weird experiences we all have from time to time turned out to be very difficult to replicate in laboratory settings, and basically, ever since its inception, parapsychology is struggling to confirm psi even exists, let alone come up with a solid theoretical background. As a result, nowadays, parapsychology is typically considered to be (at best) fringe science, or even pseudoscience, and is actively ignored by main stream psychology.

A small number of researchers have continued parapsychological investigations, though. Rather than aiming for dramatic demonstrations of psi by mediums, for example, modern-day parapsychologists study far more subtle effects. One of the best-studied topics is that of presentiment (see eg. Bierman and Radin, 1997): an anomalous baseline response to a stimulus that yet has to appear. An emotional picture, for example, induces a strong physiological response. In presentiment, this response appears to be temporally mirrored in the baseline: emotional picture evoke a physical reaction before they are presented, and apparently this physiological response is symmetrical in time (i.e., if the typical physiological response occurs at t = 2.5s, the ‘presponse’ will occur at t = -2.5s). Presentiment has been quite widely studied, and the effects seem to be more stable and legit than your typical psi-effect.

Presentiment has drawn wide mainstream attention with the publication of Bem’s landmark 2011 paper ‘Feeling the future’ in JPSP (Bem, 2011). Bem claimed to show solid evidence for the existence of presentiment in a whole series of experiments, in over 1000 subjects. However, the paper was met with a storm of criticism. I am calling Bem’s paper a landmark paper because it can be pinpointed as the paper that started off the massive debate on questionable research practices in (social) psychology, even before the uncovering of the fraud cases of Stapel, Hauser, Smeesters, and (likely) Foerster. The main critique on Bem’s work was that Bem appeared to ‘shop’ in his results, and did not use appropriate statistics: basically, he was accused of practices such as ‘p-hacking’ (mildly massaging your data until you hit p <.05), and ‘HARKing’ (hypothesizing after the results are known). Moreover, replications of Bem’s experiments failed, and the refusal of JPSP to publish these failed replications only added to the uproar surrounding the field of social psychology (one set of null results has been published in PLOS One, though: here).

Bem, however, did not give up. He, and his coworkers, have published two meta-analyses on the presentiment effect. Armed with the statistical techniques used by his critics (such as p-curves and Bayesian analyes), Bem and colleagues analyzed 90 experiments and show that there ‘is something going on’ in their latest meta-analysis, to be found here.

What to make of this? Does Bem’s latest work indeed support the existence of psi? Let me first say that a) I consider myself to be an open-minded skeptic (more on that later), and b) that I am far from an statistics expert. Having reviewed the latest paper I find the case compelling at first sight. I have got two major concerns, though.

1. How big is the file drawer? Bem et al. estimate that the overall effect size they find for psi can only be negated by at least 520 studies finding no effect. At first sight, this seems a large number. But is it really that unlikely that there are 520 experiments yielding no result Bem et al. did not know about and therefore did not include? Well, it is not. The Bem-experiment has drawn wide attention, and is likely to have been replicated. In our own institute several teaching assistants (ie. not academic faculty!) have used the Bem case as example in second year research practicals. Such data is typically not archived, as it is not considered to be ‘research’, and therefore goes unnoticed. Tenured and tenure-track faculty are less likely to engage in direct replications for the ‘usual’ reaons: lack of time, concerns about tenure, or simply lack of interest. In other words, I do not think 520 experiments (worldwide!) is a completely unrealistic number. The file drawer on psi might be quite substantial.

In addition, if they exist, pre-cognitive effects should also emerge in main-stream research. If a stimulus can exert retro-active influence, this should also be apparent in paradigms that are not explicitly designed to measure psi. For example, in a typical priming experiment, one could look at effects of primes (or targets) presented in trials after the critical stimulus. If precognitive effects do exist, one should be able to find effects in such datasets as well. Now, I myself have several datasets in such weird things are going on, although the most parsimonious account is still a main-stream one, obviously. However, ideally, to make a strong case for psi, main-stream research should be taken into account as well.

2. The theoretical background. The Achilles-heel of psi-research, though, is the lack of a clear theoretical framework. Just that you find a difference between two conditions in an experiment does not mean that much: laws of probability dictate you will find a difference in 1 out of every 20 experiments with p<.05. What matters if your difference fits with a given theory: were you able to predict you would find a difference? Many critics argue that parapsychology lacks a theoretical basis, and that given that psi is incompatible with the laws of physics, research into psi is by definition pseudoscience.

And that is where many critics are wrong.

Psi is not necessarily incompatible with physics. Bem et al. drag in quantum physics, which is something more scholars appear to do when stuff gets complicated (I consider it to be a bad habit), but there is no need for quantum physics to find a loophole in physics that allows psi. Psi involves the flow of information backward in time. Perhaps surprisingly, this is not impossible according to physics. Most physical laws are in fact time-symmetric. For example, in the formula x(t) = v * t, which gives the position x of a point mass at moment t given its speed v, there is nothing against entering a negative number for t. The only physical law which imposes a direction of the flow of time is the second law of thermodynamics, which postulates that the entropy of a closed system will increase over time. Basically it means that the sugar cube you put in your coffee will dissolve, but not spontaneously re-appear after it dissolved, and that your coffee will get colder over time, but not hotter (unless you heat it).

This is such a fundamental property of the universe that it dictates a direction of time on all other physical processes. However, it is also not a law, but a statistical property of the universe. Basically, potential high-entropy states of a system are far more abundant than potential low-entropy states. Consider the sugar cube in your coffee: the organized, low-entropy state in which sugar molecules are when packed into a cube is quite specific. All molecules of the sugar cube are packed into a confined spatial location (the cube), which limits the number of possible states. However, molecules of a fully dissolved cube can be anywhere in your mug, yielding a far larger number of possible states. Yet, it is not impossible that a sugar cube will rematerialize in your cup of coffee after dissolving (there is no physical law prohibiting the sugar molecules in your cup of coffee of reassembling themselves into a sugar cube), it is just impossibly unlikely. And to date, we do not know how the universe got itself into this mess – why was entropy at the beginning of everything so low?

That aside – technically, time reversals, or better, the universe adopting a state where effect appears to precede cause, are not impossible, just extremely unlikely. But does it mean that presentiment can be real? Well, there is a second problem. Even if we accept that information in extraordinary circumstances can travel backwards in time, presentiment potentially creates time paradoxes. The best-known example is the grandfather-paradox. Suppose you have a time-travelling machine, and travel back in time to when you grandfather was a child. And you shoot him. Apart from that being quite cruel in the typical circumstance where you have a loving grandpa, it creates a paradox: if your grandfather dies before he fathered one of your parents, how can you exist and travel back in time to shoot him?

The same thing applies to presentiment. Technically, it is possible to build a presentiment detector if the effect is real: Bierman and Radin (1997), for example, report an anomalous increase in galvanic skin response before presentation of an unpredictable emotional stimulus compared with the baseline response to neutral stimuli. If you sample a participants GSR to stimuli for a sufficient number of trials, it is well possible to estimate a participant’s typical baseline to neutral stimuli and estimate whether the baseline response on a given trial is typical of a subsequent neutral or subsequent emotional stimulus. That would allow you to predict the future: by a quick analysis of anticipatory GSR activity you should be able to guess (not perfectly, though) whether the upcoming stimulus will be emotional or neutral. And this is where the paradox arises: if your analysis is quick enough, you can quickly switch of the monitor, blindfold the participant, or even change the upcoming stimulus before the stimulus is presented and thus prevent the participant from seeing the stimulus that would trigger the emotional reaction. Effectively, this is the same as shooting your grandfather in the grandfather-paradox: if you erase or change the event that triggered the presentiment response in the first place, how can it trigger presentiment?

Stephen Hawking has argued that the universe ‘actively resists’ time paradoxes. Bierman (2001, and personal communications) supposes that this is one of the reasons why psi effects are so elusive: as soon as they become informative, and are able to create time paradoxes, they cease to exist. I am not sure if this is the case or not. However, I find the time paradox argument against presentiment compelling – a lot more compelling than the incorrect argument that psi does not fit with the laws of physics.

In their meta-analysis, Bem et al. cite a Bayesian analysis of Bem’s 2011 paper by Wagenmakers et al. In Bayesian statistics, one evaluates the likelihood of evidence gained by an experiment, or set of experiments, relative to the likelihood of the data given a prior hypothesis. Wagenmakers et al. set their prior to 10^20, meaning that they would be convinced of the reality of presentiment if the outcome of an experiment was 10^20 as likely to be found under the hypothesis that psi exists than under the hypothesis that it does not. Bem et al. report a Bayes-factor of 10^9 in favour of the existence of psi. Impressive – but not impressive enough for Wagenmakers et al. Bem et al. argue that a prior of 10^20 is unrealistically and unreasonably high. Well, is it?

Wagenmakers et al. base their prior on the fact that psi is incompatible with the laws of physics. With that argument I do not agree (see above) per se, but I do think the time paradox argument is convincing enough to warrant a very high prior. 10^20 seems not unrealistic or unreasonable in that sense to me

So to conclude: I am open to the possibility of psi. There are no universal laws against it, it is just extremely unlikely. Bem’s meta-analysis is convincing, but to me not convincing enough. It would be far more convincing if Bem et al. could make more specific theoretical predictions about when and how presentiment might occur. They do offer some interesting points – in particular, presentiment seems to be most prominent in ‘fast thinking’ scenarios. Now, that is actually what I would predict as well. If it exists, presentiment is most likely not a ‘conscious’ phenomenon. Personally, I have never consciously experienced a time-symmetric emotion. However, most of us will know the experience of a ‘gut feeling’ (“I knew something bad was gonna happen!”) My own research has shown that such ‘gut feelings’ are easier picked up when you’re not consciously trying to ‘listen’ to them. So, in that respect, if presentiment exists, it might indeed be easier to find it in experiments that require ‘fast thinking’.

One final remark: some critics argue that psi should not be studied at all, and ridicule any serious treatment of the topic. I have even seen some tweets arguing that psi is an excellence benchmark for novel statistical methods – if you find evidence for psi using your statistical method, your method is wrong!

This stance seems profoundly unscientific to me, in particular coming from people who claim to have strong dislike for pseudoscience. You simply cannot and should not dismiss results purely on the basis that they do not fit your beliefs. I’d say we have some bad experiences with that in the Middle Ages.

Arguably, the theoretical case for psi is not strong, but it is strong enough to take seriously. As a researcher, I study consciousness, and to be frank, we haven’t got a clue how consciousness works. The hard problem of consciousness is still far from solved, and there are profound philosophical problems with the materialist theories of consciousness. Psi (if it exists) may offer new insights into the nature of consciousness. For me, this is the main reason to regard the field with interest and not dismiss is right out of hand. In that respect, I think Bem’s work is valuable, interesting, and deserves recognition from main stream science, and I’d rather have skeptics trying to shoot at the data and analyses than actively ignoring psi altogether. Therefore, I am happy to see that many people have taken a critical look at the data; in the end, this can only give us a clearer picture of what is going on.