Sometimes researchers break into opposing camps. They may each find well respected journals to publish their work in, but neither’s work appears capable of changing the minds of the others. Adversarial collaboration is an approach for getting beyond this. The idea is that intellectual opponents co-design new studies. As much as possible, they should agree ahead of time about how the data will be analyzed, and include other neutral collaborators. Then they write down their predictions, their uncertainties, and how different results will change their opinion. When the study is done, hopefully they end up agreeing about how to interpret it. But even if they didn’t, I think examining such a study, including the collaborators’ predictions beforehand, would really help other people decide what to believe.
Philip Tetlock and Gregory Mitchell are the only people I’ve heard promote it. They call for it here, and describe it in more detail towards the end of an article titled “Unconscious Prejudice and Accountability Systems: What Must Organizations do to Prevent Discrimination.”
Here is part of their conclusion:
Of course, what makes adversarial collaboration scientifically attractive—the prospect of breaking epistemic impasses—may also render it politically unattractive. Nothing will happen if either side decides that it is better off when there is less scientific clarity. For this reason, failures to broker adversarial collaborations are profoundly informative: they signal to the policy world that the American racism debate and the sub-debate on unconscious prejudice may be politicized beyond scientific redemption. Tetlock (2006) has offered rough sociology-of-science diagnostics for judging the odds of failures of this sort. Adversarial collaboration is most feasible when least needed: when the clashing camps have advanced testable theories, subscribe to common canons for testing those theories, and disagreements are robust but respectful. And adversarial collaboration is least feasible when most needed: when the scientific community lacks clear criteria for falsifying points of view, disagrees on key methodological issues, relies on second- or third-best substitute methods for testing causality, and is fractured into opposing camps that engage in ad hominem posturing and that have intimate ties to political actors who see any concession as weakness. Tetlock (2006) calls the former community as “epistemic Heaven.” the latter “epistemic hell,” and maintains—in the spirit of Figure 4—that if adversarial collaboration is indeed unnecessary in heaven and impossible in hell, we should expect the greatest expected returns in the “murky middle” in which theory-testing conditions are less than ideal but not yet hopeless.
If you can think of a topic in between which could benefit from adversarial collaboration please suggest it in the comments.
Update: Apparently Daniel Kahneman and collaborators originated the term and successfully implemented it!