a case for single-blind review

(Cross posted from here)
When i was in grad school, at one of the academic meetings i regularly participate in, it became regular fare for 2 particular folks in my circles to engage in a prolonged debate about how we should overhaul the academic publishing system. This was so regular (i recall them having portions of this debate for 3 consecutive years over dinner) that the grad students in the bunch thought of this as a grenade in our back pockets we could toss into the fray if ever conversations took an unwelcome turn to the boring. I bring this up because there are lots of aspects of this process that i have quite a few thoughts on, but have never really formalized them too much more than is required for such elongated dinner conversations. And one particular aspect of that was raised on Facebook yesterday by a colleague – asking about the merits of single blind review. I started my answer there, but wanted to engage this a little more fully. So, i’m going to start a series of posts (not sure how many there will be at this point) on the publication/review process here, that i think could be interesting discussions. I hope others will chime in with opinions, questions, etc. These posts will likely be slightly longer than typical fare around here. I expect that some of my thoughts on these will be much more formulated than others.

So, let’s start with a case for single blind review. I think think there are quite a few merits to single blind review (for a few other takes, see here and here). I won’t presume to cover them all here, but i will get a start. Feel free to add others, or tell me i’m completely off my rocker in the comments.

The most compelling aspect as i see it (and what i posted initially in response to the FB thread) is competence of reviewers. Academia is a really small world. Within particular areas of expertise its even smaller. Within those small worlds the cycles of information flow are much faster than review processes. So it’s common, even expected, that if I have the expertise to adequately judge the merits of a paper, I’ve a decent chance of coming across it before being asked to review it (as a conference presentation, working paper, etc.). Should my expertise to review the paper then be sacrificed on the alter of ‘double blind’ review? Last year, of the ~dozen reviews I wrote, I’d estimate I knew the authors of half upon receiving the request (no googling involved). There are journals who in the interest of double blind review in their request procedures would have actually asked me to recuse myself in those cases, which I think does more to exclude the appropriate reviewers from the pool than it does to maintain meritocracy in the process.

Another important facet is the reality that science is a socially produced product.* All too often, i think the air of objectivity surrounding scientific writing attempts to remove this reality from view. People make decisions while conducting science that are an important part of the things they “discover” in this process. Pretending we’re mere measurers and note-takers of the “out there” reality obfuscates these decisions in a way that i think is quite intentional. I think putting scientific writing into its fuller context helps give reviewers/consumers the ability to more appropriately evaluate its merits. Bringing the researcher back into the writing/reading/evaluating of it is one step in that direction.

Another that’s closely related is that (virtually) all scientific papers are part of a larger scientific project. Aspects of that “larger” story are often key elements to fully understanding what’s going on in any one “digestible unit” (i.e., single paper). So i say, let’s allow for and even highlight that, rather than presuming otherwise.

For now a final point that i saw elsewhere on the web this morning, but have definitely encountered on my own before is attribution of ideas. Sometimes, i see really good ideas while reviewing in papers that ultimately get rejected. If i want to build on those ideas later, but don’t want to claim them as my own. What to do if double blind? Some would make the case to wait to build on them till they find their way into publication form, and there’s some sense to that, but that’s not always entirely feasible.

Obviously, i have not exhausted this question by any means, and hope this opens a discussion…but those are a few of the key pieces i see as for single-blind review.
_____
*There’s a reading of this version of events that suggests that scientific advance is therefore all relative and lacks any semblance of objectivity. That’s not the version i’m offering here. This is a much larger discussion than i want to engage for now, but i do think that there are scientific truths (with a capital T even) out there that we do come to knowledge of during the scientific enterprise, i just don’t think that ascribing “detached observer” status to the scientist role really helps us all that much in getting there. In fact, i think it can be a substantial impediment.

Advertisement