What are the currently under-appreciated research areas in sociology? I have my opinions, but I’d actually prefer to focus your attention on the meta-question: how might decide which research areas are under-appreciated?
There are two basic approaches, inside views and outside views. Inside views require understanding a phenomenon in terms of its component parts. In contrast, outside views require the often easier task of appropriately categorizing the phenomenon, a.k.a. choosing a reference class.
For examples, let’s return to the original question, where the phenomenon under investigation is “the appreciation/under-appreciation of research areas in sociology.” An inside view might look like the following chain of (questionable) reasoning: 1. the highest goal of sociology is to discover the knowledge which results in the greatest good for the greatest number. 2. Insects are numerous. 3. Little sociological research is done on insects. 4. Therefore, Insect sociology is under-appreciated.
An outside view might look like this:
1. It has been proven that, in the past, research areas more popular with women have been the most under-appreciated. 2. Therefore, its a good bet that research areas more popular with women are currently the most under-appreciated, even if I have no idea what those research areas are or why they are valuable.
My inside view and outside view arguments are pretty weak, they are just there to illustrate the concept of inside and outside views. Can you improve on them or make a case for inside/outside approaches more generally?
Permutations 
Hi Mike,
Not quite sure I understand your distinction between “inside view” and “outside view.” Does ‘inside view’ mean “judged according to criteria that are (socially or conceptually) internal to the object” while ‘outside view’ means “judged according to criteria that are external to the object”? So it would be a question of “appreciation according to disciplinary criteria” vs “appreciation according to extradisciplinary criteria”?
OK, actually I just looked at the wikipedia article you mentioned, and I guess what you mean by ‘outside view’ has to do with trying to make predictions for (assessments of) some X in relation to previously observed events in other members of some set Y that gives us the relevant ‘reference class.’ (I guess the most obvious reference class here would be the set of other disciplines.)
On reflection, my basic thought is that we would benefit from going a bit beyond the terms of the initial question about under-appreciated research areas. I’m not sure there can be any objective fact of the matter about which research programs are under-appreciated, because any judgment of underappreciation presupposes a judgment of the value of a piece of research, and one can only judge the value of academic work in relation to (a) a set of disciplinary standards and (b) a given historical and political context. And neither of these can be taken for granted. For example, the very people who have atypical views about disciplinary standards of research evaluation may be the very ones whose work is under-appreciated. This all leads me to think that the real question here isn’t “how do you identify under-appreciated research questions,” but rather “what social processes of legitimation determine which research is widely acceptable or narrowly acceptable, and moreover, what social processes of disciplinary differentiation *produce* disagreements over research standards?” You know what I mean? We would need an adequate sociology of sociology to really settle these questions, it seems to me.
Eli, I probably didn’t do the greatest job of pinning down the concepts of inside and outside views. The concepts are not that new, but nor is that terminology completely standard. [Mike googles…] Here’s a good blog post that I probably read long ago: http://www.overcomingbias.com/2007/07/beware-the-insi.html It contains the earliest reference to it that I’ve seen: Kahneman and Lovallo (1993).
I brought up the concept of inside and outside views because I think they can help people think about a whole range of issues, and I’ll probably reference it in future blog posts. Under-appreciated research was just one example that I thought might catch people’s interest. I hope that example makes sense regardless of what definition of “under-appreciated research” we use, but I’ll try to flesh out the idea a little more. If I say research area X is under-appreciated in sociology, I think that the world would be a better place if, on the margin, sociologists paid more attention to research area X. To the extent I am rational, I have reasons (other beliefs) that support my belief that X is under-appreciated. For example, I might believe that more attention to X (and necessarily less attention to something else) would decrease racism.
Even if judging the value of particular research is always subjective, and no objective fact of the matter can ever be proven decisively, it seems to me that my subjective judgment should respond to learning information about objective facts. If I learn something that reduces my belief that research on X reduces racism, then, ceteris paribus, I must reduce my belief that research on X is under-appreciated by sociologists.
Hi Mike,
The inside/outside view distinction seems to map in interesting, complicated ways onto other distinctions we might use to make judgements about research (intellectual vs sociopolitical criteria for instance). If I understand the Overcoming Bias post right, Hanson believes that, everything else being equal, “outside views” are generally more likely to be successful in predicting the course of events? It seems to me that there’s something deeply unspecified in the distinction between “inside” and “outside” views, though, which has to do with *how you pick the reference class*. It seems to me that you already have to have something like an “inside view” to know which reference class to pick, because without understanding the case in question, you don’t know which comparison cases would be relevant. Suppose we want to predict the next presidential election — surely we’d need to look at the qualities of the current candidates, the general political situation, the economy, the military, the changing cultural tendencies of the American electorate, etc, *just to decide which previous elections would constitute the relevant reference class.* In other words, I’m not sure we have an either/or choice between inside and outside views; it seems hard to have the latter without the former (I’m not sure if it works the other way too).
About under-appreciated research, I find it ironic that I, the non-sociologist, am the one arguing that we need a more sociological investigation of scientists’ opinion formation, while you are the one arguing that we can or at least should try to approximate some kind of rational ideal! Anyway, my point isn’t that I necessarily disagree, myself, with your suggested standard for “under-appreciated research” (though it could still stand some clarification); my point is just that standards of evaluation differ within the discipline, and that seems like a phenomenon that deserves its own investigation. You know?
Yes, I agree (and think its quite perceptive of you to notice) that a person’s outside view is necessarily influenced by their inside view and vice versa. Choosing a reference class can be difficult, there is often room to argue about which is the appropriate one. Inside/outside could be considered a spectrum, with larger/broader reference classes being more outside.
It isn’t true that the broader reference class (outside view) is always better, but it has been demonstrated that there are a lot of situations in which people make systematically biased predictions which are improved by using an outside view.
For example,
http://en.wikipedia.org/wiki/Planning_fallacy
When no single reference class is obviously best, we might do some sort of weighted average of different reference classes, or bring in more of our inside view.
Finally, I’d love to investigate scientists’ opinion formation and should have said so. I just wanted that to be seen as a complement to, rather than a precursor for, forming judgments about which research is under-appreciated. It is, perhaps, somewhat impolitic to talk about about it explicitly, but we make important decisions based on implicit judgments so I figured it deserved some attention.
I think the idea of having multiple reference classes and comparing or averaging is very interesting; I’d love to hear examples of that. But anyway this is just to say that I pretty much agree with your more recent comment and I think that in this post, at least, our opinions are approaching the convergence point!
[…] than it often gets) but Andrew focuses on points of disagreement (e.g. do statisticians drastically overvalue difficult but less useful research). Highly […]