How to talk about Association

Statistician and political scientist Andrew Gelman recently offered some thoughts on how to talk about associations that could be causal.  In my opinion, even when we limit ourselves to high quality scholarship, some work offers far more evidence of causality than other work.  The evidence for this claim, and the consequences which follow from it, should be the topic of much future research (and blog posts).  In our research, many of us want to make claims that sound like, e.g. “on average, an hour of studying improves final exam scores by 5%,” which we might consider, “a strong effect of studying on test scores.”  When is this causal language justified?  First of all, I think every paper needs to address potential threats to causal interpretations.  Randomized controlled trials, and natural experiments, have the best claim to proving causal relationships – they clearly justify the causal language above.  But with appropriate qualifications, I think a paper using propensity score matching/stratification, and in many contexts, plain old regression techniques (especially, e.g. diffs-in-diffs) can justify the use of causal language.  The truth is, the devil is in the details.  In general, I think we sociologists could be a little more careful in our use of causal language.  Of course, causality isn’t everything.  How to weigh the importance of demonstrating causality versus other important goals in our research is a very difficult question.

Paul Krugman on Metaphors and Models

Paul Krugman has spent a lot of time recently criticizing economists associated with the Chicago School because he believes they are making horrible public policy recommendations and, importantly, misusing mathematical models, e.g. the oft criticized, dynamic stochastic general equilibrium models. But Krugman has written a lot that is equally interesting, and less political.

Today I’d like to draw your attention to an article he wrote in 1994 entitled, The Fall and Rise of Development Economics.  Its absolutely worth reading the whole thing, but I’ll summarize and excerpt a huge chunk from the middle of it for those of you who are really pressed for time.  Krugman tells us that rigorous modeling is very important for economics, but he also argues that the new (1950s and 1960s) emphasis on modeling led people to forget or ignore things people knew about development economics for years, until people figured out how to model them.

After the break I’m pasting the middle section of the essay, essential reading on metaphors and models:

WWPD?

(What would Popper Do? Or perhaps more accurately, How would Popper feel?) An interesting thread on “identification” over at OrgTheory.

From the comments on the intent of the intial post: “What I [Fabio Rojas] am trying to get at is that there is a temptation to dump theory for ever more sophisticated hypothesis testing.” To me, a concern definitely worth heeding.

Mathematical models: why and what for?

An anthropologist friend of mine, Eli Thorkelson, recently asked for my comments on an article by feminist economist Julie Nelson. The article is a critique of rational choice theory (RCT) and I think it has some glaring omissions and misleading claims, but before I get to them I want to briefly offer the positive case for mathematical models, of which rational choice models are a subset.*

Markets and other social phenomena are complex.  We’re never going to have a theory that perfectly describes everything we might want to know about them.  Modeling is one way we can get some insight.  In a model we make some simplifying assumptions and then work out their implications.  Then, and this is key, we do empirical work; we analyze data to see how well our model captures various aspects of the phenomena we are interested in.  Ideally a  model will make predictions that are novel, in the sense that the researcher would not have made them before studying the model, and in the sense that they distinguish one model from another in a way that can be tested rigorously.

What about those simplifying assumptions?  If they turn out to be false, doesn’t that mean the model is junk?  

it’s not the end of the world as we know it, just yet

While hanging out at Columbia the past couple of years, i was comforted to realize that i wasn’t alone in my lack of enthusiasm for p-values. In fact, while some folks in some disciplines are completely infatuated with them, did you know there are others out there that dismiss them? i mean, entirely!? [removes tongue from cheek]

While it might be an overstatement to suggest that the variety of headlines that accompanied the news of the HIV vaccine trial in Thailand claimed that “AIDS is now a disease of the past,” it really wouldn’t be stretching the tone of some of those headlines too far. Some people really did seem to think this was the breakthrough we’d been waiting for. That is, those people who focused solely on the p-values. E.g., from one of the NYT pieces on it “Although the difference was a mere 23 people…it was statistically significant.”

But what most people seemed to fail to pick up on was the very next part of that sentence –

My Old Book Review of Six Degrees

I wrote a review of Duncan Watts book on social networks, for a class and thought I might as well share it here, even if it is a little out of date:

Networks are everywhere.  In the first chapter of Six Degrees Duncan Watts notes that gossip, power outages, epidemics, even properties of the human brain such as consciousness all emerge from the interaction of their constituent elements.  Having provided this motivation, Watts spends much of first half of the book discussing what he knows best, “small world” networks.  In the second half he presents a network perspective for a wide range of topics such as…

CMU Machine Learning & the G-20

Oh my. (ht KD)

“Repeal Power Laws” has to be my favorite. Though “Basyesians against Discrimination” is pretty good too.

is my DNA making you make me quit smoking?

With this, my first post here, i suppose i should briefly also introduce myself. i am an Assistant Professor of sociology in the School of Social and Family Dynamics at Arizona State University.

So, i’ve been puzzling for a while over my thoughts on the numerous papers that have come out over the past few years by Christakis, Fowler et al using the Framingham Heart Study data. If you didn’t catch the story that recently ran on their work in the New York Times Magazine, i’d highly recommend taking the time to make your way over there at some point. The article does a pretty good job of summarizing what they’ve published and raising some of the pertinent questions that have been posed previously (both in print and not).