You have probably heard the phrase “data is not the plural of anecdote,” meaning that evidence which has been gathered systematically is far more valuable than a collection of individual stories.
I certainly have heard it, and I have used it myself, but I didn’t realize until recently that there’s a new-ish department at Dartmouth College where the phrase is front and center.
“Anecdotes are useful but it’s difficult to know how indicative they are of a broader population trend, or whether they’re unique to the 20 people we talked to in the focus groups,” said Dr. Elsa Voytas, an assistant professor in the Department of Quantitative Social Science.
This department, which is just a few years old, takes math classes (starting with stats, my least favor math class but the one with the most real-world impact) and courses in data, computing and even game theory, then combines them with gnarly topics like “race, incarceration and politics” and “sociology of mental health” to create a specialized bachelor’s degree.
In essence, the department wants to help researchers tackle what I think of as squishy subjects. These are the sort of topics that engineers and physics majors look down their noses at – developing global peace, for example, as compared to developing global energy sources – but which in the long run are the most significant.
Consider Voytas’ specialty: Supporting reconciliation in post-violence settings. Since every single group in the history of the planet has fought with at least one other group and is still angry about it, it would be really useful to know how to help folks reconcile their differences and get along.
“The biggest issue is that we don’t know how policies that are widely implemented are working … No one’s looked at the impact on individuals who receive them, what’s the impact on the broader public,” Voytas said.
Voytas described her research around Greenwood Rising, a museum and memorial about the Tulsa Race Massacre of 1921 when angry mobs of white people destroyed one of the nation’s most successful Black communities, slaughtering people and burning buildings. Greenwood Rising is an attempt to help everybody come to terms with a horrid, shameful moment in our past, first by forcing us to acknowledge it (it was never mentioned in any of my American History classes) and then helping people have conversations about it.
Does this work?
Voytas’ team had focus group conversations with visitors – “that’s not particularly quantitative but sometimes quantitative data doesn’t show us that much unless it’s informed by conversations with real people” – and used them to design a bigger project about attitudes toward racism and racial justice, based on close-ended questions on a survey for people before and after visiting the museum.
The result: “We didn’t find a big effect (from visiting) the museum.”
So the project is a waste? Not necessarily. “This is informative, too. It may show that people choosing to visit a museum like this already have attitudes in line with the museum,” she said. It indicates that the organizers need to move the message outside of the museum, through school visits or social programs or videos or something else.
Even then, it might not do much. But it might, and the benefits are so huge we need to try.
That unsatisfying conclusion is, alas, typical of social-science problems, and explains why hard-science folks shy away from them. There are almost never clean, straightforward solutions when you’re dealing with the interaction of many human beings, at least not in the way you get solutions when dealing with the interaction of electrons or solvents. The department tries to bridge the gap by mixing the quantitative and qualitative.
“Some classes are about analyzing data in a careful way, asking questions about that data, what kind of biases it might contain,” said Voytas. “That’s a massive part of the job, really – figuring out strategies to collect data, encouraging entities to try to collect data if they’re not. … what are the limitations in the process that generated the data; what are they capturing, what are they unable to capture? Are there other sources we can collaborate this data with, get a better sense of the situation?”
That point about “what are they unable to capture” is also key. We have a strong incentive to concentrate on things to which we can attach numbers and as a result, can miss what matters.
That is the reason macroeconomics is taking the world down the wrong path: It focuses almost exclusively on the dollar value of things, ignoring squishy stuff like happiness, the environment and social stability because we don’t know how to quantify them.
If places like the Department of Quantitative Social Science can help change this attitude, we might have a chance. Although I think they should have called it Department of Nailing Down The Squishy Stuff – that would probably draw more undergrads.
Return to the Concord Monitor