They are questions so big, so incredibly complex, that until now scholars had not even attempted to answer them: What historical event of the past 250 years most profoundly shaped US political philosophy? Is democracy good for economic growth? Can we predict outbreaks of gang violence? At Columbia’s Data Institute, teams of sociologists, economists, social workers, and data scientists, using computational methods of their own design, have tackled these questions. What they have discovered may overturn long-held academic dogmas, and even save lives.
A New Take on the Great War
No historical record may capture America’s changing political consciousness better than the president’s State of the Union address. These documents, issued annually since 1790 with little variation in form, have been studied extensively by scholars. But this year, a team of researchers led by Columbia sociologist Peter Bearman, the director of the Interdisciplinary Center for Innovative Theories and Empirics, gained a valuable new perspective on the archive. Berman, along with sociology PhD student Alix Rule '13GSAS and physicist Jean-Philippe Cointet of the University of Paris, developed an innovative text-analysis program that examined all 1.8 million words in the 227 speeches, identifying the specific nouns that US presidents have used when discussing topics like foreign policy, trade, taxes, employment, immigration, and military budgets. The researchers then looked to see when new ways of framing these issues gained influence, as evidenced by a president’s lexicon being adopted by his successors.
No matter how the researchers sliced the data, 1917 stood out as the point at which the most pronounced long-term changes in political discourse occurred. Woodrow Wilson’s speech to Congress in December of that year, delivered eight months after the US declared war on Germany, was more focused than previous State of the Union addresses on promoting an interventionist foreign policy, industry regulation, and the financing of large-scale public programs — ideas that US historians today consider core tenets of American political thought. Wilson’s shift away from the more isolationist and laissez-faire policies of his predecessors was clearly precipitated by World War I; he spoke at length about the need for Americans to aid their European allies and to grant the federal government the authority to regulate agriculture prices and control the nation’s railroads and waterways in support of the war effort. Bearman’s and his colleagues’ insight is to show that large numbers of words and concepts from Wilson’s speech crop up in the addresses of subsequent US presidents. (The words “democracy,” “united,” and “peace,” become mainstays of foreign-policy discussions beginning with Wilson, for example.)
“We know what constitutes modern political thinking, but until now have been unable to say exactly when it originated,” says Bearman.
The new study is important because historians have long debated whether World War I left an indelible mark on American political philosophy; some scholars have argued that it wasn’t until the aftermath of the Second World War, when the US proposed its Marshall Plan to rebuild Europe, that Americans became believers in big government and nation-building. In fact, Bearman and his colleagues’ analysis shows that the 1950 State of the Union address, in which Harry Truman trumpeted the success of the Marshall Plan and warned of the impending threat of Communism, was the second most influential in US history.
“When you look at historical texts as one long, continuous river of words and ideas, you’re able to spot patterns that you wouldn’t see otherwise,” Bearman says. “We believe that our methods could be useful in analyzing other large historical corpuses, whether the Atlantic slave-trade records or the Proceedings of the Old Bailey.”
Curbing Gang Violence on the Toughest Turf: Facebook
Members of street gangs have been insulting and threatening one another with increasing frequency on Facebook and Twitter, a trend that law-enforcement officials say is contributing to a surge of gang-related violence in US cities. Desmond Patton, a Columbia assistant professor of social work, is determined to fight the trend using big data. This past spring, he teamed up with computer scientists at MIT and social workers at the YMCA of Chicago to create a software application that will detect when violent threats appear in publicly shared Facebook or Twitter posts in the Chicago area. As soon as a threat is posted, the application will alert local social workers so that they can try to track down the involved parties and defuse the situation.
“A lot of gang violence is committed by kids as young as fourteen or fifteen who, because of peer pressure, may find themselves in situations where they feel they have no choice but to assault or shoot someone,” says Patton, who has been working with disadvantaged youths on the streets of Chicago for more than a decade. “If we can intervene quickly, we might be able to change their minds.”
Designing a computer program that is capable of identifying credible threats from among the millions of new Facebook and Twitter messages posted daily in a major city like Chicago requires an intimate knowledge of gang members’ communication styles. Patton is an expert on the topic; over the past two years, he has interviewed dozens of current and former Chicago gang members about the codes they use to signal their intention to harm someone. He has learned, for example, that serious threats often contain the street address of a targeted person — typically with the name of the street spelled backwards — and a photograph of the person flipped upside down.
“We’ve tried to catalogue these types of contextual clues so that the software will distinguish between a real threat and, say, a quoted rap lyric,” he says.
A team of MIT researchers led by Henry Lieberman will soon integrate Patton’s findings into a text-analysis program to be used in Chicago on a trial basis. Patton and Lieberman say that if their tool helps to curb violence there, it could be adapted for other cities.
Does Democracy Pay?
Some economists say that poor countries are likelier to achieve prosperity under dictators than democratically elected leaders. Elected officials, they argue, are unable to act fast in crises, to implement unpopular reforms that may be necessary for long-term growth, or to resist signing off on partisan pork-barrel projects.
To test this theory, an interdisciplinary team of Columbia, MIT, and University of Chicago researchers analyzed a large set of economic and political data for 175 countries, covering the period 1960 to 2010. Their results, published recently in a white paper on the website of the National Bureau of Economic Research, suggest that democracy isn’t bad for business at all. In fact, the researchers found that countries that transitioned from an autocratic form of government to democracy achieved a 20 percent higher GDP per capita, on average, in the following twenty-five years. One potential explanation, they say, is that democracies invest more money in education and health care — a fact established by several previous studies — and thereby help their citizens to be productive.
“These investments appear to more than offset any inefficiencies that democracy can theoretically engender — like wealth redistribution and political gridlock,” says Suresh Naidu, a Columbia assistant professor of economics who worked on the study.
Naidu and his collaborators say that by analyzing the development trajectories of all the world’s countries over a long period, they were able to control for variables that have distorted the results of previous, smaller studies. For example, they were able to account for the temporary dips in GDP that countries usually experience when transitioning from one form of government to another; they also controlled for the fact that poorer nations tend to grow quickly, whereas countries further along the path to prosperity tend to grow more slowly.
“Having a large data set enabled us to home in on the precise question we wanted to answer: did democracy pay off, on average?” says Naidu. “In almost every case, we found that it did, big time.”
