ISERP Working Papers - Fall Issue
The following initiative showcases the working papers of Columbia's faculty in the Social Sciences departments. The papers have not been peer reviewed. To add your voice to the discussion, please submit your paper to iserp-communication@columbia.edu, or upload it here. Thank you to the faculty who have submitted a working paper for our first bulletin, we are hoping this initiative will tighten the ties of our research community.
To read and download any of these working papers, click on the paper, or visit our new site: ISERP's Working Papers. This site is UNI-protected, and only Columbia and Barnard affiliates have access to the full text.
To receive subsequent bulletins, please sign up here.
Please see below our summer issue of the Working Papers Bulletin. To view the spring issue of our bulletin, please click here.
Topics covered in this bulletin:
Trust Methods: Accounting for Who, What, When and How to Trust
Larry Au, Sociology, Cristian Capotescu, Trust Collaboratory, Gil Eyal, Sociology, Sophie Sharp, Sociology
What is trust and how should it be studied? In this paper, we argue against conventional approaches to studying trust in the social sciences and propose an alternate strategy focused on “trust methods”. Instead of treating trust as a static property that can be measured by close-format survey questions, we conceptualize trusting as a skillful act that is highly context-dependent and attuned to temporal variables such as speed, duration sequence, and timing. To illustrate this approach, we draw on interviews with Long Covid patients focusing on how they account for what makes trust in “my own research,” “other patients”, and “my doctor” responsible rather than blind faith.
keywords: Trust, Expertise, Ethnomethodology, Long-Covid
Confronting Core Issues: A Critical Test of Attitude Polarization
Yamil Ricardo Velez, Political Science, Patrick Liu, Political Science
A long-standing debate in political psychology considers whether individuals update their beliefs and attitudes in the direction of evidence or grow more confident in their convictions when confronted with counter-attitudinal arguments. Though recent studies have shown that instances of the latter tendency, which scholars have termed attitude polarization and “belief backfire,” are rarely observed in settings involving hot-button issues or viral misinformation, we know surprisingly little about how participants respond to information targeting deeply held attitudes, a key condition for triggering attitude polarization. We develop a tailored experimental design that measures participants’ positions regarding their most important issues and randomly assigns them to different mixtures of personalized pro-attitudinal and counter-attitudinal information using the large language model GPT-3. We fail to recover evidence consistent with attitude polarization across three studies. We conclude by discussing implications for the study of political cognition and the measurement of attitudes.
keywords: Attitude Polarization, Beliefs, Political cognition, Misinformation
Making Sacrifice: How Normative Control Is Enacted through Rituals of Integration
James Y. Chu, Sociology
Organizations frequently rely on robust norms to motivate and coordinate employee action, but how certain values and beliefs become authoritative organizational norms remains obscure. Drawing on fieldwork in a recently founded not-for-profit school, I show how normative control is enacted through a ritual of integration: a sequence of annual, collective activities that draws actors external to the manager-employee dyad. Across four stages, teachers were drawn into collective and emotionally charged interactions with parents, children, and each other. Sequential participation in these activities produced a robust norm of sacrificing oneself for the sake of children, which appeared to be shared by a collective of actors that transcended the boundaries of the organization. By controlling processes and outputs around this ritual, such as the timing of these stages and who would participate, the founder promoted her vision without being perceived as the author of this norm. Taken together, these results further our understanding of how normative control is created, how perceptions of meaningful work emerge through interaction with beneficiaries, and the synthetic potential of organizational rituals in integrating different types of actors and controls.
keywords: Normative Control
Political Behavior and Democratic Attitudes
James Y. Chu, Sociology, Jan G. Voelkel, Sociology, Stanford University, Michael N. Stagnaro, Sloan School of Management, Massachusetts Institute of Technology, Suji Kang, Political Science, Northwestern University, James N. Druckman, Political Science, Northwestern University, David G. Rand, Sloan School of Management, Massachusetts Institute of Technology, Robb Willer, Sociology, Stanford University (Lead authors)
Concern over democratic erosion has led to a proliferation of proposed interventions to strengthen democratic attitudes in the United States. Resource constraints, however, prevent implementing all proposed interventions. One approach to identify promising interventions entails leveraging domain experts, who have knowledge regarding a given field, to forecast the effectiveness of candidate interventions. We recruit experts who develop general knowledge about a social problem (academics), experts who directly intervene on the problem (practitioners), and non-experts from the public to forecast the effectiveness of interventions to reduce partisan animosity, support for undemocratic practices, and support for partisan violence. Comparing 14,076 forecasts submitted by 1,181 forecasters against the results of a mega-experiment (n=32,059) that tested 75 hypothesized effects of interventions, we find that both types of experts outperformed members of the public, though experts differed in how they were accurate. While academics’ predictions were more specific (i.e., they identified a larger proportion of ineffective interventions, and had fewer false positive forecasts), practitioners’ predictions were more sensitive (i.e., they identified a larger proportion of effective interventions, and had fewer false negative forecasts). Consistent with this, practitioners were better at predicting best-performing interventions, while academics were superior in predicting which interventions performed worst. Our paper highlights the importance of differentiating types of experts and types of accuracy. We conclude by discussing factors that affect whether sensitive or specific forecasters are preferable, such as the relative cost of false positives and negatives, and the expected rate of intervention success.
keywords: Democratic attitudes, political beliefs, academics
Megastudy identifying effective interventions to strengthen Americans’ democratic attitudes
Jan G. Voelkel, Department of Sociology, Stanford University, Michael N. Stagnaro, Sloan School of Management, Massachusetts Institute of Technology, James Y. Chu, Sociology (Lead authors)
Deep partisan conflict in the mass public threatens the stability of American democracy. We conducted a megastudy (n=32,059) testing 25 interventions designed by academics and practitioners to reduce Americans’ partisan animosity and anti-democratic attitudes. We find nearly every intervention reduced partisan animosity, most strongly by highlighting sympathetic and relatable individuals with different political beliefs. We also identify several interventions that reduced support for undemocratic practices and partisan violence, most strongly by correcting misperceptions of outpartisans’ views – showing that anti-democratic attitudes, although difficult to move, are not intractable. Furthermore, both factor analysis and patterns of intervention effect sizes provide convergent evidence for limited overlap between these sets of outcomes, suggesting that, contrary to popular belief, different strategies are most effective for reducing partisan animosity versus anti-democratic attitudes. Taken together, our findings provide a toolkit of promising strategies for practitioners and shed new theoretical light on challenges facing American democracy.
keywords: Democratic attitudes, Democracy, political beliefs
Affirmative Action and Educational Ranking at University
Affirmative Action in Centralized College Admissions Systems
Nano Barahona, Economics, UC Berkeley, Cauê Dobbin, Economics, Georgetown University, Sebastían Otero, Economics
We study the consequences of affirmative action in centralized college admissions systems. We develop an empirical framework to examine the effects of a large-scale program in Brazil that required all federal institutions to reserve half their seats for socioeconomically and racially marginalized groups. By exploiting admissions cutoffs, we find that marginally benefited students are more likely to attend college and are enrolled at higher-quality degrees four years later. Meanwhile, there are no observed impacts for marginally displaced non-targeted students. To study the effects of larger changes in affirmative action, we estimate a joint model of school choices and potential outcomes. We find that the policy has impacts on college attendance and persistence that imply a virtually one-to-one income transfer from the non-targeted to the targeted group. These findings indicate that introducing affirmative action can increase equity without affecting efficiency.
keywords: Affirmative Action, college admissions, equity, education
The Price of Prestige: Educational Rankings and Adolescent Perceptions of College Cost
James Y. Chu, Sociology
Public metrics like rankings and ratings are routinely criticized for imposing uniformity in how people think and behave, but rankings may also generate divergent outcomes when people interpret them differently. Here, I investigate the inequality implications of this claim by studying how educational rankings affect adolescent perceptions of college cost, and how these effects vary by socioeconomic status background. I propose that those from lower socioeconomic status backgrounds perceive rankings as a stronger signal for cost than their more advantaged peers, leading them to overestimate the cost of opportunities that they might otherwise benefit from. To test this claim, I conducted a conjoint survey experiment among a diverse sample of U.S. adolescents (n=800). Continuing-generation adolescents underestimate the cost of prestige, while their first-generation peers overestimate how expensive highly ranked schools will be. First-generation students also do not interpret rankings as strong signals of financial aid, even when prestigious colleges offer more aid. Differences in perceived net cost mediate college choices, with first-generation students are three times less likely to prefer top ranked colleges, even when told the college has admitted them. These results imply that the public metrics we use to establish consensus about the prestige of colleges may also be leading those from traditionally disadvantaged backgrounds to overestimate the costs of attendance, potentially sustaining the high degree of stratification observed within the higher education system today.
keywords: education, first-generation, socioeconomic inequalities, rankings, university
Learning, Teaching, and Grading in Economics
Improving Learning Outcomes in Introductory Level Econometrics.
Seyhan Erden, Economics
This paper presents the findings of 12 years of data from studying the teaching of econometrics. The first course on the topic of econometrics has always been a challenging course for both students and instructors. Students come from different quantitative backgrounds, and mostly with the prejudice that this is one of the most challenging courses in their academic career. We showed that using Classroom Response Systems (CRSs) such as polls closes the achievement gap between students from higher and lower quantitative levels. Besides students’ performance, we also investigated instructor performance through teaching and course evaluations utilizing data from 38 classes over the course of 12 years. We showed that the instructor performance is higher under the in-class modality compared to the online modality and showed that this gap in performance between the two modalities widens as students’ grades improve; a positive association between grades and instructor performance under the in-class modality exists. However, the association is negative under the online modality.
keywords: Teaching Econometrics, Teaching Pedagogy
Seyhan Erden, Economics
This study focuses on the relationship between undergraduate students' attitudes and performances toward Econometrics, a crucial subject in Social Sciences research. Despite its importance in the public and private sectors, many students have negative attitudes toward Econometrics, which can impede learning. This research utilizes a questionnaire to assess students’ attitudes toward Econometrics once at the beginning and again at the end of the semester and evaluates the effectiveness of methods employed throughout the semester in fostering positive attitudes. While a similar questionnaire was used in a study conducted in Spain with only an end-of-semester questionnaire, this research is unique in its analysis that combines the attitudes questionnaire with active learning methods and students’ performances in Econometrics courses.
keywords: Econometrics, Students’ Attitudes and Performance, Teaching/Learning Intro Level Econometrics, Innovative Teaching Methods
Grade Inflation and Grading Consistency in Principles of Economics
Wasseem Noor, Economics
This article tests for potential grade inflation and grading consistency in the Principles of Economics course at Columbia University. Examining grade data for over 11,000 students across 73 sections between 2010-2023, we do not find evidence of grade inflation, and we do find examples where grades consistently are given more leniently by some instructors and students in certain schools receive higher grades. Potential methods to address grade inflation are also provided.
keywords: Grade Inflation, Teaching, Economics
Advances in Estimation Methods
Estimating Separable Matching Models
Alfred Galichon, Economics and Mathematics, NYU Arts & Science, Bernard Salanié, Economics
Most recent empirical applications of matching with transferable utility have imposed a natural restriction: that the joint surplus be separable in the sources of unobserved heterogeneity. We propose here two simple methods to estimate models in this class. The first method is a minimum distance estimator that relies on the generalized entropy of matching introduced in Galichon and Salani ́e (2022). The second applies to the more special but popular Choo and Siow (2006) model, which it reformulates as a generalized linear model with two-way fixed effects. Both methods are easy to apply and perform very well.
keywords: matching, marriage, assignment, structural estimation
Andrew Gelman, Political Science and Statistics
A well-known rule in practical survey research is to include weights when estimating a population average but not to use weights when fitting a regression model—as long as the regression includes as predictors all the information that went into the sampling weights. But it is not clear how to apply this advice when fitting regressions that include only some of the weighting information, nor does it tell us what to do when analyzing already-collected surveys where the weighting procedure has not been clearly explained or where the weights depend in part on information that is not available in the data. It is also not clear how one is supposed to account for clustering in such analyses. We propose a quasi-Bayesian approach using a joint regression of the outcome and the sampling weight, followed by poststratifcation on the two variables, thus using design information within a model-based context to obtain inferences for small-area estimates, regressions, and other population quantities of interest.
keywords: models, inference, survey research, Bayesian approach
Artificial Intelligence and Aesthetic Judgment
Jessica Hullman, Computer Science, Northwestern University, Ari Holtzman, Computer Science, University of Chicago, Andrew Gelman, Political Science and Statistics
Generative AIs are objects that produce creative outputs in the style of human expression. We argue that encounters with the outputs of modern generative AI models are mediated by the same kinds of aesthetic judgments that organize our interactions with artwork. The interpretation procedure we use on art we find in museums is not an innate human faculty, but rather the result of a historical process involving disciplines such as art history and art criticism. This gives us pause when considering our reactions to generative AI, how we should approach this new medium, and why generative AI seems to incite so much fear about the future. We naturally inherit a conundrum of causal inference from the history of art: a work can be read as a symptom of the cultural conditions that influenced its creation while simultaneously being framed as a timeless, seemingly acausal distillation of an eternal human condition. In this essay, we focus on an unresolved tension when we bring this dilemma to bear in the context of generative AI: are we looking for proof that generated media reflects something about the conditions that created it or some eternal human essence? Are current modes of interpretation sufficient for this task? Historically, new forms of art have changed how art is interpreted, with such influence used as evidence that a work of art has touched some essential human truth. As generative AI influences contemporary aesthetic judgment we outline some of the pitfalls and traps in attempting to scrutinize what AI generated media means.
keywords: Artificial Intelligence
Slavery, Abolition, and the Revolutionary Era
Antislavery in the Age of Rights: Or, The Rights of Slaves Confronts the Right to Slaves
Christopher Brown, History
This paper considers the following questions. To what degree and in what ways did the language of rights figure in the antislavery campaigns at their inception during the American and French Revolutions? In what circumstances, and to what degree, did the recognition of rights or the expansion of rights figure among abolitionists’ aims? When rights did figure, what notions of rights did abolitionists have in mind? How did the principle of fundamental rights relate to the assertion of specific rights before constituted authorities in specific institutional contexts? And, finally, what difference did the attack on slavery make to broader understandings of rights in the revolutionary era? These are difficult questions to answer, and the current scholarship permits better responses to some them than others. Three points, though, may be made at the outset. The first concerns chronology. Moral objections to slavery long predated the age of revolutions. Critiques of colonial slavery date back to the first decades of European colonization itself and drew upon legal and cultural traditions that were older still. Second, rights talk constituted just one aspect of antislavery discourse in the late eighteenth century and, often, neither the most pervasive nor the most important. Third, rights played a significant role in the defense of slavery in the revolutionary era, indeed perhaps as much as in the campaigns against it. Slaves represented a form of private property in the Americas. Because, today, we regard the ownership of people as abhorrent, it can be easy to overlook the peril to property rights that antislavery represented, and understate the obstacles that an attack on those rights would face, even in the revolutionary era, especially in the revolutionary era.
keywords: Rights, Slavery, Antislavery, American Revolution, French Revolution
To read and download any of these working papers, click on the paper, or visit our site, ISERP's Working Papers. This site is UNI-protected, and only Columbia and Barnard affiliates have access to the full text.
To receive subsequent bulletins, please sign up here.