A couple of days ago I wrote about a six-year old statistical controversy in political science (in my defense, it was new to me!), where Jesse Richman and his collaboraters used a massive online survey–the Cooperative Congressional Election Study (CCES)–to argue that non-citizen voting in the 2008 and 2010 elections in the United States was non-trivial.
Ansolabehere et al. Reply
The CCES principal investigators, led by Stephen Ansolabehere, wrote a reply. They were able to investigate measurement error in the citizenship status variable in the 2010 data since respondents were followed-up in the 2012 wave. They show that a non-trivial number of people who said they were non-citizens in 2010 indicated they were citizens in 2012 (36/121), which is not a big deal as they may have naturalized. But, a non-trivial number of people who said they were non-citizens in 2012 indicated they were citizens in 2010 (20/105) which is really worrisome; people losing their citizenship while still living in the United States is probably quite rare. Ansolabehere et al. argue that these individuals are assuredly citizens who happened to misreport their citizenship in 2012.
It gets even worse for Richman et al. Among the respondents whom we are confident are really non-citizens, since they consistently reported their status in 2010 and 2012, none reported voting in 2010. Among the 141 respondents who indicated they were non-citizens in either the 2010 or 2012 waves, only four of them were validated as voting in 2010, and all four inconsistently reported their citizenship status. Three were the assuredly fake “non-citizens” (who initially self-reported as a citizen in 2010 and then switched to self-reporting as a non-citizen in 2012). The other “non-citizen” whose 2010 voting participation was validated, said they were a non-citizen in 2010 and then said they were a citizen in 2012. Although it is possible this person illegally voted as a non-citizen in 2010 and then naturalized by 2012, Ansolabehere et al. infer it is much more likely they were actually a true citizen in 2010.
Richman et al. Rejoinder
Richman and his colleagues have a response (which was written three years ago and is apparently not going to see print). First, they take issue with Ansolabehere et al.’s overstated conclusion that non-citizen voting is zero and complain about Ansolabehere’s lack of power with the 85 individuals who consistently self-reported as non-citizen in 2010 and 2012.
Second, Richman push back on the idea that non-citizens who have validated votes are really just citizens missclassified–they look at immigration attitudes in the 2012 CCES and show that the 32 non-citizen validated voters are quite similar to the 263 non-citizen validated non-voters, and both groups are quite different from citizens.
[A major argument Richman et al. make that I am glossing over is about Ansolabehere et al.’s estimate of the reliability of the citizenship status variable. Ansolabehere et al. assume that non-citizens and citizens are equally likely to misstate their status. Since citizens are a much bigger group than non-citizens, a person with an inconsistent status is much, much more likely to truly be a citizen than a non-citizen. Richman et al. argue that non-citizens are probably more likely to misstate their citizenship status, although this argument seems premised on the idea that voter fraud is widespread among non-citizens which I don’t think Richman et al. has established. I think that if you buy Richman et al.’s argument, then the implication is if one looks at people with inconsistent citizenship statuses–namely those initially self-reporting as citizens in one wave and self-reporting as a non-citizen in the next wave–half are true citizens and half are true non-citizens].
Third, Richman et al. look at voter registration and find among respondents who were consistent self-reported non-citizens in 2010 and 2012, 5 out of 47 (10.6%) had validated voter registrations in 2012 and 1 out of 47 (2.1%) were validated voters in 2012; among respondents who were consistent self-reported non-citizens in 2010, 2012, and 2014, 1 out of 16 (6.3) had validated voter registrations in 2014 and 0 were validated voters in 2014.
My take
I have a couple of comments about this debate.
First, as someone who has zero exposure to the CCES data, I was frustrated by both sides in not clearly explaining why they were examining some waves and variables but not others. For example, I was wondering why Ansolabehere et al. did not look at validated voting in the 2012 wave (as Richman et al. did in his rejoinder). One does wonder about the issue of cherry-picking.
Second, in my previous blog post I expressed frustration with the original Richman et al. article because it was not clearly conveying its analyses, and I especially felt this way with the Richman et al. rejoinder. Apparently, I am not the only one who has this reaction to Richman’s arguments.
Third, in the original Richman et al. piece, 27 and 13 so-called non-citizens reported they voted in the 2008 and 2010 elections, respectively. Richman et al. use these quite small samples to estimate levels of Democratic voting nationwide among non-citizen voters, and then assume these generalizations applied to non-citizen voters in specific contests (e.g. Minnesota in 2008). They did not ever acknowledge sampling variability in this estimate. In their rejoinder to Ansolabehere et al., they make a cheeky comment about how they were forced by Ansolabehere et al. to engage in analyses of small analyses: “If their critique, based as it is on such small samples, has any validity, then our response mu[st] join it on this terrain” (p. 10). In reality, Richman et al. were the first to bring us to this terrain and for them to criticize Ansolabehere et al. for power issues seems like a case of the pot calling the kettle black.
Fourth, the comparison of 2012 immigrant attitudes of non-citizen voters, non-citizen non-voters, and citizens was probably the best evidence in favor of Richman et al.’s point of view (although I wish they broke out the citizens into voters and non-voters to have clearer comparisons). I wonder why they did not carry out such analyzes in the 2010 data–their original article actually used the 2010 data on immigration attitudes, but without a citizen/non-citizen voter/non-citizen non-voter comparison.
Fifth, I have to reiterate the point of my original blog post. Richman et al. want to chalk up measurement error in the citizenship variable to non-citizens wanting to cover up their illegal participation in the U.S. electoral system. It just raises issues about the generalizability of the CCES sample of non-citizens to the population of non-citizens in the U.S. If a non-citizen has illegally participated in the electoral system, and they feel compelled to have to lie about their citizenship status on a survey, why are they even participating in a survey about voting behaviors in the first place? My point is not to deny it has ever happened–I am sure it must have–but that the people doing this must be pretty unusual. Richman et al. made a case that non-citizen voting was largely driven by ignorance as education did not predict non-citizen voting as it does for citizen voting. A very logical inference is that the CCES is drawing on a select group of non-citizens combining high levels of ignorance and political interest, and again my suspicion is that the sample is systematically overestimating non-citizen voting.