Quantifying police killings

One-third of all Americans killed by strangers are killed by the police.

Patrick Ball, Granta

…the Washington Post’s database of fatal police shootings showed 14 unarmed Black victims and 25 unarmed white victims in 2019.  

Heather MacDonald, USA Today

Someone turning to data to try to quantify the problem of police killings (or police brutality more broadly) would probably be frustrated by these statements which are not necessarily contradictory, but they differ in their general thrust.  One the one hand, Patrick Ball, writing in 2016, notes that police killings are so numerous they make up a good portion of “stranger homicides” that Americans give so much disproportionate attention to (as opposed to more common killings with the killer and victim know each other).  More recently, Heather MacDonald writes in early July 2020 that the number of police shootings of unarmed individuals is a very rare event and we devote disproportionate attention to them rather than all of the other killings of Black Americans.

As best as I can tell, both authors are accurately reporting what data tells us about police violence in the United States.  But I still want to talk to talk about the sources of that data because I teach a quantitative literacy course and understanding where data comes from is so important for evaluating how much trust we should place in data analyses.

Patrick Ball’s piece gives a good overview of two official sources: the FBI’s Supplementary Homicide Report (SHR) which is really part of the Uniform Crime Report (UCR) and the “Arrest-Related Deaths” (ARD) data collection program.  The UCR essentially involves law enforcement agencies (LEAs) reporting data on crimes to the FBI.  It appears this is submitted to the FBI not in one large info dump at the end of a year, but continuously.  Except for Mississippi, all states in the U.S. have a centralized office to which LEAs submit data (apparently Mississippi LEAs, if they choose to report crime data, report it directly to the FBI).  Looking over the UCR documentation, as best as I can tell LEA data submissions are voluntary and there is some level of noncompliance; the methodology document for the 2018 Crime in the United States report states that the FBI imputes crime statistics for an unspecified number of areas where the LEA supplied less than 12 months of data (including areas with less than 3 months of data!).  As another indicator of LEA noncompliance with the UCR, the FBI has had two reporting systems co-exist because LEAs stuck with a more antiquated system (the Summary Reporting System) rather than move to the more flexible National Incident-Based Reporting System, which will apparently be the only reporting system in 2021.

Essentially, crime reporting data is a mess, and we have not even come to data on police shootings!  As Patrick Ball states in his Granta piece, the UCR only reports on “justifiable homicides” committed by law enforcement personnel (given how much we have seen the police act with impunity in inflicting violence on civilians in the past few months, it seems doubtful to me there is a large reservoir of homicides committed by police that their departments judge to be non-justifiable).  The current UCR website reports around 440 homicides committed by law enforcement per year in the 2014-2018 period.

The other data source Ball discusses–ARD–seems like it was designed to fail from the start but ironically it probably did a better job than the UCR ever did.  The ARD program tasks coordinators for each state to conduct a census of deaths (not of police, but of civilians) occurring during an arrest.  As the Bureau of Justice Statistics (BJS) put it, “SRCs were responsible for understanding the scope and definition of the ARD program, identifying eligible cases in their state, and working within available resources to collect and report information about those cases. The manner in which these data were collected varied from state to state [my emphasis], and often depended on the data systems available to the SRC, the involvement of local law enforcement agencies or medical examiner’s/coroner’s offices, and other support that the SRC may have had to conduct the data collection.” As Ball and Kristian Lum pointed out in their own analysis of the SHR and ARD data, most states relied on media searches to inventory the number of police killings.  Not surprisingly, the BJS report concluded the ARD data were crap (but despite that, as Ball points out, it actually listed more police homicides than the supposedly more standardized SHR).  ARD was discontinued in the early 2010s; as best as I can tell, the last year of data collection was 2011 and ARD reported 689 police killings.

Ball discusses his and Kristian Lum’s efforts to compare the ARD and SHR counts to estimate what Joel Best calls the “dark figure”: the number of police killings not reported.  They think that in the jurisdictions covered by ARD and SHR, the total number of annual police killings is around 1,250 per year in the 2003-2011 period; Ball bumps that up to 1,500 to account for jurisdictions not covered by the ARD/SHR data.

Since Ball wrote his Granta piece, there has been yet a third official data collection effort, the “National Use of Force Data Collection” which supposedly went online in 2019 (but I have not seen any reports using data collected from it).   This is also another voluntary data collection effort (LEAs are not compelled to contribute data) where LEAs will directly submit information on police killings (as well as severe police beatings and police shootings where no one was shot) to an online portal.  Count me skeptical that this will yield much better data than the SHR or ARD.

Private entities have launched their own attempts to track police killings which appear to be based mostly or entirely on news reports.  This includes Fatal Encounters and the Washington Post database that Heather MacDonald referenced.  The Washington Post database has a fairly narrow scope: “only those shootings in which a police officer, in the line of duty, shoots and kills a civilian…The Post is not tracking deaths of people in policy custody, fatal shootings by off-duty officers or non-shooting deaths.”  In contrast, Fatal Encounters is trying to measure a broader concept: “We try to document all deaths that happen when police are present or that are caused by police: on-duty, off-duty, criminal, line-of-duty, local, federal, intentional, accidental–all of them.”  Not surprisingly, the Washington Post consistently reports around 1000 fatal police shootings every year since 2015; the Fatal Encounters database tends to find many more police killings.  They find around 1600-1900 police killings a year since 2015; in 2019 they reported 1,795 police killings.  The Fatal Encounters counts for the same time period as the Lum and Ball analysis is around 1210 (recall that Ball and Lum estimated police killings of around 1250 per year in their analysis, so the Fatal Encounters estimate is very close to theirs).

This brings me to Patrick Ball’s and Heather MacDonald’s statistics.  Both are telling the truth (in the sense they are accurately relaying data) yet use statistics in service of very different stories: Ball to play up the dangers of U.S. police; MacDonald to downplay it.  This is a very nice illustration of Best’s point in Damned Lies and Statistics: that activists broaden or narrow their conceptual definition of a problem to further their own ideologies.

In this moment, statistics seem beside the point.  We routinely see video evidence of police unnecessarily inflicting violence and brutality on civilians.  I suspect that MacDonald is correct that police killing of unarmed civilians is rare (although her statistics omit non-shooting fatalities like the cases of Eric Garner and George Lloyd), but it is probably the tip of the iceberg.  It is unlikely that we are going to get a thorough accounting of the prevalence of police assaulting peaceful protesters, but it is not necessary for the public to see that something is wrong with policing in the United States, and something has to change.

 

 

 

 

COVID-19’s effects on the heart

A study by Marc Dweck at the University of Edinburgh reporting on heart scans of COVID-19 patients has caught the attention of various outlets today.  According to the BBC’s Science Focus, “more than half of heart scans among hospitalized COVID-19 patients are abnormal, suggesting the coronavirus may have a devastating impact on this vital organ”.

The researchers, comprising a network of cardiologists, asked clinicians who conducted echocardiograms on 1,216 patients either testing positive for COVID-19 (73%) or judged as most likely having the virus (27%) to immediately fill out a survey on an app on their cell phones giving some background on the patient and the findings of the echocardiogram.

The study and the media write-ups tout that the data came from 69 different countries (I counted 68, although maybe they are counting “missing” as a country), although the UK provided a good chunk of the data (42%), with France, Italy, Spain, and the United States providing another good chunk of the data (30%).

As the BBC correctly reports, the clinicians reported for 55% of the echocardiograms that the patient had an “abnormal scan”.

The  write-up is fairly reserved.  The main point of the study is to show how pre-scan symptoms (“indications”) are related to the scan findings, to encourage doctors to think about using echocardiograms for COVID-19 patients showing these symptoms.

This may seem like a no-brainer to most of us, but the conclusion alludes to concerns that echocardiograms increase the risk of COVID-19 infection.  Clearly doctors will need to weigh the potential benefits (clinicians reporting abnormal scans were 46 percentage points more likely to report that there was a change in the course of treatment due to the scan) against the risk that the scan will spread the virus.

For that very narrow motivation, one cannot fault the study too much, especially since it is taking place during a pandemic.  Doing a comparison to non-COVID-19 patients would have been nice (they did show that the chances of having an abnormal scan were larger with clinician-reported symptom severity, and that is also higher for patients with confirmed cases of COVID-19 as opposed to probable cases), but very impractical in this context (especially since echocardiograms are feared to increase the risk of COVID-19 infection) and showing the effects of COVID-19 is not really the point of the study.  Another concern is that since clinicians are doing survey reports of their findings from the echocardiograms, clinicians’ diagnostic standards might not be consistent across all settings (remember this study is drawing on survey data from clinicians in 68 or 69 countries).  The fact that the clinicians know the subjects’ COVID-19 status and severity might bias their findings towards finding abnormalities, but again I think you can cut the researchers some slack because they are doing this in a pandemic.

Having said that, I think the authors and/or their supporting institution (the British Heart Foundation funded the study, and put out a press release about it) should be criticized for publicizing the findings.  It is inevitable that most readers will view news reports as indicating that the study supports the notion that COVID-19 affects the heart.  Take Science Focus’s headline: “Coronavirus may have a ‘devastating impact’ on the heart”. The headlines of  Sky News (“Heart scans of more than half of COVID-19 patients abnormal”) and Newsweek  (“Scans reveal heart damage in over half of COVID-19 patients in study”) are a bit more circumspect but I think even these will invite readers’ inferences that this study shows that COVID-19 affects the heart.  In reality, as the study acknowledges in its conclusion, its methods cannot speak at all to the effect of COVID-19 on heart disease.  First, most obviously, not all COVID-19 patients are hospitalized.  Second, even among those who are hospitalized, clinicians will only consider an echocardiogram for those with symptoms of heart disease.   These outlets also had similar caveats buried deep in their pieces, but they do not really convey how the findings are totally orthogonal to the issue of how serious COVID-19 affects hearts.

This study may or may not help doctors’ decision-making for ordering echocardiograms for COVID-19 patients.  I do not see the relevance of it for the general public, and when the BHF and/or the authors reached out to media outlets promising this was a study worthy of their attention, they guaranteed that lay readers would read more into the study than they should.