As best as I can tell, there are two main stages where COVID data goes awry:<\/p>\n
- \n
- Who Gets Tested<\/li>\n
- How Testing Data Is Aggregated<\/li>\n<\/ul>\n
Who Gets Tested<\/h3>\n
It goes without saying that those infected with SARS-Cov-2 but are not tested are not going to show up in the data.\u00a0 Who are these cases?\u00a0 My initial thought was that these would primarily be asymptomatic cases and people who get sick but get over the disease fairly quickly.\u00a0 But this does not take into account how testing availability is segregated by race and class.\u00a0 Science Magazine<\/a><\/em> discusses the research<\/a> of epidemiologists Wil Lieberman-Cribbin, Stephanie Tuminello, Raja Flores, and Emanuela Taioli showing that in New York City testing rates in March 2020 were highest in White-dominated zip codes (although it was not correlated with zip code SES).\u00a0 Taioli told Science Magazine<\/em> that NYC equalized testing capacity\u00a0 since March but we do not know what the situation is in other parts of the country (h\/t Vox<\/a>).<\/p>\n
How Testing Data is Aggregated<\/h3>\n
<\/p>\n
Even more depressingly, even the very simple act of combining testing data to get a picture of what’s going on in the entire United States is fraught.\u00a0\u00a0Ourworldindata<\/a> talks about their data sources for COVID tests which is the COVID Tracking Project<\/a> and the CDC<\/a> (they don’t make clear how they combine these two sources of data). As of this writing, the CDC’s “Testing Data in the U.S.” page<\/a> (updated 8\/3\/2020) lists 52.9 million tests done and 5.0 million positive tests, while the CDC COVID Data Tracker<\/a> (also updated 8\/3\/2020) lists 61.5 million tests done and 5.5 million positive tests.\u00a0 And the COVID Tracking Project (updated today 8\/4\/2020)\u00a0 lists 52.5 million tests and 4.7 million positive tests.<\/p>\n
Sigh.<\/em><\/p>\n
As best as I can tell, all COVID-testing labs are required<\/a> to report their testing results to “the appropriate state or local public health department”.\u00a0 [I am not sure about this, but I believe that the recent kerfuffle<\/a> over HHS being the sole recipient of hospital COVID data is not related to this basic issue of COVID testing and case counts but I could be wrong].\u00a0 In turn, these state and local public health departments report their data to the CDC (as well as make it public, so the COVID Tracking Project picks that up).\u00a0 We also know that the testing aggregation is screwed up as some states are combining viral tests (which measure if someone has a current infection) and antibody tests (which measure if someone was infected in the past) which really screws things up (the focus of most international comparisons and time trends have been viral tests).\u00a0 According to the CDC Data Tracker, the states still doing this as of now are Delaware, D.C., Maine, Mississippi, Missouri, Oklahoma, Puerto Rico, Tennessee.\u00a0 It appears this aggregation is where we are getting these different estimates.<\/p>\n
Back to Trump<\/h3>\n
Trump’s argument that increased COVID cases is just an artifact of increased COVID testing is not a totally crazy idea.\u00a0 As Joel Best points out in\u00a0Damned Lies and Statistics<\/em>, often times when researchers start counting incidents of a new social problem, they inevitably show dramatic increases early on just because surveillance has gotten better, but not necessarily because the actual problem got worse).\u00a0 Best illustrates this with the example of FBI statistics on the incidents of hate crimes.<\/p>\n
Sharon Begley at StatNews<\/a> wrote an article taking on this argument.\u00a0 Essentially, she treats the percentage of tests that are positive as a more “valid” measure of prevalence.\u00a0 So in Florida (the state that saw the biggest relative increase in COVID cases from May to July), .002% of the population tested as positive in mid-May; in mid-July, the corresponding percentage was .059%, 25 times the percentage in May.\u00a0 But Florida’s testing also increased over time, from 479 tests to 65,567 tests.\u00a0 \u00a0 That 2,400 percentage increase in positive cases could just be an artifact of the increased tests.\u00a0 So Begley looks at the percentage of tests that were positive over the same time frame; in mid-May, it was 3.160 percent; in mid-July, it was 19.254 percent, 5 times the percentage in mid-May.\u00a0 So the prevalence of COVID increased even among the tested which undermines Trump’s argument.<\/p>\n
According to Begley’s calculations, there is only a handful of states seeing increased cases of COVID but steady (or even declining) percentages among those tested: Colorado, Indiana, Michigan, Missouri, North Carolina, Ohio, and Wisconsin.<\/p>\n
I find this exercise a bit unsatisfying though.\u00a0 Although Begley did not do it, we can do the same calculation for the United States.\u00a0 In mid-May, .008 percent of the population tested positive; in mid-July it was .019 percent of the population, 1.4 times that in May.\u00a0 \u00a0 And guess what? The percentage of tests that came back positive help pretty steady from mid-May to mid-July, from 7.4 to 7.6 percent.\u00a0 So if you really think the percentage of positive cases among the tested is a more valid measure of COVID prevalence, you would have to accept Trump’s argument that overall, the increased number of COVID cases in the United States is due to greater testing.\u00a0 I guess the state-level analysis is a bit more valid because it is getting “closer to the ground” but it seems like an arbitrary unit of analysis (why is the state-level better than, say, the county-level?).<\/p>\n
[to be clear, I am skeptical of the argument that trends in the percentage of positive cases among the tested is a more valid estimate of trends in the population overall; it seems to me that the number of tests is responding to more and more people getting the disease]<\/p>\n
I think more compelling would be looking at COVID hospitalizations and deaths.\u00a0 Both trends increased since mid-June, but the levels are still quite lower than what the United States was experiencing in April (although hospitalization data has been disrupted<\/a> due to the switchover to HHS reporting so the recent hospitalization counts are probably downwardly biased).<\/p>\n
![\"\"](\"https:\/\/sites.temple.edu\/klugman\/files\/2020\/08\/COVID-19-NET_Image.png\")
<\/a><\/p>\n![\"\"](\"https:\/\/sites.temple.edu\/klugman\/files\/2020\/08\/coronavirus-data-explorer.png\")
<\/a><\/p>\n<\/p>\n
To be honest, I am a bit pessimistic that data could really put Trump’s argument to rest, because our data is so poor (which I would guess is the legacy of our federal system combined with willful negligence in the executive branch).\u00a0 Vox<\/a> had an interesting argument about the need for “surveillance testing” which, instead of aggregating all tests ever done,\u00a0 would involve outpatient clinics recording data on all patients showing symptoms.\u00a0 This would supposedly yield trends that would be more comparable over time.<\/p>\n","protected":false},"excerpt":{"rendered":"
Donald Trump’s disastrous interview with Jonathan Swan centered around COVID testing data which I want to take up here and just organize my thoughts on the subject.\u00a0 I am looking for a good overview of COVID case data and where its potential biases are and I am a bit flummoxed. As best as I can … <\/p>\n