A couple of weeks ago I wrote skeptically about a CDC study by Mark Czeisler et al. reporting very high rates of mental health issues among young people in 2020 due to COVID. I wrote:
If it was me doing the study, I would have minimized mention of COVID-19 and tried to mimic questions about substance abuse and suicidal ideation that are fielded in other recent surveys (pre-pandemic) and just do a pre/post comparison.
I am chagrined to report that they actually did try to do this. A write-up in the Philadelphia Inquirer alerted me to this and sure enough in their conclusion they say this:
Elevated levels of adverse mental health conditions, substance use, and suicidal ideation were reported by adults in the United States in June 2020. The prevalence of symptoms of anxiety disorder was approximately three times those reported in the second quarter of 2019 (25.5% versus 8.1%), and prevalence of depressive disorder was approximately four times that reported in the second quarter of 2019 (24.3% versus 6.5%) (2). However, given the methodological differences and potential unknown biases in survey designs, this analysis might not be directly comparable with data reported on anxiety and depression disorders in 2019 (2). Approximately one quarter of respondents reported symptoms of a TSRD related to the pandemic, and approximately one in 10 reported that they started or increased substance use because of COVID-19. Suicidal ideation was also elevated; approximately twice as many respondents reported serious consideration of suicide in the previous 30 days than did adults in the United States in 2018, referring to the previous 12 months (10.7% versus 4.3%) (6).
So the numbers regarding anxiety and depressive symptoms come from the National Health Interview Survey (NHIS). The NHIS’s sample is quite different from that of the Czeisler et al. study. While Czeisler et al. relied on a Qualtrics panel and an online survey, the NHIS actually randomly sampled households to do their interviews and use computer-assisted personal interviewing (CAPI).
The Czeisler et al.’s measures appear comparable to those used by the NHIS, but it is hard to tell. Here is what Czeisler et al. say about how they measured anxiety and depression:
Symptoms of anxiety disorder and depressive disorder were assessed via the four-item Patient Health Questionnaire (PHQ-4). Those who scored ≥3 out of 6 on the Generalized Anxiety Disorder (GAD-2) and Patient Health Questionnaire (PHQ-2) subscales were considered symptomatic for these respective disorders. This instrument was included in the April, May, and June surveys. [p. 1050]
Here is how NHIS measured anxiety and depressive symptoms:
They are derived from responses to the first two questions of the eight-item Patient Health Questionnaire (PHQ-2) and the seven-item Generalized Anxiety Disorder (GAD-2) scale.
In the PHQ-2, survey respondents are asked about how often in the last two weeks they have been bothered by 1) having little interest or pleasure in doing things, and 2) feeling down, depressed, or hopeless. In the GAD-2, survey respondents are asked about how often the respondent has been bothered by 1) feeling nervous, anxious, or on edge, and 2) not being able to stop or control worrying. For each scale, the answers are assigned a numerical value: not at all = 0, several days = 1, more than half the days = 2, and nearly every day = 3. The two responses for each scale are added together. The NHIS indicators in the table are the percentages of adults who had reported symptoms of anxiety or depression that resulted in scale scores equal to three or greater. These adults have symptoms that generally occur more than half the days or nearly every day.
I think both studies used two items each to measure anxiety symptoms (the questions about frequency of feeling nervous, anxious, or on edge, and not being able to stop or control worrying) and depressive symptoms (the questions about having little pleasure in doing things and feeling down, depressed or hopeless), all four of which were coded on a 0-3 scale (0=not at all; 3=nearly every day).
The questions about suicide ideation come from the 2018 National Survey on Drug Use and Health (NSDUH); the sampling strategy and interviewing method seems very similar to the NHIS, although NSDUH uses audio computer-assisted self-interviewing for sensitive questions. (Fun fact: I think I was interviewed for the 2016 NSDUH; I was definitely part of a very similar study at least). It does appear that Czeisler et al. did use the same question wording regarding suicide ideation than did NSDUH, with the exception of shortening the time frame (from seriously thinking committing suicide within the past 12 months to the past 30 days).
So Czeisler et al. did their due diligence ensuring comparable measurement across the two studies, so we are left with two possibilities explaining the different prevalences of mental health issues across the two studies: the population did change (probably due to COVID-19) or the Czeisler et al.’s Qualtrics panel was biased towards representing people with mental health issues. This is plausible but I have talked before about Stephanie Fryberg’s research on Native American attitudes towards Native American mascots for sports teams and showed her Qualtrics panel of Native Americans tended to be more highly educated than the general population of Native Americans. I suppose a Qualtrics panel for the nation as a whole might skew towards people really struggling with COVID but it seems a bit of a stretch.
So I have to reluctantly conclude that I think Czeisler et al. are correct that people are more likely to say they have seriously considered suicide, or experienced anxious or depressive symptoms, now than they did last year or in 2018. Having said that, the description of the PHQ makes me wonder if the measure is really tapping into the debilitating natures of clinical anxiety and depression.