public health

What the Coronavirus Models Can’t See

A nurse works on a computer while treating patients with the coronavirus in the intensive-care unit at MedStar St. Mary’s Hospital on May 1, 2020 in Leonardtown, Maryland. The coronavirus death toll in D.C., Virginia, and Maryland surpassed 2,000 people on Friday as the District recorded its largest number of daily infections thus far. Photo: Win McNamee/Getty Images

We’re committed to keeping our readers informed. 
We’ve removed our paywall from essential coronavirus news stories. Become a subscriber to support our journalists. Subscribe now.

On April 20, less than two weeks ago, Donald Trump predicted the ultimate coronavirus death toll would be between 50,000 and 60,000 Americans. The prediction was strategic — he wanted credit when those targets were hit, and was betting it would be useful to remind people how much better those numbers were than even the lower-end estimates had looked just weeks before. “Now we’re going toward 50 — I’m hearing, or 60,000 people. One is too many. I always say it. One is too many, but we’re going toward 50 or 60,000 people,” he promised. “As you know the lower was supposed to be 100,000 people.”

At the time, the bet, however grotesque morally, seemed relatively safe politically. A week and a half earlier, Anthony Fauci had told NBC News that the final toll “looks more like 60,000 than the 100,000 to 200,000.” Earlier in the month, the University of Washington’s IMHE model — which has served through most of the pandemic as the most trusted and widely cited projection — had revised its estimates first from 93,531 to 81,766, then to 60,415.

But on Wednesday, the total death toll crossed 60,000, on a day that represented a new peak in single-day American deaths from COVID-19: 2,700. The data is a bit noisy right now, in addition to being almost certainly very partial, accounting for perhaps only one-half of total deaths. Though the public narrative is that the country has turned the corner and gotten a handle on things, enough to begin slowly “opening up,” the data tells a different story. A new daily peak means that even if deaths declined as rapidly now as they grew earlier this spring — when in a month total deaths grew from 3,834 to 62,860 — we would be due for at least as many more deaths as we’ve had to this point. In other words, another 60,000 people. A peak is not an end; it is, in optimal circumstances, a midpoint. Considering we just reached that new peak, it is not a foolproof assumption that daily deaths won’t grow again — far from it. And even if we do assume that this week was the overall peak, there are many reasons to think the decline from that peak will be slower than the ascent — meaning considerably more than 60,000 more deaths.

Through early April, though, the projections from epidemiological models kept shrinking, producing a wave of “good news” updates and a whole narrative among COVID skeptics that the projections were criminally alarmist all along. But the death figures have simply not cooperated with that optimism. The IHME model, in particular, has come under increasing criticism — “It’s not a model that most of us in the infectious-disease epidemiology field think is well suited,” Harvard’s Marc Lipsitch told Stat News. That’s especially true after a paper found that on 70 percent of days, the death count fell outside the IHME’s 95 percent confidence interval (this sounds wonky, but it means that the model gave a range it expected to contain 95 percent of all possible outcomes, and only 30 percent of real outcomes actually fell within it). But in part because it made a name for itself early in the epidemic, IHME remains a trusted name in the media (on the FiveThirtyEight podcast last week, Nate Silver interviewed its main author and offered praise for the model) and, in fact, its projections have ticked back up recently, though just by a few ticks, with a present estimate of 72,433 deaths by August 4. If the country stays on Wednesday’s pace, it will hit that number by May 4.

Pulling up short of 75,000 deaths means, in other words, an incredibly abrupt conclusion to the pandemic, with deaths going all the way to zero very soon and staying there permanently. That is not going to happen. For about the last two weeks, the country has been on a roughly flat trajectory of about 2,000 deaths per day. If it stays on that plateau through August 4, it would mean not 12,000 more deaths, but 180,000. And the pandemic wouldn’t simply end on August 4 just because the modeling does.

Of course, we may well not stay on that pace, but decline. How quickly? According to a New York Times survey of five major models, published last week, all of the models project a quite rapid decline — as rapid as the ascent was. As for how quickly that decline would begin, a model based at the University of Texas, which has won praise as an alternative to the IMHE, now says with 100 percent certainty that the country has passed its peak — this despite the fact that just on Wednesday we reached a new peak, and despite the likelihood that no more than 5 percent of the country, at most, has been exposed to the disease.

So, what is happening? Why is it that nearly all efforts to project the future shape of the pandemic seem unable to see more than a week or two into the future? And why, even in that time frame, are they almost unanimous in projecting a precipitous decline that is almost every day contradicted by the number of new deaths?

There are two big explanations. The first is that even under present conditions, in which the spread of infection is being dramatically constrained by shutdowns, the disease is not behaving as we expected.

What do I mean by expect? Clinical understanding of COVID-19 is evolving, which means our grasp of the timeline of sickness and recovery isn’t perfectly firm. But the basic rule of thumb is that it takes a little less than a week from the time of infection to the arrival of real symptoms; another week or so until those symptoms become serious enough to require hospitalization for those who do require it; and a week or two from there until those who ultimately die will die. That means the full life cycle of the disease, as we understand it, is about a month. A perfectly effective quarantine would therefore reduce practically to zero the number of new cases within a month of being implemented — after 30 days, anyone who’d been sick before would have either recovered or died, and, because of the quarantine, very few would’ve gotten sick in the meantime.

By March 30, the overwhelming majority of U.S. states had issued stay-at-home directions — in many cases, those directions had already been in place for weeks. Which means that by April 30, one month later, anyone who had caught the disease before the lockdowns began should have already passed through the entire life cycle of the disease, either recovering or dying, and that all the new cases we are currently seeing are the result of infections since the shutdown. On April 30, after a month in which new cases ranged between 25,000 and 30,000, there were 29,500 new reported cases.

It is not entirely clear what accounts for this. It could be there is some feature of the disease we do not yet understand, or something about how long it incubates that we have failed to recognize. It is possible the disease remains infectious even after a patient has recovered, perhaps for a significant amount of time. It is possible that sheltering at home is a flawed quarantining method, since family members can infect one another, and that some meaningful share of the new cases come from within quarantines. It is possible that some meaningful share comes from within hospitals, with doctors and nurses and administrators all serving as vectors of the disease. It is possible that some meaningful share comes from the 50 million exposed frontline workers — employees at grocery stores and meat plants and pharmacies and the post office and so on — who are laboring (without hazard pay) to make the quarantines possible for the rest of us. Quite likely, a few of these factors, and possibly all, are contributing, to some degree (and 900 workers at a single Tyson Foods plant did just test positive). Together, they are preventing anything resembling a decline of deaths or cases as rapid as the ascent a month or so ago. Instead, under conditions that you might have thought — in theory — would bring the caseload close to zero in this amount of time, national deaths and caseloads have merely flattened out. Perhaps, depending on how you process the data, they’ve moved somewhat, but not dramatically, downward.

But there is a second, more fundamental problem with the IHME model, indeed with nearly all the models you may have encountered, in one form of media or another over the last few months. Each is processing data — positive cases, hospitalizations, deaths — produced under those extraordinarily stringent social conditions. Each is generating a curve based on that data, which is to say, those conditions — the lockdown from which you are almost certainly reading this story. And the projections produced as a result reflect those conditions, too: Seeing a flattening in the number of cases or deaths, the models project a decline, even if the flattening is not a function of the disease itself but the extraordinary social conditions we have imposed on it. This is fine, so far as it goes, because it is useful to know the trajectory of infection under conditions of quarantine if you are under conditions of quarantine. But it only goes so far — as far as the quarantine goes.

Why is that? Traditionally, epidemiological modeling has been used to project the natural course of a disease as it spreads through what is called a “virgin” population. But there is nothing natural about the course of COVID-19 over the last few months. Its spread has slowed so dramatically from its early phase of exponential growth because of the national lockdown. Especially in hot spots like New York and Washington, the curves look encouraging — they’ve reached plateaus or even begun to decline. But even in those hot spots, the large majority of the population remains “virgin” — an estimated three-quarters of the population of New York City, for instance. Elsewhere in the country, the proportion is much, much higher, and already case declines in the country’s hot spots are being offset by growth in more “virgin” territory. The models may seem to be projections about the future course of the disease, but they are only projections about what the future course of the disease would be if present conditions continue. In other words, those curves expire, basically, as soon as lockdown sunsets. This is why, though even the sunniest COVID optimists would acknowledge reopening heightens risk somewhat, just about all the models now trace an elegant slope downward, with no uptick at all to account for any “reopening” in the near future.

But that reopening is already starting, which means the conditions that have produced those elegant (and encouraging) curves are ending. According to one estimate, Georgia’s reopening, which began this week, has already produced a thousand new cases in a span of 24 hours and is expected to double deaths by August. Other places could see an even more dramatic spike with only a partial, or “midway,” return to nonessential business:

Even in those states where there has been relatively little spread, the reopening is taking place from a much, much higher baseline of current infection than is optimal — or, I think, assumed. Take Texas, for instance:

And while some states are taking a more cautious approach — in California, for instance, Governor Gavin Newsom is taking things quite slowly — the disease isn’t very easily quarantined in suffering communities. “That’s like having a peeing section in the swimming pool,” as Seattle public-health official Jeffrey Duchin put it recently. “It doesn’t stay where you started.”

Some next-generation, “agent-based” models are sophisticated enough to parse these complexities, and make projections that reflect a broader array of human inputs. But, being considerably more complex, they have taken longer to build and refine to reflect the changing dynamics of this pandemic. As a result, they are only now really beginning to come online, and were left out of much of the conversation about modeling and projection early in the course of the disease. This has led some of those “agent-based” modelers to criticize the mathematical models. But while the agent-based approach is more sophisticated, it also requires those running the models to input policy parameters — what social-distancing guidelines are in place, what amount of testing, what percentage of “normal” human contact has been reduced — in order to generate predictions. Which makes them more useful in considering which policies to implement — which businesses can safely reopen, and to what degree, say — than in projecting the final shape and toll of the disease. Which all means, when you see on your television or computer screen an encouraging downslope of projected cases or deaths, you are not really seeing a depiction of a country turning the corner on this pandemic. You are seeing a depiction of a country turning the corner on a small fraction of a pandemic, with the future course of the disease very much unknown.

What happens after gradual reopening? Since the enormous majority of the public remains unexposed and therefore vulnerable, an enormous amount depends on what measures are taken to protect them — or to expose them. Which means the uncertainty is quite high — all those human variables piled on top of the disease variables. According to one model highlighted by the CDC, the death toll could reach as high as 293,381, or be as low as 88,217. Its median projection is 150,760, which matches quite neatly with the weekly survey conducted by UMass’s Thomas McAndrew of 150 epidemiologists and forecasters; the most recent edition yielded a median projection of 150,454 deaths.

Even these figures reflect some considerable optimism about reducing the impact of the disease through one of three methods: mass testing, at the moment still a distant dream for nearly all cities and states; treatment, where the closest thing we have to a “miracle drug” is remdesivir, which does not statistically improve the death rate of those taking it; or a vaccine, which, on the absolute most optimistic timeline, would be ready only in the fall. In the absence of breakthroughs in any of these three areas, the math looks considerably bleaker: Since we are only about a tenth of the way to herd immunity, we’d be expecting, probably, ten times more deaths than we’ve seen to date, for a total north of 600,000. That’s roughly in line with some agent-based projections for “unmitigated spread,” including this one, projecting a median outcome of 556,807 deaths (with a high end of 890,242).

In all likelihood, the future course of the disease won’t follow an unmitigated trajectory — at least some amount of additional testing will allow us to control the spread a little more effectively, and presumably treatments will arrive that will at least lessen the lethality of the disease somewhat, if not dramatically. And so it is possible, perhaps even probable, that we will not again exceed the peak of daily deaths reached just this week, even after “opening up” — that social-distancing guidelines, phased reopening, better hygienic practices, and improving treatments will keep the total number of cases at any one time from surpassing that 2,700-per-day figure. But while they have dominated talk about the state of COVID so far, peaks are most important when assessing how acute a medical crisis is or will become, and especially how best to plan for and allocate care. The total volume of cases is a much more significant signal in terms of the ultimate toll of the disease — that is, not the height of the curve at its peak but the area underneath the curve as it stretches out not just through the summer but into the fall and possibly into 2021 and even 2022.

Over the past few months, this possibility has often been described as a “second wave.” But such a wave would not be the result of disease mutation or a seasonality effect — the virus dying out in the summer, only to come roaring back during the fall and flu season. It would be because of human choices to gradually ease our way out of our bunkers and back into something more closely resembling “normal” life, though perhaps 95 percent of us have yet to even be exposed. It would be less like a second wave, arising suddenly from a placid equilibrium, than the opening, or the breaking, of a dam. All those people vulnerable to potential infection, and protected by quarantine measures, exposed by human choice. “That’s the risk we face here,” former FDA commissioner Scott Gottlieb said this week, “that this doesn’t really go away because we don’t get rid of this round. That the mitigation steps weren’t quite robust enough, as painful as they were, and we continue to have spread right into the fall.”

A report released Thursday by epidemiologists at the University of Minnesota was even bleaker, projecting that the disease would continue to spread for another 18 to 24 months. In their study, the authors laid out three possible paths forward: one scenario in which the end of lockdown is followed by a successive series of smaller peaks, each producing a re-intensification of mitigation measures; one in which the end of lockdown is followed by an even bigger peak next fall and winter, far beyond what we’ve seen so far; and a “slow burn” scenario, in which mitigation measures as successful enough to moderate future peaks and avoid additional shutdowns, but in which there continue to be infections and deaths for an extended period of time.

“This thing’s not going to stop until it infects 60 to 70 percent of people,” one of the lead authors told CNN. “The idea that this is going to be done soon defies microbiology.” On April 30, NBC News reported the federal government, even while signaling optimism about the course of the disease, had ordered 100,000 new body bags. On May 1, the CDC quietly removed the IHME model from its website, and the country reached a new peak of daily deaths: 2,909.

What the Coronavirus Models Can’t See