Apr 2 2020

Understanding COVID-19 Numbers

Expontential growth, lagging indicators, robust metrics.... confirmed case count is meaningless

Disclaimer: I am not an epidemiologist. By profession I am a software consultant and developer who also does data analysis and visualization.

I would love to add more visuals to this. For now if you want to see this more visually, checkout my Observable Notebook that I created to show how Death Count as a Lagging Indicator.

As with many people right now, my mind thinks a lot about COVID-19 and the massive impact it continues to have on our lives and our world. This is a world-changing event.

I often fall into the trap of irrationally thinking other people think in similar ways as I do and have come to similar conclusions. This is obviously not true. Here are four things I've been thinking about when looking at COVID-19 numbers.

  1. Exponential Growth: things can go from looking fine to being completely overwhelmed in a very short time.
  2. Lagging Indicators: we want to know who is infected, but we only confirm cases two or more weeks after the actual infection happens
  3. Confirmed Case Count is Not a Robust Metric: many people with symptoms are not being tested and it seems many people are asymptomatic. Variations in testing capacity and testing criteria make the confirmed cases metric almost meaningless.
  4. Exponential Growth + Lagging Indicators: two or three weeks is a long time for something to grow when it is growing exponentially

1. Exponential Growth

By now you have probably heard the term exponential growth. Exponential growth is hard to really understand. Our brains do not think in exponentials.

One thought exercise that tries to prove this point involves thinking where you would end up if you went for a short walk of 30 steps. You can probably accurately predict where you would end up if you took 30 normal (linear) steps. However, where would you end up if you took 30 exponential steps in which each step is twice as long as the previous one?

Let's assume one step is one meter.

  • 30 linear steps take you 30 meters away.
  • 30 exponential steps take you around the earth 26 times (1,073,741824 meters)

Most people don't think about exponential growth in everyday life. One group that has more experience than most is investors that focus on high-growth startups. Below is a great quote I saw on Twitter from Paul Graham who is co-founder of the startup accelerator and seed capital firm Y Combinator.

Hopefully by now you realize it is not too early to act. But if you are reading this and still on the fence, hopefully this article helps you understand that with exponential growth you may think things are looking fine one moment, but very quickly you can be completely overwhelmed.

2. Lagging Indicators

Ideally we would know in real-time when a new person is infected by the coronavirus. We would then have these people self-isolate immediately so they cannot transmit the virus to anyone else. This is not reality. In reality all of the metrics we can measure are lagging indicators of actual infections.

Number of confirmed cases is the metric that is most prominently reported in the media. Confirmed cases lag behind actual infection due to several things:

  1. Lag in symptoms: currently the CDC reports that COVID-19 symptoms appear between 2 and 14 days after exposure. Let's assume the average is 8 days delay in symptoms.
  2. Lag in seeking medical attention: it may take someone a few days to develop symptoms that are severe enough to seek medical attention. Let's assume it only takes 2 days for someone to seek medical attention (likely higher)
  3. Lag in test results: it takes time to get test results back (If you can even get tested at all. More on that later.) Optimistically, let's assume it takes 4 days to get test results (likely higher).

This somewhat optimistic view gives us 14 days, or two weeks of lag in the confirmed case metric for COVID-19.

It is important to realize that the results of any actions (social distancing, lockdown, etc.) will only be seen after you wait for the lag. Any action that we take now to affect confirmed cases will only be seen in two weeks.

3. Case Count is Not a Robust Metric

It seems obvious to say at this point that there are a lot of people in the world who have or had COVID-19 and were not tested. These people are not included in the confirmed case metric. With the huge variation in testing capacity and testing criteria, even within the same country, using the simply confirmed case metric is almost meaningless. This metric relies on the assumption that most cases are being confirmed.

  1. Many people are not being tested due to a lack of testing capacity.
  2. Many people are not being tested because they are asymptomatic or their symptoms are not severe enough to seek medical attention.

Several metrics that are more robust than confirmed cases

  1. Death Count: morbid, but more robust. It is less likely for a COVID-19 death to be missed than a simply COVID-19 case. Unfortunately, the lag in death numbers is higher even than confirmed cases. It may be three to four weeks from the time a person is infected until the time they die and the death is confirmed to be COVID-19 related.
  2. Hospitalizations: we are less likely to miss hospitalizations than we are non-hospitalizations. However, we still have to deal with lag in symptoms, seeking medical attention, and testing results.
  3. ICU admittance: this metric is important because it is the highest strain on the health care system. Again, we are dealing with lag in symptoms, seeking medical attention, and testing results..

Notice that waiting for a test to confirm a COVID-19 diagnosis is a challenge for all metrics. Given the issues with testing capacity and efficiency, presumptive diagnoses are probably more helpful than waiting for official test confirmations. Increasing testing capacity and efficiency is critical!

4. Exponential Growth + Lagging Indicators

As I outlined above, I am not a fan of confirmed cases as a metric. Hospitalization and ICU admittance data seems hard to find for most places. Therefore I have been looking at death counts. This is a rather morbid metric and also has significant lag. While confirmed cases has an optimistic lag time of two weeks, my thinking is that death count has an optimistic lag time of three weeks.

Most countries seeing death counts that double around every three days. This means if you have not made any changes in the last three weeks (new social distancing or lockdown measures), your death count will double seven times in the next three weeks.

If you have done nothing for the last three weeks, and your death count is currently 100, it will be 12,800 in three weeks.

Checkout my Observable Notebook I created with an example of Death Count as a Lagging Indicator.