The SARS-CoV-2 coronavirus disease identified in late 2019 rapidly became a global pandemic, and the associated death count a critical issue. International comparison of death statistics allows us to study the dynamics of the pandemic and the effect of health policies. Given that each country has set up its own counting system and that these systems have evolved over the months, are the differences in mortality observed in time and space really comparable and attributable to epidemiological variations? Using information on COVID-19 deaths for about 15 European countries, the United States, and South Korea, the authors provide valuable documentation to better understand and interpret the observed differences.
1Monitoring the current COVID-19 pandemic is critical for designing public health measures to combat it and for evaluating their effectiveness. National and international authorities have been seeking the best real-time indicators available to measure its spread and assess mitigation policies and health interventions. The numbers of positive cases, hospitalizations, and deaths stand among the most common indicators. The first two rely directly on available resources, illustrating a country’s capacity to regularly perform large numbers of tests as well as the availability of inpatient and intensive care unit (ICU) beds. Death counts are less dependent on country-specific resources. To keep track of the pandemic, many countries have thus made available series of death counts attributable to COVID-19. Several attempts have been made to gather international COVID-19 data (number of cases, hospitalizations, ICU admissions, and deaths) for comparative analysis (Harvard GenderSci Lab, 2020; Riffe et al., 2020; Roser et al., 2020). As part of this effort, INED has developed a database available on the Demography of COVID-19 Deaths website, [1] which focuses on death counts by age and sex with special attention paid to critical aspects of the data (INED, 2020).
2Early on, demographers underscored the importance of quantifying data on the pandemic through a population-based approach (e.g. Dowd et al., 2020; Verhagen et al., 2020). Such an approach takes population age structure into account because significant differences in population size and structure across countries affect the number of deaths, given the greater vulnerability of older people (especially men) to COVID-19.
3Computing age- and sex-specific COVID-19 death rates provides the most relevant indicators for international comparisons when available data are of good quality and where similar criteria are used to attribute a death to COVID-19 (Gutierrez et al., 2020; Pearce et al., 2020). However, even in high-income countries with a long history of demographic data collection, the available statistical information on COVID-19 has changed over time, with definitions of COVID-19 deaths and coverage of data continuing to vary from place to place. Statistics reported by health-care systems are usually limited to cases of COVID-19 infections confirmed through laboratory testing, while statistics reported by death registration systems usually include any death for which COVID-19 is mentioned on the death certificate. Most systems that monitor mortality data are designed for annual reviews, such as the vital statistics compiled by national civil registration systems. Such data must pass through rigorous and time-consuming quality protocols, and are typically not made available until 12 to 18 months after the close of each year.
4Other systems, such as those designed for epidemiological surveillance, process ad hoc and ongoing information. However, they report less accurate data and typically collect less detailed sociodemographic characteristics. They are designed to identify the start of epidemics (like the flu) and monitor their spread, peak, and decline. Because they rely on warning signals, which provide an immediate view of changes, these systems are not designed to count case numbers exhaustively and precisely.
5The COVID-19 pandemic demanded that a new data collection system be developed or that existing systems at least be expeditiously updated. Consequently, collected data vary not only between but within countries as collection gradually improves and coverage becomes more extensive. This process has induced artifactual changes in the pandemic’s patterns, thus making the available real-time demographic data imperfect. To avoid introducing biases in international comparisons, any analysis of COVID-19 statistics must factor in variations in data coverage and representativeness.
6We use COVID-19 death counts by age and sex from 16 countries (Appendix Table A.1), available from the Demography of COVID-19 Deaths database. We qualify these data by highlighting the critical points to consider when using these data for international analyses and comparisons. We focus on three main elements: (a) data definitions e.g. cause of death, testing strategies, case-confirmation mechanism, and consideration of “probable cases,” (b) data collection, e.g. system type, coverage by place of death, verification, and reporting time lag, and (c) data publication, e.g. reference date and frequency.
7We center our analysis on characterizing the publicly available official data by age, sex, and reference date. We first use examples to illustrate the potential biases in the collection systems, thereby drawing attention to data differences and shortcomings to avoid when using the data. Second, we classify countries’ data sources into groups with comparable data, thus providing a recommendable means for using these imperfect statistics reliably.
I – Critical points for international comparison
1 – Definition of deaths attributable to COVID-19
Criteria
8Cause of death may be attributed to COVID-19 following biological testing, clinical diagnosis, imaging, suspicion of infection due to symptoms or proximity to known cases, or death-certificate mention. Typically, patients who die in hospitals are tested. Deaths from COVID-19 at home or in nursing homes have been qualified in various ways, such as testing (the most common) and identifying typical symptoms (for suspected or probable cases). The quality and completeness of COVID-19 mortality statistics depend on the ability of national systems to carry out viral tests on patients and, in some places, timely autopsies (Centers for Disease Control and Prevention, 2020). Therefore, COVID-19 death counts may differ across countries and over time depending on the criteria countries use for classifying the cause of death.
9Initially, countries reported only COVID-19 deaths confirmed by laboratory tests. As the pandemic developed, some countries began adding the word “probable” to their confirmed COVID-19 death statistics. The criteria for suspecting COVID-19 without testing included whether the deceased experienced typical and acute symptoms, lived in or traveled through a high-risk area, or had contact with a confirmed case. However, countries vary greatly in the degree of flexibility granted to reporting physicians when determining the probability of a COVID-19 death.
10Broadly speaking, we can identify three groups of countries in this respect. A first group includes Sweden and Austria, which report numbers of deaths attributable to COVID-19 according to two different definitions and sources. The public health entities in Austria (Bundesministerium für Soziales, Gesundheit, Pflege und Konsumentenschutz [BMSGPK] [2]) and Sweden (Folkhälsomyndigheten [FoHM]) report deaths of individuals with positive polymerase chain reaction (PCR) tests, [3] regardless of the virus’s role in the morbid process. For example, a person who died in a car accident but had a previous positive PCR test is counted as a COVID-19 death (BMSGPK, 2020; FoHM, 2020). Austria’s epidemiological reporting system (Epidemiologisches Meldesystem [EMS]) and Sweden’s National Board of Health and Welfare (Socialstyrelsen [NBHW]) report only deaths for which a medical doctor identified COVID-19 as the underlying cause on the death certificate (EMS, 2020; NBHW, 2020).
11The second group of countries covers those that have changed their definitions of deaths attributable to COVID-19: Spain, England and Wales, and Belgium. The third group consists of countries using one data source with a consistent definition since the beginning of the pandemic, namely Canada, Denmark, France, Italy, Norway, Portugal, Scotland, South Korea, the Netherlands, and the United States. Figure 1 shows how the definition of a COVID-19 death has changed in the first two groups. When the pandemic began, Spain’s health ministry (Ministerio de Sanidad, Consumo y Bienestar Social [MSCBS]) published COVID-19 death counts estimated from the aggregate number of confirmed cases reported by local authorities. Autonomous communities were asked to report PCR-confirmed cases. Starting on May 11, 2020, with the adoption of new diagnostic, surveillance, and control strategies, the Spanish authorities revised the reported death counts. These communities were then required to report, by date of occurrence and by region, individual cases confirmed by PCR or IgM serological tests (though only when symptoms pointed to the disease). [4] These policy changes in the reporting and definition introduced disruptions in the data series. For instance, the old system counted 28,752 COVID-19 deaths reported in the new system between March 3 and May 23, 2020 (date of the last death count published under the initial surveillance system), compared with the 27,527 deaths reported in the new system between February 12 and May 23, 2020, based on figures published on September 15, 2020.
12A different approach was followed in England and Wales and in Belgium. Initially, only PCR-confirmed death counts were reported by the Office for National Statistics (ONS) in England and Wales and by Sciensano, Belgium’s national public health institute. As the pandemic developed, death counts began to include presumed or probable cases, following symptoms and/or documented contacts with a positive case (in Belgium) or death-certificate mention of COVID-19 as either the underlying or contributing cause of death (in England and Wales). Thus, the overall COVID-19 death counts in these two countries considerably exceed the figures based only on laboratory-confirmed cases. Of the 9,765 deaths attributed to COVID-19 in Belgium as of July 2, 2020, only 60% (5,828) had been confirmed by PCR (Sciensano, 2020).
Figure 1: Definitions of deaths attributable to COVID-19 by source for selected countries, January–September 2020
Figure 1: Definitions of deaths attributable to COVID-19 by source for selected countries, January–September 2020
Note: Each point on the graph represents an official report, by reference date.13In the Netherlands, the National Institute for Public Health and the Environment (Rijksinstituut voor Volksgezondheid en Milieu [RIVM]) published official COVID-19 death counts that include only laboratory-confirmed cases. From the beginning of the pandemic to June 30, 2020, 6,113 deaths were attributed to COVID-19. However, over the same period, the vital statistics system of Statistics Netherlands (Centraal Bureau voor de Statistiek [CBS]) reported 7,797 deaths confirmed by a positive laboratory test and an additional 2,270 suspected COVID-19 deaths (CBS, 2020). Had the Netherlands applied the Belgian definition that counts both laboratory-confirmed and suspected cases, the country would thus have reported about 30% more deaths than it did.
Testing strategies
14National testing strategies are important for identifying COVID-19 as a cause of death beyond the sheer number of tests performed, with both random and selective testing strategies having been implemented. Early on, testing was concentrated on identified cluster areas, but because of reduced capacity, tests were later limited to individuals experiencing acute symptoms. During this period, most countries concentrated their testing efforts on hospitals. Then, as capacities increased, testing expanded to include nursing-home residents and health-care personnel, while some countries also tested those with milder symptoms or who had high-risk contact with a confirmed case. Figure 2 summarizes the six types of testing strategies implemented in the 11 countries with consistently available information and which gradually relaxed their definition of the target population.
15Combining death attribution criteria with testing strategy characteristics can thus provide a framework for assessing COVID-19 mortality coverage in each country and time period. For instance, Belgium, which counts both probable and laboratory-confirmed COVID-19 deaths, implemented a testing strategy based on experiencing symptoms and high-risk contact with a confirmed case. The proportions of confirmed and presumed cases among the total reported death counts differ considerably, depending on where the deaths occurred. For example, while 95% of hospital deaths are test-confirmed (vs. 5% presumed), the proportion drops to 26% for nursing-home deaths (Sciensano, 2020), mostly because the Belgian testing strategy did not particularly focus on nursing care facilities beyond cases having experienced symptoms and high-risk contacts. Belgium illustrates how testing strategies directly influence reported death counts. With a testing strategy targeting only hospital inpatients, as most countries had done during the peak of the pandemic, the case of Belgium gives us an informed idea of uncounted deaths by place of occurrence when only laboratory-confirmed deaths are counted. The high COVID-19 death rates in Belgium may be partly [5] due to its comprehensive definition rather than any higher underlying risk for COVID-19 mortality. Belgium’s higher COVID-19 mortality than in other countries will be confirmed only after complete vital statistics data for 2020 have become available.
Figure 2: Testing strategies by country, type, and period, February–September 2020
Figure 2: Testing strategies by country, type, and period, February–September 2020
16Another example is Norway, which has carried out the most widespread testing among the 16 study countries. On June 4, 2020, the Norwegian Institute of Public Health (Folkehelseinstituttet [FHI]) reinstated mandatory testing for anyone with symptoms or suspected by a doctor of infection (for instance, close contact with a confirmed case). In addition to those with typical COVID-19 symptoms, FHI encouraged testing the following people, even if they were asymptomatic: persons admitted to a hospital; nursing-home or other health-care residents; health-care employees directly exposed to COVID-19 patients; persons with underlying conditions considered risk factors for COVID-19 complications, as well as their relatives; persons in quarantine because of close contact with a confirmed case or after traveling; employees, children, or pupils in in-person childcare, schools, or after-school programs; and others with suspected COVID-19 infection (HSRM, 2020). This large testing campaign has improved the identification of COVID-19 deaths outside hospitals (Sperre et al., 2020).
2 – Data collection systems
Types of systems
17Various systems have been implemented to reduce delays in reporting COVID-19 deaths. COVID-19 death counts originate from civil registration systems, surveillance systems, and health agencies (Appendix Table A.1). Data sources and coverage are related in that civil registration systems provide information on all deaths in the population, including those occurring outside the health-care system. Furthermore, civil registration systems publish the statistics after completing data collection and thus include, for example, late registrations of deaths. While health agencies may report deaths daily, the data may be incomplete because of the design of their monitoring or surveillance systems.
18Figure 3 compares the death counts reported by different sources in France and in England and Wales. In England and Wales, death counts are simultaneously reported by three entities using two systems: the ONS, via their civil registration and vital statistics system (CRVS); and the National Health System (NHS) and Public Health Wales (Iechyd Cyhoeddus Cymru [ICC]), via their surveillance systems. NHS and ICC data cover all laboratory-confirmed COVID-19 hospital deaths. ONS data report all deaths that mention COVID-19 on the certificate, regardless of where they occurred. Combined NHS and ICC data consistently yield fewer COVID-19 deaths than ONS data for the same geographical territory. France is in the same situation, with Santé publique France (SpF) reporting fewer counts from hospital data than those from the Centre d’épidémiologie sur les causes médicales de décès (CépiDc), which processes and provides all countrywide cause-of-death statistics through their CRVS system.
Figure 3: Cumulative COVID-19 deaths in England and Wales and in France by source, January–September 2020
Figure 3: Cumulative COVID-19 deaths in England and Wales and in France by source, January–September 2020
Note: England and Wales: ONS = Office for National Statistics, NHS = National Health System, ICC = Public Health Wales. France: SpF = Santé publique France, CépiDc = Centre d’épidémiologie sur les causes médicales de décès. CRVS = civil registration and vital statistics.Data coverage
19It is not straightforward to assess whether the COVID-19 death counts by age and sex reported in the study countries are exhaustive and representative of all COVID-19 deaths. In principle, the statistics published in South Korea, Portugal, and Germany cover death counts for all places of death, although the degree of reporting completeness is not specified. Figure 4 shows the proportion of death counts that provide information on age and sex in six countries. In the Netherlands, the number of COVID-19 deaths by age and sex reported on March 23, 2020 (213 deaths) corresponds to only half of the total deaths eventually found to have occurred on that date (417). Incompleteness arises from various causes. In some cases, the data by age and sex are delivered by either the reporting or publication date with no subsequent updating, while the daily total is reported by date of occurrence. The latter usually increases over time, as deaths reported late are added to the totals by date of occurrence.
Figure 4: Proportion of COVID-19 death counts providing information on age and sex, by country, March–September 2020
Figure 4: Proportion of COVID-19 death counts providing information on age and sex, by country, March–September 2020
20Completeness issues can also arise due to the same country having multiple data sources. At the beginning of the pandemic, Spain reported COVID-19 deaths by age and sex from two sources whose counts overlap somewhat: the National Epidemiological Surveillance Network (Red Nacional de Vigilancia Epidemiológica [RENAVE]) and the MSCBS. RENAVE reports all recorded deaths on its dedicated electronic platform, while MSCBS reports only hospital deaths. Neither source provides an exhaustive count (MSCBS, 2020a; MSCBS, 2020b). As MSCBS data seem to be more complete than RENAVE data, we compare the daily totals to MSCBS age- and sex-specific death counts, finding that the MSCBS data coverage has improved over time. However, it covers at most two-thirds of the cumulative daily death counts. RENAVE and MSCBS stopped reporting their death-count data by age and sex on May 21, 2020. Italy’s national health institute (Istituto superiore di sanità [ISS]) consistently reported fewer COVID-19 deaths by age and sex than the daily totals reported by Protezione Civile up to the beginning of August, after which the ISS revised its figures and began publishing data by age and sex that slightly exceeded Protezione Civile’s daily totals (see Figure 4).
Reporting delays
21The cumulative daily total of COVID-19 deaths logically excludes the number reported for that very day, either totally or partially. The “new deaths announced” include those already reported in the information system and those that occurred on previous days but were recorded in the system after the preceding daily update. Each day, most countries correct a previous day’s cumulative totals by reassigning deaths reported late to their day of occurrence.
22However, some countries do not promptly adjust their data, which skews their daily death counts downward, which should be taken into account. For instance, Denmark’s Statens Serum Institut (SSI) initially reported 161 cumulative total deaths on April 3, 2020, which increased to 170 by April 8; thus, nine additional deaths were reassigned to April 3 over the following days. Such updates are important for monitoring the daily dynamics at different stages of the pandemic, and the amount by which deaths are adjusted depends on delays in their registration and in the publication process at the national level.
23The United States derives its COVID-19 death counts from information on death certificates. According to the National Center for Health Statistics (NCHS), 63% of deaths from all causes are reported within 10 days of occurrence, with substantial variations by cause of death and between states. COVID-19 deaths typically take longer to record because they cannot be automatically coded and must be processed manually, while this is only the case for 20% of the deaths from other causes (National Center for Health Statistics, 2020). Figure 5 compares the percentages of deaths reported in the United States and in Sweden between April and July 2020. We examine the proportions of COVID-19 deaths that occurred on 4 selected days (mid-April, mid-May, mid-June, and mid-July). These deaths were promptly reported and are proportional to all deaths published for these 4 days as of September 14, 2020, the reference date up to which the proportions have changed over time. This suggests that, in the United States, 50% of COVID-19 deaths have been reported within 10 days and 90% within 12 to 18 days, with little change between April and August. In total, around 4 weeks are required to record most of the deaths that occurred on a given day.
Figure 5: Data coverage (in %) by date of publication in the United States and Sweden, September 14, 2020
Figure 5: Data coverage (in %) by date of publication in the United States and Sweden, September 14, 2020
Note: Death counts by date of occurrence reported on September 14, 2020 serve as the baseline for estimating completeness of earlier reports. April–September reports are represented on the x-axis by a line corresponding to the date of publication. The y-axis indicates total reported death counts at four mid-month dates. Line colors show death percentages for each selected date relative to the baseline. Thus, June 15 deaths reported on July 22 by Sweden’s FoHM eventually came to represent 50%–90% of that day’s total deaths.24By contrast, the proportion of late-registered COVID-19 deaths appears to have declined in Sweden. At least 1 week was needed to report 50% or more deaths occurring in April and May, while reporting 90% required an additional 3–4 days in these months. This 10- to 12-day lag dropped to 6 days for July deaths. The FoHM data are based on numbers of deaths with a positive PCR test and cover all places of death (hospitals, nursing homes, other health-care institutions, private residences, etc.). These data are continually revised, and the number of COVID-19 deaths may increase or decrease as new updated laboratory (re)confirmations are received. The deceleration of the pandemic reduced the pressure on the statistical office and allowed authorities to catch up with previous delays, as fewer ad hoc reports were needed.
25Reporting delays may vary by place of death. In France, for instance, from the beginning of the pandemic to April 5, 2020, only hospital deaths were included in the cumulative daily number of COVID-19 deaths, which significantly increased when nursing-home deaths were added (Figure 6). Specifically, the cumulative number of hospital deaths reached 12,900 by April 21, then jumped to 20,796 total deaths after adding nursing-home deaths, which put France on par with Spain’s and Italy’s numbers.
26Figures recorded on weekends can also result in artificially low numbers. For example, in Figure 6, the cumulative number of COVID-19 deaths by date of reporting in countries such as Scotland, where fewer deaths are reported on Saturdays and Sundays, translates into a staircase effect on the curve. The COVID-19 death numbers for these 2 days appear constant, then surge beginning on Mondays as registration catches up.
Differential in coverage depending on where deaths occur
27The share of deaths in hospitals, nursing homes, and at home varies from country to country, depending on how the health-care system is organized and how it interacts with care facilities, as well as on residential arrangements and patterns. In addition, the information system for counting deaths may vary according to the place of occurrence, which only a few countries reported up to the end of September. Table 1 shows the distribution of COVID-19 deaths by place of occurrence in countries reporting this information. In Norway, 59.5% of COVID-19 deaths occurred in nursing homes, 37.6% in hospitals, and only 2.9% in private residences.
28In the United States, the proportion of COVID-19 deaths in nursing homes (24.1%) is much lower than in Norway. Cross-country variations in the distribution of COVID-19 deaths by place of occurrence may result from differences in the population age structure (since older populations typically comprise a larger proportion of the care-home population); older-adult care policies (Genet et al., 2011; World Health Organization, 2012; Goodwin et al., 2014); the proportions of carehome COVID-19 patients sent to hospitals or other health-care facilities; care-home versus hospital testing policies (Bianchetti et al., 2020; Comas-Herrera et al., 2020); and measures for protecting older people in care homes against COVID-19 (e.g. protocols for staff and visitors). Figure 7 represents the distribution of COVID-19 deaths by place of occurrence and by age group in the United States up to October 3, 2020. As expected, most young adults die in hospitals, while about half of the deaths at the oldest ages occurred in care homes.
Figure 6: Reporting delays of COVID-19 deaths in France and Scotland, March–September 2020
Figure 6: Reporting delays of COVID-19 deaths in France and Scotland, March–September 2020
Table 1: Distribution (%) of COVID-19 deaths by place of occurrence in countries with available data
Table 1: Distribution (%) of COVID-19 deaths by place of occurrence in countries with available data
Figure 7: Cumulative COVID-19 death counts by age and place of occurrence in the United States
Figure 7: Cumulative COVID-19 death counts by age and place of occurrence in the United States
29Hospital deaths are likely to exhibit a younger mortality structure than those occurring in care homes and in private residences. The larger the proportion of deaths outside the health-care system, the larger the age bias when analyzing incomplete data. Thus, reports from hospital-based surveillance systems, as in Italy, Spain, and Ukraine, provide a biased idea of the age and sex distribution of COVID-19 deaths.
3 – Data publication
Date of reference
30Depending on the country, COVID-19 deaths are reported by date of occurrence, of reporting (i.e. when a death is declared to the local or central administration or health authorities), or of publication. Most countries use the date of reporting to the local vital statistics office or to the central health agency as the reference date. A few countries (Belgium and Ukraine) provide data by the exact day of occurrence, and many (including the United States and England and Wales) aggregate death counts by week of occurrence. It is important to consider these different dates for international comparisons, given the time lag between occurrence and publication.
31International comparisons should be based on days since the pandemic began in each country rather than calendar days, since the dynamics differ across countries. When comparing various countries on a fixed date x, each country’s cumulative number of COVID-19 deaths by that date conceals differences partly due to the stage of the pandemic (for example, whether it is just starting, increasing, peaking, or other). Thus, international comparisons should refer to the day when the pandemic began in each country (pandemic Day 1) and start from there. Thus, countries can be compared at Day 30, for example, although the calendar date for this day will vary from country to country.
32One way to determine Day 1 is to choose a reasonable threshold for the cumulative number of COVID-19 deaths, assuming they have been reported properly since the onset of the pandemic. Table 2 uses 20 deaths as this threshold to show the calendar dates for Days 1 and 30 in each study country, as well as the publication date for Day 30’s death counts. We selected this threshold because it is large enough to determine the initial spread of the disease (sporadic cases do not necessarily lead to an epidemic) but small enough to be independent of a country’s population size.
Table 2: Dates when different countries reached at least 20 COVID-19 deaths, February–May 2020
Table 2: Dates when different countries reached at least 20 COVID-19 deaths, February–May 2020
*Approximate dates, as data are provided weekly (middle of the week selected).** Based on data by reporting or publication date. The others are based on date of occurrence.
33Not all countries update their counts daily, and vital registration data tend to be published less frequently than those from other official sources (Pison and Meslé, 2020). This variation in data availability must be considered for international comparisons of mortality on specific calendar or pandemic days. For example, death counts by age and sex are currently published once a week in the United States, England and Wales (ONS), Sweden, Scotland, and the Netherlands; while Austria, Belgium, and Germany publish these daily.
II – Reasoning from imperfect statistics
1 – Classifying groups of countries with comparable data
34To compare trends in COVID-19 deaths across countries, one should carefully consider the following issues. First, as we have just described, the timeline for comparing countries’ distinctive experiences of the pandemic’s dynamics (start and pace) is crucial. If differences in dynamics (e.g. the starting date) exist across countries, then comparisons should be based on the time since the pandemic began instead of specific calendar days. Second, comparisons must consider differences in population size and age-sex structure, due to the highly variable rates of COVID-19 mortality for younger versus older adults and between men and women. Third, the exact reporting date of the data under analysis must be considered, as the data are continuously revised and the death counts may increase or decrease with new updates. Updates logically affect the overall death count, but they can also differentially impact specific geographical areas, ages, or sexes, depending on the degree of differential reporting for these characteristics. Fourth, the definition and data coverage of COVID-19 deaths are not homogeneous across countries. In addition, all the considerations described in Section I of this paper must be incorporated into the analysis. International comparisons should thus (ideally) cover countries with similar definitions and data coverage. For instance, our study countries may be grouped as providing the following:
35Comprehensive death counts. This group comprises countries whose data include statistics from the vital registration system, where COVID-19 is mentioned on the death certificate, or surveillance systems or health agencies that report both laboratory-confirmed and suspected COVID-19 deaths. Among our study countries, this group includes England and Wales (ONS), Scotland (National Records of Scotland [NRS]), Belgium (Sciensano), France (CépiDc), and the United States (NCHS), all of which periodically update and publish death counts by age, sex, and date of occurrence.
36Conservative death counts. The data for these countries include statistics from either the vital registration system, where COVID-19 is declared as the underlying cause of death, or surveillance systems or health agencies that report only the deaths of laboratory-confirmed cases. This group includes Austria (EMS), Norway (FHI), Denmark (SSI), Portugal (Direção-Geral da Saúde [DGS]), Germany (Robert Koch-Institut [RKI]), the Republic of Korea (Korea Disease Control and Prevention Agency [KDCA]), Sweden (FoHM and NBHW), and the Netherlands (RIVM). In these countries, the reference dates for the death counts vary because some data sources refer to the date of occurrence and others to the date of reporting.
37Restricted death counts. These countries publish data reported by surveillance systems or health agencies, which provide only partial death counts (e.g. hospitals only) of laboratory-confirmed cases. This group includes France (SpF), Italy (ISS), Spain (MSCBS), Ukraine (Center for Public Health [CPH]), and England (NHS).
38Certainly, other (sub)groups could be proposed for this classification, such as testing availability or strategy, population size or density, or any other relevant characteristics. Two fundamental reasons underlie the need to group countries in this way. First, many factors can cause issues in comparative analyses, and these need to be taken into account. The distinction between COVID-19 definitions adopted by data sources is the main one. For example, as mentioned before, had the Netherlands applied the Belgian definition for attributing deaths to COVID-19—which included probable cases—the country would have reported about 30% more deaths than it did. Second, depending on their purpose, studies should concentrate on data sources rather than countries with similar characteristics, most notably the data sources’ definitions and coverage. For example, England’s vital statistics data can be used to classify this country within the comprehensive group, while NHS surveillance system data would categorize it under the restricted group.
2 – Age structure of COVID-19 deaths
39To illustrate our classification, we use data reported on September 15, 2020, to analyze the cumulative COVID-19 death counts up to July 15, 2020. We selected the latter date because it can be regarded as the end of the first wave of the pandemic in most of the selected countries, where, with the notable exception of the United States, very few new daily COVID-19 deaths were reported after mid-July and for the rest of the summer. [6] The age and sex distributions of COVID-19 deaths are fundamental and can usefully be compared with the distributions of deaths from all causes for two reasons. First, as previously discussed, differences in coverage and definition may skew the age and sex distribution of COVID-19 deaths, especially when care-home deaths are not included in the national statistics. Second, comparisons may reveal patterns in the age and sex distributions of COVID-19 deaths. We compare COVID-19 deaths with the latest available year’s age and sex distributions of deaths from all causes in each country. Deaths from all causes by age and sex are taken from the Human Mortality Database (HMD, 2020).
40The bar plots in Figure 8 show the death counts for England and Wales (ONS), the Netherlands (RIVM), and Italy (ISS), which we use here to illustrate the three groups of countries classified according to the abovementioned characteristics (a similar figure for all the study countries can be found in Appendix Figure A.1). COVID-19 deaths (represented by the solid segments) are compared with deaths from all causes in the previous year (represented by the shaded segments) for each sex and age group in each country.
Figure 8: Comparison of cumulative COVID-19 death-count distributions (%) by age and sex with deaths from all causes, using examples from each data-source group
Figure 8: Comparison of cumulative COVID-19 death-count distributions (%) by age and sex with deaths from all causes, using examples from each data-source group
41The comprehensive group is characterized by the largest proportion of COVID-19 deaths at ages 90 and over, while restricted data sources correspond to younger age distributions. In the example above using the United States (Figure 7), the age distribution of COVID-19 deaths varies according to place of occurrence. Deaths attributed to COVID-19 in nursing homes or private residences are largely underrepresented in data originating from surveillance or health systems due to the old and frail population possibly dying soon after infection and before admittance to a hospital. Thus, because the age distribution is a relative representation, the undercoverage of deaths in the oldest age group is associated with a higher proportional death count at other ages. Likewise, the more complete the data, the smaller the difference between all-cause and COVID-19 deaths at the oldest ages. Focusing on the data in the comprehensive group, we see that the age distribution of COVID-19 deaths is almost identical to the age distribution for all-cause mortality. The largest difference is for ages 80–89.
3 – International differences in COVID-19 mortality
42Due to data limitations, our analysis is performed at the national level and uses each country’s most recent estimates of the total population by age and sex. Note that we are not considering differences in each country’s population density and their geographical distribution of active clusters, despite their relevance. However, such comparisons might be somewhat hazardous, as in the case of Belgium with the United States or even with all of France. Indeed, once the pandemic hit Belgium’s small and dense population, the whole country was immediately affected, while it remained localized in some French regions and U.S. states for quite some time before spreading. The use of subnational data would be preferable for these large countries, but the quality and availability issues described in the Introduction are even more challenging for regional data. Nevertheless, evaluating the pandemic’s impact remains useful at the national level, at which most countries are making decisions about and implementing nearly all public health measures for controlling the disease.
43The pandemic’s impact in each country can be roughly estimated using standardized mortality ratios (SMR), which allow for international comparisons even when the age distribution of COVID-19 deaths is unavailable. [7] Figure 9 compares COVID-19 SMRs by data-source group and for both sexes combined for July 15, 2020. SMRs allow us to assess the extent to which COVID-19 death counts deviate from what would be expected assuming the same standard age- and sex-specific death rates for all countries. In this case, we use the Netherlands, where age- and sex-specific death rates from COVID-19 can be calculated. An SMR of 1.0 indicates that the observed and expected numbers of deaths are equal. A value lower than 1.0 indicates lower-than-expected death counts from COVID-19, while higher than 1.0 is higher than expected. Figure 9 reveals that the ratios are generally highest in countries with comprehensive data sources and lowest in countries with conservative data sources.
44International comparisons of COVID-19 mortality should be limited using sources in the same group. Comparing sources from the restrictive and comprehensive groups for the same country, for example (France and England and Wales), shows the risk of underestimating mortality if one uses surveillance system data instead of vital statistics. In France, the estimated SMR attributed to COVID-19 using health-system data (SpF) represents only two-thirds of what could be estimated using CépiDc vital statistics (Figure 9). In England and Wales, ONS data indicates a COVID-19 SMR that is 40% higher than the combined NHS and ICC data. Unfortunately, we cannot use these ratios in countries where only restrictive death counts are available, such as Italy or Spain, because definitions and limitations vary. However, the estimated impact of COVID-19 in Italy and Spain is much higher than in most of the other study countries, including those in the other groups with more complete data, which demonstrates the pandemic’s severity in these two Southern European countries.
Figure 9: COVID-19 standardized mortality ratios by data-source group on July 15, 2020
Figure 9: COVID-19 standardized mortality ratios by data-source group on July 15, 2020
Note: Indirect standardization using the Netherlands as reference. From left to right, ENG-W ONS = England and Wales (Office of National Statistics); BEL = Belgium; SCO = Scotland; USA = United States of America; FRA CépiDc = France (Centre d’épidémiologie sur les causes médicales de décès); SWE NBHW = Sweden (National Board of Health and Welfare); SWE FoHM = Sweden (Public Health Agency); NLD = the Netherlands; PRT = Portugal; DNK = Denmark; DEU = Germany; AUT = Austria; NOR = Norway; KOR = Republic of Korea; ENG-W NHS+ICC = England (National Health Service) and Wales (Public Health Wales); ESP MSCBS = Spain (Ministerio de Sanidad Consumo y Bienestar Social); ITA = Italy; FRA SpF = France (Santé publique France); UKR = Ukraine.45The situation for countries with conservative death counts is similar in that the limitations in those data differ greatly among countries. The definition and coverage of data from Sweden (FoHM) are similar to those from Portugal, South Korea, Germany, Austria, and Norway, while the data from the other Swedish source (NBHW) are closer to the reported deaths in Denmark. Among the countries in this group, variations in test and laboratory resources as well as in testing criteria introduce substantial differences in the coverage of COVID-19 deaths. Further subgrouping according to the testing strategies implemented by the countries in the conservative group could guide more accurate comparison among them.
46Focusing on the comprehensive group, by the end of the pandemic’s first wave in these countries (July 15th), England and Wales appeared to be the most severely impacted by COVID-19, followed by Belgium and Scotland, while the United States and France had lower COVID-19 mortality levels (Figure 9).
47For this comprehensive group, we refined the analysis by looking into time trends and estimating COVID-19 age-standardized death rates (ASDR) of the cumulative counts for each day from March 2020 to September 2020 (Figure 10). The ASDR is calculated using a standard age structure (Eurostat’s 2013 European Standard Population) and is directly comparable across populations with very different age structures (Eurostat, 2013). The resulting rates are then summed from one day to the next to obtain the ASDR plotted in Figure 10. The figure thus illustrates trends in the total expected number of COVID-19 deaths over time, for a standard population of size 100,000 and given the specific mortality levels observed in France, England and Wales, Belgium, Scotland, and the United States.
Figure 10: COVID-19 cumulative age-standardized death rates in comprehensive-group countries
Figure 10: COVID-19 cumulative age-standardized death rates in comprehensive-group countries
48In England and Wales, France, and Belgium, the pandemic began simultaneously and spread at a similar pace. The pandemic slowed down quickly in France, perhaps due to the country having imposed an earlier and stricter lockdown (March 17). In Scotland, the onset occurred about 2 weeks behind England and Wales, but once it began, the curve of cumulative COVID-19 deaths increased at the same pace despite remaining at a lower level than in England and Wales. In mid-May, the cumulative COVID-19 death counts started to flatten out in France and in Belgium, but the latter maintained a much higher level. Mortality also stopped increasing in Scotland 2 weeks later and remained at the same level as in Belgium. England and Wales took more time to stabilize, which explains the higher level of mortality in this country at the end of the first wave.
49Unlike that of the European countries in Figure 10, the cumulative COVID-19 death-count curve in the United States has not flattened. The trend indicates a continuous increase in COVID-19 deaths, slowing down only briefly during June, which gives the impression of two waves: one starting at the end of March, the other at the beginning of August. The curve has continued to rise continuously since then, with no end in sight at the time of writing this article (November, 2020). The two apparent peaks in the total number of deaths in the United States do not correspond to two different waves of the pandemic but to differential patterns in the spread by state. In Northeastern states such as New York and New Jersey, the pandemic began early but was quickly contained, while it started later in other states, particularly in the South and West, such as in Texas and California, and only very recently in those in the Upper Midwest, like the Dakotas and Minnesota (Hawkins, 2020).
50Our data are sufficiently detailed for comparing the age structure of COVID-19 mortality in countries within the comprehensive group. Figure 11 displays the mortality curve by age, as observed on July 15. Initially, the curve corresponds to a classic mortality pattern with relatively high child mortality (at least in the United States and England and Wales, for which data are available for younger ages). Then the rates decline until reaching a minimum at around age 10, followed by an exponential increase (or a linear log-scale increase) up to the oldest ages. As mentioned above for total mortality, mortality rates at any age are highest in England and Wales and in Scotland, and lowest in France, at least up to age 80. Despite its higher mortality between ages 25–40, the United States becomes the best performer at the oldest ages because the pace in the increase of the mortality rate begins to slow down at age 70. The particularity of the U.S. curve may reflect an underestimation of COVID-19 deaths at very old ages; and it is indeed true that the United States reported a low percentage of nursing-home deaths (see Table 1, above).
51Belgium stands in contrast, with relatively low levels of mortality until age 50, followed by a rapid increase above this age until the death rates at ages 70 and older reach the same levels as in Scotland. By including probable COVID-19 deaths in Belgium’s national mortality surveillance system, the COVID-19 death toll increases by approximately 30%. The proportion of probable cases among the reported COVID-19 deaths is even higher in nursing homes, where probable COVID-19 deaths reach 48%. In Belgium, this strategy resulted in a recorded number of COVID-19 deaths that closely approximates the estimated excess deaths (Bustos Sierra et al., 2020).
52Despite data-related issues, these differences may be real. Furthermore, they may reflect the various policies implemented in these countries and their impact on vulnerable populations, namely U.S. working-age populations and Belgium’s poor older adults in nursing homes. At this stage, further investigation is impossible, given the lack of detailed data by place of death and socioeconomic status.
Figure 11: COVID-19 mortality age pattern by country on July 15, 2020
Figure 11: COVID-19 mortality age pattern by country on July 15, 2020
53For these same countries in the comprehensive group, we examined the sex ratios of COVID-19 ASDRs over time (Figure 12). Before commenting on our findings, it is worth bearing in mind that numerous mechanisms have been proposed to explain the greater vulnerability of men to COVID-19, some rooted in biology, [8] others in behavior. [9]
54Broadly speaking, the trend is similar in all five countries. It starts erratically (because of the low number of total deaths), then reaches a peak in early April, followed by a decline and stabilization from the beginning of May. The April peak is associated with higher COVID-19 mortality for men during the first months of the pandemic; then, mortality becomes higher for women, leading to a lower sex ratio than was previously the case. From about June onward, the sex ratios stabilize at a low level compared to the March and April peaks. Despite these similarities, some differences exist in the male disadvantage across countries. Once the sex ratios in these five countries stabilize in the range of 1.4 to 1.8 in favor of women, the highest ratios are observed in France and England and Wales, and the lowest in Belgium and Scotland.
Figure 12: Sex ratios of age-standardized cumulative death rates by country
Figure 12: Sex ratios of age-standardized cumulative death rates by country
Conclusions
55We have highlighted various characteristics of COVID-19 mortality data that should be considered for international comparisons. In our review of critical points, we explained and illustrated the main data issues: the varying definitions of COVID-19 deaths, testing strategies, data collection systems and their coverage, reporting delays, and the reference dates for published death counts. Given the cross-country variations in these aspects, it is essential to consider the particularities of COVID-19 mortality data to better interpret the results from any statistical or demographic analysis.
56Despite the numerous data limitations we have identified for international comparison, we have also showed that it is still possible to extract meaningful information from country comparisons when using such imperfect statistics. Comparisons are particularly possible for countries with similar data, at least regarding their definitions as well as the type and coverage of the data collection system. Indeed, these characteristics severely affect both the overall estimated death toll and the age and sex distribution of deaths. One way to resolve these differences is to group countries with similar COVID-19 mortality data and limit the comparisons to countries within the same group.
57As this article demonstrates, comprehensive data sources provide information mostly from vital statistics (except in Belgium, where high-quality data originate from the health system). When this group is used to attribute a death to COVID-19 in the official counts, reported deaths are the least dependent on testing capacities. These countries include both laboratory-confirmed and probable COVID-19 deaths, as well as all deaths that mention COVID-19 on the death certificate. Here, we used data for this group of countries to analyze the age and sex patterns of COVID-19 mortality at the national level. [10]
58Among the populations with comprehensive data, England and Wales is the most affected by the pandemic, followed by Scotland and Belgium. In this group, the age and sex distribution of COVID-19 deaths is similar to that for deaths from all causes. The differences are driven primarily by excess COVID-19 mortality at ages 80–89, especially among men. The male-to-female differences in COVID-19 mortality rates seem to be greater than the usual sex differences in all-cause mortality in these countries.
59The critical points highlighted and illustrated in this study indicate that the data provided by some countries do not allow for direct and accurate measurement of the COVID-19 pandemic, especially where the data on its mortality are produced by surveillance systems or health agencies. Although these data collection systems allow for the more rapid production of estimates of COVID-19 deaths by facilitating daily monitoring of the pandemic’s trends and pace, their limitations call into question the degree to which such data enable proper assessment of the pandemic’s magnitude in a specific country. The underreporting inherent in these types of data collection systems depends on the share of deaths that occur within and are registered by the health-care system; equally, the share of in-hospital deaths is directly related to the scope of the system’s resources (numbers of hospitals, hospital beds, and intensive care beds, etc.). To ascertain the pandemic’s spread in each country, supplementary analyses must be conducted on the effectiveness of using surveillance-system data for international comparisons.
60A comprehensive analysis of the pandemic is necessarily based on comprehensive data from national vital statistics systems, the only sources of exhaustive, standardized, and verified mortality information for the population as a whole. The criteria used by the national vital statistics systems to attribute causes of death are highly consistent and cover all deaths, including those that occur outside the health-care system, such as in private residences and nursing homes. Unfortunately, such data are published with much delay. The lag between occurrence and publication is driven primarily by regular data collection protocols and revisions. A compromise between the rapid availability and completeness of mortality data is needed in countries where COVID-19 death counts are simultaneously reported by both surveillance and vital statistics systems, as is the case in a minority of countries (for example, England and Wales, Scotland, and France).
61As the COVID-19 mortality toll continues to increase and as additional and possibly deadlier pandemics are already anticipated, national vital registration systems must become more efficient, publish data more rapidly, improve coverage, and provide more detailed information (such as by single age, minor geographical divisions, and some socioeconomic characteristics). Had such enhancements been made for the first wave, they would have allowed public health authorities to monitor the scale and spread of outbreaks more closely.
62Finally, at this stage of our research, the results provide some clues regarding the general characteristics of COVID-19 mortality, although we urge the scientific community to develop more analytical approaches that introduce contextual variables describing the public health measures implemented to control the disease. We demonstrated that any comparative analysis should take into account country-specific indicators of health-care expenditures, access to nursing homes, and end-of-life practices (at home, in hospitals, and in nursing homes). The COVID-19 mortality data provided by the Demography of COVID-19 Deaths database lays a strong and unique foundation for such studies, but the data must be used cautiously given the potential biases described extensively in this paper.
Appendix
Notes
-
[1]
The Demography of COVID-19 Deaths (2021), Institut national d’études démographiques (distributor), https://dc-covid.site.ined.fr/en
-
[2]
See Appendix Table A.1 for a list of institutions publishing COVID-19 mortality data in each studied country, as well as their corresponding acronyms, coverage, definitions, and other characteristics.
-
[3]
Polymerase chain reaction (PCR) tests are the most common way to detect SAR infection. They require swabbing the nasopharynx (part of the throat behind the nose), nasal cavity, saliva, or throat. They are highly accurate, and the results can be provided on the same day the swab is taken, depending on laboratory capacity.
-
[4]
IgM serological tests measure immunoglobulin M (IgM), found mainly in blood and lymph fluid. IgM is the first antibody the body produces to fight a new infection.
-
[5]
More complete coverage of deaths may also be a factor. See Section I.2.
-
[6]
At the time of writing, the second wave had only just begun and could not be included in this analysis.
-
[7]
Standardized mortality ratios compare a given population’s observed number of deaths (from COVID-19 in this analysis) to the number that would be obtained by using a set of age- and sex-specific death rates (from a reference population) as the standard and applying them to the age and sex distributions of the considered population.
-
[8]
For example, the hypotheses of hormonal, inflammatory, immunological, and phenotypic differences between the sexes, by which men are potentially more vulnerable than women to the most severe forms of the infection (Sharma et al., 2020).
-
[9]
For instance, lifestyle and medication use, as speculated by Bhopal et al. (2020).
-
[10]
Admittedly, relative to the overall population, population density and cluster locations may more closely determine the size of the population at risk of contracting and thus dying from the disease. However, any discussion on the optimal level of granularity for calculating COVID-19 mortality indicators is beyond the scope of our study.
