1 Cause-specific mortality statistics are essential for understanding a population’s health situation and for developing and evaluating public policies. Yet collecting these data is not straightforward, and their reliability is often questioned. Different approaches can be used to assess their quality. Using cause-of-death data from four countries of different sizes, histories, and mortality levels, the authors propose an original method for assessing the quality of these health statistics based on regional variations within a country’s mortality structure.
2 Cause-specific mortality analysis is a powerful diagnostic tool that can clarify the mechanisms of mortality change and the epidemiological patterns in a population. This tool’s validity and usefulness depend on the quality of the data used. Thus, evaluating data quality is an important step in performing cause-specific mortality analysis.
3 The term cause-specific mortality statistics usually refers to the underlying cause of death. For the International Classification of Diseases (ICD), the World Health Organization (WHO) defines underlying cause as “the disease or injury that initiated the train of morbid events leading directly to death or the circumstances of the accident or violence that produced the fatal injury” (WHO, 1992). Which underlying cause will be reflected in statistics for a particular death event depends not only on the quality of the investigation and the diagnostics but also on the sequence of choices made in the certification and coding processes. Here, by “certification process,” we mean the completion of the medical death certificate. The following factors may be relevant: the certifying practitioner (attending physician, pathologist, forensic expert/coroner, or other medical practitioner); availability of antemortem medical records; performance of an autopsy; and the diseases or injuries the certifier considers sufficiently relevant to be mentioned on the death certificate. By “coding process,” we mean the further processing of the completed medical death certificate both to determine the sequence of causes of death and to select the underlying cause. These factors may affect the coding process: how the cause-of-death sequence is examined (using the automated coding system [ACS] or manually); which ICD updates are taken into account; and how the coder interprets ICD rules on selecting the underlying cause (when manual coding is applied). To ensure that cause-of-death statistics are accurate and consistent, the approaches used for certification and coding should be of good quality, and they should be standardized.
4 Several frameworks have been proposed for evaluating the quality and usefulness of cause-of-death data based on statistical outputs, i.e., the data collected and published (Ruzicka and Lopez, 1990; Mahapatra and Rao, 2001; Mathers et al., 2005; Rao et al., 2005; Phillips et al., 2014). Most of these frameworks are designed for countries still establishing their vital registration systems. Applying criteria such as coverage, timeliness, completeness, or the absence of incorrect or improbable age or sex dependency is not very informative for evaluating mortality statistics in countries whose registration systems are already established and generally follow WHO recommendations for gathering and reporting mortality data. In these countries, the vital registration system covers all death events, and the diagnosis of the causes of death is typically based on the results of an autopsy or medical records. The reliability of cause-specific statistics for developed countries is often assessed merely by examining the proportion of ill-defined or “garbage” codes (Mathers et al., 2005; Mikkelsen et al., 2020). [1]
5 Still, even in the absence of apparent flaws, questions may arise on whether cause-of-death data are reliable. Studies analyzing additional data sources— including those in the four countries examined in this study—often report misclassifications in mortality statistics by causes of death. These include:
6 (1) Reinspection studies. Trained specialists carefully check the original medical death certificates for the correctness of the cause-of-death statement and the selection of the underlying cause (Gavrilova et al., 2008; Ives et al., 2009; Aouba et al., 2011; Zack et al., 2017).
7 (2) Studies assessing the level of agreement across certifying doctors and medical coders. Study participants either fill out the medical death certificate based on information from clinical records and autopsy reports (certifying physicians) or select the underlying cause based on information reported on the death certificate (medical coders). The level of agreement between the participants is then evaluated (Lakkireddy et al., 2004; Klug et al., 2009; Winkler et al., 2010).
8 (3) Studies comparing the cause-of-death statements in clinical and autopsy records (Modelmog et al., 1992; Sinard, 2001; Marshall and Milikowski, 2017).
9 Studies that include additional data sources provide much information on the quality of cause-of-death certification and coding. In addition to producing more accurate estimates of the prevalence of miscoding, these studies can show how miscodings originated. However, such studies are laborious and prohibitively expensive, and thus impossible to perform on a routine basis. Moreover, the extent to which their findings are generalizable is seldom clear, since most are performed on small samples of death certificates or with the participation of a small number of medical practitioners.
10 We aim to gain new insights into the reliability of data by causes of death in developed countries, without referring to any sources other than the underlying cause-of-death data derived from vital statistics offices. We examine statistical uniformity at the subnational level to identify abnormal and unusual values across subnational entities in a given country. Although a couple publications have discussed validating cause-of-death data at the subnational level as a criterion for reliability (Mahapatra and Rao, 2001; Rao et al., 2005), our examination of the literature indicates that this approach has never been implemented in a multicountry study.
11 Like most studies that use indirect methods to assess the quality of causeof-death data, ours can only draw attention to some suspicious or abnormal patterns. We cannot identify the factors responsible for these abnormalities with certainty, as we cannot determine whether they are driven by specific differences in etiological factors or by biases and inconsistencies in certification and coding practices. However, the approach we employ can be used to identify some potentially problematic areas in the data that deserve further scrutiny.
12 The approach is based on an earlier study that identified potential flaws in Russia’s cause-specific mortality data by analyzing their consistency at the subnational level (Danilova et al., 2016). In implementing a visualization technique to detect suspiciously large deviations from the average (for the particular combinations of regions and causes of death), it was also possible to identify causes with very high levels of interregional dispersion and regions with unusual cause-of-death patterns.
13 Here, we extend this work by evaluating the subnational consistency of cause-of-death statistics for Russia, France, Germany, and the United States. We based our selection on two main considerations. First, since our aim was to examine a detailed list of causes of death, we considered only the countries with large populations to minimize random fluctuations for not-so-frequent causes of death. Second, we wanted to study the countries that have implemented different systems for collecting and distributing cause-of-death information. Our analysis is designed to assess how these different approaches have affected data consistency within each of these countries. The study analyzes the subnational consistency of cause-of-death data in each country during the 5-year period 2008–2012, for which we have subnational data for all four countries.
14 In what follows, we describe the system for collecting and producing causeof-death statistics in each country, the data and methods used in the study, and the visualization technique for identifying large variations in causes of death across the subnational entities. We then present the results for each country, outlining the most notable features identified. Finally, we highlight the strengths and limitations of our approach, discuss the specific characteristics of within-country diversities that should be taken into account when comparing the four countries, and reflect on the factors that potentially explain some peculiar patterns in the data.
I – Systems for collecting information on causes of death
1 – Russia
15 Before 1999, the process of selecting and coding the underlying cause of death in Russia was partially centralized, i.e., it was carried out centrally within each region of the country. [2] The certifier wrote the cause-of-death sequence on the medical death certificate, which was then sent to the regional statistics service, where a trained statistician would select the underlying cause and code it according to the brief Soviet classification of causes of death (Danilova et al., 2016; Danilova, 2018).
16 In 1999, Russia introduced the 10th revision of the International Classification of Diseases (ICD-10). Simultaneously, coding responsibilities were transferred from statistical offices to medical organizations that issued death certificates and became responsible for specifying the correct cause-of-death sequence and for selecting and coding the underlying cause according to ICD-10 (Danilova et al., 2016). Thus, the current coding process is largely decentralized in Russia, and efforts are made to ensure quality and consistency: a standard death certificate is used uniformly across the country, and the accuracy of medical death certificates is supposed to be confirmed by the medical organization as well as by the municipal and regional authorities (Ministry of Health and Social Development of the Russian Federation, 2009). However, it is not clear whether or how these checks are performed in each region. In 2012–2014, only 2.4% of medical death certificates in Russia were replaced (Andreev, 2016). Thus, in most cases, the underlying cause of death appears in the statistics just as it was defined by the medical establishment issuing the death certificate. Moreover, although some of the certificates could be replaced after verification at the municipal or regional level, the revision could also occur on the medical organization’s own initiative; for example, when information on the death certificate needs to be changed.
17 Quality control in the coding process relies in part on an ACS that is not centralized at the national level. In some regions, the ACS is used throughout to select and code the underlying cause, while in other regions medical organizations are free to decide whether they want to use the software to assist certifying doctors. Our analysis, however, refers to the 2008–2012 period, during which ACSs were seldom used in Russia.
2 – Germany
18 Germany’s coding process is partially centralized, i.e., at the level of the 16 federal states. Despite some general recommendations (Leitliniengruppe der Deutschen Gesellschaft für Rechtsmedizin, 2017), the legislation on postmortem examination differs slightly across these states (Pechholdová, 2008; Dettmeyer and Verhoff, 2009), and death certificate formats also vary (Schelhase and Weber, 2007). The most significant point is the presence or absence of a field (Epicrisis) in which the certifying physician can provide free-form text with additional information on the causes of death or accompanying diseases (Schelhase and Weber, 2007).
19 Each medical death certificate is sent to the health department, where a public health officer checks the completeness of the cause-of-death information and its consistency with the sex and age of the deceased. The certificate is then sent to the federal state’s statistics office, where a trained coder again checks the information before selecting and coding the underlying cause (Schelhase and Weber, 2007).
20 Germany is transitioning to automated coding. Although an ACS was initially implemented in 2007 (Eurostat, 2019a), automated coding became routine only a few years later (Eckert and Vogel, 2018). Each federal state decides independently whether to use the coding software and when. In some states, all death certificates are coded automatically, while others use automatic coding only to process particularly difficult cases (Eurostat, 2019b).
3 – The United States
21 The death certificate coding process in the United States is centralized at the national level, with the National Center for Health Statistics (NCHS) coordinating with all 50 states, the District of Columbia, and the country’s six main territories and possessions to code and distribute mortality and fertility data. Each office of vital records in the 57 jurisdictions sends death and birth certificate information to NCHS. Before 2011, some states coded their data independently, though they used standard software. Since 2011, NCHS performs coding for all territories.
22 Despite the centralized procedure for selecting and coding the underlying cause, the laws on investigating deaths vary across states (CDC, 2015a, 2015b). Furthermore, the format of the medical death certificate is not the same across all states. Most states use the 2003 revision of the U.S. Standard Certificate of Death, but a few continue to use the old format based on the 1989 revision. In 2010, 16 states still used this old format; by 2015, only two did (Xu et al., 2016; Murphy et al., 2017). The cause-of-death section did not change between the two revisions. Still, some discrepancies between certificate formats may influence the selection of the cause-of-death sequence. Unlike the revision of 1989, the 2003 certificate specifically asks through a series of checkboxes about smoking and its contribution to death, pregnancy status, and the circumstances surrounding transport fatalities. In some states, the death certificate also contains a checkbox for diabetes (Hempstead, 2009), although these boxes are not always accurately checked (Davis and Onaka, 2001; Hempstead, 2009; National Research Council, 2009; MacDorman et al., 2016).
23 A pioneer in implementing an ACS for selecting and coding underlying causes, the U.S. has used this type of system since 1968 (Israel, 1990).
4 – France
24 The process for coding causes of death is centralized nationally in France. The French Epidemiological Center for the Medical Causes of Death (CépiDc– Inserm) is responsible for coding all the country’s death certificates, the format of which is uniform throughout the country. Since 2000, France has used an ACS to process medical death certificates (Eurostat, 2019a). In 2007, the electronic death certification system was introduced to better standardize the certification process and to assist medical practitioners in the certification process by providing explanations and examples of how the certificate should be completed (Lefeuvre et al., 2014). However, the electronic death certification system (specific to France and nonexistent in the other three countries during the study period) is not yet used consistently throughout the country. In 2010, only 5% of all deaths were certified electronically (Lefeuvre et al., 2014), increasing to 10% by 2015 (Rey, 2016).
II – Data and methods
25 We used the cause-specific mortality data from France, Germany, the United States, and Russia that were available at the first (top) level of the administrative division (hereafter, “region”). To reduce the uncertainty associated with low death counts, we included only those areas with an average annual population of over one million. Thus, the analysis was performed on 15 German federal states, 20 French regions, [3] 43 U.S. states, and 52 Russian regions. These regions cover more than 97% of all the deaths that occurred between 2008 and 2012 in Germany, the United States, and France (99.1%, 98.1%, and 97.6%, respectively). However, the proportion was lower in Russia (88.5%) due to the country’s greater number of regions with populations of under one million.
26 We grouped ICD-10 items into broad nosological categories designed to ensure that each category included at least 1 death (for all ages and both sexes) per 100,000 person-years in each country. [4] The resulting shortlist included 63 groups of causes (listed with corresponding ICD-10 codes in Appendix Table A.1). Cause-specific age-standardized death rates (SDR) for both sexes were calculated for the 5-year period 2008–2012 while using the past European standard population (1976) as the standard age structure.
27 We estimated the share of each of the 63 categories in the mortality structure of each region as follows:
29 where SDRc,r is the age-standardized death rate for cause c in region r and SDRr the all-cause age-standardized death rate in region r.
30 Next, for each possible combination of region and cause, we calculated the deviation of Sc,r from the cross-regional mean:
32 where n is the number of regions and (∑nr=1 Sc,r)/n is the mean of the shares of cause c in regional mortality structures for a particular country.
33 After computing 
, we obtained four matrices D (one matrix for each country) with 63 rows (the number of causes of death) and r columns (the number of included regions in the country). The results were plotted on heat maps for legibility. Each heat map corresponds to a table with colored cells indicating how far the cause-specific share for the corresponding region stands from the mean, based on the values of vc,r. Each cell represents an element of matrix D, and the respective value vc,r is depicted in colors; the darker the color, the higher the value of vc,r. As in the original matrix, the rows of the heat map correspond to causes of death and the columns to the country’s regions. We also introduce the following notation: 
indicates the average deviation of particular cause c across all regions in the sample (representing the mean value in row c of the country’s matrix D), and 
indicates the average deviation across all analyzed causes in a particular region r (the mean value in column r of the country’s matrix D).
34 The importance of a high level of variability for a particular cause-of-death category depends on its weight in the overall mortality structure. If a high degree of variability exists for a leading cause of death, it can bias the results of the cause-specific mortality analysis more than a high degree of variability for a “small” cause. The bars on the right-hand side of the heat maps indicate what percentage of the all-cause SDR is represented by each cause-of-death category.
35 Very high levels of average interregional deviation for a particular cause of death (
) indicate a large dispersion in mortality due to that cause across regions, and they further suggest that the values calculated for the entire country are not representative.
III – Results
36 Here, we present heat maps for each country. Since we are interested in identifying potential inconsistencies in the data that may be attributable to peculiarities in certifying or coding practices, we established a system of color gradation for all four countries, by which only cases that deviate significantly from the average are visible. Deviations of less than 40%–50% are not as easy to discern, and they appear as low values. To facilitate the readability of the figures, we do not distinguish between upward and downward deviations. We are more interested in the magnitude of the deviations than in their direction, but we provide relevant information on whether the deviation is upward or downward.
37 Horizontal structures containing a large number of dark cells can be identified. These lines indicate large variations in the proportion of the corresponding causes of death across subnational areas. Similarly, regions exhibiting distinctive cause-of-death patterns (deviating from the interregional average for a large number of causes) are identified as dark vertical bands.
1 – Russia
38 The Russian heat map (Figure 1) shows a significant number of cells with very large deviations. Both horizontal and vertical patterns stand out. The largest variations are associated with mental and behavioral disorders due to alcohol use (an average deviation across regions 
of 89.7%), other mental and behavioral disorders (86.7%), [5] and senility (79.3%).
39 The highest degree of consistency is found for the groups of causes from ICD-10 Chapter II, which classifies neoplasms. Almost all types of neoplasms exhibit an average regional deviation of less than 20% from the mean; but only seven causes from other ICD-10 chapters meet this threshold. Among the top 20 least varying causes in Russia, only three were not neoplasms, of which only one was among the top 10.
Figure 1. Interregional variability of the cause-of-death structure in Russia
Figure 1. Interregional variability of the cause-of-death structure in Russia
Interpretation: Each row represents a particular cause of death, and each column a particular region. Cells are shaded according to corresponding vc,r values. Gray horizontal bars on the right show the share of overall mortality attributable to the corresponding cause of death. Colors for “Causes of death” are used to separate the ICD chapters.40 The cities of Moscow and Saint Petersburg, the Republic of Dagestan, and the Chechen Republic exhibit distinctive mortality patterns with large deviations in many cause-of-death categories. To determine whether our results depend on the sizes of the territories and on the level of territorial disaggregation, we repeated the analysis for Russia, using the 15 macroregions allocated in the Russian Federation’s spatial development strategy of 2019 (Russia, 2019). [6] This analysis resulted in some lightening of the heat map and a reduction in the average deviation due to the mutual compensation of upward and downward deviations in regions belonging to the same macroregion. Thus, our finding suggests that analyzing variability at a certain level of territorial disaggregation can hide the peculiarities of cause-specific mortality that exist at a finer level of disaggregation. A detailed description and the results of this additional analysis are presented in Appendix B.
2 – Germany
41 The top 10 most consistent causes in Germany include only one category other than neoplasms, and the top 20 include only five such categories (Figure 2). Although the regional patterns are much less stark in Germany than in Russia, Schleswig-Holstein stands out as having a larger number of darker cells than the other regions. The average value of deviation across all 63 causes (
) reaches 39.3% in Schleswig-Holstein, with deviations of more than 40% for 16 causes. Among the other German federal states included, the highest corresponding values are in Sachsen-Anhalt (
= 23.3%, and 12 causes with deviation > 40%). The darker cells for Schleswig-Holstein correspond to a diverse set of cause-of-death categories. Apart from senility, atherosclerosis, and events of undetermined intent (which have high degrees of variability across the federal states), the largest deviations from the average are found in Schleswig-Holstein, where both pneumonia and mental and behavioral disorders due to alcohol use show an upward deviation, while accidental threats to breathing, except drownings and submersion, and the group of other cerebrovascular diseases are downward.
Figure 2. Interregional variability of the cause-of-death structure in Germany
Figure 2. Interregional variability of the cause-of-death structure in Germany
Interpretation: Each row represents a particular cause of death, and each column a particular region. Cells are shaded according to corresponding vc,r values. Gray horizontal bars on the right show the share of overall mortality attributable to the corresponding cause of death. Colors for “Causes of death” are used to separate the ICD chapters.3 – The United States
42 While the representation of different cause-of-death categories among the most homogeneous causes is more balanced in the United States than in Russia or Germany, a clear shift toward the predominance of neoplasms still occurs (Figure 3). Among the top 20 of the least varying causes in the United States, there are eight non-neoplasm causes, but only two in the top 10.
43 Although the vertical patterns are not as obvious for the United States as they are for Russia, partial structures can be identified for some states. Thus, in Hawaii, the shares of malignant neoplasm of stomach and malignant neoplasm of liver and intrahepatic bile ducts, rheumatic diseases, subarachnoid hemorrhages, and pneumonia are substantially higher than the interstate average, while the share of chronic obstructive pulmonary disease is substantially lower. In New Mexico, most external causes have substantially higher shares than the average. Mortality in this state is also characterized by a noticeably higher share of liver diseases. Large deviations in mortality from liver diseases are also observed in Washington State, although in this case the deviations are in the opposite direction. While the share of alcoholic liver disease is much higher in Washington than the interstate average, the share of other liver diseases is lower. Utah stands apart from the other states due to the much smaller shares of malignant neoplasms of respiratory organs in its mortality structure.
Figure 3. Interregional variability of cause-of-death structure in the United States
Figure 3. Interregional variability of cause-of-death structure in the United States
Interpretation: Each row represents a particular cause of death, and each column a particular region. Cells are shaded according to corresponding vc,r values. Gray horizontal bars on the right show the share of overall mortality attributable to the corresponding cause of death. Colors for “Causes of death” are used to separate the ICD chapters.4 – France
44 Unlike the other three countries, France does not exhibit strong horizontal or vertical patterns. The relatively few cells indicating large deviations from the average are scattered throughout the map, with no clear alignment (Figure 4).
45 Only two causes deviate from the average by more than 20%: fibrosis and cirrhosis of the liver (20.6%) and events of undetermined intent (22.9%). However, as the latter category in France is below our threshold of at least 1 death per 100,000 person-years, it might be subject to large random fluctuations. The distinction between neoplasms and other groups of causes is not as prominent in France as in the other countries. Causes other than neoplasms represent half of the top 20 stablest causes, and six of them are in the top 10.
46 Although vertical patterns cannot be identified on the French heat map as clearly as for the other countries, the Nord-Pas-de-Calais is characterized by the largest number of strongly deviating causes of death. For seven causes, the deviation from the average is more than 40% (malignant neoplasms of lip, oral cavity, and pharynx; malignant neoplasm of esophagus; peptic ulcer; alcoholic liver disease; and fibrosis and cirrhosis of the liver; and downward for transport accidents and events of undetermined intent). Among the other French regions, 11 have no cause that deviates more than 40% from the average; and four, one, and three regions have, respectively, one, two, and three causes deviating more than 40%.
Figure 4. Interregional variability of the cause-of-death structure in France
Figure 4. Interregional variability of the cause-of-death structure in France
Interpretation: Each row represents a particular cause of death, and each column a particular region. Cells are shaded according to corresponding vc,r values. Gray horizontal bars on the right show the share of overall mortality attributable to the corresponding cause of death. Colors for “Causes of death” are used to separate the ICD chapters.IV – Discussion and conclusion
47 The levels of geographic variability in the cause-of-death structure differed in the countries we studied, with the degree of variability found to be the lowest for France and the highest for Russia. For each country, we identified the groups of causes with the lowest and highest levels of heterogeneity across the country’s regions. Overall, we observed less variability for neoplasms, but more for garbage causes. Our analysis also allowed us to pinpoint the regions with the most specific cause-of-death structures. Although pronounced patterns of regional deviations were found only in Russia, regions with some peculiarities in their cause-specific mortality structure were identified in the other countries as well.
1 – Strengths and limitations
48 Several limitations should be considered regarding our approach, which is based on an indirect assessment of the quality of cause-of-death data. First, analyzing routinely collected cause-specific mortality data without consulting additional relevant information (medical records or autopsy reports) only allows us to detect anomalous patterns. We cannot provide a definitive assessment of whether the unusual patterns we observed are due to true variations in epidemiological profiles or to differences in certification and coding practices. Nonetheless, our approach has the major advantage of relying on official data to identify potentially problematic areas in the data. The anomalies we identified can then be investigated in more depth within each country using classic validation methods (e.g., Benavides et al., 1989; Lahti and Penttilä, 2001; Rao et al., 2007; Alfsen and Lyckander, 2013). Neither does our approach allow small deviations to be deemed a definite indicator of either good data quality or consistency in certification and coding practices. Indeed, when substantial differences in environmental or lifestyle factors exist across the territories, they are expected to drive differences in cause-specific mortality profiles. The absence of the latter may then indicate some insufficiencies in the data.
2 – Heterogeneity of the four national contexts
49 The levels of internal diversity in the four countries differ significantly. First, differences in all-cause mortality were larger in Russia (with a ratio between the highest and the lowest age-standardized mortality rates across 52 regions, reaching 1.8 and a 10.2% coefficient of variation) and in the United States (1.6; 13.1%) than in France (1.4; 8.1%) or in Germany (1.3; 5.8%). One may reasonably expect that the within-country differences in cause-of-death structures would also be smaller in France and Germany than in Russia and the United States. However, while some evidence suggests this was the case in France, no such evidence was found for Germany.
50 The degrees of within-country heterogeneity in many characteristics besides mortality also differ substantially across the four countries. France is a unitary state, while the other three countries are federal states. Presumably, subnational entities in federal states have more independence over their internal affairs. Differences in local policies in health care, education, and social welfare can affect regional mortality patterns. The unitary tradition of France’s political system may also explain why the process of collecting data on causes of death appears more homogeneous and centralized than in Germany, the United States, or Russia.
51 OECD regional statistics show that interregional differences in key indicators of economic well-being (e.g., income per capita, unemployment, and wealth inequality) are generally greater in the United States than in Germany and France (OECD.Stat, 2019). While we do not have fully comparable statistics for Russia, the data collected by the Federal State Statistics Service suggest that the differences may be even more pronounced in Russia than in the United States (Rosstat, 2018). Geographic diversity across regions is also greater in Russia and the United States than in France and Germany.
52 The 1945–1990 division of Germany has affected several generations and is still reflected in differences, including those in mortality, between the East and West German populations (Kibele, 2012; Grigoriev and Pechholdová, 2017; Pechholdová et al., 2017). The procedures for collecting data on causes of death and the certification and coding practices also differed substantially before reunification (Grigoriev and Pechholdová, 2017).
53 Given these specificities in internal diversity, we recognize that it is difficult to compare the four countries directly and to generalize our findings. However, we assume that France’s centralized and automated system of collecting causeof-death data indeed provides that country with advantages regarding the internal consistency of the statistics. More than half of the causes in our list (39 of 63) exhibit an average deviation of less than 10% from the mean level across the French regions. By contrast, the number of categories that meet this threshold is 15 in the United States, 11 in Germany, and zero across the Russian regions. The United States also seems to benefit from its centralized and automated system. Despite the highly heterogeneous economic and natural characteristics in the various regions of the United States, the number of causes of death with suspiciously high variations is lower there than in Germany and much lower than in Russia.
3 – Differences in epidemiological profiles or differences in certification and coding practices
54 As has been emphasized above, the analysis restricted to the statistical outputs provides no certainty and can reveal only potential inconsistencies in the data. It may be differences in epidemiological patterns due to population characteristics or environmental factors that drive the distinct mortality levels from a particular cause of death; it may be differences in approaches to certification and coding; or both possibilities. Analyzing data on the underlying causes of death is not sufficient to disentangle these two factors. It is, for instance, very likely that the high proportions of malignant neoplasm of stomach and malignant neoplasm of liver and intrahepatic bile ducts mortality found in California and in Hawaii are real phenomena driven by the large Asian population in both states and by the large Native Hawaiian and other Pacific Islander populations in Hawaii, as these ethnic groups have been shown to experience a higher incidence of such cancers (Gomez et al., 2013; Torre et al., 2016).
55 Some of our cause-of-death categories are more sensitive to conjunctural factors and may therefore vary more across both space and time, while other categories result from structural factors and should be stabler. In particular, two alcohol-related causes that we isolated in our analysis—alcohol abuse and alcoholic liver disease—are among the causes with the highest variability in all four countries, but the degree of variability was much lower in France (
= 18.0% for mental and behavioral disorders due to alcohol use and 17.5% c for alcoholic liver disease) than in Germany (31.1% and 29.5%), the United States (34.9% and 36.7%) and especially in Russia (89.7% and 53%). We cannot immediately reject the hypothesis that these patterns differ considerably across subnational entities within the same country, and they may therefore lead to distinct levels of mortality from alcohol-related causes.
56 Despite these examples, we assume that when large deviations genuinely reflect underlying health issues and very specific epidemiological patterns in a given region, these peculiarities will in most cases have been previously documented in the literature. Similarly, when some causes of death vary greatly at the subnational level, this variation can also be anticipated. Cases without known reasons for very large deviations warrant additional examination. The assumption that they can be attributed to a lack of coordination in certification and coding approaches should be considered, among others.
57 In Russia, Germany, and the United States, some causes associated with chronic diseases (for which large variations were not expected) varied more than causes that are particularly sensitive to short-term negative factors (for which a more uneven distribution would have been anticipated). For example, variations in mortality from infectious diseases including pneumonia, from transport accidents, and from violence were found to be lower than those associated with mortality from rheumatic diseases (Germany and the United States), myocardial infarction, hypertensive diseases (United States), endocrine, nutritional and metabolic diseases, and diseases of the nervous system (Russia). This observation supports the hypothesis that the high levels of cross-regional variability identified for some causes of death on the heat maps for these countries were at least partly caused by artificially induced deviations.
4 – Regions with specific cause-of-death profiles
58 Among the four countries, the most pronounced regional pattern is clearly seen on the heat maps for Russia. Darker vertical bands characterize the cities of Moscow and Saint Petersburg, as well as the Chechen and Dagestan republics. These four regions are unique in our sample, and it is very likely that a significant share of the deviations for these regions can be explained by real peculiarities in the cause-of-death structure, as was discussed in Danilova et al. (2016). First, these regions exhibit the lowest levels of all-cause mortality in our sample. Second, while Moscow and Saint Petersburg are two entirely urbanized regions, the Dagestan and Chechen republics are the only regions in our sample that are more than half rural. Chechnya and Dagestan are also two North Caucasus republics and have large Muslim populations, with much lower levels of alcohol consumption observed in that area than in the rest of the country.
59 Despite the many peculiarities in the four regions that may result in distinctive epidemiological patterns, the influence of specific certification and coding practices cannot be ruled out. The autopsy rates of the Dagestan and Chechen republics and the city of Saint Petersburg stand out from those of other regions. In 2011, 47.9% of deaths in Russia were subjected to autopsy. In the city of Saint Petersburg, this share reached 82.7%, the highest value among the 52 regions we included. The autopsy rates in the Dagestan (1.0%) and Chechen (5.2%) republics were the lowest. The autopsy rate in the city of Moscow did not differ substantially from the rate for all of Russia (55.3%). However, other peculiarities in the system of cause-of-death certification and coding were identified in this region. The city of Moscow had the highest share of deaths from “other symptoms, signs, and abnormal clinical and laboratory findings,” which appears to be attributable to the practice of not replacing preliminary with final death certificates in the published statistics (Pustovalov et al., 2019). The system of transmitting preliminary death certificates and replacing them in the statistics with final certificates was properly organized in Moscow only in 2016. Consequently, the share of ill-defined deaths (ICD-10 Chapter XVIII, “Symptoms, signs, and abnormal clinical and laboratory findings, not elsewhere classified”) in the mortality structure of the city of Moscow dropped from 10.5% in 2015 to 3.9% in 2016. The share of deaths from the other major ICD chapters increased in parallel.
60 Vertical patterns as pronounced as those on the Russian heat map cannot be easily identified for the other countries. Still, some less obvious and consistent deviations can be discerned. Further analysis is needed to understand whether these peculiarities are attributable to geographic variations in the epidemiological profile or to certification and coding practices. For example, the high deviation in mortality from malignant neoplasms of respiratory organs in the U.S. state of Utah (with a share substantially below the interstate average) can be explained by the state’s smoking rate, the lowest in the country (King et al., 2012). However, Utah’s high prevalence of garbage codes (senility, other symptoms, signs, and abnormal clinical and laboratory findings, and events of undetermined intent) is much higher than the average and might be explained by differences in certifying practices.
5 – Causes of death with substantial subnational heterogeneity
61 Our analysis shows that easily diagnosed underlying causes (e.g., neoplasms, certain conditions originating in the perinatal period, and congenital malformations, deformations, and chromosomal abnormalities) are reported more consistently across the subnational entities in all four countries. By contrast, causes associated with garbage codes (ill-defined codes, events with undetermined intent, and atherosclerosis [7]) are among those with improbably high levels of variability in Germany, Russia, and the United States. These findings may indicate that medical professionals have different practices and differ in their understanding of when it is legitimate to select these garbage codes for the underlying cause. The contrast between Russia on the one hand and Germany and the United States on the other is also very telling. In Germany and the United States, only causes associated with garbage coding exhibited extremely high levels of variability (
more than 40%), while high levels of variability were also found in Russia for a number of well-defined causes that are supposed to have explicit diagnostic criteria.
62 We must consider not only the level of interregional variability for a particular cause of death but also the share of that cause in the mortality structure of the country. Senility exhibited large disparities across subnational units in both Russia and the United States, but the share of senility in total mortality was 2.3% in Russia (from 0.0% in five regions in the sample to 10.8% in Saratov Oblast) and only 0.2% in the United States (from 0.0% in 12 states to 1.4% in Utah). Using different approaches at the subnational level for certifying senility as an underlying cause thus appears to affect data reliability more severely in Russia than in the United States.
63 The level of consistency in the contribution of each cause of death would increase with a broadening of the cause-of-death categories. For example, the average deviation across the 52 Russian regions was 30.4% for the share of myocardial infarction and 16.0% for other forms of ischemic heart diseases. When these causes are combined into the larger category of ischemic heart diseases, the average deviation is only 13.8%. Similarly, across the 52 regions, 
for all cerebrovascular diseases combined would be only 15.5%. For four out of the five finer groups of cerebrovascular diseases included in our analysis, this value is higher, with 
for unspecified stroke reaching 57.9%, indicating that certifiers and coders may experience difficulties in selecting between medically similar diagnoses.
64 The consistency in cause-of-death data at the subnational level may serve as one indicator of the quality and reliability of national mortality statistics. We used heat maps as a visualization tool to identify large variations in the cause-of-death structure at the subnational level, which may indicate flaws in the data. Our evidence was indirect and could not unequivocally differentiate between actual variations in the epidemiological patterns and artifactual variations caused by a lack of coordination in certification and coding. However, in using this straightforward and transparent visualization technique—together with the numbers and lists of deviating causes and characteristics of variability—we are provided with an instrument to instantly evaluate the consistency of cause-specific mortality statistics at the subnational level. An advantage of our approach is that it can be used to analyze mortality data as they are routinely collected and published by national statistics offices—without requiring additional data sources. Our analysis points to nosological categories and potentially problematic areas that call for further research. Such research should investigate whether the suspiciously high deviations revealed at the subnational level can be caused by specific approaches to the certification and coding of causes of death, the stage at which such discrepancies occur. This investigation would require not only an in-depth examination of the certification and coding processes but also access to medical documentation or more detailed data. Such research could estimate whether the level of cause-of-death consistency depends on the place of death (e.g., is the comparability of causes of death across regions higher for in-hospital deaths than for out-of-hospital?), on whether an autopsy was performed, or on the type of certifier. The role of other factors, such as the number of diagnoses specified on the medical death certificate or the demographic (age, sex) and socioeconomic characteristics of the deceased, could also be assessed. Doing so would provide a clearer picture of which actions and measures might be appropriate and should be introduced to increase the quality and consistency of the collected cause-of-death data. Another important future step would be to assess the consistency of cause-of-death data at the subregional level. While our study looked at the countries’ top-level of subdivision, the specificities and deviations may exist at a finer level of disaggregation as well. The detection of small territories with abnormal mortality levels from some specific causes can be a more precise foundation for further examination of the mechanisms causing these deviations.
Acknowledgments
Dmitri A. Jdanov’s and Vladimir M. Shkolnikov’s participation in the study was partly supported within the framework of the HSE University Basic Research Program. Magali Barbieri was supported for this work by the U.S. National Institute on Aging (R03-AG058110-01A1) and by contributions to the Human Mortality Database project from the Society of Actuaries in the United States, the United Kingdom Institute and Faculty of Actuaries, SCOR, AXA, Milliman, and RGA. Pavel Grigoriev was partly supported by funding from the European Research Council (ERC) under grant agreement No. 851485. However, any opinions, findings, conclusions, and recommendations expressed in this material are those of the authors alone and do not necessarily represent the official views of the National Institute on Aging and other funders.Appendices
Appendix A
Table A.1. Causes of death selected for the analysis and corresponding ICD-10 codes
Table A.1. Causes of death selected for the analysis and corresponding ICD-10 codes
Appendix B
65 The population and territorial sizes of the regions varied widely between and within each country. The average population of the U.S. states included in the analysis was 7.1 million, and the average area was 164,200 km2. The corresponding values were 5.4 million and 23,800 km2 for the German federal states; 3.1 million and 25,900 km2 for the French regions; and 2.4 million and 192,500 km2 for the Russian regions. The larger the region, the more diverse the territories lying within its borders; so we expect some of the subregional peculiarities in mortality patterns to disappear when analyzing larger areas. We conducted a simple experiment to investigate how different the results obtained for Russia would have been if we had analyzed much larger regions. Thus, we combined the 80 Russian regions into 15 macroregions that had an average population of 9.5 million and an average area of 1,140,000 km2. [8] A macroregion in Russia, however, is not a real administrative unit and has no single governing body.
66 The aggregation of the regions into larger territorial areas results in a lightening of the heat map, with a declining share of cells marked as deviating significantly from the average. While the average of all values in Figure 1 (for 52 Russian regions) is 28.8%, it declines to 21.9% in Figure B.1. This decrease is attributable to the internal heterogeneity of the macroregions, which combine regions with distinct mortality patterns. For example, the share of atherosclerosis in two of the five regions forming the Central Black Earth macroregion is 58% and 69% lower than the interregional average; but it is 62%, 76%, and 88% higher in the other three regions. [9] Combining these regions results in the deviations balancing each other out. Thus, the deviation for the Central Black Earth macroregion amounts to only 18.4%. Our finding for Russia—the degree of variability across the macroregions is lower than it is at the regional level—supports our hypothesis that in large or diverse regions, distinct cause-of-death patterns that may exist within the region could become invisible at a less granular geographic level.
67 However, cells with large deviations are still noticeable at this higher geographic level. The causes with the largest variations are the same as those found when the analysis is performed at the regional level. The vertical bands for the cities of Moscow and Saint Petersburg are still evident on the heat map. A darker line is also visible for the North Caucasus macroregion, which includes the Dagestan and Chechen republics, regions that display a peculiar cause-ofdeath structure in Figure 1. Thus, even though subnational variability decreases when analyzing larger (aggregated) regions, our overall findings remain essentially unchanged.
Figure B.1. Inter-(macro)regional variability of the cause-of-death structure in Russia
Figure B.1. Inter-(macro)regional variability of the cause-of-death structure in Russia
Interpretation: Each row represents a particular cause of death, and each column a particular region. Cells are shaded according to corresponding vc,r values. Gray horizontal bars on the right show the share of overall mortality attributable to the corresponding cause of death. Colors for “Causes of death” are used to separate the ICD chapters.Notes
-
[1]
“Garbage codes” was first introduced by Murray and Lopez (1996) to describe the ICD codes that are not useful for informing public health policy. Using them may distort the reliability of cause-of-death statistics. WHO classifies the following ICD-10 codes as “garbage”: symptoms, signs, and ill-defined conditions (R00–R99); injuries undetermined whether intentional or unintentional (Y10–Y34, Y87.2); ill-defined cancers (C76, C80, C97); and ill-defined cardiovascular diseases (I47.2, I49.0, I46, I50, I51.4, I51.5, I51.6, I51.9, I70.9) (WHO, 2013).
-
[2]
Russia comprises different types of regions (federal subjects): oblasts, krais, republics, federal cities, autonomous okrugs, and one autonomous oblast. These types differ in their constitutional legal status (for example, republics have the right to adopt their own language as an official language in addition to Russian), although some differences have already disappeared (no legal difference exists between krais and oblasts).
-
[3]
In 2016, France implemented a reform that combined some regions and reduced their number from 27 to 18. Since our study refers to the period 2008–2012, we use the pre-2016 divisions of France.
-
[4]
The only exception is the category “events of undetermined intent.” In 2008–2012, France counted only 0.72 such deaths per 100,000, below our selected threshold. We kept this category separate because it is important for determining the reliability of reporting for other external causes. An unusually large share for this group may indicate underreporting in other external causes. In the other three countries, the crude death rates from events of undetermined intent meet the threshold (1.6, 2.7, and 29.7 per 100,000 in the United States, Germany, and Russia).
-
[5]
In Russia, as in the other three countries in the analysis, most deaths in this category are due to dementia (ICD-10 codes F01, F03).
-
[6]
Each macroregion combines several regions based on their geographic proximity and similarity of climate and socioeconomic conditions.
-
[7]
Only I70.9, “generalized and unspecified atherosclerosis,” is regarded as a garbage code by WHO (2013). However, in both Russia and Germany, where the variability in the share of atherosclerosis in the mortality structure across regions was very high, most of the deaths due to atherosclerosis (I70) were coded to I70.9 (89.7% in Germany and 64.9% in Russia).
-
[8]
It was the Russian Federation’s 2019 Spatial Development Strategy that originally grouped the regions into macroregions based on geographic proximity, similar climate, and comparable socioeconomic conditions, an approach we adopted for our analysis, although we kept Moscow oblast and the two federal cities of Moscow and Saint Petersburg separate.
-
[9]
See Section III on our treatment of upward and downward deviations on the heat maps.
