Foreword | In some countries, collecting statistics about the occurrence of homicide is not possible, either because of a lack of resources or because of the sheer volume of incidents. Fortunately in Australia there are three main data collection systems that produce largely independent sets of statistics on homicide: the National Homicide Monitoring Program at the Australian Institute of Criminology, and the Recorded Crime Australia and Causes of Death collections managed by the Australian Bureau of Statistics. In line with the AIC's policy of constantly improving the quality of data and output, this paper provides a critical analysis of these data sources and also examines the degree to which they differ. It considers whether the differences have varied over time, and the reasons behind any differences observed.
Adam Graycar
Director
A country's rate of homicide tends to be viewed by criminologists as a relatively unbiased measure of its level of violence. There are many reasons for this, including the fact that the serious nature of homicide makes it the least likely of all offence types to be affected by the "dark figure" of crime - that is, offences that go unreported and undetected. As a result, homicide is more likely than any other offence to be the subject of specific data collection efforts. In Australia there are three main data collection systems which produce largely independent sets of statistics on homicide: the National Homicide Monitoring Program (NHMP) at the Australian Institute of Criminology (AIC), and the Recorded Crime Australia (RC) and Causes of Death (COD) collections, both managed by the Australian Bureau of Statistics (ABS).
As noted elsewhere (see Mouzos 2002), a comparison of the three sources of national homicide statistics indicates that there is a large degree of variation in terms of the breadth of information collected, as well as the extent to which information is published and available to the public. The main variation exists in the sources of the data. Both RC and NHMP data are based on police offence reports, whereas the COD data are derived from death certificates. Given the differing origins of the data sources, the purpose of this paper is to provide a critical analysis of statistics derived from the three distinct data sources, and to examine the degree of divergence between the data sets and whether this has any bearing on the actual level of lethal violence in Australia.
Overview of international literature
Most international research comparing various sources of homicide data has been undertaken in the United States, with the main focus on comparisons between the Federal Bureau of Investigation (FBI) Uniform Crime Reports (UCR) and Supplementary Homicide Reports (SHR), and the National Center for Health Statistics (NCHS) Vital Statistics. These comparisons have sought to determine the level of agreement between these two main sources of data on homicide in the United States.
One of the earliest studies was Hindelang's (1974) comparison of UCR murder and non-negligent manslaughter rates with NCHS homicide rates from 1935 through to 1970. He concluded that the agreement was "generally good. Indeed the similarity in the shape of the curves is striking" (p. 3). Cantor and Cohen (1980) undertook a more extensive investigation and analysed the agreement among eight times-series compiled by the NCHS, UCR and the Office of Management and Budget. Their results indicated that the older data were less comparable, noting that the years prior to 1949 in the FBI times-series were not as accurate as the NCHS time-series. They therefore recommended that caution should be exercised when interpreting time-series results using the earlier years.
Riedel and Zahn (1985) also examined the amount of agreement between the FBI estimates based on Return A counts (which provide information at the aggregate level, without case-specific detail), SHR and NCHS counts of homicides. They noted that the differences in the rates for FBI estimates and NCHS data were small and they paralleled one another closely. They also calculated agreement ratios for each year of the series and a mean agreement ratio for a series of comparisons. Their results indicate that there was more agreement between FBI estimates and the NCHS data than between FBI estimates and other measures within the UCR. Rokaw, Mercy and Smith (1990), however, noted that the Vital Statistics annual homicide estimates were, on average, about nine per cent higher than SHR homicide estimates.
The following four factors were attributed to the differences between the two homicide data sources (FBI estimates and Vital Statistics):
- differences in coverage of US population;
- differences in the practice or rules governing the reporting of homicide deaths to the NCHS and the FBI;
- differences in the criteria in defining a case as a homicide; and
- differences in the categories used and the rules employed to classify people among demographic subgroups (p. 451).
NCHS Vital Statistics are said to include 99 per cent of all births and deaths in the United States, whereas UCR contain ESTIMATES of the number of homicides. This is because there is mandatory reporting for all states to the NCHS for the Vital Statistics, but for the UCR program reporting is only mandatory in 44 states. The collection and dissemination of Vital Statistics is one of the main functions of the NCHS, however this is only a small part of the FBI. In addition, the Vital Statistics on homicide include civilian justifiable homicides. These deaths are not included in the UCR program.
In summarising early research efforts, Riedel (1990) noted that there was greater agreement between the two series with recent data in contrast to older data:
While it is difficult to give an answer which would be appropriate for uses and occasions, it appears that with respect to a report of total homicides from the UCR, data prior to 1960 will present greater difficulties than data from that point to the present. From 1960 onward, many of the revision problems that characterised early estimates were resolved and the agreement with NCHS appears to be very high. (p. 201)
More recently, Wiersema, Loftin and McDowall (2000) examined the agreement between homicide estimates from the SHR and National Vital Statistics System (NVSS) at the county level for 1980 to 1988. They reported that the NVSS and SHR estimates agreed in 22 per cent of the counties (68 per cent if agreement is defined as no more than a difference of four homicides but in some cases they differed substantially).1 They also found that the NVSS generally exceeded the SHR, although the pattern was not uniform across counties. About 28 per cent of counties reported more SHR homicides than NVSS homicides. They attributed the lack of agreement between the SHR and NVSS data systems as a result of two general factors: differences in case definitions, and ambiguous or failed procedures. They cautioned:
...accuracy varies according to the level of aggregation and the size of the units under study. Data that are reasonable for an analysis of large urban areas or the nation as a whole may be inadequate to study smaller units. (p. 335)
These results were somewhat consistent with those obtained by Gabor et al. (2002) who compared the Homicide Survey and the Mortality Survey between 1970 and 1997 in Canada. Similar to their counterparts in the US, they found that the figures yielded by the two homicide data sources were not vastly different. They did note, however, that the differences between the two data sources were more pronounced (the Homicide Survey yielded higher homicide counts, with differences averaging almost 15 per cent during the study period) and found contrast patterns from those studies undertaken in the US. While the figures did not differ dramatically, they yielded different volumes of homicide and, most importantly, different conclusions regarding homicide trends. For the period 1977 to 1997, figures from the Homicide Survey suggested that the trend in homicide had stabilised, whereas figures from the Mortality Survey indicated that homicide had declined over that period.
This last finding is of some concern, especially because it has the potential to impact on policy decisions, both current and past. Gabor et al. (2002, p. 352) emphasise the point that "the need for reliable measures of homicide is also underscored by the critical policy issues informed by research in this area". For example, the incidence of homicide, and specifically the use of firearms in homicide, have important policy implications in terms of the effect of legislative amendments to firearms controls. It is therefore important that the sources of data used to make assessments and policy judgments are accurate - specifically, that they arrive at similar conclusions regarding homicide trends.
Comparing Australian homicide data sources
As already mentioned, there are three main data sources for homicide statistics in Australia. Both RC and NHMP data are derived from police offence reports, whereas COD data are compiled from information that has been extracted from death certificates, provided to the ABS by state and territory Registrars of Births, Deaths and Marriages. Another important difference between the homicide data sources is that NHMP data are recorded on a financial year basis (1 July to 30 June), whereas RC and COD data are recorded on a calendar year. Direct comparisons between data collected on a calendar year versus financial year is not advised as they are more than likely to yield different counts. The years that data are available for analysis also vary. RC began collecting data in 1993,2 whereas COD data are available back to 1915. Also, COD data are published on the year of registration of the death, and not necessarily on the year that the death occurred. As a result of these differences, the main comparisons between data sources will be undertaken between RC and COD for the years 1993 to 2001 on the following:
- number of homicides for total persons;
- homicide rate per 100,000 total population; and
- absolute and percentage difference between the number and rates derived from the various data sources.
Given that NHMP data are collected on a financial year, these data will be used in the examination of homicide trends over time. For the purposes of this paper, homicide includes murder and manslaughter and excludes driving causing death. Homicide in the COD data is conceptually equivalent to murder and manslaughter in the RC collection and NHMP.
Findings
In accord with the findings of research in the United States and Canada, there were modest differences identified between the counts yielded from the three main homicide data sources (Table 1). For the period 1993-2001, the mean count of homicide was 349 based on RC data, 319 based on COD data, and 337 based on NHMP data. As already noted, NHMP data are financial year counts. Compared to COD and NHMP data, RC data have consistently yielded higher homicide counts. Differences in terms of the mean homicide rate per 100,000 for the 1993-2001 period were minimal (about 0.1 per 100,000) (Table 1).
In order to statistically measure the degree of association between the figures yielded from RC and COD data, a Pearson's correlation coefficient was calculated. The Pearson correlation between the homicide figures for RC and COD data sources is -0.29, suggesting a relatively weak negative correlation. Put simply, it indicates that there is a weak inverse relationship between the two data sources, with high counts on the RC being associated with low counts on COD. The weak association, however, may be a function of the small number of years available in the array, and the low homicide counts compared to countries such as the US or Canada, where it was noted that the homicide figures were correlated strongly and positively across the two data sources.
A comparison of homicide counts and rates per 100,000 population derived from RC and COD for the period 1993-2001 indicates that the RC yielded an average of 33 more homicides per year than the COD (349 as compared to 319) (Table 2). The average annual homicide rate in Australia, according to the RC, is 0.2 (per 100,000) higher than that indicated by COD data (0.1 higher than that indicated by NHMP data).
The percentage differences between the counts derived from RC compared to COD ranged from a low of 2.1 per cent in 1993 to a high of 21.8 per cent in 1999. To illustrate this point, based on RC data there were 386 homicides in Australia in 1999, but only 302 homicides based on COD data for the same year - a difference of 21.8 per cent. Some of the more populous states account for most of the difference between RC and COD homicide figures. For example, Queensland recorded 78 homicides in 1999 based on RC, but only 54 homicides based on COD, NSW 135 and 116 homicides respectively, Western Australia 47 and 27 homicides, and South Australia 40 and 19 homicides respectively.
During the period 1993-2001, the average percentage difference in the homicide counts derived from the two data sources was 9.4 per cent (Table 2). It also appears that the differences between the two databases have become more pronounced over time (see Table 2 and Figure 1).
Figure 1 : A comparison of homicide counts in Australia based on Recorded Crime and Causes of Death data [see attached PDF for graph]
The line graphs in Figure 1 display the Australian homicide counts yielded by both the RC and COD data sources, while the bar graph illustrates the absolute difference between the two sources from 1993 to 2001. As previously mentioned, the figures drawn from RC usually exceed those recorded by the COD. Furthermore, the RC trend line shows a stabilisation in the number of homicides following a peak in 1999. Although the 1999 peak in homicides according to RC data is not replicated in the homicide trend line based on COD data, NHMP data also illustrate a slight increase in the number of homicides for the year 1998-99 (Figure 2).
Figure 2 : Yearly homicide trends based on three homicide data sources in Australia [see attached PDF for chart]
Discussion
In accordance with the results from Canadian research, the findings from the comparative analysis of homicide data sources in Australia indicate that the RC data consistently resulted in higher counts of homicide than COD data. This differs from the results of similar studies in the United States, where it was noted that the mortality data yielded higher homicide counts than the UCR.
In Australia, the differences in homicide counts derived from the various homicide data sources were minimal (an average difference of 9.4 per cent), suggesting that the differences have little impact on the overall quantification of homicide in Australia (counts and rates). However, the findings raise some concern with regard to the reliable measurement of homicide over time (that is, trend data), with COD and RC data arriving at somewhat different conclusions regarding homicide trends (the Canadian research also noted a similar inconsistency).
One factor that could account for the differences between the two collections is that they are measuring different things. In the COD collection, deaths are classified as homicides based on the coroner's determination of the "final intent" of the incident causing death. The RC collection homicide classification is based on the initial recording of criminal incidents by state and territory police agencies. If an initial report is eventually shown to not be a homicide, the RC record may not necessarily be adjusted (NHMP data, however, will be). As RC figures are based on the initial police report, offences may be recorded when the initial report or allegation is not substantiated. It is therefore likely that some incidents initially reported as homicide turn out to not be homicide after further investigation. However, to reiterate, an adjustment may not necessarily be made to the RC records.
In addition, a small number of deaths that are recorded as homicide by the coroner may in fact be recorded as accidental deaths in the COD collection. This occurs when the coroner has not yet determined the "final intent" of the incident leading to death at the time when the ABS Cause of Death file is closed (approximately midway through the year following death).
Another possible explanation for the differences in homicide counts based on the two data sources is the fact that homicide counts derived from COD data are based on the year of death registration and not on the year of death. Data on the year of death (or occurrence) are considered to be more accurate than data on year of registration. This is mainly due to year of death data allowing for seasonal analysis, not being affected by late registrations or changes in time taken to process registrations (ABS 2002).3 RC data are based on the year that the homicide becomes known to police, which in the overwhelming majority of homicides is the same year that the death occurred.
A cross-tabulation of "year of death" by "year of death registration" for COD homicide counts between the registration year of 1991 and 2000 indicates that of the 3,226 homicides registered during that period, 88.2 per cent (n=2,844) were registered in the same year that the death occurred. A one-year lag was recorded for a further 9.4 per cent of homicides (n=303). In other words, there were 303 homicides during 1991 and 2000 that were registered in the following year after the death had occurred. Figure 3 illustrates the difference in homicide counts based on year of death registration versus year of death. There are few differences between the homicide counts based on year of death registration and year of death, with the exception of the year 2000 which could be explained by the one-year lag (12.1 per cent difference between year of death registration and year of death).
Figure 3 : A comparison of Causes of Death homicide counts in Australia based on year of death registration and year of death [see attached PDF for graph]
The year 1999 recorded the largest difference between RC and COD homicide counts (22 per cent), with RC recording 84 more homicides than COD. As with all data sources, comparability over time is affected by a number of factors. These include issues relating to the collection, classification and processing of the data (ABS 2000a, p. 80). However, given this large difference in counts for the year 1999, a number of possible explanations were canvassed in order to account for it. There were a number of noteworthy changes to both the RC and COD collections during 1999. These can be divided into two main types:
- changes in classification systems; and
- changes in recording information systems.
Changes in classification systems
Both the RC and COD data collections underwent changes in classification systems in 1999. The most important change that occurred with the Recorded Crime collection was that prior to 1999, data were coded in accordance with the Australian National Classification of Offences (ANCO). From 1999 onwards, RC data have been coded in accordance with the Australian Standard Offence Classification (ASOC). Both systems provide a hierarchical classification of offences, however the ASOC system has an extended offence classification and takes into account recent legislative changes and additions to the criminal law which inherently reduced the usefulness of ANCO.
The COD collection also underwent a change in classification systems in recent years. As of 1 January 1999, the underlying causes of death data were classified according to the tenth revision of the International Classification of Diseases (ICD-10). This has resulted in changes in the interpretation and resultant coding of a number of causes, but homicide was not affected. Comparability factors were calculated based on a comparison between underlying causes of death coded in ICD-9 and ICD-10. The comparability factor for assault (homicide) was 1.02 (ABS 2001, p. 83) indicating that there were no significant differences in coding based on the change in classification system.
Changes in recording information systems
RC statistics are compiled from data extracted from computer information systems used by police to record crime. Changes to these systems have the potential to produce data that are not comparable before and after such changes were made. During 1999, there were also some changes to crime recording information systems used to generate the RC data in some jurisdictions (see ABS 2000b, p. 122, for an overview). However, these changes should not have had any impact on the homicide counts for the year 1999.
Another noteworthy change was the introduction of the Automated Coding System for processing deaths registered from 1 January 1997. Prior to this system, COD data were coded manually. The ABS dual coded (manually and using the Automated Coding System) more than 34,000 deaths registered in 1997. Comparability factors were then calculated for groups of causes as a means of adjusting data for 1996 and previous years. The comparability factor for homicide was 1.03, indicating that there were no significant coding differences between automated and manual coding (ABS 2000b).
While there was a number of notable changes to both classification and recording information systems used to generate the homicide data in 1999, none of which should have had any impact on homicide counts for that year, it is difficult to discount the possibility that these changes may have contributed in some way or other to the 22 per cent difference in homicide counts for the year 1999 based on RC and COD data sources. This seems to merit further investigation.
When using the various data sources for homicide, one needs to be mindful that differences exist between the sources. This has important implications for policy, especially when one data source suggests an increase in homicide over time, whereas another suggests stability in the trend. In order to avoid misinterpretation of the trends, it is important that a consistent data source is used.
While this study has examined the various sources of data on homicide, and the degree to which they differ, it is important to acknowledge, when attempting to understand the divergence in homicide data, that we are dealing with two distinctive data sources, and that:
...differences between cases in the files are to a great degree the result of differences in the two programs' purposes and procedures. Basically, the UCR [Recorded Crime] measures crimes, of which death is one outcome. The Mortality System [Causes of Death] measures deaths, of which crime is one cause. (Rand 1993, p. 112)
Acknowledgments
The author gratefully acknowledges Alan Mackay, Assistant Statistician, Health and Community Statistics Branch, Australian Bureau of Statistics, for his comments on an earlier draft. Two anonymous referees are also thanked for their constructive comments.
References
- Australian Bureau of Statistics (ABS) 2002, Causes of Death Australia 2001, cat. no. 3303.0, Australian Bureau of Statistics, Canberra.
- --- 2001, Causes of Death, Australia, 2000, cat. no. 3303.0, Australian Bureau of Statistics, Canberra.
- --- 2000a, Causes of Death, Australia, 1999, cat. no. 3303.0, Australian Bureau of Statistics, Canberra.
- --- 2000b, Recorded Crime Australia, 1999, cat. no. 4510.0, Australian Bureau of Statistics, Canberra.
- Cantor, D. & Cohen, L.E. 1980, "Comparing measures of homicide trends: Methodological and substantive differences in the Vital Statistics and Uniform Crime Report time-series (1933-1975)", Social Science Research, vol. 9, pp. 121-45.
- Gabor, T., Hung, K., Milhorean, S. & St-Onge, C. 2002, "Canadian homicide rates: A comparison of two data sources", Canadian Journal of Criminology, July, pp. 351-63.
- Hindelang, M.J. 1974, "The Uniform Crime Reports revisited", Journal of Criminal Justice, vol. 2, pp. 1-17.
- Mouzos, J. 2002, Quality Control in the National Homicide Monitoring Program (NHMP), Technical and Background Paper Series, no. 2, Australian Institute of Criminology, Canberra.
- Rand, M.R. 1993, "The study of homicide caseflow: Creating a comprehensive homicide dataset", in C.R. Block & R. Block (eds.), Questions and Answers in Lethal and Non-Lethal Violence: Proceedings of the Second Annual Workshop of the Homicide Research Working Group, National Institute of Justice, Washington DC, pp. 103-18.
- Reidel, M. 1990, "Nationwide homicide data sets: An evaluation of the Uniform Crime Reports and National Center for Health Statistics data", in D.L. MacKenzie, P.J. Baunach & R.R. Roberg (eds.), Measuring Crime: Large-Scale, Long-Range Efforts, State University of New York Press, Albany, pp. 175-205.
- Reidel, M. & Zahn, M.A. 1985, The Nature and Patterns of American Homicide, US Government Printing Office, Washington DC.
- Rokaw, W.M., Mercy, J.A. & Smith, J.C. 1990, "Comparing death certificate data with FBI crime reporting statistics on US homicides", Public Health Reports, vol. 105, pp. 447-55.
- Wiersema, B., Loftin, C. & McDowall, D. 2000, "A comparison of Supplementary R and National Vital Statistics System homicide estimates for U.S. counties", Homicide Studies, vol. 4, no. 4, pp. 317-40.
Notes
- 1 As the analysis was at the county level, it is unlikely that the percentage difference would be high.
- 2 The Recorded Crime Australia collection was originally known as National Crime Statistics. The name change occurred from the 1996 publication onwards.
- 3 Commencing with the Causes of Death 2001 publication (ABS 2002), the ABS has released data on a year of registration basis, as well as summary data on a year of occurrence (death) basis for the preceding year.
About the authors
Dr Jenny Mouzos is a Senior Research Analyst at the Australian Institute of Criminology.