The impact of operational performance reviews on reported crime in Queensland

In 1994 the New York City Police Commissioner, William Bratton, implemented a new, strategic approach to crime control and order maintenance that he coined COMPSTAT (for a review of the model see Bratton 1998). COMPSTAT involves executive police officers meeting with precinct or district commanders on a regular basis (Henry 2002). COMPSTAT is a four-step process that includes accurate and timely intelligence, rapid deployment, effective tactics, and follow-up and assessment (Walsh & Vito 2004). In COMPSTAT meetings the participants review district crime statistics, discuss emerging crime problems, identify strategic approaches for controlling crime problems, and discuss progress made during the previous period. In consultation with the district commanders, police executives then set a series of goals and objectives to be achieved before the next meeting.

Common opinion in popular literature and among many in the police community is that COMPSTAT can be credited with impressive reductions in crime and improvements in neighbourhood quality of life, making important contributions to the significant crime drop in New York City during the 1990s (Bratton 1997; Safir 1998; Kelling & Sousa 2001; Maple 1999; Witkin 1998). For some, COMPSTAT is seen as the 'linchpin strategy' that binds together other policing tactics such as zero tolerance, problem-oriented policing, order maintenance policing and police efforts that seek to reduce crime and improve quality of life (Silverman & O'Connell 1997).

Since the mid 1990s, Australian police executives, ministers and others have travelled to New York City to review the COMPSTAT approach and have brought the idea back to Australia with some changes to suit local conditions. The Australian versions are referred to as Operational Performance Reviews (Queensland), Operations Crime Reviews (New South Wales), Corporate Management Group Performance Reviews (Tasmania), Organisational Performance Reviews (Western Australia), COMPSTAT (Victoria) and Performance Outcome Reviews (South Australia). Some have argued that the wide diffusion of COMPSTAT across democratic countries in recent years is testament to the faith that police put in the COMPSTAT process for reducing crime problems (Maas 1998; Weisburd et al. 2003).

The claim however, that COMPSTAT and the Australian versions can reduce crime remains largely conjecture. Some critics doubt the likelihood that COMPSTAT was in fact the 'New York miracle' that many have claimed (Brereton 1999; Dixon 1998; Grabosky 1999). There is a lack of empirical research that examines the contribution (if any) of COMPSTAT to crime reduction (Eck & Maguire 2000). The one evaluation of Operations Crime Reviews in NSW found that OCR panels were temporarily effective in reducing three of four offence categories examined. Confounded by the lead up to the 2000 Sydney Olympics it is not clear whether these effects could have been sustained in the longer term (Chilvers & Weatherburn 2004).

This paper presents the results of an evaluation of the impact on reported crime of QPS's OPRs across the 29 police districts in Queensland. OPRs in Queensland sought to reduce crime and disorder, improve quality of life and institutionalise problem-oriented and partnership policing in Queensland. Interrupted time series analysis was used to assess and isolate the direct impact OPRs had on different categories of crime across the state. The second part of the paper examines the district-by-district impact of OPRs on different categories of crime, using a random-effects, mixed model (for details of the methodological approach, see Mazerolle, Rombouts & McBroom 2006). The final section is a discussion of findings.

Statewide impact of OPRs

Data

Monthly reported crime offence data for 60 different offences were collated from the QPS administrative data system and then collapsed into 13 different crime types. Various quality assurance processes were undertaken (see Mazerolle, Rombouts & McBroom 2006 for further details). Data were gathered in the form of monthly counts of offences for each police division (n=295) from July 1995 to June 2004. In total, there were 108 observation points: 73 pre-OPR monthly counts and 35 post-OPR monthly counts. The data for each division were assigned to the appropriate police district (n=29). The date of implementation was 1 August 2001.

Analytic approach

The statewide impact evaluation of OPRs used a time series analysis to assess the magnitude and direction of the intervention on reported crime incidents. SPSSARIMA interrupted time series analysis was used to analyse the effect of OPRs on reported offences over time. As a scoping exercise, the monetary savings of reduced crime due to OPRs were calculated to contextualise the results in terms of their practical significance. Estimates of the average cost of an individual type of offence taking into account both tangible and intangible costs (Mayhew 2003) were adapted for the current study to derive an estimate of the total savings in crime occurring in the post-OPR period. Savings were produced for eight individual crime types for which costs were available. Overall cost-effectiveness of OPRs was evaluated by deriving the ratio of the cost of OPRs to the costs of various crimes.

Results

Table 1 shows that the time series model for total reported offences was regularly and seasonally differenced and regular and seasonal moving average components were modelled adequately. The results show that the introduction of OPRs was found to be associated with a significant decrease in the total number of reported offences in Queensland. This is an important policy finding, showing that when reported crime data (1995 to 2004) for the entire state of Queensland are examined and all seasonal, trend, random noise (ad hoc changes in police practice, application of laws, in- and out-migration patterns statewide, ad hoc crises etc.) are factored into the model, then the OPRs introduced by the police department in early 2001 led to a direct and statistically significant decline in crime.

To visualise the amount of crime reduced directly by the OPRs, time series forecasting techniques were used to plot the predicted crime rate assuming OPRs had not been introduced against the actual crime rate post-OPR implementation (Figure 1). Note that the dark blue line represents the predicted values of total reported offences estimated from the pre-OPR time series while the lighter blue line represents the actual total reported offences after implementation of OPRs.

Figure 1 : Forecast versus actual total reported offences in Queensland

Source: Mazerolle, Rombouts & McBroom 2006

Figure 1 shows that, had OPRs not been introduced across Queensland in early 2001, a steady, seasonally-based increase in crime would have been predicted. By June 2004 the actual number of reported offences was 8,495, whereas without the introduction of OPRs, the number of reported offences could have been expected to be around 11,700. This is a saving of about 3,200 crimes directly attributable to the introduction of OPRs. This result is particularly important given that population growth across Queensland in the past three years followed a steadily increasing pattern from around 58,000 people per year to 81,000 from 2003 to 2004 (Queensland Government 2005).

When similar time series analyses were conducted on other categories of crime, OPRs were associated with a statistically significant reduction in unlawful entry offences. Serious assaults, common assaults, sexual offences, armed robberies, unarmed robberies, and unlawful use of motor vehicle offences all exhibited non-significant decreases associated with the implementation of OPRs.

The question of whether OPRs are a cost-effective intervention may be addressed by weighing their costs against savings attributable to reduced crime. The cost of OPRs for 2004-05 included salaries ($476,500) travel expenses ($40,000) and equipment costs (around $62,000). This brings the total cost of OPRs since their implementation to approximately $1,611,500. When these operational costs were weighed against the savings of reduced crime ($2,773,675), the introduction of OPRs appears to have been cost-effective, resulting in an overall saving of $1,162,175.

It should be noted however, that across all Australian jurisdictions there have been drops in the major property crime categories including unlawful entries from about 2001 onwards. Analysis of drops in property crime in New South Wales found a fall in heroin consumption, real increases in average weekly earnings, an increase in the number of heroin users returning to treatment, and an increase in imprisonment rates for convicted burglars were significant predictors (Moffat, Weatherburn & Donnelly 2005). Significant drops in property crime in the ACT were also found to result from a sustained intelligence policing initiative targeting repeat property offenders with incapacitation effects (Makkai et al. 2004). Overall, results show that OPRs in Queensland were an important factor in reducing crime, particularly unlawful entries, yet the importance of some of the other explanations that have accompanied crime drops elsewhere cannot be ruled out.

District variations in OPR impact

The first stage, statewide analysis, did not control for variations in local conditions that may have influenced the number (or rate) of reported offences. Criminological research shows that a range of local, community-based factors such as population growth, socioeconomic disadvantage, employment, urbanisation, industrialisation, demographic transformations (Cancino 2005; Eck & Maguire 2000; Moffat, Weatherburn & Donnelly 2005) are important variables that explain significant portions of crime fluctuations that occur over time and between communities. It is to be expected that these factors would also influence local crime trends in Queensland. Further, it could be expected that important differences between the 29 police districts would contextualise the statewide research findings and provide insight into some of the nuances of how OPRs contributed to the overall crime reductions in Queensland. Indeed, the statewide results described in the Stage 1 analysis may have obscured important district-level variations in OPR implementation.

Mixed model statistical techniques were used to analyse the district-by-district variations in the longitudinal time series data while controlling for sociostructural variables (for more detail see Mazerolle, Rombouts & McBroom 2006). From the outset it was likely that OPR effectiveness would vary depending on the socioeconomic characteristics of each district and the way that OPRs might have been implemented differently in the 29 police districts, for different categories of crime throughout Queensland. As such, in the second stage analysis, the 29 police districts are considered nested units of analysis within the state. Variations in districts are accounted for according to their geographical attributes (e.g. size), sociodemographic factors (e.g. SES, racial composition) as well as policing variables (e.g. district size, leadership) (see Table 2). Rather than ignoring these potential influences, this mixed model analysis incorporates these factors to provide a more accurate assessment of the effects of local conditions influencing the impact of OPRs on a range of categories of reported offences. Data on heroin use and imprisonment rates were not available at the district level.

Data

The mixed model analysis finds significant between-district variability. There was considerable variation in the starting crime rates across the 29 police districts. OPRs influenced the drop in crime in some districts, but had the opposite effect in others. Our results suggest that OPRs contributed to reductions in crime in some districts, mitigated against likely increases in crime in other districts, helped to hold some of the crime rates stable in yet some other districts and had no impact on crime in others.

When the specific districts that contributed to the decline in total reported crime are examined, the influence of OPRs is further understood (Mazerolle, Rombouts & McBroom 2006). Two districts in particular performed the best throughout the state. Both are urban areas: one regional the other city. One's decline in crime was driven by reductions in break and enters, unarmed robbery, dangerous driving and sexual offences. In contrast, the other's decline in crime was driven by reducing assaults (common and serious), offences against the person, unarmed robbery, unlawful use, and break and enter and steal.

Some of the districts with major urbanisation and in-migration experiences over the past ten years consistently posted increases in crime. In one district, the total crime rate increase was driven by significant increases in unarmed robbery and common assault. In another, increases in crime were driven by unlawful use of motor vehicles, break and enter and unarmed robbery.

The importance of social disadvantage in influencing crime rates is well known in the criminological literature (Cancino 2005; Eck & Maguire 2000). The proportion of renters is recognised as an important predictor of community crime problems (Weatherburn, Lind & Ku 1999). Results from this study are consistent with ecological explanations in variations in crime. Although OPRs are the superior predictor there are still significant effects for some of the other variables. Results clearly show enduring problems in districts with high concentrations of renters and social disadvantage. There are marginal influences gained through effective leadership, although this is not a significant predictor. Interestingly, the district leadership measure failed to emerge as the principal reason that OPRs have been successful in reducing crime in Queensland police districts, yet it was important for explaining some very specific categories of crime reduction (e.g. unlawful use of motor vehicle).

Overall, the mixed model analysis provides strong evidence for the hypothesis that there would be important variations in crime across police districts in Queensland, that the variations would not be explained only by socioeconomic variations, and that some of the statewide crime trends post-OPR would be driven in large part by the operational successes in some districts, yet compromised by the failure of other districts.

Conclusions

This project is important for several reasons. First, the fast diffusion of COMPSTAT-like programs across Australia raises the question of whether, and to what degree, the management strategy works to reduce crime. Second, the study sought to assess the impact of OPRs by police district and by crime type. This part of the research contributes to understanding how COMPSTAT-like strategies reduce some categories of crime more than others and under what types of situations. Third, the project informs police departments across Australia of the uneven impact of OPRs on reported crime at the district level.

Three major findings emerge from this research. First, the impact of OPRs is different for different categories of crime. Second, the impact of OPRs varies considerably by district. Third, it is likely that there is further variation at the smaller units of analysis (police divisions) that is influencing statewide trends in data. Understanding the spatial and temporal patterns in the data also hold most potential for the future of COMPSTAT-like innovations. These findings support recent longitudinal hotspot analyses using trajectory analytic techniques to model different crime patterns over time for highly specific places (hotspots). Weisburd and his colleagues (2004) found that strategic targeting by police of some hotspots in Seattle altered crime trend trajectories and had a significant impact on the overall city crime rate.

These results have important policy implications. First, a handful of the worst hotspots can contribute to major crime problems in a city (or district or region), so if the police can be effective in reducing crime in these hotspots, then city-wide crime rates will show major reductions. Police can show overall success by being effective at reducing crime in just a few places.

The study shows a similar pattern emerging at a much larger unit of analysis (the police district). Some districts drove a large proportion of the decline in crime across the state. This leaves a large number of districts that could (and should) be called upon during the maturing of OPRs in Queensland to reduce specific crime problems in their districts. If say, 10 of the remaining 27 districts could even marginally reduce their crime rates, then crime could be expected to continue to fall in Queensland over the next few years.

The second policy implication centres on the extent to which highly specific, problem-oriented policing efforts can impact not only on a district's success in reducing crime, but also on reducing statewide rates of specific categories of crime. Information from one district on their operational efforts to reduce unlawful use of a motor vehicle are instructive. Results show that OPRs had a direct impact on reducing unlawful use of motor vehicles and that this district contributed considerably to the reduction in unlawful use across the state. Personal correspondence with QPS officers revealed a concentrated effort in this district to develop crime mapping technologies, crime analysis techniques and a problem-oriented policing intervention at a large shopping centre that had problems with unlawful use of motor vehicles. This suggests that highly specific problem-solving efforts have the potential to influence the state crime rate (see also the evaluation of the AFP's Operation Anchorage in the ACT: Ratcliffe & Makkai 2004). If this is the case, then there is an urgent need to promote and institutionalise problem-oriented and partnership policing across all districts in Queensland. Results suggest that the uptake of the strategy is piecemeal across some divisions in some districts and further reductions in crime will occur with ongoing efforts to induce district commanders to adopt a problem-oriented approach to dealing with their specific district (and divisional) problems. The results of this research will help police department executives to rethink, tailor and adapt their COMPSTAT-like programs in the future.

Acknowledgment

This project was supported by grant number 22/03-04 from the Criminology Research Council. The views expressed are the responsibility of the authors and are not necessarily those of Griffith University, the Queensland Police Service or the Criminology Research Council. Further details are provided in the full CRC report (Mazerolle, Rombouts & McBroom 2006).

References

All URLs were correct at April 2006

• Anselin L 2005. Exploring spatial data with GeoDa: a workbook. Illinois: Center for Spatially Integrated Social Science
• Bass BM & Aviolo BJ 1995. MLQ Multifactor Leadership Questionnaire for research: permission set. Redwood City CA: Mindgarden
• Bratton W 1998. Turnaround: how America's top cop reversed the crime epidemic. New York: Random House
• Bratton W 1997. Crime is down in New York City: blame the police. In Dennis N (ed) Zero tolerance: policing a free society, 2nd ed. London: Institute of Economic Affairs
• Brereton D 1999. Zero tolerance and the NYPD: has it worked there and will it work here? Paper presented to the Australia Institute of Criminology Conference, Mapping Boundaries of Australia's Criminal Justice System, March 1999
• Cancino JM 2005. The utility of social capital and collective efficacy: social control in nonmetropolitan settings. Criminal justice policy review 16(3): 287-318
• Chilvers M & Weatherburn D 2004. The New South Wales 'Compstat' process: its impact on crime. Australian and New Zealand journal of criminology 37(1): 22-48
• Dixon D 1998. Broken windows, zero tolerance and the New York miracle. Current issues in criminal justice 10(1): 96-106
• Eck JE & Maguire ER 2000. Have changes in policing reduced violent crime? An assessment of the evidence. In Blumstein A & Wallman J (eds) The crime drop in America. Cambridge: Cambridge University Press: 207-265
• Grabosky P 1999. Zero tolerance policing. Trends & issues in crime and criminal justice no. 102
• Henry V 2002. The Compstat paradigm: management accountability in policing, business and the public sector. New York NY: Looseleaf Law Publications
• Kelling GL & Sousa WJ 2001. Do police matter? An analysis of the impact of New York City's police reforms. New York NY: Center for Civic Innovation at the Manhattan Institute
• Makkai T et al. 2004. ACT recidivist offenders. Research and public policy series no. 54. Canberra: Australian Institute of Criminology
• Maas P 1998. What we're learning from New York City. Parade 10 May: 4-6
• Maple J 1999. The crime fighter: putting the bad guys out of business. New York NY: Doubleday
• Mayhew P 2003. Counting the costs of crime in Australia. Technical and background paper no 4. Canberra: Australian Institute of Criminology
• Mazerolle L, Rombouts, S & McBroom J (2006). The impact of operational performance reviews (OPRs) on reported crime in Queensland: final report to the Criminology Research Council. Brisbane: Griffith University
• Moffat S, Weatherburn D & Donnelly N 2005. What caused the recent drop in property crime? Crime and justice bulletin 85
• Queensland Government 2005. Population growth: highlights and trends. Brisbane: Department of Local Government, Planning and Recreation
• Ratcliffe J & Makkai T 2004. Diffusion of benefits: evaluating a police operation. Trends & issues in crime and criminal justice no. 278
• Safir H 1998. The Compstat process. New York NY: New York City Police Department
• Silverman EB & O'Connell PE 1997. Revolutionizing the police: fighting crime in New York City. Security journal 9: 101-104.
• Walsh WF & Vito GF 2004. The meaning of Compstat: analysis and response. Journal of contemporary criminal justice 20(1): 51-69
• Weatherburn D, Lind B & Ku S 1999. Hotbeds of crime? Crime and public housing in urban Sydney. Crime & delinquency 45: 256-271
• Weisburd D et al. 2004. Trajectories of crime at places: a longitudinal study of street segments in the City of Seattle. Criminology 42(2) May: 283-322
• Weisburd D et al. 2003. Reforming to preserve: Compstat and strategic problem-solving in American policing. Criminology and public policy 2: 421-456
• Witkin G 1998. The crime bust. U.S. news and world report 25 May: 28-36.

About the authors

Associate Professor Lorraine Mazerolle is the Deputy Director of and Sacha Rombouts is a research fellow with the Key Centre for Ethics, Law, Justice and Governance at Griffith University.

James McBroom is a lecturer in the Australian School of Environmental Studies at Griffith University.