Australian Institute of Criminology

Skip to content

On developing the framework

The purpose of this section is to briefly discuss the experiences of the authors in developing the framework and identify some lessons that might be of value for others working in the field in either developing or critiquing similar frameworks. The theoretical challenges that abound within performance management have already been touched on; in this section, the focus will be more on the detail of implementing such a framework. Some key points should be noted:

  • Management support. Any move towards innovation in performance management (which, by extension, may lead to criticism of units and staff) requires strong support from management. The authors were fortunate to have an innovative, future-oriented manager who was supportive of the new performance framework from the outset. Without support from the top, it is probable that staff would have been far less willing to provide the data required to construct the framework. When units delayed their reporting, communication from the Commander quickly cleared any roadblocks. The Commander was also supportive of the constant development and enhancement of the framework, rather than demanding that it stay in a fixed form. The support of the Commander was also vital to the success of the Centre for AMCOS Lessons Learned, a concept that seemed foreign (and perhaps a little frightening) to the staff being evaluated. Again, strong leadership and clear communication brought staff on board.
  • A sound foundation. The Performance Framework was built on some existing work done between 2005 and 2007 to enhance AMCOS intelligence structures. As intelligence systems and processes were improved, there was a steady breakdown of information silos, which in turn, led to a greater focus on sharing knowledge and information. Without this initial work, the barriers encountered by the performance framework would have been much greater. The work on intelligence helped introduce staff to the concept of disruption, thus making the incorporation of this into the Centre for AMCOS Lessons Learned easier.
  • Consultation. Too often, performance frameworks seem to be developed in an ‘ivory tower’, separate from the true nature of work (the authors grant that given their academic backgrounds, some readers may find the above comment to be unintentionally humorous). The authors countered this by visiting as many units as they could and talking through the issues with frontline staff. To ensure that staff feedback was continually incorporated, a survey of opinions about the framework was held at the end of 2009, after the first two performance reports had been completed. The framework was rated 3.4 out of 4 for usefulness and 3.1 out of 4 in terms of accuracy. Findings from this survey were then used to improve the next performance cycle. A more recent survey (in 2011) asked for opinions on the latest version of the performance framework, now that it has been partially integrated into the planning process. This time, the framework was rated 9 out of 10 for accuracy, but only 6.8 out of 10 in terms of real value in measuring performance. This lower score is interesting. It is hypothesised that staff, having been exposed to a more advanced mechanism of measuring performance for two years, now have heightened expectations of the framework. It is hoped that future improvements to the framework noted earlier in this report will also improve the perceptions of staff and therefore increase survey scores. Surveying staff about performance (or indeed any sort of governance or management framework) is valuable, as the findings of such surveys are an excellent way to drive further improvements. If those developing performance frameworks do not discuss the framework with those being measured, they will likely end up developing a framework divorced from reality (and thus likely sub-optimal) and are almost certain to encounter staff opposition (and perhaps many of the perverse behaviours noted earlier).
  • On a related point, at times, senior management may not be entirely sure what should be measured. Many managers have been brought up on a diet of crime rate data and may not have a clear idea what the objectives of specialist units truly are. This point reinforces the need for consultation at the grassroots, as frontline staff will be able to provide the necessary information for the development of meaningful measures.
  • Proportional use of specialist resources. This point may be specific only to AMCOS, but is assumed by the authors to be universal. It relates to the share of specialist policing services provided to other policing groups. As police forces are heterogeneous, and traditional performance management schemes often stimulate competitiveness, there will often be keen interest as to the proportion of a particular service (fingerprints, dog deployments) provided to particular groups. Any framework must include some mechanism to identify these proportional shares and indeed, the authors would recommend that the more nuanced the data, the better. This allows for additional ad-hoc reports to be quickly developed in response to concerns or complaints. It also allows for the production of interesting analyses correlating crime rates in particular areas with the usage of particular specialist policing services.
  • Theoretical issues of niche unit performance. The measurement of performance by niche units—proactive specialist investigation units in particular—is a problem that, to use the same analogy as this report’s title is a ‘nut that nobody has yet cracked’. It is the view of the authors that the only way to validly evaluate the effect of major investigative operations is to conduct operation-specific post-operational evaluations. These are incredibly resource intensive. The authors also believe that there is a need for substantial public surveying about the prevalence of organised crime. Also, understanding that the public may be reticent about commenting on such a topic, there should also be regular surveying of Covert Human Intelligence Sources as well. A combination of post-operational evaluations, public surveying and Covert Human Intelligence Sources surveying will enable us to begin developing real outcome measures for proactive specialist investigation units, in the organised crime field at least.

Those of an outcome-oriented personality who have read this report probably have a simple question—has the framework improved AMCOS performance? It is a difficult question to answer. Without going into sensitive detail, there have been substantial improvements in terms of the quantity and quality of outputs delivered by some units, although whether the visibility granted by the framework was the reason is unknown. There has been anecdotal evidence that increased reporting of operational workloads has stimulated enhanced esprit de corps. There is also evidence that some components of the framework, notably CALL, have had a directly beneficial effect on some tactics and procedures. The most telling piece of information comes from overall AMCOS Satisfaction Survey results in 2010 and 2011 (both carried out well after the Framework was first implemented). The AMCOS Satisfaction Survey asked recipients of specialist policing services how they perceived the delivery of those services. In 2010, the average score for satisfaction was 3.38 out of five; in 2011 it had improved to 3.82 out of five. It should also be remembered that any performance framework, no matter how well-designed, is only as good as the use made of it.

Conclusion

In this paper, the development of the AMCOS Performance Framework has been described. AMCOS is a specialist policing body and specialist policing—whether forensics, specialist operations, specialist investigations, or intelligence analysis—is an area that traditional police performance measurement schemes do not deal well with. Indeed, some feel that traditional police performance measurement schemes do not deal well with any police work and instead, lead to perverse behaviours that have little or nothing to do with the true value of policing.

The AMCOS model was developed as a response to criticism of the visibility of AMCOS activities. It was derived from analysis of both strategic goals and tactical workstreams, and to date has been largely focused on outputs and activities, due to the fact that AMCOS contributes to but is seldom responsible for policing outcomes. Where AMCOS is responsible for such outcomes, however, some outcome measures have been developed. Performance indicators within the framework cover all aspects of AMCOS business and in the 36 or so months since it was first developed, the framework has been subjected to several major revisions.

The value of locally developed frameworks seems obvious—they are more easily accepted by staff, more difficult to manipulate and more accurate in terms of the measures they focus on. In the future, the goal is to increase the focus on outcomes within the AMCOS Performance Framework and combine it further with planning, project and risk management activities to provide AMCOS with the basis for true policing professionalism into the 21st century.