In recent years finding a 'what works' catalogue for crime prevention action has become of prime importance for practitioners and professionals in the field. If we can just work out what crime prevention tools work best when and where, then we can successfully apply these to similar problems.
'What works' analyses are about accumulating evidence and developing evidence based policy and programs (EBPP). However, EBPP and what works analyses remain controversial (Pawson 2006). This fact sheet looks at two of the most frequently used methods for building what works program evidence bases: meta-analysis and narrative technique.
The meta-analysis approach relies on a graded assessment of the scientific strength of the evaluation design of existing program evaluations. Meta-analysis is effectively an arithmetic extraction of 'net success' that seeks out types of programs that have worked to maximum effect on the widest scale. Studies using a randomised control trial are generally considered to be the gold standard of the meta-analysis approach. This approach is often associated with the review work of Lawrence Sherman in the USA and David Farrington in the UK (Sherman et al. 2002) and underpins the design of the international Campbell Collaboration project. The approach is highly quantitative and reductionist, as the key elements of successful or effective programs are distilled down to just a few elements. Knowledge transfer is seen as a matter of identifying and imitating the most powerful classes of programs and then generalising these findings for mainstream implementation. This approach is often criticised for being too 'top down' and exclusionary to properly reflect real life experience.
The narrative technique is a more qualitative and descriptive approach of program analysis and evaluation, where stories of experiences are tallied to identify the common success factors. Programs are said to work because of the compatibility of the target group, the setting, the strategy taken, program content, implementation details, stakeholder alliances and so on. The idea is to learn from review by following the successful programs. Using this approach to cataloguing what works, any future program design should attempt to imitate the program as a whole or at least try to gather in as many similarities as possible (Pawson 2006). The narrative approach has been described as involving the intuitive extraction of 'exemplars'. A frequent criticism of this approach is that it is time consuming and expensive as well as being highly dependent on personal insights and informal knowledge.
Flaws exist in both approaches and a future AICrime reduction matters will present a recent attempt to address these deficiencies, known as the realist synthesis.
References
- Sherman L et al. (eds.) 2002. Evidence-based crime prevention London: Routledge
- Pawson R 2006. Evidence-based policy: a realist perspective. London: Sage
- Campbell Collaboration