Failure to meaningfully distinguish between ‘outcomes’ and ‘outputs’ remains widespread, despite the efforts of policymakers in some countries.
The success of employment policies, for example, may be judged by simple ‘outputs’ such as the number of new jobs or changes in the unemployment rate, rather than ‘outcomes’ like the improvement in job quality across society. The raw number of social housing units is another ‘output’ measure, yet tells us little about the improved health, educational and employment outcomes of tenants resulting from the provision of safe, secure housing.
It is often said that bureaucrats like to focus on such ‘outputs’ because they’re easy to measure. I like to think, however, that most public servants entered public service to make a positive difference in society to improve economic and social conditions, and would much prefer policy settings and activities better affecting these ultimate outcomes.
There has been much positive work about managing for ‘outcomes’, yet the perennial debate about employment and social housing numbers often falls back to simple yet arguably largely meaningless measures.
More jobs and more social houses may be a good thing, yet it is the increase in well-paying, secure, quality employment and improved social conditions across the economy that actually makes a real difference to our society. It’s the same for crime rates.
Crime Rates as a Measure of Policy Effectiveness
Crime rates have long been a simple and familiar measure of criminal activity. A combination of factors – including a focus on higher-order ‘outcomes’ by government, NZ Police’s transformational change from traditional reactive policing to a preventive model, and a significant and sustained ‘crime drop’ – suggest that debate on crime rates as a performance measure may be needed. Scholarship on policy effectiveness, outcomes and evidence-based policing may also now be sufficiently advanced that we can use new measures more directly linked to outcomes, enabling policymakers and police to better direct resources towards preventing crime, and further boosting government crime prevention objectives.
Traditional Crime Rate Reduction Targets Do Not Fully Reflect Policy Outcomes
Crime rate targets are often referred to as policy ‘outcomes’, but they might more accurately be described as ‘outputs’ or ‘intermediate outcomes’, as they don’t fully reflect the ultimate intentions of crime prevention policies.
For example, a 20 per cent reduction in crime is largely meaningless, except as an indicator of the societal benefits represented by less crime. Successive targets (say a further 10 per cent, then another 5 per cent from ever-reducing baselines) seem like diminishing returns, yet the effect or impact (i.e. ‘outcome’) of successful crime reduction activities (particularly the economic and social benefits from each successive gain) may be at least as significant as the initial target.
Even if we implicitly ‘know’ the benefits of reducing crime, percentage figures alone can be deceptive. Crime rates therefore tell us little about policy effectiveness, at least in relation to the underlying economic and social benefits from less crime.
Identifying Links Between Prevention Policies and Crime Rates
Measures of crime prevention policy effectiveness may include crime rates, the number of crimes committed (‘before and after’ comparisons), crimes prevented by the policy intervention (‘the counterfactual’), and the economic and social benefits and reduced harm resulting from less crime. The latter is arguably the ultimate crime prevention policy objective, and is used in this article as an ‘outcomes’ exemplar.
It can of course be difficult to measure the ultimate policy outcomes sought by crime prevention activities.
Three Challenges in Measuring Outcomes of Crime Prevention Activities:
- Factors not directly related to criminal justice policy affect crime. Economic conditions, technology which ‘designs out’ crime (such as vehicle immobilisers and CCTV), and effective urban design all influence crime rates.
- Activities over which central agencies have limited control also affect crime. Anti-money laundering norms expressly co-opt the crime prevention capabilities
of banks and other private sector agencies. Building community crime prevention relationships has long been part of police craft, and entrusting some crime prevention tasks to third parties, often with police guidance, is a common component in situational crime prevention and problem- and community-oriented policing.
- Independent private sector innovation also affects crime. Recent examples include an online ‘Spotlight’ tool to locate sex traffickers, and ‘real-time’ risk analysis tools to reduce retail and insurance fraud. The publisher of this magazine is another example. Auror uses technology to cut high-volume theft-related offending as “the most unsolved of all crimes” because the cost of traditional investigative methods is “prohibitively high”.
Crime rates are affected by many political, economic and social factors. This means that solving the attribution problem – of proving links between prevention activities and crime rates – is complex. In a rising crime environment a crime drop may be hailed a policy success with barely a nod to causation principles. But in a falling crime setting, it is difficult to achieve even the veneer of apparent causality. Robust, scientifically sound evaluation is critically important.
Inadequacies in Crime Rates as Performance Measures
Crime rates are dropping in many countries, making this an ideal time to discuss their use as an effective measurement tool.
It is especially timely because, even as scholars continue to grapple with its causes and policy implications, the worldwide 1990s crime drop has already prompted potentially profound policy responses in several countries.
In England, the Home Secretary denounced police arguing against proposed budget cuts as “crying wolf” in the face of falling crime rates. The budget debate is ongoing, but whatever the result, the policy effectiveness narrative
is changing. Having long relied on crime rates as a measure of effectiveness, English police now argue that such statistics don’t tell the whole story. An ‘outcomes’ focus may be emerging. The Chief Constable responsible for crime statistics has acknowledged the challenge of addressing serious and complex crime where the outcomes of the police response are not easily measured in the current system.
In Australia, the Queensland Organised Crime Commission of Inquiry found that “extreme legislation” (in that case, a ‘crackdown’ on outlaw motorcycle gangs, although representing only 0.52 per cent of crime) meant that authorities “lost visibility” of other serious crimes. The Commission also described outcomes as the ultimate effects and impact of policy interventions.
Chillingly, although the ‘outputs’ of more arrests may have been judged by some as policy success, the ultimate ‘outcomes’ recorded by the Commission included “profound” and “severe” economic and societal effects of “significant numbers” of children sexually exploited, the “immense” impact of financial crimes “allowed to flourish”, and an estimated 320,000 fraud victims for whom the state had no capacity “to provide an investigative response.”
Increasingly, a wide range of academics, lawyers, police and other criminal justice practitioners have called for a “paradigm shift” towards better ‘outcomes’ in future to help avoid some of the consequences of ‘output’- driven policies such as those identified in Queensland. Nonetheless, it remains difficult currently to discern any substantial policy shift at the political level in Australia.
Elsewhere however, a search for better long-term outcomes is gaining traction in political quarters, aided by a growing body of evidence. In the United States, the head of the White House Office of National Drug Control Policy has begun recalibrating discussion towards ultimate policy ‘outcomes’. Mr Botticelli denounced the “failed policies and failed practices” of America’s decades- long ‘war on drugs’. “Blunt force didn’t knock out the drug epidemic… We can’t arrest and incarcerate addiction out of people… It’s ineffective, and it cost us billions upon billions of dollars”.
New Zealand’s Associate Health Minister Peter Dunne similarly observed that a new national drug policy, which seeks to treat the use of drugs and alcohol as a health issue, is part of “a significant shift on the global level in the last two to three years.”
Perhaps more strikingly, particularly in light of the vehement reactions from ‘tough-on-crime’ proponents to the initiatives noted above, police are now leading a renewed focus on better economic and social ‘outcomes’.
In an “abrupt public shift in philosophy” for law enforcement officials “who have sustained careers based upon tough-on-crime strategies”, more than 130 U.S. police chiefs formed a national alliance to reframe the criminal justice conversation, away from “lock ‘em up” output measures such as mandatory sentencing and three strikes laws, towards more effective outcomes.
In New Zealand, Justice Minister Amy Adams launched a new tool to give judges and probation officers better information to help consider the effect of various sentencing options on re-offending. She reportedly believes this new ‘outcomes’ approach has the potential to “radically change the way policymakers, judges and the general public think about the balance between rehabilitation and punishment”.
The opposition Justice spokeswoman, Jacinda Ardern, reportedly supports such evidence-based policy and sentencing. Care would be needed, she added, so it is too early to suggest a paradigm shift towards a sustainable ‘outcomes’ approach. But if a renewed focus on better outcomes for society has the potential to unite politicians, it helps reinforce the optimistic belief expressed in the introduction: that policymakers enter public service to make a difference. Rather than fighting over whether sentences of imprisonment should be longer, or mandatorily imposed irrespective of their effect, debating instead “what actually works?” has the potential to achieve better economic and social outcomes, in terms of the actual effect and impact of policy choices.
Police Acutely Aware of the Harms Caused By Crime
Police clearly understand the harms caused by crime, and the value of efforts to reduce harm, yet this is not fully reflected or quantified by crime rate measures.
For example, after the seizure of packs of methamphetamine each valued at $28,000, Detective Sergeant McNeill referred to “twenty bags of utter misery, and by that I mean the flow-on effects that the sale of this drug would have had on the users, their families, friends and the rest of the community.” Falling crime rates may imply less harm, but offer little meaningful insight on the real impact of less crime.
A complex mix of the perceived failure of ‘tough-on-crime’ policies and a change in the criminal environment in many countries arguably exposes flaws in the dominant crime prevention policy effectiveness metric. New measures offering a more direct line of sight between policy interventions and the economic and social benefits and reduced harm from less crime might better reflect crime prevention ‘outcomes’.
Ultimate Policy Effectiveness ‘Outcomes’
Supplementing basic crime rate data with measures of the seriousness of crime has a long history. As one scholar noted it is now arguably possible to “distinguish the harms of different crime events based on a financial assessment of societal impact.”
However, whether it is appropriate to do so, and if so how, is not without debate. Some argue that early economic cost-of-crime analyses influenced the “out-of- control emotionalism and mean-spiritedness” of extreme sentencing in the 1980s and 1990s, and represents only the “dead end of a road that should never have been taken”.
A range of improvements have now emerged, but for any new measure to be effective in practice, it must extend beyond theoretical policy evaluation. It must be robust, evidence-based, and capable of being ‘operationalised’ into ‘real-time’ policing.
Policy Effectiveness Meets Harm-Focused Policing
A strong body of literature now offers frameworks for assessing and evaluating harm, “as distinct from the perceived seriousness or costs of crime.” The eld is still developing, but a focus on the consequences of crime offers proactive police leaders and policymakers opportunities to lead a robust evidence-based, outcomes-oriented debate more effectively focused on reducing harm and crime.
Harm-focused policing linked to existing deployment and performance measures also offers a more nuanced hot-spot analysis than traditional intelligence-led policing techniques alone.
Harm-Focussed Policing and Outcomes-Oriented Metrics
Crime rates are effective in part because they are easy to use and understand. Increasingly complex cost-of- crime models suffer from a lack of simplicity, because attempts to make them simpler often add more layers of complexity and make them harder to use.
Distilling the academic literature and practical experience reveals a roadmap for measures with a more direct focus on outcomes to help achieve better results, consistent with an emerging political consensus in many countries that simplistic ‘tough-on-crime’ rhetoric is costly and sometimes counter-productive. If the intention of crime prevention policies is to make a real and demonstrable impact in cutting crime, reducing the harms from crime, and improving the economic and social outcomes from less crime, there is now a deep well of knowledge from which simple and effective new measures can be designed to better help meet those goals.
Conclusion: Realising Better Crime Prevention Outcomes
Crime rates retain legitimacy as a simple ‘output’ measure because it is often difficult to identify direct links between policy initiatives and crime, but the use of basic crime rates as an effectiveness metric has not kept pace with the modern change towards preventive policing—particularly when crime rates in general are falling.
Quantifying the economic and social benefits from crime prevention targets, and demonstrating policy outcomes beyond broad statements about reducing harm is the obvious next step for modern crime prevention using crime science to “understand the crime problem [and] make sound decisions that lead to good outcomes”.
Stepping beyond the comfort of easy-to-measure outputs, the future of policy effectiveness in the public sector generally may also be defined by measuring and achieving better economic and social outcomes. Realising those resulting from less crime would be a great start, and perhaps world-leading.