"Highlights" of a criminological career: Anthony Doob and the state of evaluation research in Canada.

• Author: Wortley, Scot
• Position: University of Toronto Canada Criminology Centre professor

Introduction

After examining the current state of crime prevention in North America, Ronald Clarke (2001: 28) noted that "[w]ithout evaluation, crime prevention cannot be improved, and without hard evidence about effectiveness, reliable information about best practices cannot be disseminated." Professor Anthony Doob, whose career has been devoted to the principles of empirical evaluation and evidence-based policy development, would undoubtedly agree with this statement. Since the early 1970s, he has disseminated criminological knowledge through a wide variety of forums, including peer reviewed articles, books, government reports, expert testimony, conference presentations, lectures, and the popular media. Perhaps the best illustration of Doob's commitment to making criminology relevant to policy makers, criminal justice practitioners, and members of the public, however, is Criminological Highlights.

Organized and edited by Doob, Criminological Highlights (CH) has been published since 1997 by the Centre for Criminology and Socio-legal Studies, University of Toronto. Each of the four issues distributed annually includes concise summaries of eight, high quality articles, published in one of the hundreds of academic journals that cover crime and criminal justice issues. Doob, with the help of the CH editorial board, scours this vast literature to uncover research particularly relevant to policy makers and practitioners who may not have the time, money, or expertise to conduct their own review. Recent topics covered in CH include the effectiveness of treatment programs for young offenders, the consequences of mandatory minimum sentences, the effects of imprisonment on future offending, racial profiling, the globalization of organized crime, public perceptions of the police, and the relationship between fear of crime and civic engagement. Criminological Highlights is read widely by government officials at the local, provincial, and federal levels, and by judges, lawyers, and police and correctional officials. Professionals from over 35 countries subscribe to this service and describe it as "invaluable," "indispensable," and "enormously credible." (1) However, despite its international reputation for excellence, the federal government stopped funding Criminological Highlights in 2011. Thankfully, the Government of Ontario ultimately stepped in and renewed funding--at least for the next few years (Gardner 2011).

The federal government's decision to end funding for Criminological Highlights should send a chill through the academic community and raise questions about the importance of criminological research in the eyes of government officials. To what extent will research findings inform the development of criminal justice policy in Canada? Will governments continue to fund research designed to evaluate the effectiveness of various crime prevention strategies? In this paper, we first review the state of evaluation research in Canada and then discuss various obstacles faced by criminological researchers interested in the issue of evaluation. We conclude by arguing that, for criminology to matter outside of academia, the next generation of criminologists must make their research both relevant and accessible to the general public as well as policy makers. In other words, we must learn from the example set by Doob--and follow in his footsteps.

Evaluation research in Canada

Doob and other Canadian academics are not alone in their support for evaluation research and evidence-based criminology (see, e.g., Hudson and Roberts 1993; Hudson 2005; Hastings 2005; Tremblay 2006; Waller 2008; Gendreau, Smith, and Theriault 2009; Linden, Mann, Smart, Vingilis, Solomon, Chamberlain, Asbridge, Rehm, Fischer, Stoduto, Wilk, Roerecke, Trayling, and Wiesenthal 2010; Slinger and Roesch 2010; Wormith 2011; Cook and Roesch 2012). For example, a recent study, sponsored by the National Crime Prevention Centre, asked over 200 Toronto stakeholders to express their views on program evaluation (Wortley and Tanner 2012). Respondents included local politicians, police officials, judges, lawyers, social workers, educators, NGO directors, program staff, and residents of socially disadvantaged neighbourhoods. Every one of these respondents--100% of the sample--felt that program evaluation is important. The comments presented below are typical: (2)

Evaluation? I think it has to be done. We've spent years and countless millions of dollars throwing our money at projects that we're never sure actually work. And I think it's a valid and reasonable thing to say, well, is this effective? We need to study these things to see what we should be funding and what programs are a complete waste of our time and money. (City Administrator)

If you're not evaluating, you're not really seeing what works and what doesn't work. And being able to look at the statistics and see that this program, on paper, is working or not, it's really important to do that. A lot of the times, the higher ups don't actually see what's happening on the ground level, so it's a good way to measure things and document things and a good way of ensuring good practices. (Youth Program Director)

I think that it's probably a good idea. Because otherwise, how will you know if a program is good if you're not keeping track, if it's not being evaluated from time to time? I think you need evaluation to be held accountable and I think that those evaluations help keep you accountable. And at the end of the day evaluation can help create better, stronger programs that might actually work at preventing violence. (Program Case Manager)

It's about ethics. You need to evaluate to know if a youth program or an anti-crime strategy is working or not. You need to know how to make it work better. You need to know if you are using tax payers' money correctly. It helps the government decide where it should put its resources. That makes it an ethical issue as well as an issue about knowledge production. (Member of City Council)

I strongly believe in evaluation. We need to evaluate law enforcement efforts, sentencing policies, correctional policies as well as community crime prevention programs. This is the only way we will be able to develop a united front against crime and fight violence in an intelligent manner. Evaluation also makes organizations accountable. When you know that you are being evaluated, you work that much harder. (Police Official)

Although support for evaluation research may be strong, the actual Canadian evaluation record is not that impressive. An extensive search of the criminological literature on youth violence prevention, for example, identifies few published studies that document the effectiveness of Canadian law enforcement strategies or community-based programs (Wortley, Levinsky, Owusu-Bempah, Marshall, Adhopia, Samuels, Cook, Roberts, and Boyce 2008). The Ontario Government's Roots of Youth Violence Inquiry also shed light on how few Canadian crime prevention programs are subject to evaluation (McMurtry and Curling 2008). The inquiry produced an inventory of 278 youth violence prevention initiatives funded by the Ontario government. Although these programs had received over $500 million in funding between 2002 and 2006, the majority were not evaluated in any way. Indeed, only 113 (40.6%) were subject to some kind of evaluation effort and only 25 (8.9%) produced a preliminary or final evaluation report.

Experts commonly differentiate between low quality and high quality evaluation studies (Welsh and...