Evidencing Success and Successful Evidence in Restorative Work: Developing Models of “Effectiveness”, “Efficiency”, and “Impact
Benjamin Fisk has been funded by the Restorative Justice Council and University of Gloucestershire to undertake a PhD titled “Evidencing Success and Successful Evidence in Restorative Work: Developing Models of “Effectiveness”, “Efficiency”, and “Impact”. Here he outlines what his research involves and some of the issues it is investigating.
Taking referrals for the family group conference (FGC) service was a great opportunity to develop relationships with professionals, help them learn about restoratives practices, and build trust in our service. A question I heard frequently from those referring for the very first time was: “Is it true? I heard the FGC team doesn’t record anything?”. As much as I might have wished that to be the case, it simply wasn’t true.
Minimal recording is an important principle of restorative practice (RP), espoused by Ted Wachtel in the early days of professional restorative work. Few disciplines are enabled to focus on practice in the way we as restorative practitioners are. That focus on developing relationships instead of churning paperwork, building of trust so participants don’t feel like every word and nuance is being recorded to use against them. This focus is where we derive much success in our work from. This makes sense, restorative practitioners are not usually investigators into the preceding matter, although there are times when this dividing line is not always so clear, and we all have statutory safeguarding obligations that mandate formal reporting. But minimal is subjective, and services record more or less depending on the positioning of the practitioner, the general sector, specific organisational obligations, and the nature of relationship harmed. The Restorative Justice Council (RJC) does not overprescribe in this area, noting concerns about hampering service creativity and independence, although there are still obligations around registered services reporting data.
This is where my PhD research comes in. The All-Party Parliamentary Group (APPG) on RJ enquiry highlighted the explicit need for greater understanding of recording and monitoring processes, and I have been commissioned by the RJC to explore the evidencing of success and successful evidence in restorative work to develop models of effectiveness, efficiency and impact. I am currently distributing a survey for professionals to clarify their personal and organisational definitions in these areas, and in the process of publishing my initial findings from an analysis of 6 reporting templates.
So, the big questions for me are, what is success in restorative work, and how is it captured using the data processes within services? I believe that we all have our own subjective interpretations of success, and these will depend on the sector we work in, and our position in that sector as practitioner, manager, commissioner and beyond. Evaluations tell us a lot about the types of outcomes we can expect, but not about what data services are collecting on the ground. How can we draw conclusions about success if we do not understand the roots of the data used to evidence this? Within a single sector is a challenge enough, but restorative practice crosses sectors with its own values and principles. Researchers have noted the issues with restorative services aligning their data capture with that of the individual sectors, and the need for restorative practices to stand on its own two feet in terms of recording and monitoring success (Llewellyn et. al., 2012; O’Mahoney & Doak, 2017). But can success in different sectors even be compared and equated? We cannot be certain if we are measuring exactly the same things, within the same sector, in exactly the same way.
The Victim’s Code has provided an important avenue within the criminal justice sector to facilitate data about restorative justice services under the umbrella of Police and Crime Commissioners (PCC). The code ensures that access to information about RJ is available, and this is facilitated by the Victims Service Grant which is devolved to PCCs by the Ministry of Justice (MoJ). PCCs are required to report every six months and outline how this is grant spent, but access to this data is currently limited as the MoJ do not publish this directly. This data is also broadly related to victims, of which RJ is an important but singular component among other services. Commercially sensitive information is also contained in this data, which further complicates external access. Recently, a number of attempts have been made to gather and understand this valuable data source. Why Me? have utilised a number of Freedom of Information requests to access some of this data for their “Valuing Victims” reports, and the work of Dr Kerry Clamp (2021) contained in the “RJ and the Code of Practice for Victims of Crime” report provides a comprehensive academic analysis using primary data obtained directly from a survey of 88% of PCCs. These reports offer an important snapshot of the types of data reported, but more work is needed to unpick the on-going role of data in restorative services that truly captures the breadth of work being undertaken and the myriad of successes that can be achieved. As the Victim’s Code develops into the Victim’s Bill, guidance may be forthcoming that can illuminate and facilitate this even further.
The Youth Justice Board is currently exploring how they can capture effective data about restorative work that falls under their umbrella. In other sectors this is less clear, reporting could be contained at the school level, or reported to the local department of children and family services. What is done with data is frequently shrouded in mystery or left to managers to manage alone. Who decides what to record? That information could even be lost to service history as staffing turns over. Is it arbitrary or nuanced, linked to the service itself, or some wider organisational expectations? In terms of success, whoever decides what measures to use will presumably have done so because they know the outcomes they are looking for. Many may argue that success is more than simply counting referrals or completed processes, and there are intangible successes in our work that are simply not recorded at all. When we talk about service satisfaction, are we really measuring what we think we are?
The academic sphere knows so little about how recording and monitoring is undertaken, and practitioners and services are working so hard, so it is not simple or easy for us to get in the thick of it and find out more. The restorative sector has focused on achieving the best outcomes, through the best practice ensuring the processes themselves are of the best quality. Recording and monitoring has taken a back seat. But now, given the position we have found ourselves, with the APPG on RJ refocusing our work, it is important to look at how we are using data, the measurements in play, and how these can be transferred across sectors to advocate for restorative work as a whole wherever it is taking place.
Ben is currently working on two strands of research: He has a publicly available survey for professionals that asks questions about personal and organisational definitions of success in restorative work, which can be found HERE:
Ben would also like to connect with services to discuss their recording and monitoring processes and procedures. If you are interested in taking part or would like more information, you can e-mail Ben at benfisk@connect.glos.ac.uk
References:
Clamp, K. (2020). RJ and the Code of Practice for Victims of Crime. A Scoping Review of Service Provision in England and Wales. University of Nottingham, UK.
Llewellyn, J.J., Archibald, B.P., Clairmont, D. and Crocker, D. (2013). Imagining Success for a Restorative Approach to Justice: Implications for Measurement and Evaluation. Dalhousie Law Journal, 36(2): 282–316. Retrieved from https://core.ac.uk/download/pdf/288305365.pdf.
O’Mahony, D. and Doak, J. (2017). Reimagining Restorative Justice: Agency and Accountability in the Criminal Process. Hart, UK.