By Dr Annabel Prescott, CEO, TRACTION for Young People
In September this year, I got to do something many modern parents never get the space to do: bring my children along to watch me present at a conference! It was wonderful to have their support and having them with me was a timely reminder of what I was there to achieve.
I was invited to present at the Australian Evaluation Society (AES) Annual International Evaluation Conference ('AES26') in Darwin. The theme was about making space for different perspectives, different contexts, and different ways of understanding data.
TRACTION works with young people aged 10 to 16 who are experiencing disengagment from education. Many of them have already been assessed, measured and reported on, over and over, by systems that never once reported back to them directly.
Our new evaluation framework and the way we've built youth voice directly into it, does something most community organisations still don't: it puts the person, not the funder or the system, at the centre of their own research and data. This was one of the reasons I was invited to present.
Having my kids in the room sharpened my thinking while I was there. If this were my child, how would I want their story told using data? Much like you, I'd want it to show potential, strength and progress, not a list of deficits, labels and diagnoses. Every young person in our programs deserves exactly the same!
Here’s a summary of how we do things at TRACTION and the top-level concepts I presented. A link to the full presentations for more context can be found on the AES26 website soon.
Evaluation laddering up to strategy
Our organisational strategy is our "why," which in turn shapes who, what and how we measure impact. We report outcomes at the individual participant level but we also wanted to tell a much wider story about disengagement and how systems respond to young people.
Working with Ithaca Impact, we've built an organisational Monitoring, Evaluation and Learning (MEL) framework that has grown out of our strategy, and I'm very proud of it.
Here's a top line summary of our approach that I shared in my conference workshops:
Young people's experience comes first. We ask whether a program is right for young people before asking whether it works, and safety comes first. If a young person doesn't feel safe and heard, no other outcome counts.
The data young people read is the data we report. The same results a young person reads in conversation with their mentor are those that flow through to ‘cost per outcome’ and funder reporting.
Their reading shapes our interpretation. When a number looks wrong, the first suspect is the instrument, then the environment, and only then the young person. A young person's own account of their progress is evidence, not an exception to be explained away.
Our influence is held to their reality. We evaluate our advocacy and policy work, not just our programs, and ask whether it reflects young people's lived experience rather than our own convenience. Data and reporting off the back must not be used to reinforce strategy; it should inform it.
The right tools
Most screening tools in everyday use across youth-based sectors are built to detect ‘risk’, ‘deficits’, and ‘disorders’. We take a more positive and constructive angle, choosing strengths-based tools and refusing deficit language in our surveys and program notes.
We use WEMWBS (Tennant et al., 2007) in our flagship BUILD program, and KIDSCREEN-10 (Ravens-Sieberer et al., 2010) in our IGNITE program.
Tools like these are harder to find than they should be. Both frameworks ask about wellbeing, including whether a young person has been feeling useful, feeling confident, and able to make up their own mind when they start and finish a TRACTION program.
We complete these measures through conversations with a mentor, chatting through progress along the way, and at the end we see before-and-after results side by side. As one young person put it: "You actually care enough to sit with me and ask these questions!" They keep their own copy and decide who else sees it.
Our Building Deadly. Riding Deadly program supports First Nations young people, co-designed with Gunya Meta, under the guidance of Aunty Faith Green. Participants answering their survey alongside mentors told us 96 per cent felt listened to in the program, compared with 56 per cent at school. This gap reinforces something we've always known: the problem is not the individual; it's the environment. A small example of how data-driven insight can reinforce our wider purpose.