Our early years qualitative developmental evaluation with The Centre for Education and Youth

• Since our pilot back in 2015, we have worked in early years settings supporting children and families to be ready for school and learning
• We share the thinking behind our exciting partnership with The Centre for Education and Youth (CfEY) who are currently evaluating our early years programme
• Now 9 months into our two year evaluation with CfEY, we hope to shed light on how our qualitative developmental evaluation is designed to provide rich insights into the impact of our work and support our programme learning and development

Monitoring and evaluation is at the core of West London Zone, with our evidence-led value driving the way we support young people on our programme. Andrew Berwick kicked off our series on evaluation last month, reflecting on some of the challenges of using experimental methods for complex social programmes. Because of these kinds of challenges, we intentionally employ a range of evaluation approaches and methods internally and with external partners. In this post, we share the thinking behind our exciting partnership with The Centre for Education and Youth (CfEY) who are evaluating our early years programme: we hope to give some insight into how a qualitative, developmental evaluation can provide rich insights into the impact of our work with early years children and families.

Our early years journey

We started working in the early years settings in 2015 as part of the West London Zone (WLZ) pilot and have been learning and developing ever since. Initially working in standalone early years settings supporting children aged 3 to 5, we have since learnt that our programme works best as a school-based programme, focusing on children in nursery, reception and year 1.

We know that children on the programme particularly improve in language and cognition, and behaviour: around 70-80% of ‘at risk’ children improve on these measures within The Brief Early Skills and Support Index (BESSI). However, before considering any potential expansion of our early years work we wanted to understand better what activities were helping to drive this change, and whether there were other opportunities to engage with all of the other organisations involved in supporting early years children.

A development evaluation

To answer these questions we have commissioned a qualitative developmental evaluation. Developmental evaluation emerged to facilitate ongoing learning, decision making and adaptations in complex and dynamic programmes, systems and environments - like the early years system that we are working in (Patton, 2011)*. We also believed we would benefit from working with an external partner to support us to design and implement the evaluation and foster shared learning. The ideal team needed to have experience working with children, families and early intervention programmes, and bring strong expertise in approaches that are participatory and collaborative – and we found that in CfEY!

The ideal team needed to have experience working with children, families and early intervention programmes, and bring strong expertise in approaches that are participatory and collaborative – and we found that in CfEY!

Our partnership with CfEY

CfEY is a ‘think and action tank’ who believe all children and young people should receive the support they need to make a fulfilling transition to adulthood. They provide rigorous research to support organisations to increase their impact and to get under the skin of issues affecting young people in order to shape the public debate and advise the sector.

We’re now 9 months into our two-year evaluation with CfEY. The evaluation takes a holistic approach: it draws insights from the full range of stakeholders involved in our programme (from children and families to school staff and local service providers). It is rooted in qualitative fieldwork, including observations and participatory activities with children in a small number of settings, plus wider reaching surveys and interviews.

Unlike a formal quantitative research design which is typically stated up-front, and then held static, this evaluation uses a process of ongoing reflection and discussion to adapt the methodology in response to emerging findings. Our starting point was an early years themed workshop, where the WLZ team came together with CfEY to establish initial hypotheses on what works well and where we could improve, based on our experiences and monitoring and evaluation of the programme to date.

In an early years themed workshop: Brown ‘seed packets’ represent activities; green ‘leaves’ represent impact; blue ‘water droplets’ represent factors that facilitate impact; and grey ‘slugs’ represent limiting factors.

We are really excited by this project, as it puts the perspectives of children, families and professionals at the forefront of the evaluation. We believe this is particularly important for this programme: the data suggests we are having a positive impact, but we want to understand how, and whether there are other opportunities within the early years space that we haven’t made the most of. We believe this shows the value of taking a mixed approach to evaluation that uses data alongside the lived experience of children, practitioners and families to understand how change is happening.

The CfEY evaluation will conclude in Summer 2024 – we look forward to sharing the insights from the evaluation with you all then. In the meantime, if you would like to find out more about CfEY or you are interested in some similar support and learning for your own project, Ellie Mulcahy, Director of Research and Operations, would be glad to help (ellie@cfey.org).

*Patton, M. Q. (2011) Developmental evaluation: applying complexity concepts to enhance innovation and use. New York: The Guilford Press.