As we emerge from the pandemic, calls for action to improve social mobility are growing louder. The Social Mobility Commission (2021) has warned that Covid-19 will exacerbate the “already stagnant” social mobility in the UK. By autumn 2020, disadvantaged pupils in primary school were a total of seven months behind their more privileged peers.
Children excluded from school face a particularly treacherous uphill battle. They are more likely to suffer long-term mental health problems, struggle to gain qualifications, become long-term unemployed, exhibit unhealthy behaviours (e.g. overweight, smoking), and commit crimes. These behaviours aren’t just costly for the individual, but costly for the state to cover the healthcare and criminal justice costs.
In this context, how can we tell what actually works? What interventions really make an impact? Evaluating projects effectively requires organisations to build it in to their projects from the beginning. Asking someone at the end of the project if they feel things have changed does not yield an accurate picture – people’s memories are often fallible. And, to be sure that any changes that are seen are not just chance, government guidance highlights the need to consider what would have happened anyway.
Simetrica-Jacobs is keen to support organisations looking to rigorously evaluate their impact. Simetrica-Jacobs was commissioned by the social impact company, Evolve, to evidence the impact of one of their programmes to support vulnerable pupils’ health and wellbeing.
Evolve works with schools, local authorities, and charities to enhance children’s life outcomes. They aim to break the cycle of inequality and support children at risk of being removed from formal education through their Project HE:RO (Health Engagement: Real Outcomes). The project is an annual mentoring programme targeting pupils with medium-to-high learning, emotional and/or behavioural needs. It aims to improve self-regulation and learning behaviours, which may have been compromised by early childhood trauma, through early interventions in primary schools delivered by their workforce called Health Mentors.
Evolve programmes have been successfully administered in over 500 schools since 2012, also deploying Health Mentors on a full-time basis for over 50 schools across England and Wales. Participating pupils are selected based on their behavioural needs or issues with self-confidence and are usually in the lowest 10% of the school population for health and wellbeing, as measured by Evolve’s Wellbeing Compass diagnostic tool.
In order to establish the impact of Project HE:RO on vulnerable school children, we tracked the same pupil’s wellbeing over the course of the school year. We compared this group against pupils in selected schools not participating in the programme. Such an approach, known as difference-in-difference, allows us to see changes in time against any wider trends. It is recognised as a robust mean of evaluating projects of this nature.
Pupils were asked to complete a survey of 20 questions asking them to report their wellbeing across five domains: physical activity, diet, personal wellbeing, emotional development, and cognitive functioning. The resulting Wellbeing Compass score ranged from 20 to 100 (most favourable). The data for this project was collected from 10 participating schools over the 2018/19 schoolyear for Year 4 to Year 6 pupils (aged 8 to 11). We collected data on a pupil’s school year, class group, and gender to then identify under which conditions the programme works best. Wellbeing data was collected before the project started, and at the end of each of the three school terms.
Our Key Findings
Our analysis found that Project HE:RO significantly increased pupils’ wellbeing as measured by the Wellbeing Compass. The above figure shows that before the project intervention, pupils selected into the programme had on average a wellbeing score 8.4 points (on a scale of 20-100) lower than non-participating pupils (60.7 vs 69.1). Analysis showed that by the end of the school year, the wellbeing gap was closed to the extent that we see participants are as happy as their peers with an average increase in wellbeing of +9.7 (scale of 20-100), as seen in the table below.
And the benefits don’t just stop with the pupils who are directly involved. We also found that schoolmates who were not in the programme also benefitted through spill-over effects from the presence of a health mentor, with increased wellbeing compared to pupils in non-participating schools. Furthermore, those spill-over effects increase with the number of days the health mentors spent in the school.
What is the implication of our findings? These results are important because the research targets children at risk of school exclusion and highlights the importance of early intervention which can bring about at least short-term wellbeing improvements. These preliminary findings suggest positive impacts on children’s wellbeing and cognitive function through Project HE:RO and consequently, much improved likelihoods for at-risk pupils continuing in stable, formal education.
Given that the economic cost of school exclusion has been estimated to be at least £370,000 per young person, this is a problem worth solving. If future research proves the long-term effectiveness of Project HE:RO and similar intervention programmes, wide-spread implementation would promise profound societal impact.
References
Gill, K. et al. (2017) Making the Difference: Breaking the link between school exclusion and social exclusion, Institute for Public Policy Research.
Social Mobility Commission (2021) State of the nation 2021: Social mobility and the pandemic.
World Economic Forum (2020) The Global Social Mobility Report. Equality, Opportunity and a New Economic Imperative.
Want to learn more about how we used statistical analysis to quantify the impact of this programme on wellbeing, or find out how we can help you prove your programme’s impact? Please get in touch with information@simetrica-jacobs.com.
