Measures and impact
Evaluate the changeMethods for assessing our work and its impact
Outcome-focused self-evaluation, is a positive part of all our work with communities. How and what we choose to measure should be part of the initial planning for a piece of work, however there are challenges involved in deciding the appropriate measures to assess our work.
What works for one piece of work may not be right for another, or we may have measures imposed upon us as a condition of funding. Careful consideration should be given to how we collect insight, intelligence and data, as well as when this should be done and by whom, so we can understand the difference our work has made to communities and how our work contributes to building inclusion.
Partners' understanding of good practice
During Design Circle 3, which was one of 3 design circles formed to creating the framework, partners shared examples of how and what they measure to assess the impact of their work, as well as their understanding of the good practice considerations. These build on the pillars and inclusive approaches within the toolkit and are our starting point when thinking about how to measure and evaluate our work with communities.
- As we are planning and delivering work, we will build in time for the right kind of assessment methods, with a focus on 'how will we know we have made a difference?'
- These are not afterthoughts or add-ons to be done hurriedly after the work is completed but an integral part of what we are doing.
- Whenever appropriate, we will make time for listening, reflection and relationship-building.
- We will gather and draw understanding from both quantitative and qualitative information. We will make use of the value that comes from both small-scale (intensive) and large-scale, extensive, assessment processes. This could include questionnaires and surveys, focus groups, anecdotal comments, mystery shopping, reporting mechanisms, demographic information and asking people to share their experiences through written stories or videos.
- In our engagement, assessment, and evaluation work and in publicising and promoting our activities, we will use different channels and methods, face to face, online and paper-based, considering the principles of environmental sustainability and digital inclusion.
- People from different agencies and organisations, at different levels within the organisations, will share information with each other as appropriate, whilst working in line with relevant data protection legislation and good practice on confidentiality. In this way, all of us will be able to make use of what each of us know.
- Using what is already there including knowledge in the faith sector, local Councillors and the voluntary and community sector.
- We will be alert to such questions as, "who is not in the room?" Are there people with an interest in the issues we are working on, who we feel we have not heard from? On this basis, we will take proactive steps so that people who have not so far been involved, can choose if they want to participate.
- During assessment and evaluation activities, in our engagement and service delivery, we will keep the language simple and free of jargon.
- We will show what we mean by using examples and stories, not just data and concepts.
- We will use categories and language which will be recognised by the people we are talking about and with, and seen as respectful by them.
- We want to be innovative and brave; we want to understand and get underneath the 'hard to define' challenges in local neighbourhoods and communities. This might include issues of trust, influence and leadership that can be complex and embedded.
- The quality and quantity of social connections; the levels and forms of social mixing between people from different groups, places and communities; attitudes towards difference, diversity and shared values, are all considerations.
- We will notice differences and distances between the way that official organisations are talking about an issue and how community members see things, for example when we discuss an area being safe even when local people tell us they don't feel safe,
- We will acknowledge the problems and challenges which we and our communities are facing, and the challenges we face together in addressing them.
- Honest discussions, listening to each other and working out who is best placed to make a difference, may be more impactful than: 'doing what we always do and getting what we always get'.
- We will write up and promote good news stories, sharing the successes together, which will often be about how we are addressing the problems and challenges.
- We will organise celebration events and use other ways to highlight the positive steps we are all taking and the good work that is going on.
Ashbrow Community Youth Support Strategy
The area was experiencing a high level of gang related activity and violence, involving young people, and although there was lots of positive activity taking place to try and counter this, the impact wasn't clear. A series of meetings were held involving ward councillors, residents, community groups, schools, Kirklees Council, the police and others.
From this, a community-based strategy to tackling the issue was developed. The purpose of this was to develop a way of working collaboratively with young people and communities in the locality, improving understanding, intelligence and the ability to meet needs and tackle the issues. It also aimed to support the community response and understand how agencies can best work together to engage, work with, support and respond to these locally identified needs.
From this work a local network of around ten community organisations has developed, which has led to greater collaboration and co-ordination of activity locally, with groups supporting each other. The voice of young people has informed the approach through those community groups and the Ashbrow Youth Panel, ensuring the insights of young people are central.
As well as ensuring a strong network of activities, the approach has also involved developing local learning, volunteering and employment opportunities to provide positive pathways for young people.
There is still more work to do to tackle the issue of serious violence in our community, but the work to date provides a strong foundation on which to build.
How we will know we're making a difference
The Inclusive Communities Framework (ICF) sets out a way of working, and has a toolkit that helps organisations review and improve how they work with communities.
It also recognises that in Kirklees we have shared outcomes and measures. For instance, we have all committed to working towards and achieving, partnership shared outcomes for the district: Our Council Plan 2021/23. Many of us also have organisational measures, such as corporate plans or measures required by funders. Others have national and, or regulatory measures i.e. Ofsted, Police Performance Outcomes and Equality Impact Assessment.
Measuring the impact of a 'way of working' can pose challenges to more traditional approaches to measuring service impact. With this in mind, we have intentionally avoided including prescribed outcomes for partners within the ICF. What partners have all agreed on is that 'If local people feel safer, better able to influence decisions and connected to their local communities' then the ICF is doing its job. We need to ensure that we are tracking actions that we, and all individual organisations, are taking to achieve this against our inclusive approaches.
In order to measure the impact that the ICF is having overall, we have developed a high level self-evaluation tool.
