Establishing a shared measurement system is embedded in the Collective Impact framework. As we continue to experiment with and refine our approach to this defining feature of Collective Impact, we have learnt that each Collective Impact initiative utilises a variety of measurement and evaluation methodologies. What is consistent across all community change initiatives in South Australia, however, is the importance of evaluating progress and measuring impact. To do this, we need reliable, consistent, qualitative data sets from such sources as the Australian Early Development Census and the Australian Bureau of Statistics, in additional to local qualitative data collected from communities themselves. Qualitative and quantitative data create a unique community narrative to inform the work of a Collective Impact initiative.
We continue to develop our understanding of measurement and evaluation and how it continuously informs community initiatives. Below are a few snapshots highlighting where measurement and evaluation have played a key role in our Collective Impact work.
SHARED Measurement System
As the SHARE initiative commenced work on Playgroup attendance, they discovered that there was no system in place to collect and collate this data across the area. Although some individual data sets were collected, data collection systems varied and information was not shared between Playgroup providers. To address this, Together SA lead the co-design of a tool to consistently capture this important data, with the following parameters:
Make it as simple as possible:
Our primary goal was to have people get used to the concept of providing data without overcomplicating the process. Therefore, we began by collecting as minimal amount of data as possible.
It must be low-cost:
We needed a low-cost option that allowed us to trial the approach before spending money on developing anything. The current tool (a Google form) is helping build our understanding of other requirements that may inform ongoing development.
Creating a Community Definition
of Belonging
In 2017, Together in the North decided to undertake a consultation to understand what a sense of belonging meant for the community. A project team worked with Together SA to design and trial consultation and engagement tools that would enable them connect with the community and collect essential qualitative data. The team created a framework for consultation and a combination of tools for each cohort of the community – children, young people and families.
The tools were:
- Chains of togetherness (children)
- Madlibs (children and young people)
- Survey and group interviews (older young people and adults)
The data from these consultations indicated that for the Playford community, belonging meant being listened to and accepted in the wider community. Opportunities to engage local activities and events were seen as good ways to help foster belonging.
blahblahblah
Data Dive
In early 2018, WAY ran a workshop with content experts from 7 organisations to explore key data sets relating to the wellbeing of young people. We called this the ‘Data Dive’. To enable WAY to select population level indicators, or data sets to measure the wellbeing of young people in the City of the Adelaide, we first needed to know what data sets already exist and which may be the most appropriate for monitoring the impact of WAY’s work. The ‘Data Dive’ revealed that, while there were several data sets available at state and national level, very few were relevant to the 18-24 year old age cohort in the City of Adelaide.
Consequently, WAY will utilise available (quantitative) data sets in conjunction with local knowledge (qualitative) to inform decision making. Future work will include investigating the creation of new data collection methodology to allow the WAY to establish its own tailored data set.
Data Portal
In 2016/17, Together SA developed a pilot of a Data Portal for communities. The Data Portal provided a platform for population level data to be collated and presented. This would allow communities to:
- Access information about their community context; and
- In the longer term, demonstrate initiatives are making an impact.
As we tested the first iteration with communities, we gained several insights:
- Being unable to access data from other government departments, limited the use of the portal for communities.
- The data presented did not always provide clear conclusions.
- Information, such as randomised student addresses, meant there was uncertainty about sharing data publicly.
Holding the portal in a government department meant that as priorities changed within the department, there were challenges in updating and maintaining the portal.
However, with over 14 initiatives across South Australia, there is still the demand for identifying, accessing and analysing population data sets.
Based on our initial pilot, we are much more aware about what approach would be useful to communities and how to make the portal more sustainable.
For more information visit our Resources page…