Measuring the Success of Collective Impact Initiatives | NAAEE

The hub for environmental education professional development

Measuring the Success of Collective Impact Initiatives

Nicole Ardoin will be monitoring the discussion thread this week to follow up on her July webinar on collective impact. Webinar participant Katie Campbell asked, "Can you give an example of a successful EE project that your collaborative has done?" Following up on this, Aleeza Oshry asked, "How are you measuring 'successful'?" Nicole will answer these questions and more throughout the week, but please feel free to jump in with your own thoughts and responses!

Great question, Katie! Our Bay Area collaborative, ChangeScale, has undertaken a number of initiatives, with several different objectives in mind. One example of an ongoing “project” that we have undertaken as a collaborative addresses one of ChangeScale's primary goals: building the EE sector in the Bay Area. With that in mind, we regularly host gatherings that provide EE-related professional development and offer opportunities for connection among our collaborating partners. We host an annual regional EE Congress; we also host three smaller gatherings during the year that focus on specific issues. Thus far, topics of the gatherings have included collective impact; the Next Generation Science Standards and EE; the structure of local and state funding for EE; and diversity, equity, and inclusion. The Congresses each have an overarching theme, but also leave space for participants to bring questions, issues, and interests. More than 60 organizations from around the region regularly attend these gatherings; in total, more than 150 organizations have attended at least one of the events in the past year.

In the past year, ChangeScale has also launched the School Partnerships Initiative. Through this initiative, we are working with three Bay Area school districts as well as a group of regional environmental education providers to create an environmental literacy leadership team. The team is assessing current environmental literacy efforts; determining feasible district-wide environmental literacy goals and strategies for reaching those goals; and establishing a district-wide Environmental Literacy Plan.

Within the School Partnerships Initiative, ChangeScale has assumed a leading role in the environmental literacy planning process and providing enhanced resources and facilitation. This expanded role allows ChangeScale to develop resources that can be used to support partnerships and inform the evolution of the Initiative in subsequent years.

Related to Aleeza's question about how we are measuring "successful": We are working on measures of success for all of these projects and initiatives, as well as for the overall collaborative. As mentioned on the webinar, measurement and evaluation is one of the areas that’s of great interest to ChangeScale and it’s also where we see an opportunity for growth. At the very least, we’re tracking “satisfaction” with our events. But this year, we hope to think more systematically about quantitative and qualitative measures that roll up collectively and help us consider whether, when, and how our initiatives are moving us toward reaching our desired outcomes. As mentioned, the gatherings are working toward the outcome of building the professional field of EE at the regional scale; concurrently, the school partnerships are working more directly toward building the environmental literacy of young people in the districts.

Specifically, with the School Partnerships Initiative, we are conducting a landscape analysis of environmental education in the three partner districts. Our landscape analysis has involved surveying teachers as well as environmental education providers to assess what the current EE landscape looks like in the districts. The landscape-analysis results will provide a baseline against which we can measure progress toward our goal of expanding access to environmental education in subsequent years. The measurement system we put in place this year will also be used in subsequent districts in future years as we scale up this Initiative.

Concurrently, we are refining our logic model framework to also include qualitative indicators of success. This framework, which is tied to our theory of change, will help assess what has changed in each district related to the implementing the new environmental literacy plans.