Decoding Media Impact: Insights, Advice & Recommendations

"[E]ven in a rapidly and unpredictably changing landscape, media projects will continue to have the power to change the world. The question we must continually answer is how can we best track that impact and harness its potential for social good."

This report provides an examination of the current state of the field of media impact assessment and is meant to serve as a practical resource for funders to inform their funding practice. It was prepared by Media Impact Funders (MIF) based on feedback from their network and seven years of experience working in the impact space. It provides examples of successful media impact evaluation, offers tools and frameworks for assessment, and highlights the challenges of defining and measuring impact in a rapidly shifting media landscape. In particular, it shares four key insights that are designed to help guide media funders.

This report was compiled because MIF learned, following a yearlong strategic process designed to determine the needs of the fields of media and philanthropy, that "funders who care deeply about measuring the impact of their investments are still looking for more clarity on how best to evaluate them. In various conversations, we've learned that many are still trying to get a better handle on how different frameworks are used. They want a pared-down list and explanation of resources, and need help navigating our robust selection of content."

The report begins with an explanation of what media impact is, explaining that "The field of media impact assessment has emerged in order to shed light on the relationship between media and social outcomes. Drawing on a wide variety of sources, methods and tools, media impact assessment considers how content, messages and ideas contribute to shaping views, decisions and actions. This information helps media producers make informed decisions about editorial, distribution and engagement strategies for media projects with social change objectives, and to help stakeholders explain the social (vs. marketing/financial) returns on their investments." It also looks at what media impact assessments are good for and what they not good for. Media impact assessments are, for example, good for creating "buy-in" for the power of storytelling, and for comparing like projects, but these assessments not good at demonstrating lasting change from short-term interventions/events or drawing big conclusions from small data.

The report shares the following four insights, highlighting one key recommendation for funders under each insight:

1. There are many different frameworks for measuring media impact for different areas of practice.
The report explains how frameworks for measuring media impact tend to track similar things, such as reach, awareness, engagement, attitudes, behaviour, amplification, influence, corporate practice, and policy change. Frameworks for measuring impact differ depending on the type of media project. The report looks at examples of frameworks for measuring the impact of documentary films and journalism projects. In relation to documentary films, one example cited is the Active Voice's Horticulture framework, which uses garden tools as metaphors to help media makers think strategically about effective storytelling. In relation to journalism projects, examples cited include the American Press Institute's proprietary software, Metrics for News, which is a paid analytics service that aligns journalism metrics and editorial values, and the Impact Tracker developed by the Center for Investigative Reporting to streamline impact measurement and analysis for journalism projects. The report also offers examples of frameworks used by other fields of media such as podcasts.

MIF makes the point that while many useful tools and frameworks exist, there is no "magic impact tool" that is equally useful for every media funder and every media project. There is a limit to what static frameworks can do, especially when they require manual tracking and input. When employing particular frameworks, funders should still consider the appropriate strategies and metrics for their grantees.

Recommendation: Fund evaluation and work with grantees to determine appropriate frameworks.

2. Funders should be mindful of power dynamics, and thoughtful in determining appropriate impact strategies with their grantees.
The report makes the point that there is sometimes a lack of clarity as to what specific measurements tell us, especially since so many new media projects are now in uncharted waters. "The rapid expansion and adoption of digital, social, mobile, streaming and now immersive content can create numerous complications when it comes to both strategy and assessment. Funders and grantees can work together to make sure that the impact tracking strategy is actually providing valuable information."

In relation to this, the report discusses a number of emerging frameworks for measuring impact that move away from summative evaluation frameworks (those that "sum up" what a particular project accomplished at the end of a designated time frame) to more developmental evaluation, where funders and grantees are both learning together in real time, which is a change from traditional power dynamics. Adopting such approaches requires a shift in funder perspective away from monitoring and compliance and adherence to predetermined criteria towards understanding impact as it unfolds.

Recommendation: Beware of power dynamics when integrating innovation metrics and adaptive evaluation approaches.

3. Digital analytics tools provide a wealth of useful data, but grantees require financial and logistical support in implementing them.
As stated in the report, "One of the most rapidly-developing areas in impact measurement is the world of digital analytics tools and dashboards that track impact across many different areas. But due to the proliferation of digital analytics tools, some organizations are awash in too much data - and they tend to be costly for grantees with smaller budgets." Funders should be aware of this limitation and consider what kind of support they could offer - for example, funding subscription costs for analytical services or providing discounted training and education on how to use such services for grantees.

Recommendation: Fund and create best practices for digital analytics.

4. There are opportunities for funders to collaborate with each other to share best practices and increase collective impact.
As media impact is an area that is still evolving, the report recommends that funders work together to discuss best practices and approaches and perhaps agree upon shared metrics for grantees who are reporting to many different funders.

Recommendation: Collaborate with other funders and fund shared infrastructure for media grantees.

The report concludes with the following guest essays by experts in the field:

• Measurement Frameworks for Journalism - by Hannah Eaves, CEO, Multiply Bureau
• Understanding Media Impact Beyond "Big Documentary" - by Andrew Lowenthal, Co-founder & Executive Director, EngageMedia
• Breakthrough and Innovations in Media Impact Assessment - by Johanna Blakley, Managing director of the Norman Lear Center at the University of Southern California's Annenberg School for Communication and Journalism