Written for evaluators, this 17-page report is a very nice guide to communicating about culture in evaluation practices and may be used as a reference in data collection. The report emphasizes that cross-cultural competency is a necessary skill for evaluators to have. The author encourages evaluators to think of culture as a factor to be considered as much as sampling and measurement. In addition to the introduction and conclusion, the paper has three main sections: culture, social identity and group membership, and privilege and power.
Resources
The Annotated Guide to Tools & Resources provides tools, frameworks, and resources to help you develop and implement your evaluation. It’s a repository of useful, practical materials that can help you create an evaluation plan; design your evaluation approach; develop or adapt tools and instruments; and otherwise move your evaluation forward.
This Guide was originally assembled from many sources and fields and annotated by evaluator Suzanne Callahan of Callahan Consulting for the Arts. We continue to add resources. Your suggestions are welcome!
Do you have a useful tool or resource to add? Contact animatingdemocracy@artsusa.org.
This 33-page document calls itself a study but can serve as a guide to indicator development. Though done in 1996, it an extremely thorough, thoughtful overview of indicator consideration and development. It defines indicators, gives examples of types, and talks of the challenges of measuring difficult areas such as political development. It discusses issues related to indicators in governmental agencies versus NGOs.
This 33-page document calls itself a study but can serve as a guide to indicator development. Though done in 1996, it an extremely thorough, thoughtful overview of indicator consideration and development. It defines indicators, gives examples of types, and talks of the challenges of measuring difficult areas such as political development. It discusses issues related to indicators in governmental agencies versus NGOs.
From the Centers for Disease Control and Prevention, this 71-page report defines process evaluation and describes the rationale, benefits, key data collection components, and program evaluation management procedures. Within the framework of discussing tobacco use prevention, this paper is a very good primer for process evaluation for readers in a variety of fields. It provides clear and well-presented charts, graphics, principles, and summaries to help guide readers.
From the Centers for Disease Control and Prevention, this 71-page report defines process evaluation and describes the rationale, benefits, key data collection components, and program evaluation management procedures. Within the framework of discussing tobacco use prevention, this paper is a very good primer for process evaluation for readers in a variety of fields. It provides clear and well-presented charts, graphics, principles, and summaries to help guide readers.
Resource Format: book / article
Continuum: Awareness, knowledge, understanding
Understanding how knowledge moves through communities can help evaluators select the components of a program that are best suited for scaling up. “Scaling” is the practice of replicating programs for implementation at new locations, by other organizations, or to serve larger groups of people. Knowledge-for-action theories address how learned information becomes the measureable impact of arts for change work. This article highlights five theories of knowledge transfer: utilization, diffusion, implementation, transfer, and translation.
Publication Date: August 31, 2008
Resource Format: practical tool
This set of resources is useful to organizations that want to increase their social impact. The Kellogg Foundation asks, “Ever wonder if you are getting through to people?” They offer twelve principles in the Knowledge to Understanding framework that aid in knowledge transfer. An introductory video provides a snapshot of the framework and a report and interactive slideshow provides more depth into all twelve guiding principles.
Resource Format: book / article, case study
This 35-page report provides a summary of findings and lessons learned by the Neighborhood and Family Initiative (NFI), a comprehensive community initiative launched by the Ford Foundation that began in 1990. NFI was a 10-year effort that sought to strengthen a single neighborhood in each of four cities and to improve the quality of life for the families who live in them. The report is divided into two sections; the first provides a brief overview of NFI while the second section distills the lessons learned by NFI over the course of implementation.
This piece is an excellent primer on how to think about outcomes and the hurdles that may arise in measuring them. It addresses and deals with challenges such as measuring intangibles. Published by GrantCraft (a division of the Ford Foundation), the eleven-page guide is written for grantmakers to describe outcomes-based evaluation. It defines key terms and makes a case for why outcome measurement is important.
This piece is an excellent primer on how to think about outcomes and the hurdles that may arise in measuring them. It addresses and deals with challenges such as measuring intangibles. Published by GrantCraft (a division of the Ford Foundation), the eleven-page guide is written for grantmakers to describe outcomes-based evaluation. It defines key terms and makes a case for why outcome measurement is important.
Resource Format: book / article, case study
Los Angeles Poverty Department (LAPD) is a Skid Row-based theater organization, founded and directed by artist John Malpede. LAPD has distinguished itself by its longstanding commitment to making change in L.A.’s Skid Row community, particularly regarding the homeless, through theater-based civic engagement work. Many have observed LAPD’s apparent potent effects on individuals and on social relations in Skid Row, and acknowledge its contributions to influencing structures, systems, and even policy.
"What are we doing, and why do we think it’s going to make a difference? Are we being effective?" Grantmakers ask evaluation questions like these of their grantees and themselves. This brief guide explains why grantmakers use theories of change to guide their questioning, unearth assumptions that underlie their work, establish common language, and develop strong action plans. Contributors to the guide also describe how a theory of change sets the stage for evaluation by clarifying goals, strategies, and milestones.
Written for funders, this 43-page piece is about social entrepreneurship, its relatively new place in the field of philanthropy, and implications for evaluation. Developed from research with grantees and two dozen interviews with funders, it talks about flexibility, leadership, and risk in funding innovation, and why these are key.
Resource Format: book / article, practical tool
This article from the Social Edge website (a program of the Skoll Foundation) aims to connect practitioners of the social benefit sector to network, learn, and share resources. It provides useful language and concepts about measuring outcomes in the social science field. Written to "social entrepreneurs," the piece explains that qualitative information can be effective in measuring social change, but one needs to be systematic in collecting and using qualitative data to evaluate programs.
Publication Date: December 31, 2009
Resource Format: book / article
In answer to the challenges that face corporate philanthropy in identifying a shared definition of impact measurement, the author sets out to assess current measurement practices, clarify what is needed in terms of impact evidence, and identify next steps. The article is organized into three conversations between key stakeholders engaged in corporate philanthropy.
Resource Format: practical tool
What is the difference between outputs and outcomes? Between outcomes and impact? “Measuring What Matters: The Challenge of Quantifying Social Change” steers clear of jargon and offers an accessible evaluation framework for practitioners.
Resource Format: practical tool
This tool serves as a model to align values, actions, and measures of progress for State Art Agencies. In table form, it lays out a generic base for locating concepts of participation within a framework of concepts of public value and motivating values of different groups. The table can serve as a basis for developing the types of outcomes and measures related to State Art Agencies' actions to broaden, deepen and diversify creators, stewards and spectators/participants.
Resource Format: practical tool
This tool serves as a model to align values, actions, and measures of progress for State Art Agencies. In table form, it lays out a generic base for locating concepts of participation within a framework of concepts of public value and motivating values of different groups. The table can serve as a basis for developing the types of outcomes and measures related to State Art Agencies' actions to broaden, deepen and diversify creators, stewards and spectators/participants.
Resource Format: book / article
This paper is one of a number of working papers produced for the project “Understanding the drivers of, and value and benefits afforded by, engagement in culture and sport”. The objectives of the project are to define and model the following broad relationships: 1. The impact of policy options and other factors on the level of engagement in sport and culture. 2. The outcome of engagement in sport and culture. 3. The value of these outcomes. This paper focuses on the last of these points.
Resource Format: book / article
This paper is one of a number of working papers produced for the project “Understanding the drivers of, and value and benefits afforded by, engagement in culture and sport”. The objectives of the project are to define and model the following broad relationships: 1. The impact of policy options and other factors on the level of engagement in sport and culture. 2. The outcome of engagement in sport and culture. 3. The value of these outcomes. This paper focuses on the last of these points.
Resource Format: book / article, case study
Artist Rha Goddess’s Hip Hop Mental Health Project (HHMHP) seeks to contribute to shifting the cultural paradigm of shame and alienation surrounding mental illness, and satisfy a need for a SAFE place to confront the issue and obtain vital information.
Resource Format: book / article, case study
When Flint Youth Theatre began planning for a new play addressing the local and national problem of school violence, it had no idea that, in the process of developing the project, its own community would experience a devastating elementary school shooting. A year after the tragedy, the play ...My Soul to Take, written by artistic director and playwright William Ward, became a focal point for fresh attention on this persistent and painful issue.
"In so many evaluations,” said a program officer, “no one thinks to ask the users.” Participatory action research offers grantmakers a way to do so. It engages all parties in all aspects of an evaluation, from defining the problem to gathering and analyzing data to preparing recommendations. In this guide, learn about a unique evaluation method and how grantmakers used it to evaluate programs in agriculture, early childhood development, and immigration.
Publication Date: December 31, 2009
Resource Format: book / article
This piece suggests that the accountability movement is “setting a floor for minimum standards” (p. 84) and has consequences for effective social change work. Foundations, in particular, measure impact in terms of attentiveness to accountability standards, but this is a false measure of success. Instead, the organization’s focus should be on its transformative value to society.
Resource Format: practical tool, website
Point K Learning Center ("Point K"), a website and online workstation, was created by Innovation Network (Innonet), a leader in the participatory evaluation field and one of the first to make online tools available that could be used by smaller as well as larger organizations. Point K features practical tools and resources for nonprofit planning, evaluation and action. It is a major part of Innonet’s mission of sharing know-how to create lasting social change. Point K aims to help users assess strengths, articulate goals, use data and better tell the organization’s story.