This 5-page article on Action Evaluation from the Beyond Intractability website provides a knowledge base and tools regarding social conflict resolution and community reconciliation. This article assists participants, funders, and facilitators in reaching consensus about what they seek to accomplish (people's goals), why (their values and beliefs), and how (suggested account strategies). The article addresses the process in three stages: establishing the baseline, formative monitoring, and summative evaluation.
Theory
The Annotated Guide to Tools & Resources provides tools, frameworks, and resources to help you develop and implement your evaluation. It’s a repository of useful, practical materials that can help you create an evaluation plan; design your evaluation approach; develop or adapt tools and instruments; and otherwise move your evaluation forward.
This Guide was originally assembled from many sources and fields and annotated by evaluator Suzanne Callahan of Callahan Consulting for the Arts. We continue to add resources. Your suggestions are welcome!
Do you have a useful tool or resource to add? Contact animatingdemocracy@artsusa.org.
Publication Date: May 16, 2012
Resource Format: book / article
This piece sheds light on the philanthropic sector’s efforts to improve measurement and evaluation (M&E), specifically within the context of foundations involved in social change work. The broad observations of the six contributing authors are that purpose; cost-benefit ratio; culture, context and capacity; unit of analysis; timing; feedback; and transparency matter to measurement and evaluation. Luis A.
Based on several years of field research in communities around the U.S., this six-page brief presents an insightful framework for better capturing and measuring arts, culture, and creative expression at the neighborhood level.
Resource Format: book / article, practical tool
Written for Animating Democracy's Arts and Civic Engagement Impact Initiative Working Group, this 14- page paper presents a conceptual framework (or logic model) for arts-based engagement. It offers a discussion of the components of the framework, and a list of questions to guide research explorations.
The site offers an overview of Urban Institute's Arts and Culture Indicators Project, defines cultural vitality, briefly addresses cultural vitality indicators, offers case study examples, and gives abstracts and links to high-quality relevant research studies. The site is a beautiful resource for arts and civic engagement practitioners and policymakers. Graphically strong with top notch research, it is a model for the field.
Resource Format: book / article
This 43-page report is a literature review commissioned by the Arts Council of England to support its two-year social inclusion research program. It examines the impact of arts in addressing social exclusion with the purpose of informing the design of the research and placing it in a policy context. The first section attempts to define social exclusion, offers ideas and obstacles to measuring it, and relates it to the arts.
Resource Format: book / article, case study
This 544-page book by sociology professor Robert D. Putnam makes the case that the social capital of American citizens is declining because of increasingly diminished connections with their communities. He supports this claim with extensive interviews and research, much of which is arranged in charts and graphics.
Resource Format: practical tool, website
The California Endowment believes that evaluation represents an important opportunity to assess achievements, generate knowledge and cultivate learning. It works to support grantees, evaluators and others interested in learning more about culturally competent health evaluation. Its website has a wealth of resources, reports, and articles useful to both funders and practitioners. Of particular relevance are items related to storytelling approaches to program evaluation, multicultural concerns in evaluation, and understanding impact of advocacy efforts.
Resource Format: book / article
Written for Animating Democracy's Arts and Civic Engagement Impact Initiative, this 69-page paper speaks directly to arts and civic engagement work, surveys current research, and makes recommendations for future practice. The paper has three sections. The first attempts to define and differentiate civic terms: civic engagement, social capital, public sphere, community engagement, community and civic capacity, arts, culture, humanities, social inclusion, cultural citizenship, and the cultural public sphere.
Resource Format: book / article, case study
Drawing from Craig McGarvey’s seven years of professional experience as Program Director in Civic Culture at The James Irvine Foundation, a California-wide philanthropy, Civic Participation and the Promise of Democracy explores civic engagement strategies (faith-based and community organizing, popular education, collective learning) through providing an overview of their use in California. During McGarvey’s time at Irvine’s Civic Culture, the program supported Californians who were working to build a durable pluralism from the State’s unprecedented demographic diversity.
Publication Date: December 31, 2010
Resource Format: book / article
The authors cite nonprofits collaborating in an education system to introduce the concept of collective impact, defined as, “the commitment of a group of important actors from different sectors to a common agenda for solving a specific social problem.” Collective impact is unlike collaboration efforts and partnerships in that its “initiatives involve a centralized infrastructure, a dedicated staff, and a structured process that leads to a common agenda, shared measurement, continuous communication, and mutually reinforcing activities among all participants.” In short, large-scale, complex
Resource Format: book / article, case study
Critical Perspectives is a collection of reflective, critical, and creative essays about the role of arts and humanities in civic dialogue. Twelve essays focus on three very different projects that employed the unique capacities of theater, visual art, and historic preservation to reach people and stimulate them to talk together in new ways about issues that matter in their communities.
This 104 page report (with eight chapters and seven appendices), based on extensive community research, discusses cultural vitality and gives extensive information on indicators. This piece -- primarily for evaluators, researchers, and policymakers -- is written in clear, user-friendly language. The fundamental goal of the Urban Institute's Arts and Culture Indicators Project (ACIP) is to help policymakers make better decisions for neighborhoods and cities.
Resource Format: practical tool
This checklist is a guide to incorporating democratic processes in an evaluation to assure valid conclusions where there are conflicting views. A concise checklist aims to ensure that the evaluation process, and thus conclusions, are inclusive and unbiased. The checklist is built around three principles: 1) Inclusion, 2) Dialogue, and 3) Deliberation. The Inclusion principle implies that all relevant interests, values, and views of major stakeholders are taken into account when designing and conducting the evaluation.
Resource Format: book / article, practical tool
This short book defines developmental evaluation and addresses some myths. Offering an alternative approach to linear logic of traditional methods, developmental evaluation supports the process of innovation within an organization and its activities. The book outlines developmental evaluation's uses in innovation and developing project ideas and defends the approach's credibility. It is an appropriate resource for organizations that work in an innovative manner and thus seems tailored to arts organizations.
Publication Date: May 22, 2013
Resource Format: case study
Continuum: Capacity (social capital, skills, practice, resources) | Systems, policies, places (legislation, issue/problem resolution)
Developmental Evaluation: An Overview (J.W. McConnell Family Foundation Powerpoint) presented at the Animating Democracy/Americans for the Arts' Funder Exchange on Evaluating Arts & Social Impact at the Nathan Cummings Foundation on May 22, 2013.
The initial scoping phase, so important for providing a sound and lasting framework for the data and evidence reviews, has now ended and the hard work has begun. Technical work on the modelling is under way, including reviewing over 50 different datasets in the UK that could be useful for the model. The systematic review of the research – a wide-ranging hunt for any study in our sector that addresses engagement – has unearthed over 10,000 studies. This allows us to begin the process of mapping the research onto systems dynamics model of engagement, putting evidence-base into action.
Publication Date: July 31, 2012
Resource Format: book / article
The authors suggest that traditional evaluation approaches (formative and summative) fail to meet the complex needs of social sector innovators. Instead, grantmakers should approach evaluation differently, specifically involving the use of developmental evaluation (attributed to Michael Quinn Patton). Through a review of literature, interviews, and case studies, this piece assists with putting developmental evaluation into practice. At the heart of this call for new evaluation approaches, is the encouragement of social innovation and change.
This website is a good resource for individuals and community groups working for change in their communities, specifically with issues relating to race and racial equity. The site is written for users who are new to evaluation -or perhaps those who give funds related to racial equity, but who are not yet clear on how to evaluate it. The site discusses how to apply a “racialized” perspective to evaluation, meaning using the ideas of racism, oppression, privilege and access to power as a lens through which evaluation questions are developed and results are analyzed.
To get insight into a complex community, problem or process of change, sometimes you need to look beyond conventional research or evaluation methods. Ethnography is a powerful way to step inside the culture of an organization or community, hear ongoing feedback from multiple points of view, and understand a program's real impact. This guide describes ethnography's benefits and pitfalls, and how the method is used to document, evaluate and improve approaches to youth engagement, HIV education and neighborhood policing.
Resource Format: book / article, practical tool
Written for evaluators, this 17-page report is a very nice guide to communicating about culture in evaluation practices and may be used as a reference in data collection. The report emphasizes that cross-cultural competency is a necessary skill for evaluators to have. The author encourages evaluators to think of culture as a factor to be considered as much as sampling and measurement. In addition to the introduction and conclusion, the paper has three main sections: culture, social identity and group membership, and privilege and power.
This 33-page document calls itself a study but can serve as a guide to indicator development. Though done in 1996, it an extremely thorough, thoughtful overview of indicator consideration and development. It defines indicators, gives examples of types, and talks of the challenges of measuring difficult areas such as political development. It discusses issues related to indicators in governmental agencies versus NGOs.
From the Centers for Disease Control and Prevention, this 71-page report defines process evaluation and describes the rationale, benefits, key data collection components, and program evaluation management procedures. Within the framework of discussing tobacco use prevention, this paper is a very good primer for process evaluation for readers in a variety of fields. It provides clear and well-presented charts, graphics, principles, and summaries to help guide readers.
Resource Format: book / article
Continuum: Awareness, knowledge, understanding
Understanding how knowledge moves through communities can help evaluators select the components of a program that are best suited for scaling up. “Scaling” is the practice of replicating programs for implementation at new locations, by other organizations, or to serve larger groups of people. Knowledge-for-action theories address how learned information becomes the measureable impact of arts for change work. This article highlights five theories of knowledge transfer: utilization, diffusion, implementation, transfer, and translation.
Resource Format: book / article, case study
This 35-page report provides a summary of findings and lessons learned by the Neighborhood and Family Initiative (NFI), a comprehensive community initiative launched by the Ford Foundation that began in 1990. NFI was a 10-year effort that sought to strengthen a single neighborhood in each of four cities and to improve the quality of life for the families who live in them. The report is divided into two sections; the first provides a brief overview of NFI while the second section distills the lessons learned by NFI over the course of implementation.