The Advocacy Evaluation Project, orchestrated by Innovation Network (Innonet), began in 2005 to move the field of advocacy evaluation beyond assessing policy change into one that considers more fundamental and contributory components of advocacy efforts: capacity building, network formation, relationship building, communication, issue framing, and leadership development. In Phase I the Advocacy Evaluation Resource Center was created. Now a part of Innonet's Point K Learning Center (which requires a free login), the Resource Center is a database .
Featured Resources on Indicators
These tools and frameworks offer diverse information about indicators. This list has been assembled from many sources and fields and annotated by evaluator Suzanne Callahan of Callahan Consulting for the Arts.
Resource Format: book / article
Continuum: Awareness, knowledge, understanding
In this article, author Alan Brown revisits and interprets the 2005 report, The Gifts of the Muse, published by the RAND Corporation and commissioned by the Wallace Foundation, with valuable analysis of how public value occurs from the core artistic experience. Starting with an individual experience, the cumulative impact of exposure to the arts can ripple through a community. Brown argues that artists and practitioners need a consistent language to communicate the transformative experience of the arts to funders and policymakers.
The site offers an overview of Urban Institute's Arts and Culture Indicators Project, defines cultural vitality, briefly addresses cultural vitality indicators, offers case study examples, and gives abstracts and links to high-quality relevant research studies. The site is a beautiful resource for arts and civic engagement practitioners and policymakers. Graphically strong with top notch research, it is a model for the field.
Resource Format: book / article
This 43-page report is a literature review commissioned by the Arts Council of England to support its two-year social inclusion research program. It examines the impact of arts in addressing social exclusion with the purpose of informing the design of the research and placing it in a policy context. The first section attempts to define social exclusion, offers ideas and obstacles to measuring it, and relates it to the arts.
Resource Format: case study
Arts for All, a partnership between the Los Angeles County Arts Commission, Office of Education, and local partners, developed the School Arts Survey to measure the availability and quality of arts education throughout Los Angeles County. The survey was administered as a pilot to five Los Angeles County schools. Using 16 indicators of quality, access, and equity, the survey can be used to compare arts programs at different schools. Indicators of pedagogy, student learning, school environment, and community engagement were evaluated by school staff using a 10 point scale.
Publication Date: December 31, 2007
Resource Format: book / article
In order to better understand film as an agent of social change, this article offers examples of documentary films that have led to change in viewer behavior, public policy, and discourse. After examining the successful impact of these examples, the authors offer planning tools for outreach campaigns, an approach to assessing impact, and lessons learned. This article may be useful to the strategic design of programs and outreach efforts that aim to affect change.
Resource Format: practical tool
The Boston Youth Arts Evaluation Project (BYAEP) Framework responds to the need for an evaluation model that considers the combination of education, youth development and social services outcomes and that does justice to the beauty, complexity and holistic nature of such integrated work. The BYAEP model has the potential to provide useful comparisons of program evaluations between organizations and across disciplines.
Resource Format: practical tool, website
Published by the Alliance for Justice, this highly useful, interactive guide has two major parts: the Advocacy Capacity Assessment Tool (13 pages) and the Advocacy Evaluation Tool (11 pages). The graphically pleasing guide is made for foundations and grantees to help improve evaluation methods given the difficulties of showing cause-and-effect between advocacy and policy change. A user-friendly introduction explains the purpose of the tools, suggested uses, a definition for advocacy, how the tools can guide grantmaking decisions, and how the tools can be customized.
Resource Format: book / article
Written for Animating Democracy's Arts and Civic Engagement Impact Initiative, this 69-page paper speaks directly to arts and civic engagement work, surveys current research, and makes recommendations for future practice. The paper has three sections. The first attempts to define and differentiate civic terms: civic engagement, social capital, public sphere, community engagement, community and civic capacity, arts, culture, humanities, social inclusion, cultural citizenship, and the cultural public sphere.
Resource Format: case study
Collective Encounters (CE), a theater based in Liverpool, England, uses evaluation tools to measure its efforts to affect social change. This seven-page primer details CE’s evaluation policy framework, which distinguishes between feedback, monitoring, evaluation and advocacy.
An online service based at the University of Kansas, this extensive site aims to promote community health and development by connecting people, ideas and resources. With over 7,000 pages presented in user-friendly language, the Community Tool Box (CTB) hopes to build capacity for those who wish to change their communities for the better. Users can approach the site in five different ways: To read about specific skills in community work, users may click on the Table of Contents to locate the 46 chapters and nearly 300 distinct CTB sections.
Resource Format: practical tool, website
This site is an easy-to-follow guide for developing and enhancing arts programs for older adults. Essentially an online book with ten chapters, all of which are available as pdfs for easy printing, the toolkit is a good example of a "soup to nuts" approach to program design, including evaluation.
Resource Format: practical tool
This PDF is related to two resources: (1) the Cultural Planning Toolkit, and (2) a related Web Resource . The related web resource on integrating cultural and community planning offers more of an overview and is only nine pages long. It opens with a series of nice, concise definitions of cultural planning and some related terms.
This 104 page report (with eight chapters and seven appendices), based on extensive community research, discusses cultural vitality and gives extensive information on indicators. This piece -- primarily for evaluators, researchers, and policymakers -- is written in clear, user-friendly language. The fundamental goal of the Urban Institute's Arts and Culture Indicators Project (ACIP) is to help policymakers make better decisions for neighborhoods and cities.
Resource Format: book / article, practical tool
This short book defines developmental evaluation and addresses some myths. Offering an alternative approach to linear logic of traditional methods, developmental evaluation supports the process of innovation within an organization and its activities. The book outlines developmental evaluation's uses in innovation and developing project ideas and defends the approach's credibility. It is an appropriate resource for organizations that work in an innovative manner and thus seems tailored to arts organizations.
Resource Format: book / article, practical tool
These appendices are lists of indicators for evaluating the impact of leadership development programs.
This website is a good resource for individuals and community groups working for change in their communities, specifically with issues relating to race and racial equity. The site is written for users who are new to evaluation -or perhaps those who give funds related to racial equity, but who are not yet clear on how to evaluate it. The site discusses how to apply a “racialized” perspective to evaluation, meaning using the ideas of racism, oppression, privilege and access to power as a lens through which evaluation questions are developed and results are analyzed.
Resource Format: practical tool
An outstanding piece, this 51-page handbook presents a great discussion of outcomes as they relate to policy and advocacy. Intended for grantmakers but useful for nonprofits as well, it encourages users to think about measurement of advocacy and policy efforts toward social change. It offers outcomes lists, examples of data collection tools, and many sample instruments from which users may benefit.
This 33-page document calls itself a study but can serve as a guide to indicator development. Though done in 1996, it an extremely thorough, thoughtful overview of indicator consideration and development. It defines indicators, gives examples of types, and talks of the challenges of measuring difficult areas such as political development. It discusses issues related to indicators in governmental agencies versus NGOs.
From the Centers for Disease Control and Prevention, this 71-page report defines process evaluation and describes the rationale, benefits, key data collection components, and program evaluation management procedures. Within the framework of discussing tobacco use prevention, this paper is a very good primer for process evaluation for readers in a variety of fields. It provides clear and well-presented charts, graphics, principles, and summaries to help guide readers.
Publication Date: August 31, 2008
Resource Format: practical tool
This set of resources is useful to organizations that want to increase their social impact. The Kellogg Foundation asks, “Ever wonder if you are getting through to people?” They offer twelve principles in the Knowledge to Understanding framework that aid in knowledge transfer. An introductory video provides a snapshot of the framework and a report and interactive slideshow provides more depth into all twelve guiding principles.
This piece is an excellent primer on how to think about outcomes and the hurdles that may arise in measuring them. It addresses and deals with challenges such as measuring intangibles. Published by GrantCraft (a division of the Ford Foundation), the eleven-page guide is written for grantmakers to describe outcomes-based evaluation. It defines key terms and makes a case for why outcome measurement is important.
Written for funders, this 43-page piece is about social entrepreneurship, its relatively new place in the field of philanthropy, and implications for evaluation. Developed from research with grantees and two dozen interviews with funders, it talks about flexibility, leadership, and risk in funding innovation, and why these are key.
Resource Format: book / article, practical tool
This article from the Social Edge website (a program of the Skoll Foundation) aims to connect practitioners of the social benefit sector to network, learn, and share resources. It provides useful language and concepts about measuring outcomes in the social science field. Written to "social entrepreneurs," the piece explains that qualitative information can be effective in measuring social change, but one needs to be systematic in collecting and using qualitative data to evaluate programs.
Resource Format: practical tool
What is the difference between outputs and outcomes? Between outcomes and impact? “Measuring What Matters: The Challenge of Quantifying Social Change” steers clear of jargon and offers an accessible evaluation framework for practitioners.