First, Do No Harm . . . Then Do More Good
Most nonprofits view collecting information on outcomes for their clients as a daunting task, a waste of resources, or both. However, the process of data collection and outcomes measurement is a critical activity for any nonprofit that seeks to improve the quality of services it provides. Without knowing what they do well and what needs to be improved, nonprofits can end up providing the same services for years without ever really knowing if they could be doing something different that would lead to greater benefits for the population they are serving.
For a nonprofit to provide the best services possible to its clients, it must measure its outcomes. This is easier said than done; frequently the entire culture of the organization must change to become more accepting of the regular collection of outcomes. Fortunately, there are steps that a nonprofit can take to make this culture change more feasible and more lasting.
Changing the Conversation
The first (and perhaps most critical) step in creating a culture of outcomes measurement is getting everyone to understand this simple statement:
A nonprofit should measure outcomes for a single reason: to improve the quality of services for clients.
Far too often, nonprofits think of data collection and evaluation as a chore that has to be done to satisfy funding organizations. This line of reasoning, unfortunately, drives nonprofits to collect only what’s required by funders in the short term, rather than information that would allow the organization to determine how to improve services for clients over time.
To avoid this trap, a nonprofit’s leadership must change the conversation entirely. Leaders must recognize and then clearly communicate that outcomes measurement is not about simply counting things or gathering information. And it is not about satisfying funders. It is an internal effort aimed at figuring out what works and what doesn’t, so that the organization can provide the best possible services to its clients. This approach usually resonates with nonprofit staff, nearly all of whom share a deep commitment to making a difference for those who need assistance.
How Do You Know That Your Organization Is Not Hurting Clients?
Every nonprofit assumes that its programs and services are doing good for its clients. Unfortunately, no organization is perfect. No program is perfect. No individual is perfect. Despite the best of intentions, nonprofits will make mistakes, and those mistakes can cause harm to clients or participants. At the Latin American Youth Center (LAYC), a youth-development agency with multiple locations in the National Capital Region, we learned this lesson the hard way.
In 2007, one of LAYC’s parenting programs added some lessons to an existing curriculum. The additional lessons focused on domestic violence issues with the intent of teaching parents that domestic violence is not appropriate in any culture and that there are safe ways to escape domestic violence situations.
When the programming was completed, I analyzed the tests we administered before and after the program. The results were shocking. LAYC’s parenting programming, with the additional domestic violence lessons included, actually changed the participants’ attitudes toward domestic violence in the wrong direction.
After finishing our programming, a greater number of participants believed that domestic violence is an appropriate expression of love between partners, that domestic violence is an acceptable part of the Latino culture, and that there is no safe way to leave a violent partner. In a very real sense, our program caused harm to our participants, despite the best of intentions.
Fortunately, because LAYC was collecting information on the participants’ attitudes before and after the program, we were able to make important changes to this program before starting with the next group of participants. In the original domestic violence classes we had provided the instruction in a mixed-gender environment. Aft seeing the negative results, we consulted with domestic violence experts and then split the classes into separate classrooms for men and women so that each could feel more comfortable expressing their feelings. This change, along with others, brought positive, statistically significant changes in attitude in every single cohort.
Making Good Use of the Data We Collect
Once staff members have bought into the idea of outcomes measurement, the next critical step is getting them to actually use the data they are collecting. This means creating reports and data summaries that staff can easily utilize to make decisions. Collecting data is important, but if the data are never used to influence decisions or change programs, then they do not benefit clients.
In LAYC’s residential and housing programs, staff members have taken this message to heart. Every six months staff examine nineteen independent-living skill areas (e.g., personal hygiene, money management, housekeeping) and the progress made by residents. For those areas where residents fail to show progress or actually demonstrate regression in skills, residential staff increase the amount of instruction (at the group and individual level) to offer greater reinforcement of lessons and skills. These extra hours are redistributed from instruction on skill areas where residents are showing significant progress. In this way, staff can change their instructional patterns to match the needs of residents.
By providing staff with information to help them refine and adjust their work, an organization can empower staff to continually improve the quality of services they provide to clients. Data can be a tool to allow staff to serve their clients better, rather than a burden to overcome. This ultimately is how the culture change can be maintained over time.
He is the Director of Data and Evaluation for the DC Promise Neighborhood Initiative
This article originally appeared in the book, Leap of Reason, which is available in full at leapofreason.org