Not Testing Development Impact? You May Be Wasting Everyone’s Time.
“Are we doing good work?”
I knew the question was coming. This donor started every meeting that way. I even practiced an answer, so I wasn’t sure why my response felt so weak. “We are planting lots of trees.”
Somehow, I knew that didn’t answer his question. I tried again. “We are making lots of microcredit loans.” That felt even weaker.
The truth is, we really didn’t know. Being relatively new to nonprofit work, I hadn’t given much thought to impact, beyond a vague sense of wanting to do good and help poor people. Furthermore, like many small nonprofits (and a few big ones), we had confused activities with impact.
Yet to be good stewards of the gifts God has given us, nonprofit professionals have a responsibility to our clients, our donors, and most importantly, to God, to effectively and honestly measure, understand, and report the impact of our work.
Periodically, I would visit our program participants, who would describe how proud they were of their farms and how much they liked our staff. But after a couple of trips, I began to notice we always visited the same five farms, despite working with hundreds of families.
Additionally, there was no way to gauge the relative importance of the program to their lives. (Growing more bananas, for example, might be life-changing to some, but nothing more than a hobby for others.) Third, and most disconcertingly, stories of dissatisfaction began to creep in.
In the late nineties, we got a dose of reality when the local price of oregano collapsed. It was the primary cash crop we had been promoting in the Dominican Republic and for several years farmers had been successful. However, a warehouse full of unsold oregano led to an ugly confrontation. Farmer after farmer shared their frustration.
I will never forget one man in a battered hat, his toddler son standing on the folding chair next to him, as he gestured angrily at the sacks of oregano around the room. “You told us to grow this! Now you can’t sell it, and he can’t eat it,” he indicated the boy. “What I am supposed to do?”
I had the horrifying realization that I had been steering the organization without a clear sense of our objective, or any way to measure progress towards it. If we were asking people to join this journey with us, whether as clients or donors, we owed it to them to know the destination, and update them on progress.
Tabulating activities told us nothing of the changes occurring in local communities, or even whether they were positive or negative. In our case, those activities included trees planted, loans made and oregano produced. For others, perhaps it might be patients seen, meals served, vitamins distributed, gardens planted, wells dug or mosquito nets distributed. However, without measuring impact, there is no way of knowing the utility of those activities.
Many of our donors simply assumed their gift was “for a good cause” and that was enough. (Afterall, our overhead was low!) My own expectations were not significantly higher. However, whether we realized it or not, once the organization engaged donors and clients, we incurred a significant moral obligation to fulfill our promises to both, whether implied or explicit.
In a nonprofit organization, the product is change. Unlike a for-profit business, or an organization that relies primarily on service fees, in a donor-funded nonprofit, the person buying the product (the donor) is not the same person using it (the client or beneficiary). That often makes the donor completely dependent on the organization’s own testimony to know whether he or she is paying for something worthwhile.
The client, on the other hand, rarely has any choice. Therefore, the donor will have to place complete trust in the reporting integrity of the organization. It is possible, and perhaps even common, to fail both client and donor, while operating in ignorance and sharing a few happy stories.
We had been asking people to invest their resources in our work, without any idea of the real impact it was having.
Even more troubling, we didn’t know if our activities were helping or hurting the people we served, despite asking some of the most vulnerable people in the world to risk their limited resources on what we were offering. That is not uncommon and sadly, almost everywhere I have traveled, I have seen damage from bad development, whether it is people who have become jaded and dependent on handouts, unfinished church mission projects, or abandoned and impractical “income-generating” schemes that ultimately cost the time, energy and optimism of the very people they were intended to serve.
When we finally did conduct a comprehensive study of our program in 1999, the results weren’t encouraging. We were helping a few people – the families we always visited and interviewed – while leaving most of the rest behind. Some we were even hurting, with debt accumulating and crops they couldn’t sell.
This news was not welcomed by our board, nor our local partners. It can be extremely painful to come to terms with the fact that your work isn’t effective, especially when you have put so much of your heart into developing and executing it. However, in addition to our donors and our clients, we owe it to ourselves and to God to make the absolute best use of the resources and talents with which we have been entrusted.
Today, an important part of our organizational culture includes the willingness to unflinchingly examine ourselves in the light of unflattering information. We have learned this is as important as celebrating the positive results, because it is the only way we can continue to improve our program.
As anyone who has ever been lost can attest, knowing where you are is just as important as walking. Only then do you know what corrections to make to get closer to your objective. It is an ongoing learning process.
Although we still track activities, as we seek to reverse deforestation and poverty, we measure and compare tree cover, rather than trees planted, soil conservation rather than farmers trained, and change in living standards rather than loans. As a result, the paths we use to achieve those objectives are very different from what they were 20 years ago. Furthermore, as we continue learning, I hope what we are doing in five or ten years will be different from what we are doing now.
As sift through the data from our most recent impact evaluations it has been exciting to learn about positive impact. Today, I can say with confidence that we are doing good work. But there are sobering results as well and these may ultimately be the most important, because they give us the opportunity to ask what we could be doing better.
God has never asked us for anything less than our best, and if we are to truly love our neighbors as ourselves, we owe them the best as well.
Scott Sabin has served as CEO, and formerly Executive Director, of Plant With Purpose (www.plantwithpurpose.org) since 1995. Plant With Purpose is an international Christian organization that empowers smallholder farmers around the world in places where poverty and deforestation intersect. Combining savings groups with agroforestry and regenerative agriculture, Plant With Purpose operates long-term watershed-focused programs in the Dominican Republic, Haiti, Mexico, Tanzania, Burundi, DRC, Ethiopia, and Thailand. Plant With Purpose has empowered farmers in more than 1000 communities to plant 43 million trees and dramatically reduce poverty in their watersheds. Scott is the author of Tending to Eden: Environmental Stewardship for God's People. He holds a BS in Political Science from Oregon State University and an MA in International Relations from the University of San Diego. He and his wife Nancy, a nurse practitioner, live in San Diego and have two children, Amanda (20) and Danny (17). Scott blogs at https://www.scottsabin.com/.