Apollo M. Nkwake - Working with Assumptions in International Development Program Evaluation: With a Foreword by Michael Bamberger

This Springer imprint is published by the registered company Springer Nature Switzerland AG

The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

All development programs and policies are based on a wide range of assumptions. For example, one program for inner city youth may be based on the assumption that combating teenage drug addiction will be a more effective way to reduce urban poverty than improving the poor writing skills of high school graduates, while another program in a similar community might assume that providing youth with religious guidance is a key element in combatting urban poverty. It is quite common to find that the guiding principles of both programs are based on unspoken assumptions. It is not uncommon for program organizers to become offended when an evaluator tries to identify these basic assumptions as they may feel that their personal values are being called into question.

The evaluations of development programs and policies are also based on a wide range of interlocking assumptions. Evaluators may make assumptions about the real purpose of a program (which may be different from what is stated in the documents given to the evaluator), what the program is likely to achieve and what problems are likely to arise, the real motivation for commissioning the evaluation, and how the results will be used (and perhaps misused). An evaluators personal values and political orientation may also lead to assumptions about, for example, whether the program is worthwhile; whether it is likely to have negative consequences for certain groups of women, ethnic minorities, or other vulnerable groups; and what the real purposes of the agencies supporting the program are. Additional sets of assumptions also relate to the evaluation methodology: what is the best or appropriate methodology to use? What are the appropriate output, outcome, and (perhaps) impact indicators to measure? Do we need to base the evaluation on a program theory? Does it make sense to think about causality? Evaluators also align themselves on the quantitative/mixed-method/qualitative evaluation spectrum, with all of the assumptions that these positions entail.

The agencies commissioning and using evaluations also have their own assumptions about what an evaluation is, what it can achieve, why it is being commissioned, and how it will be used. Some might assume that evaluators should be skilled scientists who can provide precise statistical answers to questions such as whether the program worked and how much quantitative difference it made to the lives of the intended beneficiaries. Others might assume that evaluators should be management consultants who can provide useful guidance on how to improve program performance. Clients may also have assumptions about the role of evaluators. Is evaluation a service that you can shop for until you find an evaluator who will provide the answer you are looking for (and who will not criticize your program)? Are evaluators really working for the funding agency (whatever they may say about being there to help you improve your program)? Are they willing to ask questions and provide information of interest to the implementing agencies and national policymakers?

Given the wide, and often crucial, sets of assumptions that underlie all development programs and their evaluations, one might expect that all program documents and their corresponding evaluation designs would include a detailed statement of their underlying assumptions. One might also expect that it would be standard practice for evaluators to discuss and clarify these assumptions before the evaluation begins. Even more important, one might expect members of an evaluation team to discuss and reach an agreement on the key assumptions underlying the proposed evaluation hypotheses and research designs. However, as Apollo Nkwake reminds us, most of these assumptions are not made explicit, and in many cases, the agencies supporting programs, the managers and staff of implementing agencies, and the evaluators are often not even aware of some of these key assumptions. Based on a review of over 200 program evaluations during the past 20 years, he tells us that ... nothing has stunned me like the pertinence of assumptions to evaluation viability. What a resource and a risk assumptions can be! I have found them a great resource when they are explicatedtheir validity not withstandingand a great risk when they are not explicated.