Blog

The role and politics of evidence in development

4 April 2014.
By David Hulme and Pablo Yanguas.
The Independent Commission for Aid Impact released today a report criticising the selective use of evidence in DFID and the reporting bias that stems from the pressure to demonstrate “results” (see the article on The Guardian). This resonates with a point that we made at a workshop on evidence held at LSE last year. Here’s how we see the dilemma of using evidence in development.

1. The two faces of evidence

Progresa-Oportunidades is a Mexican programme of conditional cash transfers to the poor whose success stems from the central role of evidence in both its design and operation: empirical data were produced at all stages in order to determine whether performance failures were due to faulty theory or implementation, thus allowing for constant adaptation in search of greater effectiveness.
The World Bank is a major development agency which has invested heavily in generating evidence for better policy. However, a 2006 evaluation found that the large volume of high-quality research produced at the Bank had not been used to improve policy on a technical level, but to proselytise and generate political support: research that supported pre-existing policies was publicised, whereas unfavourable research was ignored.
This contrast illustrates the two faces of evidence in development, demonstrating that its role in policy-making ultimately depends on the nature of the politics surrounding it.

2. The measurement imperative

Evidence-based policy-making has the potential to increase effectiveness and value-for-money as well as strengthen accountability. But it can also generate an organisational imperative to measure impact as a result of a clear chain of political incentives connecting electorates through cabinet ministers and senior agency management all the way to programme officers and technical specialists. There is a politics of evidence in development, which can lead to certain perverse effects:

• A pressure to exaggerate the availability and quality of evidence;
• An unrealistic public understanding of the complexity of public policy; and
• An illusory form of accountability based on “experts” and “data”.

3. The experimental constraint

There seems to be a “hierarchy of evidence” in aid programming which prioritises experimental methods. This hierarchy is problematic on both conceptual and practical grounds. First, the experimental logic of discovery, although desirable, might not be feasible for many social processes, due to the complexity and contingency of causation: ingrained social phenomena like poverty, discrimination, corruption or bad governance are more akin to complex systems subject to stochastic dynamics than to atomizsed observations easily assigned to treatment and control groups. Second, there is a risk that prioritising experimental methodologies will severely constrain the policy space: in essence, a large portion of the professional brief of governance and social development advisers would be excluded from a policy space due to the lack of clear “evidence”.

4. Theory and ideology

There is also pretence in evidence-based policy that results – “hard data” – are enough to increase our confidence in the success of a given intervention, provided that the methodology behind them is robust. This view entirely misses the fact that theory is inherent to every methodology: underlying concepts and assumptions always impact the selection of indicators, variables and models. When theory is avoided or disregarded, something else must take its place; something else must justify the assumptions and methodology behind the evaluation of a given intervention. The only other thing that can provide a coherent justification for evidence is ideology, whether it is an ideology of co-operation (e.g. “ujamaa”) or one of selfishness (e.g. reductionist rational choice). Only by making concepts and theory explicit can such ideological creep be avoided in evidence-based development policy.

5. What to do

In order to reap the potential benefits of evidence with none of its perverse effects, the development community has to actively preempt its political implications while pursuing two broad objectives:
Better evidence-based policy-making:

• Increase the use of evidence in developing countries, for instance clarifying the incentives for politicians, and not just in aid agencies;
• Build stronger disciplinary-to-professional relationships, in particular between politics and the governance profession and anthropology/sociology and the social development profession; and
• Ask for the theoretical underpinnings of any evidence base.

Better dissemination and use of knowledge:

• Make the evidence debate a Southern issue rather than a Northern imposition;
• Improve the use of evidence in the media, especially in developing countries; and
• Re-frame evidence as part of democratic processes rather than elite exchanges.

 
Finally, the use of evidence in development should be guided at all times by one overarching concern: focus on what is important, not merely on what can be easily measured or tested.
 
Note: This post was originally published on the Justice and Security Research Programme blog.