New report aims to scrutinise spending on contractors – and fails miserably

Independent evaluation is as important for donor agencies as for individual programmes. That’s why the Independent Commission for Aid Impact (ICAI) is a fantastic idea. It’s responsible for scrutinizing large chunks of DFID’s spending, and giving an independent judgement on impact and effectiveness. Their reviews have ranged from water and sanitation in Sudan, to bilateral aid to Pakistan, and offered important critiques of DFID’s work.

ICAI’s latest review is of “private sector contractors”. Responding to renumeration scandals that hit the press last September, ICAI set out to answer key questions such as: does the use of contractors affect the ability of developing countries to build up expertise? Is DFID too reliant on buying in core skills rather than developing them in their own staff? And are consultants’ profits justifiable?

Unfortunately, ICAI completely fails to answer these questions. The review is technocratic and dull, favouring bland advice such as “update DFID’s bid evaluation process” and “strengthen learning from contractor-delivered programmes”. Useful, but hardly transformational.

Despite the lack of substance, ICAI actually offer firm conclusions, saying that “contractors are an effective option for DFID”. However, they do not present any evidence to justify this. In fact, the authors say that “DFID does not have a way of assessing and learning whether contractors are a better way of delivering aid than other channels open to it”, which seems to somewhat contradict their initial assertion that contractors are good value for money.

This is because of a basic problem with ICAI’s methodology. To assess contractor’s value for money you need a comparison, which in this case should be another delivery method such as multilateral, direct CSO awards, or self-delivery. While the report gives five case studies of consultant delivery, it fails to consider or compare with any other approaches. Consequently, it is unable to offer any kind of sensible judgement. It’s like trying to award an Oscar after watching one film. You might have some intelligent things to say about the film, but there’s no way you can make any kind of comparison.

I’ll leave the question of whether contractors are actually a sensible way to deliver aid for another time. There are good arguments to be made either way – visit this blog from INTRAC for an informed and balanced opinion. As DFID prepares to spend £1.3 billion on consultants in 2013, it’s a shame that the ICAI report left us no wiser about whether they are an effective use of money.


ROUGH NOTES: New report aims to scrutinise spending on contractors – and fails miserably

No hugely controversial rough notes this week – can you do better? Give your thoughts in the comments above.

M: The comms manager for ICAI is really struggling to get attention on the reports. He has called upon UK NGOs to support him with this particularly when the reports have positive conclusions. If we wanted to take this a step further, and perhaps its more appropriate in a separate blog we could call on the aid sector community to hold the ICAI to account in actually holding DFID to account by writing substantive reports which use real (field based) evidence and analysis.

M: Also, the Cabinet Office is currently reviewing ICAI’s role. This is part of the gvnt wide commitment to evaluate the role and governance structure of all non-departmental-public-bodies, quangos, independent institutions etc. We could call others to feed in to this process – the call for evidence may have already closed so we will need to check that first – or call upon the Cabinet Office to use this, not as an opportunity to dissolve ICAI (very unlikely), but to strengthen it.

N: Good points, M!