private sector

Can contractual agreements deliver change in complex systems?

/ aidleap / 2 Comments

For me, the most exciting message coming out of the debates around complexity is that development agencies should focus less on planning and delivering pre-determined outputs, and more on understanding the system that they work in and the part that they could play in changing it. They should throw away detailed implementation plans, embrace iterative management, and merrily accept that they are just one of many factors contributing to change. This blog suggests that this way of working is inconsistent with the use of contractual agreements to deliver aid.

Most large donors are heavily reliant on contractors to implement their projects. These contractors typically commit to conducting activities and producing an agreed number of outputs, in exchange for money. This agreement is formalised in logical frameworks, contract agreements, or just scrawled on the back of an envelope. The donor monitors and manages these programmes against these activities and outputs. For example, DFID conducts annual reviews of each project and scores each output from A+ to C. (With B, inexplicably, being a fail.) This blog refers to any type of organisation managed through this relationship – whether NGO, government, or private sector.

This management style requires outputs to be clearly defined and fully attributable to the programme. It encourages implementers to focus on meeting these quantitative targets rather than broader development outcomes. It is difficult to change the targets and outputs, since the donor will inevitably suspect the contractor of trying to get out of what they promised.

This can work well when the project is to deliver pre-specified services. However, it is generally incompatible with any programme that seeks to influence complex systems. Quantitative results may not be easily specifiable – and any quantitative targets could set perverse incentives. For example, a requirement to reduce the incidence of malaria by 20% can incentivise a health provider to distribute bednets, rather than strengthening health systems. Activities and outputs are likely to change as the programme learns more about the system it works in and experiments with new approaches, making it very difficult to set targets up-front. Information asymmetries between the implementer and the donor make it very difficult for the donor to effectively manage this relationship. How do you know if the implementer is revising targets based on a greater understanding of their potential to leverage systemic agents of change, or if they’re just making excuses for bad performance?

Could we deal with this by improving our monitoring and evaluation systems? This is what I spend my life doing, so I’d love to say that the answer was yes. For example, we might hold contractors accountable to outcomes rather than outputs. In some cases (such as payment by results pilots) this seems to work – but I think will remain the exception rather than the rule. Outcomes typically are only partially attributable to programme activities, if at all. The donor will never really know if missed targets reflect poor implementation or other factors, and so will be unable to hold contractors accountable for them. Perhaps more significantly, a good monitoring system is fundamentally dependent on organisational culture. Staff must want the data they collect to be a fair reflection of their performance, and to have incentives to report on failures as well as successes. They must want to know when things are not going well, and how it can be improved. In a contractor-client relationship, where present and future jobs are at stake, that’s simply not going to happen.

So what can be done? There are some simple steps which could be taken; donor staff could (and should) spend more time managing projects, and staff should have a deeper understanding of the context, which would allow them to better make judgement calls regarding the responsibility of the implementer for success or failure. Perhaps more significantly, they could take evaluation findings to reflect on the implementer – not (as is currently the case) the project design.

However, I don’t think that’s enough. Ultimately, working in complex environments isn’t susceptible to a simple contractual agreement. Transforming social and economic systems isn’t the same as contracting out dustbin collection, or even direct delivery of healthcare services. Instead of managing contractors through activity or output-based contracts, donors need to select partners who share their values, and offer longer-term support to help them achieve their goals. This can be NGOs, governments, civil society, or direct delivery. It could even be the private sector – though it is unlikely to include profit-driven aid contractors. By paying more attention to the organisational culture and motives of implementing agencies, donors might move a step closer to some of their loftier ambitions.

P.S. Regular AidLeap readers (hi Mum!) might notice that this blog is a a mish-mash of two previous blogs– the Risks of Complexity and Why the Private Sector Shouldn’t Deliver Aid. Both sparked fascinating discussions which informed this post, so please continue that in the comments.

DFID’s Private Sector Development Programme Receives Poor Rating from Aid Watchdog

/ aidleap / 1 Comment

ICAI have just released a report on private sector development, packed full of interesting nuggets and insights. It’s a great read, and we recommend that you go and read the report straight away. Just in case you don’t, we summarise some of the key conclusions, recommendations, and add our own thoughts below.

Three interesting conclusions from ICAI:

DFID’s ambition is ‘immense’, compared to what it can realistically achieve. DFID aims to transform economies and market systems, which are dependent on a load of factors which DFID can’t really do anything about. Instead, DFID “may need to adopt the role of a more modest partner, market convenor and intelligent customer”. It should be clearer about what value it can add as a development agency.

DFID is not fully considering the risks of private sector development. The authors applaud DFID for being prepared to take risks and innovate, but suggests that they need to better understand and manage these risks. Inadequate risk management may be related to DFID’s poor project management; existing processes (such as the business cases) ‘currently place too much emphasis on the initial project design and relatively little on supervision and learning’.

Simplistic targets can be counter-productive. The team note that current tools to monitor private sector development programmes are poor, as they do not account for the complex theories of change. In particular, targets can focus staff on ‘quick wins’ at the expense of longer-term changes.

ICAI’s Recommendations:

Recommendation 1: DFID should clearly define and articulate where it can add most value in PSD relative to other stakeholders. It should be more realistic in its ambitions and the impact it seeks to achieve.

Recommendation 2: DFID should provide clearer guidance to its staff on how to design a coherent and well-balanced PSD country portfolio that matches its goals for an end to extreme poverty through economic development and transformational change.

Recommendation 3: DFID needs better to calibrate and manage the risks associated with PSD and so innovate in a more informed fashion.

Recommendation 4: DFID needs to work harder to understand the barriers and business imperatives faced by the private sector in participating in development. Wherever it operates, DFID needs to be clear how and where its interventions can address these barriers

Our comments:

How much ambition is right? DFID is criticised throughout the report for being too ambitious, relative to what it realistically can achieve. But do we not want ambition? If DFID listed its ultimate goal as ‘slightly increasing income for a few people, probably’ would we criticise it for a lack of ambition? Obviously there’s a thin line between ambitious stretch targets and outright stupidity, but it’s not clear to me where the balance should lie.

How should results be measured? The potential negative effect of simplistic targets is an important finding, and we’ve discussed the ‘evidence agenda’ on this blog before. However, the report is less clear on what the solution could be – and the recommendations don’t directly address monitoring. There’s a tentative suggestion of an ‘assessment of the cumulative impact of a balanced PSD portfolio on the private sector’ in each country. I don’t really understand what such an exercise would look like, or whether it could realistically produce interesting findings given the huge problems of attribution and scale.

Easy, big spender. DFID’s rapid scale-up in private sector development is a concern, and one left largely untouched in the report. At its best, private sector development should require relatively little money. Flooding the private sector with money can have quite marked negative effects, as incentives are distorted and inclusive business suddenly becomes less profitable than chasing donor-driven initiatives. Is DFID’s spend in this area appropriate?

Why the private sector shouldn’t deliver aid

/ aidleap / 12 Comments

Delivering development and humanitarian projects is a big business, and a very profitable one. DFID will spend well over a billion pounds through the private sector in 2013-14, about 11% of its budget. This blog argues that using the private sector to deliver aid exacerbates some of the worst aspects of the development sector, leading to short-term, unaccountable aid projects.

Private sector contractors are typically selected through an open tender; competitors bid to win a contract, demonstrating that they can deliver quality outputs at the lowest possible price. Motivated by profit and drawing upon private sector expertise, contractors could be more efficient than NGOs, more reliable than local partners, and cheaper than direct delivery by DFID. It’s a seductive logic, and something we’re all familiar with; I go to the burger joint down the road because it sells the tastiest burgers at the lowest price.

However, open tenders encourage the winners to deliver aid in the worst possible way. Businesses typically win contracts for five years or less, and so have an incentive to focus on the short term. Cynically put, their economic interest is served by perpetuating the situation which they benefit from, rather than building local capacity or handing over to local partners.

Moreover, the accountability of private sector contractors is exclusively to their funder. There is no incentive to involve local communities in their work, which slows programmes down and costs money. Effective monitoring and evaluation can be deprioritised; why spend money on something which might make you look bad?

Of course, you could level similar accusations against other methods of delivering aid. In particular, NGOs are also dependent on funders, and tender for bids. Perhaps NGOs have a moderate countervailing force; they’re generally set up with an explicit social rather than financial purpose, which can affect the incentives that the staff work under. At some level, most senior managers in NGOs are required to care about more than burn rates and deadlines. In private sector contractors, by contrast, directors are often not from the development sector, and will have no institutional incentive beyond profit (even if they do have individual beliefs or desires to help). (See the comments below for more discussion on this.)

A lot of the current problems in development and humanitarian work come from misaligned incentives. The people who deliver aid have little incentive to think about the long term, to listen to or involve the people who they are delivering aid to, or effectively monitor and evaluate programmes. The only way the sector will improve is if donors think harder about how to improve these incentives, perhaps through longer-term partnerships with organisations sharing the same aims. Spending more and more money through tenders to the private sector is a big step in the wrong direction.

Footnotes:

By ‘private sector contractors’ we mean operators that are primarily driven by a financial imperative. This doesn’t necessarily include all those working in the private sector – though if they’re not primarily driven by financial imperatives then I would view them more as NGOs.

This certainly isn’t an argument against engaging with the private sector. The private sector is a key source of income and employment, and a more functional private sector in developing countries (including increased access to multinational firms, I think) should be a key aim of development projects. That’s a very different case from saying that the private sector should be a significant deliverer of aid.

Finally, as you can probably tell, I am quite short of ideas for ways that donors and governments could genuinely align incentives so that deliverers of development aid are not encouraged to focus on short-term, unaccountable projects. Ideas welcome in comments!