Cringe-worthy from the get go, Rosamund Urwin’s Evening Standard piece starts by drawing an uncomfortable parallel between DFID’s go-getting CHASE team of “humanitarian heroes” and the Magnificent Seven – quick recap of the plot of the Magnificent Seven for those of us under 70:

A Mexican village is at the mercy of a band of outlaws. The townspeople, too afraid to fight for themselves, hire seven American gunslingers to free them from the bandits’ raids.

Thank god for brave foreign heroes who swoop in and save poor local folk from bad hombres right? Unfortunately, what this comparison unhelpfully ignores, is the important ideological shift over the last decade which has seen the sector attempt to move (albeit slowly) from an internationally led top-down model towards more contextually appropriate, locally led and delivered responses to humanitarian crises. 

The image that this articles projects of young British superheroes parachuting in to fix the world’s problems is a lazy trope which ignores the complexity of providing responsible international support during humanitarian crises AND simultaneously dismisses the very real fact that it is the people affected by disasters and crises who are the real humanitarian heroes.

‘If there’s a crisis in the world — and everyone’s trying to flee a country — these guys are the ones flying in’

Well sure, I mean except for the millions upon millions of affected individuals who don’t try to flee. You know the ones who actually live in crisis affected communities.  You know the ones who were there before the crisis, the ones who responded first (and often best) despite feeling the impact themselves. You know the ones who will continue to live there long after the glare of the spotlight fades, the money dries up, and the CHASE team have conducted their impact evaluations and moved on? Yeah, those heroes.

Which brings me neatly to the Secret Seven’s Magnificent Seven’s dazzling array of super powers: Mega multitasking? Please. Catching a helicopter? Uh-huh. Being a woman? Could we have tried a little harder with that one? Being Sherlock (the Benedict Cumberbatch incarnation)? Specific, but he really was super wasn’t he, god damn you Moriarty *shakes fist*.

All jokes aside, I truly respect and admire those who work in the humanitarian sector.  Those who spend their lives supporting people and communities to better prepare for, respond to and recover from the multitude of disasters and humanitarian crises affecting our globe today.  I applaud those who seek to change the system and build a better one in its place, very much including those who work in the DFID CHASE team.

However, this article contributes little to that cause – it reads like a thinly veiled attempt to reassure the British public that the much celebrated (but let’s be honest still pitiful) 0.7% foreign aid budget is being spent on PR wisely. Its patronisingly simplistic approach implies the reader can’t handle the truth about the complex political, economic and cultural issues surrounding humanitarian response activities.

Reductionist portrayals of the sector which frame humanitarian aid as the international provision of life saving goods and services, fail to contribute to the shifting debates and discourse that actually drive the sector forward. More than that, they actively undermine the work of many and the lives of many more.

…But wait a second, now that I think about it, isn’t the Magnificent Seven a western remake of a Japanese original? Maybe I’ve missed the point entirely and the article is actually a scathing comment on the sector’s “white-washing” of humanitarian responses? Touché Rosamund. Touché. 

Author: Duke Miller, hardened aid worker and author

Aid Leap outlines problems and then offers solutions and that is the professional way. Yet, year after year, we are confronted with the same old difficulties requiring wheel reinventions. Refugees die, money is chased, programs collapse, agencies and governments position themselves, wars continue, stars fly in, and R. Kipling smiles. We are caught in a continuous loop of assessments, proposals, M&E reports, coordination meetings, and training workshops created by donors, headquarters, and paid consultants.

When was the last time you met a really happy and satisfied aid worker?
I have an observation. Try not to judge me too harshly. Most aid workers should be subversives. I make this statement only with the finest intentions that a good bottle of tequila can engender. My sentiments are absolutely glowing.

I see a “Books” tab on the Aid Leap home page. I suggest the administrators add “Schindler’s Ark” to the list. Draw your own conclusions from the book, but in my opinion Oskar Schindler is the greatest aid worker to have ever lived. (Apologies to Fred Cuny, but if he was still around, I think he’d agree.)

Talk about bad agencies to work for; Schindler’s couldn’t have been any worse.
Program activities: fooling Nazis.
Security: nonexistent.
Goals and Objectives: fuzzy, but intense.
Budget: none, except misappropriated assets and expendables.
Mandate: nope.
Project replication: difficult.
Implementing partners: yes, but they didn’t know it.
Implementation strategy: on the fly, every day.
Salary: unpaid volunteer.
Accomplishments: hero of humanity and 1,200 lives.

Maybe none of us will ever be in Schindler’s class, but that doesn’t mean we shouldn’t try. You need to notice how your country/donor/agency policies play out around you. Sometimes those guidelines will be wrong or inefficient or stupid and they will make you angry and frustrated. At some point you might consider yourself as part of the problem. If that happens, then congratulations—don’t quit, go to work. Yeah, I mean really go to work.

I once knew an old guy who helped get refugees out of East Berlin. He was famous in certain circles. Here is a summation of what he sent me off with right before a particularly messed up assignment: ‘Despite the odds, work with what you’ve got. Sometimes enemies can be allies. Have a reputation for honesty, but lie artfully. Be political. Use your luck and daring. Concentrate on individuals. Don’t be overwhelmed by the enormity of what you face. Start small, but think big, and always keep moving.’

The last thing he said to me was, “You need to read about Oskar Schindler.”

A few weeks later, the man with all that advice, died. I went out and bought the book. It became my bible. If you haven’t looked hard at the life of Oskar Schindler, then maybe you should. It will help next time your brain begins to freeze at a donor conference or when you wake up in the middle of the night and your cot is soaking with sweat and you think, this is totally impossible.

A manager in a field programme that I evaluated recently showed me the glowing findings from his latest monitoring trip – based on a total sample size of two farmers. When I queried the small sample size, he looked shocked that I was asking. “It’s OK”, he explained, “We’re not aiming for scientific rigour in our monitoring.”

I regularly hear variants of this phrase, ranging from the whiny (“We’re not trying to prove anything”), to the pseudo-scientific (“We don’t need to achieve 95% confidence level!”) It’s typically used as an excuse for poor monitoring practices; justifying anything from miniscule samples, to biased questions, to only interviewing male community leaders.

I think managers use this excuse because they believe there is a difference between what we do (monitoring, finding things out, investigating), and what other people do (science, evidence, proof, academia). This is, however, a false dichotomy. There is no magic bar which you jump over, and suddenly find yourself conducting scientific or academic research.

Of course, there are some techniques which require high levels of time or expertise, and so are more appropriate for specialists in that area. This might include randomised control trials, or ethnographic analysis. These techniques are not, however, somehow more ‘scientific’ or ‘rigorous’ than others, but more suitable for certain questions.

For example, if you want to know the effect of a new medicine, then a controlled trial is the best approach. If you want to understand society from the point of view of the subjects of the study (and have plenty of time on your hands) then ethnography is more suitable. If you want to understand how a community-based intervention affected a community, then a focus group discussions mixed with a wider survey might be appropriate. Academics typically asks different research questions, and so use different techniques to those used in monitoring and evaluation.

This doesn’t mean that monitoring and evaluation departments are somehow justified in conducting terrible research. If your focus group discussions are dominated by men, a sampling method poorly constructed, or your analysis relies on cherry-picking data, it’s not suddenly OK just because you’re monitoring rather than an academic. Bad research is bad research, no matter who does it.

Monitoring and evaluation would get a lot better if donors, programme staff, and even some M&E professionals stopped comparing monitoring practices to some kind of imagined scientific ideal. Instead, it would be more profitable if they thought more clearly about the impact of the monitoring, and what it is designed for. Do you need to estimate the attributable impact of an intervention? Do you want to understand how this change has happened? Or do you plan to understand the context in which you work?

All are reasonable purposes, and all have different implications for the type of research that you might want to do. Pretending that basic principles of rigour and good research practice don’t apply, however, perpetuates the idea that monitoring and evaluation is a kind of glorified feedback form, a box-ticking exercise that gathers ‘results’ but doesn’t worry about what they mean.