Wednesday, July 29, 2009

Defining Success and Failure, Managing Risks

When I started working as a NASA contractor more than a year ago, I quickly noticed the differences between NASA and international development. No kidding! I didn't expect much in common but I wasn't sure at all what to expect with NASA.

NASA's daily routines, all the hard work that gets missions into space, revolves around minimizing the risk of failure. Risk management is a big deal. You build a spacecraft that costs a lot of money, you can't afford to have it blow up or become floating space debris. A lot of attention is paid to ensuring success by analyzing every possible failure mode and coming up with mitigation strategies when there is a residual risk. There are methodologies and full time risk manager positions handling all this.

Failures tend to be obvious, even when they are not catastrophic. Either something is working as planned or it is not. You can have a partial failure but you know exactly what is and what isn't working. It's not something you can hide.

As in the transportation industry, catastrophic failures are studied extensively to understand the causes of the failure and to make sure that this particular type of accident isn't repeated. NASA tries very hard to learn from its failures. Accidents in the human spaceflight program may get the most public coverage, but all accidents and failures in space and on the group are dissected to understand root causes. Ensuring the lessons learned from such detailed studies are embedded in project routines to avoid repeat failures is a more difficult task. Many of the contributing factors to a failure are soft issues like team communications that don't have an easy technical solution ready to apply uniformly across missions.

On the success side, the typical discourse sounds very much like PR and has little to do with trying to understand what when right when a mission is successful. There is little attention being paid to all the factors that made it possible for a particular mission to succeed. Success is defined as the absence of failure and doesn't seem to require extensive "study." Success is normal, failure is the anomaly to focus on.

I should add that success is defined very clearly and early on in a mission's development. How that success is defined early on has important implications on how the mission is designed and the types of risks and mitigation strategies that are developed, including what gets chopped down when the budgets are cut.

Turning to the field of international development, it's as if all of that is reversed. Failure is something projects / donors hardly ever admit to because they can get away with it. Failure is often far away, relatively invisible, easily forgotten. Success and failure are not clear cut because, among other things, projects (and their multiple stakeholders) often fail to come to a common understanding of what will constitute success. Monitoring and Evaluation (M&E) meant to document progress and, eventually, "success" is underfunded or not funded at all, as well as very difficult to execute in a meaningful and unbiased way. There are no incentives to openly talk about failures and why they happen.

There is no "risk management" beyond perhaps spelling out some assumptions in the early design of a project. In a NASA project, risk management is an ongoing process, not something handled during the project design phase.

Writing the statement above about the absence of risk management in international development prompted me to go check. I discovered that AusAID, the Australian development agency, does talk about "risk management." See AusGuidelines: Managing Risk. And this turns out to be very timely since the Australian Council for International Development is doing a workshop on the topic July 29th in Melbourne and July 30th in Sidney.

My previous experience in international development circles has been that few project managers have been trained as project managers and are aware of or applying project management methodologies such as those promoted by the Project Management Institute (PMI). That's just not how international development projects are designed and implemented. I did see a trend, especially in IT-related projects, where more sophisticated project management approaches were becoming a requirement.

So, if NASA needs to learn how to analyze its successes as much as it analyzes it failures, I would suggest that the international development community needs to pay more attention to defining what success and failure mean for any given project or program, and start applying risk management principles more systematically. Risk management methodologies would need to be adapted to existing international development practices and requirements and to the specifics of different types of projects.

See also:
Charles Tucker, "Fusing Risk Management and Knowledge Management," ASK Magazine, Issue 30.
Reblog this post [with Zemanta]

1 comment:

Darshana Patel said...

Dear Barbara,

Thank you for this very thoughful blog post. I have referred to it in a blog post that I have written for the CommGAP program in the World Bank as well.(
I look forward to reading more from you. Darshana