Addressing uncertainty as a part of cost-benefit studies

By , September 26, 2012

When we talk about uncertainty, we’re not describing data or a study or other inanimate objects. We feel uncertain when we don’t know enough. In cost-benefit analysis (CBA), most of our uncertainties arise because of two factors:

  • The impacts of policies or programs may be hard to measure or predict; and
  • The value of those impacts may be difficult to monetize—that is, assigning dollar values to them can be challenging.

Cost-benefit analysis isn’t supposed to guarantee precise costs and benefits. If an analyst told you he was dead certain that a proposed project would save a jurisdiction exactly $1 million, wouldn’t you be suspicious or at least doubtful? A CBA that considers a range of possible scenarios and explains the likely outcomes may be much more credible despite uncertainty.

We can think about how CBAs deal with uncertainty in two main ways. The first relates to the methods a study uses to take uncertainty into account. Sensitivity analysis techniques are the best methods for addressing uncertainty, as our recent webinar and this tool describe in detail. When a CBA includes sensitivity analysis, the study may generate a realistic range of estimates for the net present value, a sturdier and more conservative approach than making a single point estimate. Or it may state the likelihood that you’ll get a result, for example, there’s only a 20 percent chance that this project will break even.

The second way cost-benefit studies can deal with uncertainty is in the way practitioners make their work transparent to others. Although the concept may seem paradoxical, practitioners come across as more dependable when they explicitly address uncertainty. They can do this by acknowledging the limitations in their work, explaining the assumptions and choices they made (such as which perspectives they included or omitted in their study), and thoroughly documenting their work. Authors who communicate clearly about their decisions, their data, and their calculations offer readers and decision makers a deeper, more nuanced, and more meaningful look into a CBA’s results.

A solid cost-benefit study will use caution in forecasting without tossing readers into a sea of doubt. A CBA that puts uncertainty in context will put readers in a better position to understand and act on the information provided.

2 comments

  1. Do you know of literature that talks about CBA using a range of uncertainties, not just base, best & worst case scenarios but a whole range given some variables of uncertainty? And not Monte Carlo either?

  2. Hi Christen. It sounds as if you are referring to partial sensitivity analysis, sometimes called one-way sensitivity analysis, which is discussed briefly in the webinar and tool kit that are cited above:
    http://bit.ly/IgVb0c
    http://bit.ly/I5qR9q
    Partial sensitivity analysis examines how changing one input affects the CBA result when all other inputs are held constant (usually at their base case or expected value). The technique is useful for identifying which inputs or assumptions have the largest impact on the estimated net present value. (In other words, which uncertain inputs the results are “most sensitive to”). If you’re looking for more information, introductory texts on CBA almost invariably devote some space to describing partial sensitivity analysis:
    http://works.bepress.com/david_weimer/4/

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>