This article is part six of a running series on the question How can you build a budget before you know the work involved?

I’ve covered four strategies likely to boost the success quotient: working in phases, using risk premiums, doing up-front analysis, and turning the question around.

Today I’ll talk about the fifth strategy. (Remember, don’t read anything into the order in which I’m writing about them.) The remaining articles in this series will cover some “strategies” to avoid and one last potentially successful but risky approach.

Strategy 5: Mine the Wisdom of Crowds

The phrase “The Wisdom of Crowds” was popularized by an excellent book released six years ago by economist and columnist James Surowiecki, The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations. It in turn takes its title from the still-relevant 1852 book by Charles McKay, Memoirs of Extraordinary Popular Delusions and the Madness of Crowds.

McKay’s thesis is that the sum of us can be dumber than each of us, citing such examples as tulipomania and the South Sea bubble.¹ Surowiecki, on the other hand, sets out to demonstrate that at least in some circumstances the sum of us can be smarter than any of us.

One of these wisdom-of-crowds circumstances often occurs in determining in advance the cost of a project. Given sufficient predictors possessing some information about the work required, you can pool and average (or take the median of) the predictions, and you’ll find, according to Surowiecki, that you have a pretty good estimate.

Is it a perfect estimate? No, but then you lack the information to deliver such an estimate; remember, we’re talking about trying to deliver an estimate before you truly have the requisite information.

There are two problems with this method, neither of which is necessarily a showstopper. First, according to Surowiecki, the following conditions must hold for this method to produce a reasonable prediction:

1. Diversity of opinion, where each person has “private information,” with different data or different views on the problem.
2. Independence, meaning that these opinions or estimate are delivered without consultation with or reference to the opinions/estimates of others.
3. Decentralization, meaning that participants have specialized or local knowledge.
4. Aggregation, a means to collect these opinions or estimates and derive a collective decision.

Second, gathering the wisdom of the crowd takes time, both clock/calendar time and the time spent by the individuals thinking about the estimates. That time could often be better used in accumulating the data needed to make a good estimate — and indeed, part of the reason we’re in this situation is that the client wants the estimate now, not in two days.

That said, there are times, especially on a large/complex matter, where neither of these objections is a barrier. The client will accept an estimate in a few days or a week, knowing she’s posed a complex problem. The practice already plans to assign a number of people to the matter. These workers meet first three Surowiecki criteria and you, as project manager, can cover the fourth. And most of all, you don’t have a single clean way to gather or interpret what hard data you can collect.

Consider a large litigation in an area where there is some data for comparison — you’re not working blind in a new field of law — but there isn’t sufficient data to find clear patterns. Obviously, large retail operation slip-and-fall cases don’t meet these criteria; indeed, standard early case assessment (ECA) tools and protocols exist for just such we’ve-seen-it-many-times-before cases. I don’t think an issue such as the first Vioxx defense would qualify either, since Merck has many similar pharma examples on which to draw, even if the particular set of facts and hypotheses surrounding Vioxx is new.

But what about the situation BP now faces, to take an extreme example? I have no doubt the CEO and board of BP are asking the GC for estimates — and no one yet knows such basic facts as the extent of the damage because the well is still leaking into the Gulf. The GC can probably give a range, best case to worst case, but that’s not terribly helpful. (Worst case might be useful for the board to think about reserves — cash reserves, not oil reserves — but it’s not hard to conceive admittedly outlier scenarios that cannot be solved by cash reserves… such as the Arthur Andersen debacle.) However, the CEO probably wants something like a 75% point on the outcome curve — “give me an estimate for bad stuff happening, worse than average, but not at the blow-up-the-company level.”

I’m not privy to this case in any way, so I don’t know how BP is proceeding. That said, were I the finance lead for the GC, I’d consider asking every experienced attorney in the litigation area, in house and at their firms, for a best estimate on the ten-year cost of this litigation, both actual litigation costs and settlement/penalty amounts. I’d also pull in their lobbyists and some still-active attorneys who worked on the Exxon Valdez case. (I believe for various reasons that Exxon itself would participate.) Why lobbyists? Because ultimately the Exxon penalties were significantly reduced through various appeals, and lobbyists have a good feel for the political process. Not all lobbyists, by the way, are simply retired politicos scarfing up easy bucks; many, especially the ones you don’t read about, are extremely savvy professionals.

I bet that ten years down the road, the median numbers gathered in this manner will prove strikingly prescient (absent an Andersen-style outcome, or black swan, which is another article entirely).

This is an extreme example, but there are occasionally much smaller variations on this theme — a little time, a lot of complexity, and multiple players each with different insights. Consider asking for wisdom-of-crowds estimates and creating prediction markets in these cases. Do read Surowiecki’s book first, though; a brief summary, whether mine or Wikipedia’s, isn’t sufficient to deliver the data.

Wideband Delphi Methods

Wideband Delphi methods are also worth considering in this context. I’ve written of them before. This tool is often used by project managers, but most often in building estimates where you have — or can build — a clear task list or work breakdown structure. I’ve never applied it to trying to create a budget for a project where you’re not sure about the extent of the project itself.

That said, wideband Delphi might work if you do have a reasonable understanding of the project and you know what the specific outcome looks like. The latter is the key. In the BP example, no one knows what the specific outcome might be. But in the Merck Vioxx example, where it was clear early on that Merck was going to take early cases to trial rather than settle, they or their firms might have been able to use wideband Delphi to predict the eventual cost of, say, the first five trials, and from that estimate deliver a budget everyone could live with.

——————

¹Tulipomania is the name given to the ridiculous — in hindsight — run-up on the price of tulip bulbs in Holland in the 1630s, a run-up to which the dotcom implosion has been compared. The South Sea bubble in 1720 was a case of a stock wildly and intentionally overhyped by company owners. In both cases, the “greater fool” theory was at work — “I’ll pay too much for this good or this stock because there will always be a greater fool to whom I can unload it at a profit.” Overpriced real estate, anyone?