Cost estimates, budgets and reserves

Three sound ways of estimating the same school software rollout gave 4,037, 3,899 and 3,510 thousand, a spread of 15 %, before any money had been spent. The budget needs one figure, a plan for spending it month by month and money held back for what will go wrong; get any of the three wrong and the project looks late or overspent from its first report.

You will estimate the rollout analogously, parametrically and bottom-up, each with a range, spread the chosen estimate into a monthly cost baseline, size two reserves for two kinds of uncertainty, and test the result against the four characteristics the U.S. Government Accountability Office (GAO) uses to judge an estimate.

Three estimating methods give three numbers, and each needs a range

A school group in Nagpur, 1,500 students on two campuses, wants admissions, fees, attendance and report cards on one school ERP system within six months. Before the sponsor funds it, someone has to say what it will cost. GAO’s cost guide names three common methods. An analogous estimateanalogous estimate: An estimate that takes the actual cost of a similar past project and adjusts it for the known differences, such as size or scope; quick, but it carries the quirks of that one project. takes the cost of a similar past project and adjusts it for the differences. A parametric estimateparametric estimate: An estimate that applies a statistical relationship between cost and one or more drivers, fitted to comparable past projects, to the new project's values of those drivers. uses a cost estimating relationshipcost estimating relationship: A formula, usually fitted to past projects, that predicts cost from a driver such as users, students or floor area; it holds only over the range of the data behind it., a statistical link between cost and a driver such as the number of students, fitted to several past projects. A bottom-up estimatebottom-up estimate: An estimate built by pricing every work package of the work breakdown structure and adding them up; GAO calls it engineering build-up. Anything left off the list costs nothing in it., which GAO calls engineering build-up, prices every work packagework package: A lowest element of a work breakdown structure, small enough to estimate, give to one owner and check as complete. More in “Scope statements and work breakdown structures” and adds them up.

Past costs cannot be compared as they were paid. The program first moves five earlier rollouts to this year’s price baseprice base: The year whose prices an estimate is stated in. Costs from other years are moved to it with a cost index before they are compared, and inflation is added for later years. with a cost index, so Rollout A’s 1,410 from four years ago counts as 1,780, and then estimates the new rollout all three ways:

"""Estimates an invented six-month school ERP rollout (1,500 students, two campuses) three ways, each with a range.

Money is in thousands, at this year's prices. The five past rollouts are first moved to this year's price base with an
invented cost index (base year = 100), as GAO's cost guide asks before data are compared.
- Analogous: the most similar past rollout, scaled by students, plus 10 % for the second campus; the range comes from
  the cost per student of all five past rollouts.
- Parametric: a straight-line cost estimating relationship (cost = a + b x students) fitted to the five rollouts by
  least squares; the range is the largest miss of that line on its own data, applied both ways.
- Bottom-up: the sum of the work packages, each with a low, most likely and high cost.
"""

from decimal import ROUND_HALF_UP, Decimal

STUDENTS = 1500
CAMPUS_UPLIFT = Decimal("1.10")

# Past rollouts: (school, years ago, students, cost then, in thousands); index of each year, this year = 100.
PAST = [
    ("Rollout A", 4, 600, 1410),
    ("Rollout B", 3, 900, 2120),
    ("Rollout C", 2, 1100, 2460),
    ("Rollout D", 1, 1400, 3230),
    ("Rollout E", 0, 2000, 4430),
]
INDEX = {4: Decimal("79.2"), 3: Decimal("84.0"), 2: Decimal("89.0"), 1: Decimal("94.3"), 0: Decimal("100")}

# Work packages of this rollout: (name, low, most likely, high), in thousands.
PACKAGES = [
    ("Requirements and fit-gap", 300, 360, 450),
    ("Configuration and integration", 700, 810, 1050),
    ("Licences, first year", 900, 900, 990),
    ("Data migration", 240, 300, 450),
    ("Training, both campuses", 300, 360, 430),
    ("Go-live and support", 230, 270, 340),
    ("Front-office devices", 140, 150, 170),
    ("Project management", 330, 360, 400),
]


def r(x):
    return x.quantize(Decimal("1"), rounding=ROUND_HALF_UP)


base = [(name, students, Decimal(cost) * 100 / INDEX[ago]) for name, ago, students, cost in PAST]
print("Past rollouts at this year's prices (thousands):")
for name, students, cost in base:
    print(f"  {name}: {students:>5} students  {r(cost):>6}  {r(cost / students * 1000):>5} per 1,000 students")

# Analogous: the nearest rollout by size, scaled by students, plus the second campus.
nearest = min(base, key=lambda b: abs(b[1] - STUDENTS))
analogous = nearest[2] * STUDENTS / nearest[1] * CAMPUS_UPLIFT
rates = [cost / students for _, students, cost in base]
a_low, a_high = min(rates) * STUDENTS * CAMPUS_UPLIFT, max(rates) * STUDENTS * CAMPUS_UPLIFT

# Parametric: least squares on (students, cost).
n = Decimal(len(base))
mx = sum(Decimal(s) for _, s, _ in base) / n
my = sum(c for _, _, c in base) / n
b = sum((Decimal(s) - mx) * (c - my) for _, s, c in base) / sum((Decimal(s) - mx) ** 2 for _, s, _ in base)
a = my - b * mx
misses = [abs((a + b * s) - c) / c for _, s, c in base]
worst = max(misses)
parametric = (a + b * STUDENTS) * CAMPUS_UPLIFT
p_low, p_high = parametric * (1 - worst), parametric * (1 + worst)

low = sum(Decimal(p[1]) for p in PACKAGES)
likely = sum(Decimal(p[2]) for p in PACKAGES)
high = sum(Decimal(p[3]) for p in PACKAGES)

print()
print(f"Cost estimating relationship: cost = {r(a)} + {b.quantize(Decimal('0.001'))} x students "
      f"(largest miss on its own data {(worst * 100).quantize(Decimal('0.1'))} %)")
print()
print(f"{'Method':<12} {'Low':>6} {'Point':>6} {'High':>6}  Basis")
print(f"{'Analogous':<12} {r(a_low):>6} {r(analogous):>6} {r(a_high):>6}  {nearest[0]} scaled to 1,500 students, +10 % campus")
print(f"{'Parametric':<12} {r(p_low):>6} {r(parametric):>6} {r(p_high):>6}  line of five rollouts, +10 % campus")
print(f"{'Bottom-up':<12} {r(low):>6} {r(likely):>6} {r(high):>6}  sum of {len(PACKAGES)} work packages")
spread = (max(analogous, parametric, likely) - min(analogous, parametric, likely)) / likely * 100
print()
print(f"The three points differ by {spread.quantize(Decimal('0.1'))} % of the bottom-up point.")

Past rollouts at this year's prices (thousands):
  Rollout A:   600 students    1780   2967 per 1,000 students
  Rollout B:   900 students    2524   2804 per 1,000 students
  Rollout C:  1100 students    2764   2513 per 1,000 students
  Rollout D:  1400 students    3425   2447 per 1,000 students
  Rollout E:  2000 students    4430   2215 per 1,000 students

Cost estimating relationship: cost = 745 + 1.866 x students (largest miss on its own data 4.8 %)

Method          Low  Point   High  Basis
Analogous      3655   4037   4896  Rollout D scaled to 1,500 students, +10 % campus
Parametric     3713   3899   4085  line of five rollouts, +10 % campus
Bottom-up      3140   3510   4280  sum of 8 work packages

The three points differ by 15.0 % of the bottom-up point.

The analogy scales Rollout D, the nearest in size, to 1,500 students and adds 10 % for the second campus: 4,037. Its range, 3,655 to 4,896, comes from the cheapest and the dearest of the five rollouts per student, because the quirks of one project travel with an analogy. The line fitted to all five predicts 3,899 with the same uplift and misses its own data by at most 4.8 %, which gives the narrowest range; it knows only students, so a cost the driver cannot see, such as a second campus, has to be added by hand. The bottom-up sum is the lowest point, 3,510, with a range from each package’s low and high costs.

The lowest number is not automatically the best one. A bottom-up estimate prices only what is on its list, and an item left off the list costs nothing in it. That is why GAO’s best practices ask for major cost elements to be cross-checked with another method. Here the 15 % gap is the first finding: the team asks whether its packages miss something the past rollouts paid for, such as a second round of data cleaning.

Which method leads depends on what is known when the estimate is due. Answer the three questions for a project of your own:

Early on only the top-down methods are possible, and their ranges are wide; once the work breakdown structure reaches work packages, the bottom-up estimate becomes the budget and the others become checks on it.

A budget spreads the estimate over the months it will be spent

The sponsor releases money month by month and progress will be measured month by month, so the estimate becomes a time-phased budgettime-phased budget: A budget with each work package's cost placed in the periods its work is planned in, so that planned spending can be compared with progress period by period.: each package’s cost placed in the months its work is planned in. The program spreads the eight packages over the six months, adds the reserve for each named risk in the month the risk could strike, and draws the running total:

"""Turns the bottom-up estimate of the invented school ERP rollout into a time-phased budget with two reserves.

Money is in thousands, at this year's prices. Each work package is spread over the months it is planned in. The
contingency reserve is the expected cost of the identified risks (probability x cost), placed in the month each risk
could strike, and it is part of the cost baseline. The management reserve, for what nobody has identified, is 5 % of
the baseline, held outside it by the steering group. The chart draws one # for every 100 of the cumulative baseline.
"""

from decimal import ROUND_HALF_UP, Decimal

MONTHS = 6
# Work package: (name, cost, {month: share of its cost})
PACKAGES = [
    ("Requirements and fit-gap", 360, {1: "1"}),
    ("Configuration and integration", 810, {2: "0.5", 3: "0.5"}),
    ("Licences, first year", 900, {2: "1"}),
    ("Data migration", 300, {3: "0.4", 4: "0.6"}),
    ("Training, both campuses", 360, {4: "0.5", 5: "0.5"}),
    ("Go-live and support", 270, {6: "1"}),
    ("Front-office devices", 150, {4: "1"}),
    ("Project management", 360, {m: "1" for m in range(1, MONTHS + 1)}),
]
# Identified risks: (id, event, probability, cost if it happens, month it could strike)
RISKS = [
    ("R1", "data migration needs a second pass", "0.4", 200, 4),
    ("R2", "vendor raises the licence fee", "0.3", 100, 2),
    ("R3", "exam season moves training days", "0.5", 60, 5),
    ("R4", "fee-payment link slips three weeks", "0.25", 120, 3),
]
MANAGEMENT_RESERVE = Decimal("0.05")


def r(x):
    return x.quantize(Decimal("1"), rounding=ROUND_HALF_UP)


work = {m: Decimal(0) for m in range(1, MONTHS + 1)}
for name, cost, spread in PACKAGES:
    shares = {m: Decimal(s) for m, s in spread.items()}
    if name == "Project management":  # level of effort: the same amount every month
        shares = {m: Decimal(1) / MONTHS for m in shares}
    assert sum(shares.values()) == 1, name
    for m, s in shares.items():
        work[m] += cost * s

estimate = sum(work.values())
reserve = {m: Decimal(0) for m in range(1, MONTHS + 1)}
print("Contingency reserve from the risk register:")
for rid, event, p, cost, month in RISKS:
    expected = Decimal(p) * cost
    reserve[month] += expected
    print(f"  {rid} {event:<36} {p:>4} x {cost:>3} = {r(expected):>3}  (month {month})")
contingency = sum(reserve.values())
baseline = estimate + contingency
mr = baseline * MANAGEMENT_RESERVE
print()
print(f"Cost estimate (work packages)   {r(estimate):>5}")
print(f"+ contingency reserve            {r(contingency):>5}")
print(f"= cost baseline                  {r(baseline):>5}")
print(f"+ management reserve (5 %)       {r(mr):>5}  held outside the baseline")
print(f"= project budget                 {r(baseline + mr):>5}")
print()
print(f"{'Month':<6} {'Work':>5} {'Reserve':>8} {'Baseline':>9} {'Cumulative':>11}")
total = Decimal(0)
for m in range(1, MONTHS + 1):
    month = work[m] + reserve[m]
    total += month
    bar = "#" * int(total / 100)
    print(f"{m:<6} {r(work[m]):>5} {r(reserve[m]):>8} {r(month):>9} {r(total):>11}  {bar}")
print()
print(f"Half the baseline is spent by month {next(m for m in range(1, MONTHS + 1) if sum(work[k] + reserve[k] for k in range(1, m + 1)) >= baseline / 2)}.")

Contingency reserve from the risk register:
  R1 data migration needs a second pass    0.4 x 200 =  80  (month 4)
  R2 vendor raises the licence fee         0.3 x 100 =  30  (month 2)
  R3 exam season moves training days       0.5 x  60 =  30  (month 5)
  R4 fee-payment link slips three weeks   0.25 x 120 =  30  (month 3)

Cost estimate (work packages)    3510
+ contingency reserve              170
= cost baseline                   3680
+ management reserve (5 %)         184  held outside the baseline
= project budget                  3864

Month   Work  Reserve  Baseline  Cumulative
1        420        0       420         420  ####
2       1365       30      1395        1815  ##################
3        585       30       615        2430  ########################
4        570       80       650        3080  ##############################
5        240       30       270        3350  #################################
6        330        0       330        3680  ####################################

Half the baseline is spent by month 3.

The cumulative column is the cost baselinecost baseline: The authorised, time-phased budget that cost performance is measured against: the work packages plus the contingency reserve, without the management reserve. drawn as a curve: 420 after month 1, 1,815 after month 2 and 3,680 after month 6. Its steep second month is the first year’s licences, 900 paid at once, which is why half the baseline is spent by month 3. A finance team that reads “six months, 3,680” and expects about 613 a month would see an overrun in month 2 that is really the plan. Earned value, the next lesson, measures progress against this curve; draw it with the line graph maker to see its shape.

The same numbers stack up into the budget the sponsor sets aside. Step through them:

Two figures that are easily mixed up come out of it. The cost baseline, 3,680, is what performance is measured against; the project budget, 3,864, is what the sponsor funds, and the difference is the reserve nobody may spend without a decision.

Two reserves cover two kinds of uncertainty

The rollout’s 170 and 184 are not padding; they answer different questions. The contingency reservecontingency reserve: Budget for identified risks, usually sized as the sum of each risk's probability times its cost and held inside the cost baseline in the periods the risks could occur. pays for risks the team can name, such as a second pass of data migration: each risk’s probability times its cost, added up over the register. That sum is a pool, not a budget for each risk. If the second pass happens it costs the full 200, not the 80 it added, and the shares of the risks that pass without happening help to pay for it. The management reservemanagement reserve: Budget held outside the cost baseline for problems nobody has identified, released by the sponsor or steering group through a recorded decision. Some organisations, GAO among them, use the terms differently. pays for what nobody has named yet, and its size is a decision of governance, best informed by a risk analysis of the whole estimate. The table puts the two side by side, with the rollout’s figures:

GAO’s earned value chapter also keeps management reserve outside the measured baseline until part of it is allocated to a problem. It reports research finding that programmes typically hold back 5 to 10 % of their contract value for it, adds that some need more and others less, and names a risk analysis of cost and schedule as one way to set the amount. The PMP examination outline lists the same two jobs under planning and managing finance: quantifying risk and contingency allocations, and managing financial reserves.

Agree on the words before the first budget report, then keep the two apart in every report that follows.

Hidden padding also ruins later measurement: a package that finishes under its padded budget looks efficient when it was only overestimated.

Four tests separate a reliable estimate from a guess

According to GAO’s research, a reliable estimate has four characteristics: it is comprehensive, well documented, accurate and credible. The guide’s 18 best practices and its 12-step process are there to produce them. Comprehensive means the whole life-cycle costlife-cycle cost: The total cost of a product or system over its whole life: development, deployment, operation, support and disposal, not only the project that builds it., because a system costs money long after the project that builds it ends. The rollout’s first draft had three weaknesses planted in it, and each fails a different test:

Two of the fixes are already in this lesson’s programs: the cost index puts every past rollout on this year’s prices, and the three methods check each other. A sensitivity check would vary the licence fee, 900 of the 3,510, and show how far the total moves. Long projects also state the price base of the estimate and the inflation they assume, because a budget in this year’s prices runs short in later years, and a cost paid in another currency names the exchange rate it was converted at; the inflation calculator shows how fast a rate compounds.

Practise: estimates, baselines and reserves

The questions reuse the rollout’s figures and add a few new cases. Work each number out before you choose an answer:

Interview questions

Warm-up: how do a contingency reserve and a management reserve differ?

A contingency reserve covers risks that have been identified: it is sized from the risk register, usually as the sum of each risk’s probability times its cost, sits inside the cost baseline and is used by the project manager when one of those risks occurs. A management reserve covers problems nobody has identified: its size is set by governance, it sits outside the baseline and only the sponsor or steering group releases it, by a decision that moves budget into the baseline. In the school rollout they were 170 and 184 thousand.

Warm-up: what makes an estimate parametric?

It uses a statistical relationship between cost and one or more drivers, fitted to data from comparable past projects, and applies it to the new project’s value of the driver. The rollout’s relationship was 745 plus 1.866 for every student, fitted to five earlier rollouts on one price base. It is quick and repeatable, but it sees only its drivers.

Two harder questions follow: a sponsor who wants the budget to match a figure, and three estimates that disagree. Decide what you would check first, then open each first step:

Key takeaways

• Analogous, parametric and bottom-up estimates suit different stages of definition, and each should be stated as a range that comes from its own data.
• A bottom-up estimate is often the lowest because anything missing from its list costs nothing, so major elements are cross-checked with another method.
• A time-phased budget places each package’s cost in its months, and the cumulative curve is the cost baseline that progress is measured against.
• A contingency reserve for named risks is one pool inside the baseline, while a management reserve for unknowns sits outside it and is released only by a decision.
• GAO judges an estimate reliable when it is comprehensive, well documented, accurate and credible, which includes the whole life cycle, one price base and a test of its major assumptions.