Attribution, incrementality and marketing mix

Pune Bakes spent 30,000 rupees on Instagram ads in Pune for four weeks, and the ad platform reported 61 orders from them, almost exactly the 62 the ads needed to pay for themselves at a margin of 486 an order. A test that kept the ads out of Nashik says they added about 52 orders, somewhere between 35 and 69, so whether they paid is still open.

The platform’s number is attribution, a share of the credit for orders that happened; the test’s number is incrementality, an estimate of the orders that would not have happened without the ads. By the end of this page you can read GA4’s attribution models and what each gives a channel, explain why every platform’s total differs, design a holdout or geo test and report it as a range against break-even, and decide whether a marketing mix model is worth building.

Attribution shares the credit for an order among the visits before it

Most orders come at the end of a path of visits from several channels, and an attribution modelattribution model: A rule or a model that decides how the credit for a conversion is shared among the channels on the path before it, such as last non-direct click or data-driven attribution. decides how much of each order each channel gets. GA4 offers three: data-driven attribution, the subject of the next section; paid and organic last click, which gives everything to the last channel that is not Direct; and Google paid channels last click, which gives everything to the last Google Ads channel and falls back to the paid and organic rule when there is none. Direct gets credit only when the whole path is direct. First click, linear, time decay and position-based were removed in 2023 (Get started with attribution). Step through one of the bakery’s orders:

The program draws 500 synthetic paths of the bakery’s customers and shares their orders out under last non-direct click and under two of the removed rules, shown only to see how a rule moves credit. Run it, then press Edit and raise the chance that a path ends with a Google search from 0.6 to 0.9:

"""Share the credit for 500 of the bakery's orders among the channels on each order's path, under three rules.

The paths are synthetic, drawn from a seeded generator. "Last non-direct click" is GA4's paid and organic last click:
all the credit goes to the last channel that is not Direct, and an all-direct path keeps it for Direct ("Get started
with attribution", Analytics Help). Linear and position-based are rule-based models that GA4 removed in 2023; they are
shown only to see how a rule moves credit, never as what GA4 reports. The bakery is fictional.
"""
import random

rng = random.Random(42)
CHANNELS = ["Instagram ads", "WhatsApp", "Organic Search", "Email", "Direct"]


def path():
    """One order's path: Instagram ads often open it, WhatsApp and Email help, Search and Direct often close it."""
    steps = []
    if rng.random() < 0.55:
        steps.append("Instagram ads")
    for channel, chance in (("WhatsApp", 0.45), ("Email", 0.2)):
        if rng.random() < chance:
            steps.append(channel)
    if rng.random() < 0.6 or not steps:
        steps.append("Organic Search")
    if rng.random() < 0.35:
        steps.append("Direct")
    return steps


def last_non_direct(p):
    others = [c for c in p if c != "Direct"]
    return {others[-1] if others else "Direct": 1.0}


def linear(p):
    others = [c for c in p if c != "Direct"] or ["Direct"]
    return {c: others.count(c) / len(others) for c in dict.fromkeys(others)}


def position_based(p):
    others = [c for c in p if c != "Direct"] or ["Direct"]
    if len(others) <= 2:
        return linear(others)
    credit = {c: 0.0 for c in others}
    credit[others[0]] += 0.4
    credit[others[-1]] += 0.4
    for c in others[1:-1]:
        credit[c] += 0.2 / (len(others) - 2)
    return credit


paths = [path() for _ in range(500)]
models = {"last non-direct": last_non_direct, "linear*": linear, "position-based*": position_based}
totals = {name: {c: 0.0 for c in CHANNELS} for name in models}
for p in paths:
    for name, model in models.items():
        for channel, share in model(p).items():
            totals[name][channel] += share

print(f"{'orders credited':16}" + "".join(f"{name:>17}" for name in models))
for c in CHANNELS:
    print(f"{c:16}" + "".join(f"{totals[name][c]:17.0f}" for name in models))
touched = sum("Instagram ads" in p for p in paths)
print(f"\nInstagram ads appear on {touched} of the 500 paths; * = removed from GA4 in 2023, shown for teaching only")

orders credited   last non-direct          linear*  position-based*
Instagram ads                  39              128              136
WhatsApp                       67              109               99
Organic Search                359              219              227
Email                          35               44               38
Direct                          0                0                0

Instagram ads appear on 253 of the 500 paths; * = removed from GA4 in 2023, shown for teaching only

Instagram ads appear on 253 of the 500 paths. Last non-direct click gives them 39 orders, because Search usually comes last and takes 359. Spread the credit evenly and Instagram gets 128; weight the first and last visits and it gets 136. The orders are the same in every column. A rule decides the credit; it never discovers it, and none of the three says whether those 253 customers would have ordered without the ads.

Data-driven attribution learns from the paths that did not convert

Data-driven attributiondata-driven attribution: GA4's attribution model that shares credit by each touchpoint's estimated contribution, learned from paths that converted and paths that did not. It can revise recent conversions for up to 7 days. does not apply a fixed rule. Google says it uses the account’s own data, and its algorithms evaluate the paths of people who converted and of people who did not, with a counterfactual approach that “contrasts what happened with what could have occurred”. For Google ad exposures, the gains come from “training on data from randomized controlled trials”, and where an account has little data the models may draw on aggregate data from its data sharing settings (Get started with attribution). Follow the steps Google describes:

Two practical points follow from that description. Recent numbers move, because conversions “can be reattributed for up to 7 days after the conversion”: the bakery’s Instagram ads had 31 of last week’s orders on Monday and 34 on Friday. And the model is still attribution, run inside your reports on the paths GA4 could see. It gives an opening channel such as an Instagram ad credit that last click never does, but for one business’s ads it runs no experiment, and the bakery cannot see inside it. The reporting model is a setting, under Admin, Data display, Events, Attribution settings, beside the key event lookback window that decides how far back a touchpoint can still earn credit.

Every ad platform counts with its own rules, so the totals never agree

Each ad platform measures from its own side, with its own windows. Google Ads counts a conversion that follows a click within 30 days by default, adjustable from 1 to 90; after an engaged view of a video ad within 3 days; and after an ad that was only seen within 1 day (About conversion windows). View-through conversions, where someone “saw your ad but did not click it”, are kept out of the main Conversions column (About conversion measurement). Other platforms set their own windows and count views their own way, so read each one’s attribution settings before you compare. Above all, every platform counts the orders that follow its own ads, while GA4 gives each order to one channel by one rule, so the platforms’ totals added together can exceed the orders the business took. The bakery’s reports for one week:

Pick one source of truth for each question and say which: the bakery’s own order records for how many orders there were, GA4 for how they spread across channels, each platform’s own numbers for tuning the campaigns inside it, and a test for what the ads added. Consent widens every gap, since GA4 measures less of the visitors who refused, while a platform may still count its own clicks.

Incrementality asks what would have happened without the ads

Incrementalityincrementality: The difference between what happened with the marketing and what would have happened without it, measured against a group or a region kept from the ads. is the difference between what happened with the ads and what would have happened without them, and the only way to see the second is to keep some customers from the ads. An audience holdout keeps a holdout groupholdout group: The people, or the region, deliberately kept from a campaign during a test, so that their results show what would have happened without the ads. of people out of the campaign and compares their conversion rate with the rest, a comparison of two proportions with a standard test (NIST/SEMATECH e-Handbook). A geo test runs the ads in one region and not in a similar one. A staggered start switches regions on one after another. A comparison of before and after with no control proves nothing on its own, because the season moves both.

The bakery ran its ads in Pune and held them back in Nashik for four weeks, after four ordinary weeks. The program computes a difference in differences, Pune’s change minus Nashik’s, so that whatever moved both cities cancels out, and a range by redrawing the weeks 2,000 times. Run it, then press Edit and add two more weeks to each list of Pune and Nashik, copying the last value:

"""Estimate what the bakery's Instagram ads added, from a test that ran them in Pune and held Nashik back.

A difference in differences: the change in Pune's weekly orders minus the change in Nashik's over the same weeks, so
whatever moved both cities (the season, a holiday) cancels out. The range comes from a bootstrap, which redraws the
weeks of each period with replacement 2,000 times from a fixed seed. The orders are synthetic; the margin of 486 per
order is the goals lesson's. The bakery is fictional.
"""
import random
from math import erf, sqrt
from statistics import mean, stdev

PUNE = {"before": [52, 48, 55, 50], "during": [66, 70, 63, 69]}  # ads on in Pune during the test
NASHIK = {"before": [30, 28, 33, 29], "during": [33, 31, 35, 32]}  # ads held back
SPEND, MARGIN, WEEKS = 30_000, 486, 4


def did(test, control):
    return (mean(test["during"]) - mean(test["before"])) - (mean(control["during"]) - mean(control["before"]))


rng = random.Random(7)
draws = sorted(did({k: rng.choices(v, k=len(v)) for k, v in PUNE.items()},
                   {k: rng.choices(v, k=len(v)) for k, v in NASHIK.items()}) for _ in range(2000))
low, high = draws[49], draws[1949]
estimate, spread = did(PUNE, NASHIK), stdev(draws)
break_even = SPEND / MARGIN / WEEKS  # extra orders a week the ads must bring to pay for themselves
paid = sum(d >= break_even for d in draws)


def normal_between(a, b, mu, sigma):
    """P(a <= X <= b) for a normal curve with mean mu and spread sigma, the model the page's figure draws."""
    def cdf(x):
        return 0.5 * (1 + erf((x - mu) / (sigma * sqrt(2))))
    return cdf(b) - cdf(a)


print(f"Pune: {mean(PUNE['before']):.2f} -> {mean(PUNE['during']):.2f} orders a week")
print(f"Nashik: {mean(NASHIK['before']):.2f} -> {mean(NASHIK['during']):.2f} orders a week")
print(f"Added by the ads: {estimate:.2f} orders a week, 95 % range {low:.2f} to {high:.2f} (bootstrap spread {spread:.2f})")
print(f"\nOver {WEEKS} weeks: {WEEKS * estimate:.0f} extra orders, range {WEEKS * low:.0f} to {WEEKS * high:.0f}")
print(f"Their margin: {MARGIN * WEEKS * estimate:,.0f}, range {MARGIN * WEEKS * low:,.0f} to {MARGIN * WEEKS * high:,.0f}, "
      f"against {SPEND:,} of ad spend")
print(f"To pay for the ads: {SPEND / MARGIN:.1f} extra orders, {break_even:.2f} a week")
print(f"Redrawn tests in which the ads paid for themselves: {paid} of {len(draws)} ({100 * paid / len(draws):.1f} %)")
print(f"Normal curve with mean 13, spread 2.2: P(15.4 <= X <= 30) = {normal_between(15.4, 30, 13, 2.2):.4f}")

Pune: 51.25 -> 67.00 orders a week
Nashik: 30.00 -> 32.75 orders a week
Added by the ads: 13.00 orders a week, 95 % range 8.75 to 17.25 (bootstrap spread 2.21)

Over 4 weeks: 52 extra orders, range 35 to 69
Their margin: 25,272, range 17,010 to 33,534, against 30,000 of ad spend
To pay for the ads: 61.7 extra orders, 15.43 a week
Redrawn tests in which the ads paid for themselves: 298 of 2000 (14.9 %)
Normal curve with mean 13, spread 2.2: P(15.4 <= X <= 30) = 0.1377

The ads clearly added orders: 13 a week, with a 95 % range from 8.75 to 17.25 that stays far from zero. Over four weeks that is 52 extra orders, between 35 and 69. Paying for 30,000 of spend at 486 an order needs 61.7, or 15.43 a week, and the range runs on both sides of it. Only 298 of the 2,000 redrawn tests, 14.9 %, paid. The normal curve below has the estimate’s mean, 13, and its spread, 2.2, and its lower end starts at 15.4, the break-even: read the share of the curve beyond it.

The shaded share is 0.1377, close to the 14.9 % of the redrawn tests. Move the lower end down to 13, the break-even of a cheaper campaign, and half the curve lies beyond it. The honest report gives the range, the method and what would change the decision: “The ads added 35 to 69 orders over four weeks in Pune against Nashik, measured by a difference in differences; at 30,000 of spend they paid for themselves in about one test in seven; a longer test, more cities or a lower spend would settle it.”

Marketing mix models need years of varied spend

A marketing mix modelmarketing mix model: A statistical model that explains a business's sales over time by its spend on each channel, with carry-over and saturation, without tracking individual people. works from the other end: it explains weekly sales by the spend on each channel, with carry-over and saturation, and estimates what each channel adds without tracking anyone. Two open-source ones are widely known. Google’s Meridian, Apache-2.0, is a Bayesian framework that “enables advertisers to set up and run their own in-house models” (Meridian); its guide asks for weekly data, “a minimum of two years’ worth of weekly data for geo-level models” and three years for national ones (collect and organize your data). Meta’s Robyn, MIT-licensed, uses ridge regression to explore carry-over rates and saturation curves and is “built for granular datasets with many independent variables” (Robyn).

Both need what a small business rarely has: years of weekly sales and spend, with spend that varied enough for the model to see what each change did. The bakery has under a year of steady Instagram spend, which no model can split into causes. Answer the table’s questions for a business you know:

For most small businesses the order is attribution in the reports from day one, a test whenever a channel’s budget is big enough to matter, and a mix model only once there are years of varied spend across many channels.

Practise: explain the reports, then measure the ads

The lab works through the bakery’s numbers in two steps: first say why three reports disagree, then write the functions that estimate what the ads added and whether they paid:

"""Three gaps between the bakery's reports for one week, and their causes. The bakery is fictional."""
CAUSES = {"own-credit", "views", "reattribution", "consent"}

GAPS = {
    "g1": "The social ad platform reports 61 orders for its campaign; GA4's last non-direct click gives Paid Social 22, "
          "and most of the other 39 orders ended with a Google search.",
    "g2": "Of the platform's 61 orders, 15 came from people who saw an ad in their feed and never clicked it.",
    "g3": "On Monday GA4's data-driven report gave Instagram ads 31 of last week's orders; on Friday it gave them 34.",
}

ANSWERS = {"g1": "consent", "g2": "consent", "g3": "consent"}

WHY = {}

from gaps import ANSWERS, CAUSES, GAPS, WHY


def test_every_gap_has_a_known_cause():
    """gives each of the three gaps one cause from CAUSES"""
    assert set(ANSWERS) == set(GAPS)
    assert all(cause in CAUSES for cause in ANSWERS.values()), ANSWERS


def test_platform_counts_its_own_orders():
    """sees that the platform claims every order after its own ad"""
    assert ANSWERS["g1"] == "own-credit"


def test_views_are_invisible_to_ga4():
    """sees that orders after a view without a click exist only in the platform"""
    assert ANSWERS["g2"] == "views"


def test_recent_numbers_still_move():
    """sees that data-driven attribution revises recent conversions"""
    assert ANSWERS["g3"] == "reattribution"


def test_every_gap_has_a_sentence():
    """explains each gap in a sentence a colleague could check"""
    assert set(WHY) == set(GAPS)
    for gap, sentence in WHY.items():
        assert len(sentence.split()) >= 8, f"{gap}: write a full sentence"

"""What the bakery's Instagram ads added in Pune, measured against Nashik, and whether it paid. The bakery is fictional."""
from statistics import mean

PUNE_BEFORE, PUNE_DURING = [52, 48, 55, 50], [66, 70, 63, 69]
NASHIK_BEFORE, NASHIK_DURING = [30, 28, 33, 29], [33, 31, 35, 32]


def did(test_before, test_during, control_before, control_during):
    """The difference in differences of average weekly orders."""
    return mean(test_during) - mean(test_before)


def profit(extra_orders, margin, spend):
    """What the extra orders leave after the ad spend."""
    return extra_orders * margin


def verdict(low, high, needed):
    """'pays', 'does not pay' or 'cannot tell yet'."""
    return "pays"

from lift import NASHIK_BEFORE, NASHIK_DURING, PUNE_BEFORE, PUNE_DURING, did, profit, verdict


def test_did_on_the_bakerys_test():
    """finds 13 extra orders a week in Pune, after Nashik's own rise of 2.75"""
    assert abs(did(PUNE_BEFORE, PUNE_DURING, NASHIK_BEFORE, NASHIK_DURING) - 13.0) < 1e-9


def test_did_subtracts_the_control():
    """takes away what moved the control city too"""
    assert abs(did([10, 10], [15, 15], [20, 20], [22, 22]) - 3.0) < 1e-9
    assert abs(did([40, 42], [41, 43], [30, 30], [25, 25]) - 6.0) < 1e-9


def test_profit_pays_for_the_ads_first():
    """leaves a loss of 4,728 for 52 extra orders at 486 against 30,000 of spend"""
    assert profit(52, 486, 30000) == -4728
    assert profit(70, 486, 30000) == 4020


def test_verdict_reads_the_whole_range():
    """says pays, does not pay or cannot tell yet from both ends of the range"""
    assert verdict(35, 69, 61.7) == "cannot tell yet"
    assert verdict(65, 80, 61.7) == "pays"
    assert verdict(10, 40, 61.7) == "does not pay"
    assert verdict(61.7, 90, 61.7) == "pays"

Finally, test yourself on the five ideas, from shared credit to mix models; the questions lead with your shakiest one:

Interview questions

Warm-up (fresher to 2–5 years): attribution or incrementality, and the question each one answers

Attribution shares the credit for conversions that happened among the channels on their paths, by a rule such as last non-direct click or by a model such as GA4’s data-driven attribution; it answers which channels to credit in the reports. Incrementality asks how many conversions would not have happened without the marketing, and it needs a comparison with a group kept from the ads, such as an audience holdout or a geo test with a control region. The two can differ widely: a channel that closes most paths gets much credit from last click, while the test may show it added little. Attribution is in the reports every day; incrementality needs a test, and gives a range rather than one number.

Two more questions follow, about a platform that claims more orders than it caused and about designing a test. Choose your first move for each, then open it:

Key takeaways

• An attribution model shares the credit for an order among the channels on its path; GA4 offers data-driven, paid and organic last click and Google paid channels last click, gives Direct credit only for all-direct paths, and removed first click, linear, time decay and position-based in 2023.
• Data-driven attribution learns from paths that converted and paths that did not, revises conversions for up to 7 days, and is still attribution, not a test of your ads.
• Every ad platform counts the orders after its own ads with its own windows and views, so totals disagree by design; choose one source of truth for each question.
• Incrementality needs a group kept from the ads: a holdout or a geo test, read as a difference in differences with a range, and judged against the extra orders needed to break even.
• Marketing mix models such as Meridian and Robyn need years of weekly, varied spend; most small businesses learn more from simple tests first.