Last-click affiliate attribution looks great. But how much of that revenue would have happened anyway? Incrementality testing answers the question every CMO eventually asks: is affiliate actually adding value, or just claiming credit?
The Incrementality Problem in Affiliate Marketing
A consumer discovers your brand via a Google search, visits your site, leaves without buying, clicks a cashback publisher's link a week later, and purchases. Last-click attributes the sale to the cashback publisher. But would that consumer have purchased anyway if the cashback publisher didn't exist? Almost certainly yes — they already had purchase intent from the Google search. The cashback publisher captured last-click credit for a conversion that would have occurred regardless.
This is the classic affiliate incrementality problem: not all attributed revenue is incremental revenue. For brands with heavy cashback and coupon publisher mixes, 30-50% of attributed affiliate GMV may be non-incremental.
Why This Matters for Program Investment Decisions
If your affiliate channel attributes $2M GMV/month but only $1.2M is incremental, your actual CPA is 67% higher than your attributed CPA suggests. This means:
- ◆You may be over-investing in the affiliate channel relative to its true contribution
- ◆You may be under-investing in channels with better incrementality
- ◆Your commission spend on non-incremental conversions is pure margin erosion with no incremental revenue offset
Incrementality Testing Methodology
Method 1 — Holdout Testing (Gold Standard)
Create a randomly selected holdout group (10-20% of your target audience) who are systematically NOT shown affiliate publisher content or offers for a defined test period (typically 4-8 weeks). Compare conversion rates between the holdout group (no affiliate exposure) and the treatment group (normal affiliate exposure).
Incremental lift = (treatment CVR - holdout CVR) / treatment CVR
Requires: precise audience segmentation, clean traffic isolation, and sufficient sample size for statistical significance (typically 50,000+ unique visitors per group).
Method 2 — Publisher-Level Holdout
Turn off a specific publisher (e.g., your largest cashback publisher) for 4 weeks and measure total channel GMV. If total channel GMV drops by the same amount as the publisher's attributed GMV, the publisher is incremental. If total channel GMV barely changes, the publisher was claiming credit for conversions that other channels (or direct) would have captured anyway.
This is simpler than a full audience holdout but limited to testing publisher-level incrementality, not channel-level.
Method 3 — Geographic Holdout
Turn off affiliate in specific geographic markets (e.g., the Pacific Northwest) for a test period and compare GMV performance vs. control markets. Less precise than audience holdouts but achievable without sophisticated audience segmentation infrastructure.
Interpreting Incrementality Results by Publisher Type
- ◆Content publishers (editorial, review): typically 70-90% incremental — they introduce buyers to your brand
- ◆Email newsletter publishers: 60-80% incremental — they re-engage buyers who may have forgotten the brand
- ◆Coupon publishers: 20-50% incremental — they capture buyers already mid-funnel from other channels
- ◆Cashback publishers: 10-40% incremental — the lowest incrementality of any publisher type
- ◆Loyalty publishers: 15-45% incremental
Using Incrementality Data to Optimize Commission Strategy
Adjust commission rates based on incremental value, not just attributed GMV:
- ◆Pay higher commissions to high-incrementality content publishers (they're creating genuine new revenue)
- ◆Reduce commissions for low-incrementality cashback publishers (they're capturing revenue that would have occurred anyway)
This rebalances your publisher mix toward top-funnel discovery and away from bottom-funnel credit capture. A program that shifts 20% of its GMV mix from cashback (30% incremental) to content publishers (80% incremental) can improve overall channel incrementality by 10-15 percentage points — without changing total attributed GMV at all.
Operationalizing Incrementality Measurement
Most brands run incrementality tests quarterly. A standard cadence:
- Q1: Baseline holdout test — establish channel-level incrementality benchmark
- Q2: Publisher-level holdouts for top 3-5 publishers by GMV contribution
- Q3: Re-test after commission restructuring — validate that changes moved incrementality
- Q4: Annual program review with incrementality-adjusted ROAS as primary metric
The programs that build incrementality measurement into their regular operating rhythm are the ones that avoid the common trap of optimizing for attributed GMV while letting true channel contribution erode. Last-click attribution will always flatter your affiliate program. Incrementality testing tells you what it's actually worth.
