Using Geo Experiments to Validate Your Attribution Model

Attribution models are opinions. Geo experiments are evidence.

Every attribution model makes assumptions. First-touch assumes the first interaction matters most. Last-touch assumes the final interaction is decisive. Data-driven models weight touchpoints based on statistical patterns. All of these are interpretations of the same data -- and none of them come with a guarantee of accuracy.

Geo experiments provide something attribution models cannot: causal evidence. By measuring what actually happens when you change ad spend in specific regions, you generate ground truth that can be used to calibrate and validate your attribution model.

For agency owners managing significant ad budgets, this calibration process is the difference between an attribution model you hope is right and one you know is right.

Why attribution models drift

Attribution models don't stay accurate forever. Several factors cause them to drift from reality:

Channel mix changes. When you add TikTok to a Meta + Google mix, the interactions between channels change. Your attribution model, trained on the old channel mix, may misattribute conversions.

Audience evolution. As you scale, you reach less qualified audiences. The conversion patterns that worked at $50K/month spend may not hold at $200K/month.

Platform algorithm changes. Meta and Google constantly update their targeting and delivery algorithms. These changes affect which users see your ads and how they convert, which changes the accuracy of your attribution model's assumptions.

Seasonal shifts. Black Friday shopping behavior is different from July shopping behavior. An attribution model calibrated during Q4 may be wrong in Q2.

Without periodic validation, you could be making budget decisions based on an attribution model that was accurate six months ago but isn't anymore.

The calibration framework

Step 1: Record your attribution model's predictions

Before running a geo experiment, document what your attribution model says about the channel you're testing.

Example: Your attribution model says Meta drives 35% of your total conversions, with an attributed ROAS of 3.2x.

Record this prediction in detail:

• Attributed conversion count from Meta
• Attributed revenue from Meta
• Attributed ROAS
• The attribution model used (last-touch, data-driven, etc.)

These numbers become your benchmark for comparison.

Step 2: Design the geo experiment

Design a geo holdout test for the specific channel you want to validate. Pause the channel in control regions and measure the actual revenue impact.

Follow standard geo test design principles:

• 10+ matched regions per group
• 3-4 weeks test duration
• Backend revenue data as the measurement source
• Pre-test power analysis to ensure you can detect meaningful effects

The experiment should isolate the channel in question. If you're validating Meta attribution, pause Meta only -- keep Google, TikTok, and everything else running identically in both groups.

Step 3: Measure actual incremental impact

After the test, calculate the incremental revenue driven by the channel:

Incremental revenue = (Test region revenue - Control region revenue) adjusted for baseline differences

Incremental ROAS = Incremental revenue / Channel ad spend in test regions

Incremental share = Incremental conversions / Total conversions in test regions

Step 4: Compare predictions to reality

Now compare your attribution model's predictions with the geo experiment's measurements:

| Metric | Attribution Model | Geo Experiment | Gap | |--------|------------------|----------------|-----| | Share of conversions | 35% | 28% | -7 pts | | ROAS | 3.2x | 2.4x | -0.8x | | Revenue contribution | $140K | $112K | -$28K |

In this example, your attribution model overcredits Meta by about 20%. This is a common finding -- most attribution models give Meta more credit than geo experiments confirm, primarily because they don't fully account for the conversions that would have happened without Meta ads.

Step 5: Calibrate the model

Use the gap between prediction and measurement to create a calibration factor:

Calibration factor = Geo experiment value / Attribution model value

In our example: 2.4 / 3.2 = 0.75

Apply this factor to your attribution model's Meta numbers going forward. When the model says Meta ROAS is 3.2x, your calibrated estimate is 3.2 x 0.75 = 2.4x.

Important: Calibration factors change over time. Re-validate quarterly or whenever you make significant changes to your channel mix.

Advanced calibration techniques

Multi-channel sequential testing

Don't calibrate all channels at once. Run sequential geo experiments:

1. Month 1: Test Meta (pause Meta in control regions)
2. Month 3: Test Google (pause Google in control regions)
3. Month 5: Test TikTok (pause TikTok in control regions)

Each test gives you a channel-specific calibration factor. Combined, they tell you how your full attribution model needs to be adjusted.

Cross-channel interaction measurement

A more advanced approach: test what happens when you pause Meta but keep Google running.

If pausing Meta reduces Google Search conversions by 15%, that tells you Meta drives demand that Google captures. Your attribution model should account for this interaction -- credit some of Google's conversions to Meta's demand generation.

This cross-channel insight is uniquely available through geo testing and impossible to get from attribution models or platform lift studies alone.

Incremental ROAS curves

Run multiple geo tests at different spend levels:

• Test 1: Reduce Meta by 25% in control regions
• Test 2: Reduce Meta by 50% in control regions
• Test 3: Pause Meta entirely in control regions

The results create an incremental ROAS curve -- showing how returns change at different spend levels. This curve is invaluable for budget optimization because it shows you exactly where diminishing returns begin.

What well-calibrated attribution looks like

A well-calibrated attribution model produces numbers that are within 10-15% of geo experiment results. Perfect agreement isn't the goal -- that's not realistic given the different methodologies. But directional alignment with tight bounds is achievable.

Signs your model is well-calibrated:

• Channel-level ROAS from your model is within 15% of geo-tested incremental ROAS
• Channel share of conversions matches geo-tested incremental share within 10 percentage points
• The model's channel ranking (best to worst) matches geo test rankings

Signs your model needs recalibration:

• Model ROAS diverges from geo-tested ROAS by more than 25%
• The model ranks channels differently than geo experiments
• Model predictions have gotten progressively worse over time (widening gap)

Building calibration into your workflow

Quarterly validation cycle

1. Q1: Run geo test on your largest channel (usually Meta)
2. Q2: Run geo test on your second largest channel (usually Google)
3. Q3: Test your newest or fastest-growing channel
4. Q4: Validate blended model accuracy against full-funnel geo test

Calibration documentation

Maintain a simple document tracking:

• Date of each geo experiment
• Channel tested
• Attribution model prediction vs. geo experiment result
• Calibration factor calculated
• Date factor was applied to the model

This document becomes your measurement credibility artifact. When clients or stakeholders ask "How do we know your attribution is accurate?", you have documented evidence.

Automation opportunities

Some attribution tools (including Go Funnel) allow you to apply calibration factors directly within the platform. Instead of manually adjusting numbers in spreadsheets, you can set channel-level calibration weights that adjust reported metrics automatically.

This means your team sees calibrated numbers in their daily dashboard without needing to remember to apply correction factors manually.

FAQ

How much does it cost to run calibration experiments?

The primary cost is the revenue sacrificed in control regions. For a 4-week test covering 30% of regions, expect to lose approximately 5-10% of total Meta-attributed revenue during the test period (30% of regions x 35% Meta share x some organic baseline). The insight lasts 3-6 months, making the ROI strongly positive.

What if geo experiments show my attribution model is already accurate?

Celebrate, then validate again in 3-6 months. Market conditions change, and a model that's accurate today may drift. Also consider that close agreement between your model and geo experiments is a strong signal that your tracking methodology is sound.

Can I calibrate without running geo experiments?

You can compare attribution data against platform lift studies or backend conversion analysis, but these approaches have their own biases. Geo experiments remain the gold standard for calibration because they use your own backend data and capture cross-channel effects. If geo experiments aren't feasible, second-best is running platform lift studies and cross-referencing with backend order data.

---

Go Funnel uses server-side tracking and multi-touch attribution to show you which ads actually drive revenue. Book a call to see your real numbers.