Geo Testing Explained: How to Run Geographic Experiments

The only way to know if your ads actually work

Attribution models tell you which ads get credit for conversions. Geo testing tells you whether those conversions would have happened without the ads.

That's a fundamentally different -- and more important -- question.

Every attribution model, no matter how sophisticated, makes assumptions about causation. A customer saw your Meta ad and then bought your product. Did the ad cause the purchase? Or would they have bought anyway? Attribution models assume causation. Geo tests measure it.

For agency owners managing significant ad budgets, geo testing is the gold standard for proving that your work generates real, incremental revenue.

What geo testing is

A geo test divides geographic regions into two groups:

• Test group: Regions where you run ads (or change ad spend)
• Control group: Matched regions where you don't run ads (or maintain existing spend)

After a defined test period, you compare outcomes (revenue, conversions, store visits) between the two groups. The difference is your incremental lift -- the conversions that happened because of ads, not just alongside them.

It's the same logic as a medical trial. Give the treatment to one group, a placebo to another, and measure the difference. Geography is the randomization mechanism.

Why geo testing matters

It answers the incrementality question

Platform-reported ROAS tells you how much revenue ads are associated with. Geo testing tells you how much revenue ads actually caused. The gap between those numbers is often 20-50%.

A Meta campaign might show a 4x ROAS in the ad manager, but a geo test might reveal that only 2.5x is incremental -- the rest would have happened through organic search, direct traffic, or word of mouth.

It validates your attribution model

If your attribution model says Meta drives 40% of your conversions and a geo test shows Meta's incremental contribution is actually 25%, your attribution model needs calibration. Geo tests provide ground truth that you can use to adjust and improve your attribution.

It protects budgets during reviews

When a CFO asks "How do we know this ad spend is working?", attribution reports are an assertion. Geo test results are evidence. The difference matters during budget review cycles.

How to design a geo test

Step 1: Define your hypothesis

Start specific. "Meta awareness campaigns drive incremental revenue" is better than "Our ads work." The more specific the hypothesis, the more actionable the results.

Good hypotheses:

• "Increasing Meta spend by 50% in test regions will increase total revenue by 15-25%"
• "Pausing TikTok ads in control regions will decrease new customer acquisition by 20%"
• "YouTube pre-roll drives incremental top-of-funnel awareness that converts within 30 days"

Step 2: Select and match geographic regions

This is the most important step. Bad matching produces unreliable results.

Matching criteria:

• Similar population sizes
• Similar historical revenue/conversion patterns
• Similar demographics and income levels
• Similar competitive landscapes
• Similar seasonality patterns

Common geographic units:

• DMAs (Designated Market Areas) -- standard for TV and large-scale tests
• States or provinces -- simpler but less granular
• Cities or metro areas -- good for localized businesses
• Zip codes -- highest granularity but more noise

How to match: Use historical data (6-12 months) to find regions with similar revenue trajectories. If Region A and Region B have tracked closely over the past year, they're good candidates for test and control groups.

Minimum regions: Use at least 5-10 regions per group. More regions reduce the impact of random variation on your results.

Step 3: Determine test duration

Geo tests need enough time to achieve statistical significance. The duration depends on:

• Conversion volume: Higher volume = shorter test needed
• Expected lift: Smaller expected lifts require longer tests to detect
• Seasonality: Avoid running tests across major seasonal shifts
• Budget cycle: Align with natural business cycles

Rule of thumb: Most geo tests need 2-6 weeks to produce reliable results. Tests measuring brand awareness effects may need 8-12 weeks.

Step 4: Set power requirements

Before running the test, calculate the statistical power -- the probability that your test will detect a real effect if one exists.

You need:

• Historical baseline data for each region
• An estimate of the minimum detectable effect (e.g., "we want to detect at least a 10% lift")
• Your acceptable significance level (typically 90-95% confidence)

If the power calculation shows you need more regions or a longer test than you can afford, adjust your hypothesis to test a larger effect size or a bigger spend change.

Step 5: Execute the test

During the test period:

• Test regions: Implement the change (increase spend, launch new channel, pause a campaign)
• Control regions: Maintain existing conditions exactly
• Monitor daily: Watch for anomalies that could contaminate results (local events, competitor actions, supply issues)
• Don't peek and react: Making mid-test adjustments based on early results invalidates the experiment

How to analyze geo test results

Calculate incremental lift

Incremental lift = (Test group outcome - Control group outcome) / Control group outcome

If test regions generated $500K in revenue and control regions (adjusted for size) generated $420K:

Incremental lift = ($500K - $420K) / $420K = 19%

Assess statistical significance

Use a statistical test appropriate for your design. For most geo tests, a paired t-test or a difference-in-differences analysis works. The key question: is the observed difference larger than what random variation would produce?

If p-value is below 0.05 (or your chosen threshold): The lift is statistically significant. Your ads drove incremental revenue.

If p-value is above 0.05: The result is inconclusive. Either the ads don't drive incremental revenue, or your test didn't have enough power to detect the effect.

Calculate incremental ROAS

Incremental ROAS = Incremental revenue / Incremental ad spend

This is the truest measure of ad effectiveness. It tells you the return on each marginal dollar spent -- not the return on dollars associated with ads.

Common geo testing mistakes

Contamination between regions. If customers in control regions see ads through national campaigns (TV, podcasts, influencer content), the control isn't clean. Either exclude national campaigns from the test or account for them in your analysis.

Too few regions. Using 2-3 regions per group leaves you vulnerable to random variation. A local event (a factory closing, a sports championship) in one region can skew your entire result. Use 5+ regions per group.

Testing during anomalous periods. Black Friday, back-to-school, or any period with unusual buying patterns will distort results. Run tests during normal business periods.

Peeking at results early. Looking at data after 3 days and drawing conclusions is the fastest way to get wrong answers. Commit to the full test duration before analyzing.

Confusing statistical significance with practical significance. A statistically significant 2% lift might not be worth the ad spend it took to achieve. Always pair statistical analysis with ROI calculations.

When to use geo testing

Validating a new channel. Before committing major budget to TikTok or YouTube, run a geo test to measure incremental impact.

Justifying current spend. When a CFO questions whether Meta is worth the investment, a geo test provides proof.

Calibrating attribution models. Use geo test results to check whether your attribution model's channel weights match measured incrementality.

Testing spend changes. Before scaling a channel from $50K to $100K/month, test the incremental return of the additional spend in select regions.

FAQ

How much does a geo test cost to run?

The primary cost is the revenue you forgo in control regions (where you reduce or pause spend). For a 4-week test pausing Meta in 30% of your regions, you'd lose approximately 30% of those regions' Meta-attributed revenue during the test period. The cost of the lost revenue is the investment -- the insight you gain about true incrementality is the return.

Can small brands run geo tests?

Brands with fewer than 500 conversions per month will struggle to achieve statistical significance with geo tests. The numbers need to be large enough for meaningful comparisons. For smaller brands, lift studies offered by Meta and Google are a simpler alternative, though less rigorous.

How often should I run geo tests?

Quarterly for major channel validation. Whenever you're considering a significant budget change (20%+ increase or new channel launch). And annually as a general health check on your overall marketing effectiveness.

---

Go Funnel uses server-side tracking and multi-touch attribution to show you which ads actually drive revenue. Book a call to see your real numbers.