> ## Documentation Index
> Fetch the complete documentation index at: https://docs.northbeam.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Interpreting your results

> How to read an incrementality test result and turn it into a decision

Once a test finishes collecting data, the results page summarizes what happened in the holdout and what it implies about the channel's real contribution. Reading it well means separating the statistical question (is the effect real?) from the business question (is the effect worth the spend?), and knowing which parts of the summary answer which.

## What it is

A results readout has three layers: a **headline verdict** that classifies the outcome, a set of **key metrics** that quantify it, and a **confidence interval** that bounds it. Together they describe the range of true lift consistent with what the test observed.

The verdict is one of **Significant lift**, **No detectable lift**, or **Inconclusive**. Each is a claim about signal versus noise, not about profitability. A significant result means the exposed and holdout groups differ by more than chance would explain; it does not, on its own, mean the channel clears your efficiency bar.

<div className="nbt-chip"><span className="nbt-chip-label">Scope</span><span>Applies to a single channel, in a single test window, at the holdout size the test was designed around. Results do not generalize across channels, geographies, or seasons without re-testing.</span></div>

## How it works

<div className="nbt-subs">
  <div className="nbt-sub"><span className="nbt-sub-num">01</span><span className="nbt-sub-text">The exposed and holdout groups accumulate conversions over the life of the test.</span></div>
  <div className="nbt-sub"><span className="nbt-sub-num">02</span><span className="nbt-sub-text">The gap between them is measured as **incremental conversions** and **incremental revenue**, then expressed as **lift %**, **iROAS**, and **iCAC**.</span></div>
  <div className="nbt-sub"><span className="nbt-sub-num">03</span><span className="nbt-sub-text">A **confidence interval** and **p-value** describe how precisely that gap is known, and the verdict follows from where the interval sits relative to zero.</span></div>
</div>

The time series chart plots cumulative conversions for both groups. Two properties matter: the pre-period should be flat (the lines tracking each other before day zero), and the gap should widen steadily rather than appear on a single day. A flat pre-period confirms balanced assignment; a gradual widening confirms the effect looks like media rather than a promotion, outage, or tracking change.

### Key metrics

| Metric                  | What it means                                                | How to use it                                        |
| :---------------------- | :----------------------------------------------------------- | :--------------------------------------------------- |
| Incremental conversions | Orders that would not have happened without the channel      | The numerator for every efficiency calculation below |
| Incremental revenue     | Revenue attributable to those conversions                    | Compare against spend in the test window             |
| Lift %                  | Relative difference between exposed and holdout groups       | Use for comparing across tests of different sizes    |
| iROAS                   | Incremental revenue divided by incremental spend             | The number to hold against your ROAS target          |
| iCAC                    | Incremental spend divided by incremental conversions         | Useful when your goal is new customer acquisition    |
| Confidence interval     | The plausible range for true lift                            | Plan against the lower bound, not the point estimate |
| p-value                 | Probability of seeing this result if there were no true lift | Below 0.05 is the conventional threshold             |

## How it relates to the verdict

The three verdicts correspond to where the confidence interval sits relative to zero.

|                                  | Significant lift                           | No detectable lift                                | Inconclusive                              |
| -------------------------------- | ------------------------------------------ | ------------------------------------------------- | ----------------------------------------- |
| **What the interval looks like** | Entirely above zero                        | Narrow and centered near zero                     | Wide, spanning zero                       |
| **What it means**                | The channel produced a measurable effect   | The channel did not move conversions meaningfully | The test could not tell signal from noise |
| **What to do next**              | Compare iROAS against your marginal target | Reduce or reallocate spend                        | Redesign, usually with a larger holdout   |

An interval entirely below zero is a fourth case: the exposed group underperformed. Before treating that as evidence the channel is harmful, check for assignment or delivery issues.

### When a result is inconclusive

Inconclusive is information, not failure. The usual causes:

1. **Insufficient power.** The holdout was too small, or the window too short, to detect the effect size you care about.
2. **Holdout contamination.** Users in the holdout were reached anyway, through a different placement, a retargeting audience, or cross-device delivery.
3. **Overlapping activity.** A promotion, seasonal spike, or another test running in the same window absorbed the signal.
4. **Effect below the detection floor.** The channel may work, just at a magnitude smaller than the test was designed to see.

Adjust the design before you extend the duration. A larger holdout usually buys more power than extra weeks.

## Why it matters

**A result is a range, not a number.** The point estimate sits in the middle of a confidence interval, and the honest planning number is the lower bound. Teams that treat the point estimate as precise systematically over-forecast the return on the channels they measure.

**One test describes one moment.** Incrementality shifts with saturation, creative, and seasonality. A result from six months ago describes six months ago; ongoing decisions need a re-test cadence, not a single verdict frozen in time.

**Statistical significance is not a spend decision.** A significant lift still has to clear your marginal efficiency target, not your blended average. The results page tells you whether the channel worked; your target tells you whether it worked enough.

### Common pitfalls

**Reading the point estimate as precise.** Plan against the lower bound of the interval.

**Comparing lift across differently designed tests.** Different holdout sizes, geographies, and windows produce numbers that are not directly comparable.

**Stopping early on a promising trend.** Peeking and stopping when results look good inflates false positives. Let the test reach its planned end.

**Treating one test as permanent truth.** Re-test on a cadence to track drift.

## In one sentence

A test result is the range of true lift consistent with what you observed; read the lower bound, compare it to your marginal target, and re-test on a cadence.