A/B Test Audiences Overlapped After Launch: Quarantine the Contaminated Result

A prospect who saw Variant A on Monday and Variant B on Thursday cannot cleanly represent either variant. The problem is not that the prospect received outreach twice. The problem is that the second touch may be shaped by the first.

Start with the prospect, not the aggregate reply-rate dashboard. Build one row for every prospect assigned to the test, then attach every variant, send time, connection state, reply, and booked-call outcome. That record tells you whether the audience was actually isolated.

For the wider outreach method, see the comment-led playbook. A test should preserve legitimate, consent-based business outreach and should not be used to evade platform controls or restrictions.

Which prospects received more than one test variant?

Join every test assignment and delivery event on one stable prospect identifier. Do not use a display name alone. A person can change a profile name, and one company can contain several valid prospects.

Count distinct variants per prospect, then inspect the order of exposure. A prospect who was assigned two variants but received only one may be salvageable for a delivered-message metric. A prospect who received both variants belongs in the contaminated cohort for outcomes that either message could affect.

Concurrent overlap is not automatically invalid. Microsoft Research explains that independently assigned tests can run together when the treatments do not materially affect each other, while interacting changes need isolation. Apply that distinction to the actual LinkedIn outreach surface, such as the same connection note, opening message, offer, or follow-up timing. Microsoft Research (accessed October 17, 2026).

• Deduplicate by the prospect identifier used at assignment time.
• Record the exact variant, delivery timestamp, and message stage.
• Flag every prospect with more than one delivered variant.
• Flag prospects whose control and treatment status changed after launch.

What belongs in the contamination census?

Use the census as the decision record, not as a report after the decision. Every affected prospect needs a clear salvage decision before anyone reads a winner from the test.

The table is an original operating template. It does not estimate an effect. It makes the evidence boundary visible.

Which observations remain usable after contamination?

Keep observations only when you can state the exposure history plainly. A prospect with one delivered variant and no related prior exposure can remain in a clean cohort. A prospect who received competing variants should not be used to decide which one caused a reply or booked call.

Separate operational facts from comparative results. You can still count delivered messages, identify duplicate exposures, and check whether the routing failure has stopped. Those checks do not prove one outreach variant beat another.

Restarting a test without re-randomizing can carry prior effects into the next phase. GrowthBook describes how returning users can make later results unpredictable when earlier exposure changed who comes back or how they behave. For an outreach test, use a new test key or a new assignment method when a rerun needs a fresh comparison. GrowthBook (accessed October 17, 2026).

• Keep clean single-exposure observations in a separately labeled cohort.
• Quarantine duplicate exposures from the primary comparison.
• Retain contaminated rows for the incident record and delivery audit.
• Do not combine the clean cohort and contaminated cohort in one winner calculation.

Should the team stop, rerun, or narrow the experiment?

Stop the affected comparison when the same prospect received variants that compete on the same stage or outcome, and the result would change an adoption decision. Hold the winning variant from production until the clean cohort can support that decision.

Narrow the experiment when the duplicate exposure is confined to a defined segment and the remaining prospects have complete, single-variant histories. Publish the narrowed cohort rule before reading the result, then leave the contaminated cohort out of the primary comparison.

Rerun when exposure history is incomplete, when variants touched the same conversation stage, or when a restart reused old assignments. Mutual-exclusion layers exist for the cases where the same context should not enter more than one experiment. LaunchDarkly documents this approach for experiments that may collide on the same surface. LaunchDarkly (accessed October 17, 2026).

For a current outreach process that starts with familiarity before a connection request, read why cold connection requests stopped working and how to write LinkedIn comments people actually remember. Those pages describe the method, not a guaranteed test outcome.

• Stop when contamination affects the decision cohort.
• Narrow when the clean cohort has a pre-defined, complete exposure history.
• Rerun when the team cannot reconstruct a prospect's history or isolate the treatment.

How do you prevent the next overlap?

Set the assignment rule before launch. One prospect should enter one related outreach experiment at a time. The rule needs a stable prospect ID, a test name, a variant, an assignment timestamp, an exposure event, and a release condition.

Keep tests separate when they work on unrelated stages and could not reasonably alter each other's outcome. Isolate them when they change the same connection request, opening message, offer, or follow-up path. This is a recommendation based on the interaction risk, not a claim that every concurrent test fails.

Statsig describes layers as mutually exclusive universes for experiments that touch the same surface and could interfere. It also notes that an experiment's layer is locked once the experiment starts, which is why the isolation decision belongs in launch planning rather than cleanup. Statsig (accessed October 17, 2026).

• Assign one stable prospect ID before the first test touch.
• Log assignment separately from actual exposure.
• Reserve related outreach variants for one mutually exclusive cohort.
• Block rollout when the census contains unresolved duplicate exposures.