Detection: Single creative driving most conversions

Key: creative_concentration_risk Severity: High Confidence: 80–90%

What this rule detects

We flag a campaign when a single ad creative is responsible for the bulk of its conversions. The rule operates at the campaign level: it gathers every active ad with at least one purchase in the audit window, requires at least two such ads and at least 30 total purchases across the campaign, then checks the most successful ad's share. If one ad accounts for 70% or more of the campaign's purchases, the rule fires. The finding is not that the creative is bad — it is usually the best ad in the account — but that the campaign's results have become a single point of failure. When that one creative fatigues, there is nothing behind it.

Why it matters in 2026

When 70% or more of a campaign's conversions come from one creative, the campaign's performance is effectively that one ad's performance, and creative fatigue then plays out in a predictable order: click-through rate falls first, then conversion rate, then ROAS, while frequency climbs against an audience that has already seen the ad. The 2024–2025 platform shifts make this fragility sharper. Meta's Advantage+ creative and Advantage+ campaign structures push accounts toward fewer, broader campaigns, which concentrates delivery; and the rebuilt "Andromeda" ad-retrieval and ranking model widened the candidate pool the system ranks from, so a creative that has earned a strong predicted-action-rate history is pushed harder and converges faster onto the historical winner. The same early-leader feedback loop that produces false winners inside an ad set (see false_winner_meta_bias) also keeps pushing impressions to a fatiguing hero creative across a campaign. On top of that, iOS 14.5+ App Tracking Transparency and conversion modeling mean a campaign leaning on one creative has a thinner, noisier signal to detect that the creative has started to decline. The defensible position is creative depth built before the decline starts — not a scramble for replacements after CPA has already moved.

The math

The headline statistic is the concentration ratio: the top ad's purchases as a share of the campaign's total purchases.

total_purchases = sum of purchases across all active converting ads
top_purchases   = purchases of the single best-converting ad

concentration = top_purchases / total_purchases

The rule fires when concentration >= 0.70.

concentration is a single proportion estimated from a finite window, so it carries sampling uncertainty. To express how tight that estimate is, we put a Wilson score interval around it — the same interval used across the catalog because it stays well-behaved for proportions near 0 or 1, where the normal-approximation "Wald" interval breaks down:

            p_hat + z^2/(2n)      z          /  p_hat(1 - p_hat)     z^2     \
center  =  ------------------    +/- -------- * sqrt( ----------------  +  -------- )
             1 + z^2/n           1 + z^2/n   \        n              4n^2    /

  p_hat = concentration = top_purchases / total_purchases
  n     = total_purchases
  z     = 1.96  for a 95% two-sided interval

When the lower bound of that interval still sits at or above 0.70, the concentration is not a small-sample artifact — it is real.

To compare two ads' true conversion rates rather than their raw counts, the Beta-Binomial conjugate model is the right tool. Treat each ad's true conversion rate as unknown with a Beta(alpha, beta) prior; observing s purchases in n clicks updates it in closed form:

prior:      rate ~ Beta(alpha, beta)
data:       s purchases out of n clicks
posterior:  rate ~ Beta(alpha + s, beta + n - s)

posterior mean = (alpha + s) / (alpha + beta + n)

With a weak uniform prior Beta(1, 1), the hero ad's posterior is Beta(1 + top_purchases, 1 + top_clicks - top_purchases) and a challenger ad's is Beta(1 + chal_purchases, 1 + chal_clicks - chal_purchases). The probability that a challenger's true rate is within striking distance of the hero's is the overlap of their posteriors — this is the test for "do I have a real replacement yet?" before redistributing budget.

When a campaign has many ads and you are comparing several challengers at once, the multiple comparisons need a false-discovery correction. We use Benjamini-Hochberg: sort the m p-values ascending, p(1) <= ... <= p(m), and keep the largest rank k whose p-value still clears its rank-scaled threshold:

find largest k such that:   p(k) <= (k / m) * Q

  m = number of comparisons
  Q = target false-discovery rate (we use Q = 0.10)

reject (keep) comparisons ranked 1..k; drop the rest

The shipped rule's trigger is a single concentration proportion against a fixed 0.70 threshold, so Benjamini-Hochberg is not applied to fire the finding — but it is the correction we use when ranking which challenger ads are genuinely closing the gap.

The thresholds we use and why

The 30-purchase floor is applied to the campaign total, not to each ad's "rest" bucket. This is deliberate: the whole point of the rule is that the non-hero ads are small, so requiring the rest bucket to clear 30 purchases would refuse to fire on exactly the most concentrated campaigns the rule exists to catch. A 30-purchase total still satisfies the rule-of-thumb for a single-proportion estimate. The 0.70 concentration threshold and the 0.5 fatigue factor in the recoverable estimate are tuned defaults, not statistically derived constants, and Nachiket should review them against real audit data. Note also that the catalog summary and an earlier draft of this page referenced a 20-purchase floor; the shipped code uses 30, which is the value documented here.

Known false-positive cases and how we mitigate them

• A newly launched campaign. A campaign live for only a few days will show apparent concentration that is really just a function of which ad was approved and started spending first. The 30-purchase floor reduces this, but a very young high-velocity campaign can still trip the rule. Re-running the audit after the campaign has matured resolves it.
• An intentional evergreen hero. Some brands deliberately run one hero creative for months and accept the dependency. If refreshed cuts, hooks, or formats of that hero are tracked as separate ads, the concentration is safer than it looks. The rule still surfaces it — knowing the exposure is useful even when the strategy is deliberate.
• A challenger that is already ready. If the team has tested replacement creative waiting to launch, the finding is informational rather than urgent. The finding's recoverable figure is framed as exposure, not waste happening today, for exactly this reason.
• One creative serving structurally different ad sets. If the hero ad runs across both a prospecting and a retargeting ad set, its campaign-level purchase share can look inflated. The evidence surfaces the per-ad counts so a practitioner can see the structure before acting.

A worked example

A $30K/month account runs an active Advantage+ Sales campaign with four active ads, all converting, over a 30-day audit window:

Total purchases: 200, which clears the 30-purchase floor. There are four active converting ads, clearing the minimum of two.

concentration = top_purchases / total_purchases = 152 / 200 = 0.76

0.76 is at or above the 0.70 threshold, so the rule fires. It is below the 0.80 strong threshold, so confidence is 80%.

Wilson 95% interval on the concentration (p_hat = 0.76, n = 200):

CI approx [0.696, 0.816]

The lower bound (0.696) is essentially at 0.70 — the concentration is real and not a small-sample artifact, though it sits right at the edge, which is consistent with the 80% (not 90%) confidence.

Recoverable monthly value models the hero converting at half its current rate after fatigue. If the campaign spent $25,000 in the window:

risk_factor   = concentration * 0.5 = 0.76 * 0.5 = 0.38
at_risk_spend = campaign_spend * risk_factor = 25000 * 0.38 = $9,500
monthly_cost  = at_risk_spend * (30 / days_in_range)
              = 9500 * (30 / 30) = $9,500 / month at risk

The finding tells the practitioner that roughly $9,500 a month of campaign spend is exposed to a single creative's fatigue cycle — the fix is to build two or three challenger creatives now, while Ad A is still performing.

Limitations

• The rule measures concentration of conversions, not creative fatigue itself. It identifies the fragility before the decline; it cannot tell you the hero has actually started to fatigue. Frequency and CTR-decay signals are handled by the separate frequency_saturation rule.
• It compares purchase counts, not revenue. A campaign where the hero drives 76% of purchases but only 50% of revenue is less concentrated in dollar terms than the count suggests.
• It cannot judge whether a challenger ad is a genuine creative idea or a near-duplicate of the hero. Two ads that are minor variants of the same concept count as two ads, even though they would fatigue together.
• Modeled conversions from ATT signal loss mean the per-ad purchase counts are partly estimates; the 30-purchase floor reduces, but does not eliminate, modeling noise in the concentration ratio.
• It operates per campaign. An account whose risk is spread as several moderately concentrated campaigns will not trip the rule even though the account-level creative bench is thin.

Source

This methodology page is generated from apps/api/app/services/detections/creative_concentration_risk.py. The detection code is open for inspection on GitHub. Disagree with how the rule fires? Open the file. Read the code. Tell us where we're wrong.