Deterministic bucketing
Each visitor is bucketed using a deterministic FNV-1a hash of their visitor and experiment ID. The same visitor always lands in the same group, ensuring a stable comparison without session-by-session randomness.
A/B testing
Know if the wheel 💵 made money
Not just spins and signups. A holdout group and real statistics tell you whether the popup added profit.
A holdout is a slice of your traffic the wheel deliberately skips — those shoppers browse and buy exactly as they would with no popup installed. Everyone else sees the wheel as normal. Because the same visitor always lands in the same group, the only difference between the two groups is the wheel itself, so whatever profit gap shows up between them is the wheel's effect, not a sale you already had.
What's under the hood
The stack behind the number
Every piece below runs automatically once a campaign is live.
Profit per exposed visitor
Computed from the margin and shipping cost you enter, not just orders or revenue. That's what turns a spin count into a dollar figure.
Two-proportion z-test, 95% CI
The gap between the wheel and holdout groups is measured with a two-proportion z-test, producing a 95% Wald confidence interval. You see the exact statistical range, not just which number is bigger.
Sample-ratio-mismatch detection
If the traffic split deviates, a chi-square test for sample-ratio mismatch (SRM) automatically flags the test as invalid. The check uses a strict significance level (α=0.001) to avoid false alarms.
Recommendations engine
Surfaces setup fixes grounded in your own data, like flagging a jackpot probability over 1.5% or waiting for at least 1,000 visitors before calling a result reliable.
shopifybackend / dashboard Growth+
Incremental profit / exposed visitor
+$1.81
▲ vs holdout control · 95% CI [+1.62, +2.00]
Winner ✅
Exposed 12,480
Leads 3,972
Spins 3,610
Orders 742
Demo data. Your dashboard reflects your store.
Reading the result
One dashboard, 📊 one verdict
The dashboard shows profit for the wheel group against the holdout, the confidence interval on that gap, and whether the sample-ratio check passed. You read one screen, not a spreadsheet of raw counts.
Pricing
Free on every plan
Holdout testing, profit reporting and the recommendations engine ship on Free, Growth+ and Pro+ alike. The paid tiers sell more reach and integrations, not the measurement itself.
Free · $0Growth+ · $29/moPro+ · $47/mo
Questions
Good to know
How does A/B testing for a Shopify popup work?
Profiter uses a holdout group. A slice of your traffic never sees the popup, creating a control group for comparison. The app then measures the profit per visitor for the group that saw the wheel against the group that didn't. This isolates the popup's true effect on your bottom line, not just sales you would have gotten anyway.
Is there a free A/B testing app for Shopify?
Yes. Profiter's A/B testing is included on every plan, including the $0/mo Free plan. This gives you the holdout group functionality, profit-per-visitor reporting, and the full statistical engine with sample-ratio-mismatch detection. Paid plans offer more monthly spins and integrations, not the measurement tools themselves.
How do you know if a Shopify A/B test is statistically significant?
Profiter uses a two-proportion z-test to calculate a 95% Wald confidence interval on the profit gap between the wheel and the holdout. This shows the range of likely outcomes and how sure the engine is. The dashboard won't declare a result reliable until the test has collected data from at least 1,000 exposed visitors.
How does Profiter ensure the A/B test data is valid?
First, visitor bucketing is deterministic using an FNV-1a hash, so the same shopper always sees the same experience. Second, the app runs a continuous chi-square test to detect sample-ratio mismatch. If the traffic split deviates from what it should be, the test is automatically flagged as invalid before you can draw a wrong conclusion.
Can you A/B test different discounts on Shopify?
Yes. With Profiter, you can run separate campaigns with different prize structures to see what works best. The recommendations engine will also suggest changes based on your store's data, like switching from percentage-off to fixed-amount discounts if your average order value is over $100, a common optimization.