A/B testing and lift

RAVE™ answers two different questions:

1. Do RAVE™ cards help at all? A holdout keeps a share of visitors from seeing any cards, then compares how often each group converts. The difference is the lift.
2. Which wording works best? Message variants split visitors between different headlines on the same card.

Both use the same conversion goal, so set that first.

Set a conversion goal

A goal tells RAVE™ what "converted" means for this campaign. Open the campaign and go to the A/B test tab.

• Page rule: a visitor converts when they reach a page matching the rule, for example URL contains /thank-you.
• Event goal: click + Add event goal and choose an event type (such as purchase) and how many minutes after seeing the campaign it must happen (30 by default). Events count when they can be tied to the same visitor: through Rave.track, auto form capture, or a webhook that passes ?vid=.

Without a goal, analytics still show impressions, hovers and clicks, but "Converted visitors" stays empty.

Turn on the holdout

1. 1

   Open the A/B test tab

   In the campaign editor, go to A/B test.

2. 2

   Set the holdout

   Drag the slider. The heading shows "X% of visitors see no RAVE™ cards". 10 to 20% is typical; 0 turns the test off; the maximum is 50%.

3. 3

   Add a conversion goal

   Add a page rule or an event goal as described above.

4. 4

   Save

   Click Save changes.

Each visitor is assigned to a group once and always lands in the same group for that campaign, so a returning visitor never flips between seeing and not seeing cards.

All visitors to the campaign's pages
   |
   +-- 80-90%  "Saw RAVE™" -> cards shown  -> conversion rate A
   |
   +-- 10-20%  "Holdout"   -> no cards     -> conversion rate B

Lift = (A - B) / B

Read the lift

Open the campaign and click Analytics. Choose a 7, 30 or 90-day range. The A/B lift: RAVE™ vs holdout section shows:

• Saw RAVE™: conversion rate and "X of Y visitors converted" for visitors who could see cards.
• Holdout: the same for the held-back group.
• Lift: the relative change in conversion rate. A 4% rate against 3% is +33.3% lift.

RAVE™ runs a two-proportion z-test and labels the result:

The A/B lift section is available on plans that include A/B tests. If the holdout is off, the section links you back to the A/B test tab.

Split-test message variants

On the Cards tab, each card (click Edit) has Message variants (split test). Click + Add variant to add a headline and, optionally, its own highlight line. Variants are labeled A, B, C and so on; each card can have up to 10.

Each variant has a weight from 0 to 100 that sets its share of visitors. Two variants at 50 split traffic evenly. Like the holdout, a visitor keeps seeing the same variant.

In Analytics, the Split tests section lists every card's variants with impressions, CTR (click-through rate), viewers and conversion rate. The leader gets a star, and every other variant shows a verdict against the leader. RAVE™ compares variants on conversions per viewer once conversions arrive, and on click-through rate until then.

Tips for a clean test

• Change one thing at a time: either the holdout or new variants, not both on the same day.
• Wait for Significant before acting. Small samples swing wildly.
• Keep the conversion goal stable during a test. Changing it changes what you are measuring.
• Test events never count in analytics, so they won't skew results.

Troubleshooting

• "Converted visitors" is zero: add a conversion goal on the A/B test tab, and check that your thank-you page URL matches the rule.
• Event goals never convert: the event must be tied to the visitor. Use Rave.track on the thank-you page, auto form capture, or pass ?vid= on the webhook.
• The lift section is missing: your plan may not include A/B tests. Check your plan on the Billing page.