# A/B Subject Testing A/B Subject Testing lets you send two different subject lines to segments of your contact list and automatically — or manually — select a winner to send to the remainder. It is available for both one-off mailouts and automated mailouts. *** ### One-off mailouts #### Adding subject lines In the **Content and Preview** screen of an unsent mailout, click **Add another subject line** below the subject line field. Add your second subject line — merge fields can be used in both versions. You can delete the second subject line at any time before sending if you decide to revert to a single subject. When previewing the mailout, both subject lines are shown. When sending a preview email, you are asked to specify which version to send. #### Setting segments and winning criteria Once both subject lines are saved, go to the **Schedule and Testing** screen. The page title updates to reflect that A/B testing has been detected. **Test segments** Drag the A/B testing slider to set the proportion of your contacts who will receive each test version. The range is 5% to 30% in 5% increments. The remaining contacts will receive the winning version once it is determined. You can also choose a 50/50 split — in this case all contacts receive one of the two versions and there is no winning version send. **Winning criteria** Choose whether the winner is determined by the highest **read rate** or the highest **click-through rate**. Set the testing time period — UbiQuity defaults to one day, but this can be set to between 1–23 hours or 1–5 days. Both read rate and click-through rate are calculated from delivered emails, not total recipients. If both versions perform identically, UbiQuity defaults to Version A as the winner. You can also select **Manually selected** as the winning criteria if you want to combine metrics or apply your own judgement when choosing a winner. > A 50/50 split does not have a winning version. You will only be able to schedule the mailout, not set winning criteria. #### Scheduling and confirming The schedule and send rate apply to both the initial test send and the winning version send. The Confirm screen shows a summary including the test splits, recipient counts, winning criteria, and the estimated timing for both phases of the send. #### Sending and selecting a winner When your A/B test is confirmed, both versions are sent simultaneously. As soon as the testing period ends, UbiQuity automatically sends the winning version to the remaining contacts. To manually select a winner at any point during sending, click **Select Winning Version** on the Mailout Dashboard. Select your preferred version and confirm — the remaining contacts will be sent that version immediately. You can override an automatically determined winning version using this button, provided the winning version send has not yet begun. #### Reporting The standard mailout report is extended with an A/B testing section below the overview. This includes: * Total reads and clicks for each version * A chart tracking results over the testing period * A **statistical significance** indicator showing how confident you can be in the result **Statistical significance** measures whether the difference in performance between the two versions is large enough to be meaningful rather than random. UbiQuity uses a 95% confidence threshold: * **Significant** — you can be confident the winning subject line made a difference * **Questionable** — there is likely a relationship, but the sample size may be too small to be certain * **Not significant** — the test cannot confidently conclude one subject line performed better The Mailout Folder Report is also extended to include both subject lines in the comparison section. *** ### Automated mailouts #### Creating a test In the **Content and Preview** screen of any Automated Mailout (sent or unsent), click **New subject line A/B test**. Give the test a name (used for reporting only — not visible to recipients) and add your second subject line. #### Segments and winning criteria Automated Mailout A/B tests use a fixed 50/50 split. This is because Automated Mailouts send all emails in a batch at the same time, leaving no window to delay the winning version. Automated Mailouts also typically send to smaller batches, making uneven splits impractical. Set the winning criteria — **highest read rate** or **highest click-through rate** — on the **Recurrence and Testing** screen. #### How a winner is selected Before selecting a winner, UbiQuity checks for both statistical significance and a minimum sample size (threshold). The thresholds are based on global read and click rates for Automated Mailouts. Until both conditions are met, the mailout continues sending both versions 50/50. Once a winner is confirmed, UbiQuity sends only the winning version from the next run onwards. **Selecting a winner manually** After the Automated Mailout has run at least once, a **Select as winner now** option appears in Content and Preview. Select the winning version and confirm — the mailout will send only that version from the next run. You can use this to override the automated selection at any time. #### Active vs inactive tests A test is **inactive** until the Automated Mailout runs for the first time after the test is saved. While inactive, the test name and subject lines can be edited. Once the mailout runs, the test becomes **active** and the subject lines, test name, and winning criteria are locked. To change these, you must end the test and create a new one. #### Ending a test Click **End test** on the Content and Preview screen to end an active test before a winner is selected. You will be asked to confirm. The subject line with the higher read or click rate at that point will be kept, and the report will reflect that no winner was formally selected. > Ending a test early means incomplete results. If you run the same subject lines again, you must give the test a new name and stats reset to zero. #### Reporting The Automated Mailout report is extended with an A/B testing section showing current and previous tests. Contact history records which version each contact received. You can also filter on A/B versions across all areas of UbiQuity. *** ### Tips for effective A/B testing * **Have a clear goal** — decide in advance what success looks like before you send * **Choose the right winning criteria** — reads and clicks are not always the best measure; consider what action you actually want contacts to take * **Give the test enough time** — rushing a test to meet a deadline can result in inconclusive data * **Be patient with smaller lists** — statistical significance takes longer to reach with smaller audiences * **Keep testing** — a single successful subject line is a starting point, not a destination * **Think long-term** — use A/B test insights to improve all your communications, not just the current campaign --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.ubiquity.co.nz/documentation/channels/email/a-b-subject-testing.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.