Webform A/B Testing

Webform A/B Testing

Web forms are interactive HTML forms published on web pages which users can fill in and submit. Like any other web form, the ones created in Zoho CRM can be embedded on your web pages for various purposes, such as to gather feedback, place product orders, or make service requests. The information submitted through the form is captured automatically in your Zoho CRM account.

Let's look at an example. You have a lead generation form on your landing page that is not giving you as many leads as expected. You may want to look into the reason for this as the form is the gateway between your website and visitors. There are many best practices to increase the performance of web forms. Something as simple as reducing the number of fields and changing the color and text of your CTA button can boost the rate of form submissions. If you want to check how these changes will impact submission rates, you can use A/B testing.

A/B testing is essentially running an experiment where two or more variations of a web form are shown to different segments of the website visitors. You can then compare the performance of the variants to identify which version gets the best results. Instead of making changes to your lead generation form based on assumptions, the A/B Testing feature for web forms in Zoho CRM helps you make an informed decision and use the form that gives the best results. 


The process of A/B testing for webforms in Zoho CRM

The process of A/B testing can be categorized into three parts:
  • Start A/B testing
    This part involves selecting the web form you want to execute the A/B test for and creating the variants. You can then set up A/B testing by specifying the target audience and how long to run the experiment for.
  • View A/B testing results
    Once A/B testing is completed, you can view the status and the statistics on the performance of the original form and the variants.
  • Launch the winner
    After you run this test, you can determine which form has the highest conversion rate, which is the percentage of visitors who submitted the form. You can launch the winning form to replace the original form on the web page.

Terminology

  • Original Form: The web form you want to carry out A/B testing on.
  • Variants: The versions of the original web form you want to test.
  • Visitors: The number of people who have visited the web page that hosts the web form.
  • Unique Submissions: The first form submission made by each visitor. Multiple submissions can be made but only one submission per visitor will be considered as a unique submission. 
  • Conversion Rate: The percentage of visitors who submitted the web form.
  • Abandon Rate: The percentage of visitors who started filling out the form but failed to submit it.
  • Complete Submission: The number of visitors who filled in all the fields in the web form and submitted it.
  • Partial submission: The number of visitors who did not fill all the fields in the web form and submitted it with some fields left empty.
  • Improvement Rate: The increase in the percentage of submissions received from the variant compared to the original form.
  • Leading: The webform variant with the highest conversion rate during the A/B test.
  • Winner: The web form variant with the highest conversion rate at the end of the A/B test. 

Start A/B Testing

Create variants of the original web form

You can create different versions of the original web form by including more fields, modifying the field properties, changing the Call To Action buttons, changing fonts, revising the background color of the form and so on. All the changes that you make in the variant can be viewed from the Changes List. You can also revert the changes, if necessary.

A/B testing configuration

Setting up A/B Testing involves defining the percentage of visitors that should be roped in for the A/B testing. Further, the percentage of visitors who should view the original web form and the variants should also be specified. The date and time or the number of visitors after which the A/B Testing should end determines the duration of your A/B Testing. You can also save the A/B Testing configuration as draft.

  • Test sampling: Specify the percentage of visitors that should participate in the A/B testing.
  • Split visitors: Specify the percentage of visitors who should view the original form and the variants. For example, 70% of the visitors will view the variant while the remaining 30% will view the original form.
  • Test ends: Specify the duration of the A/B Testing. The maximum duration can be three months. The duration can be specified in the following ways:
    • End after a specific day, week, or month
    • Mention the from and to dates
    • End after the visitor limit is reached. For example, after 350 visitors have viewed the web form, A/B Testing can be completed.

Status of A/B testing

Once the A/B test has started, the status of the test can be categorized as following:
  • Running: An ongoing A/B testing process.
  • Scheduled: The A/B test is scheduled to take place in the future.
  • Completed:The A/B testing is complete.
  • Paused: An ongoing A/B test has been temporarily put on hold.
  • Launched: The Winner has been launched to replace the original web form.
    Note: Even if one variant has a slightly higher conversion rate than another, you may still choose to use the version with a lower rate for other reasons. You can choose to launch whichever variant you want, regardless of the winner of the test.
  • Drafted: A preliminary version of the A/B test that you have configured but not yet executed.
Note
  1. There can only be one A/B test in the Running status at any one time.
  2. A/B testing is supported only for manual embed in case of CRM Forms used in Zoho Sites. It is not be supported for forms that have been embedded as elements.

Setting up A/B testing

To set up A/B testing

  1. Go to Setup > Developer Space > Webforms > A/B Testing.
  2. Click Create A/B Testing in the A/B Testing page.

    You can also create an A/B test by clicking the Create A/B Testing button from the Webforms page or by moving your mouse pointer over a webform on the Webforms tab and clicking More and then Create A/B Testing.
  3. In the  Create A/B Testing page, enter the following details:
    • A/B testing name: Specify a name for this experiment.
    • Form name: Choose the web form you want to test from the dropdown.
  4. Click Create.
    You will be redirected to the WYSIWYG editor where you will create a variant of the original form.
  5. Customize the variant as required. You can customize it in the following ways:
    • Click More in the Variant tab and then Clone or Delete to clone or delete a variant.
    • Click the Add icon (+) to add a new variant.
    • Click View changes to view the list of changes made to the variant compared to the original form.
    • Hover over a field in the Changes List page and click Revert to undo the change.
    • Click Preview to view what the variant looks like.
  6. Click Next.
  7. In the  A/B Testing Configuration pop-up:
    • Enter the percentage of visitors to include in the A/B testing experiment in the Test Sampling field.
    • Split the visitors between the different versions of the form by dragging the handle to the left or right.
    • Specify when the test will end.
    • Select the check box to send a notification about which form wins the A/B test through email.
    • Click Start A/B Testing.
    • Click Save as Draft to save the A/B testing configuration as a draft.
Note
  1. A/B testing works only for web forms that are hosted in Embed or iFrame format. 
  2. You can schedule the A/B testing to start at a future date.
  3. Once A/B testing has been started, the configurations or the variants cannot be edited. 
  4. Once the A/B test is complete, only the original form will be available for visitors until another variant is launched.

View A/B testing results

You can view the A/B testing result by hovering over the A/B testing name and clicking the View Results option. There are two tabs: Preview and Analytics.

Note
  1. It will take ten minutes after the A/B test is complete to declare the winner.
  2. The analytics will not be dynamically updated. You will need to refresh the page to view the updated results.

Preview tab

The Preview tab displays the original form and the variants that you have created for testing. You can preview the form variants in full screen by clicking the Expanded View icon (). You can also see the changes made to the variants compared to the original form.

Analytics tab

The Analytics tab displays the following details:

A/B Testing Details

  • The duration of the A/B test.
  • The number of days, weeks, months, or visitors for the A/B test.
  • The user who started the A/B test.
  • The status of A/B test.

A/B testing stats

  • Experiments: the original form and the variants.
  • Unique submissions by visitors.
  • The conversion rate and improvement in the conversion rate compared to original form.
  • The percentage of visitors assigned to the original form and the variants.
  • The leading form: The version of the form with the highest conversion rate during the A/B test.
  • The winner: The version of the form with the highest conversion rate after the A/B test is complete.
Points to remember
  1. If the conversion rate is the same for all the variants, the winner will be decided based on the following criteria in order:
  2. Minimum abandonment rate
  3. Highest complete submission rate
  4. Original form 
  5. If the conversion rate is 0 for all the web forms, no winner will be declared and the notification email will not be sent to the user who started the A/B test.
  6. If the conversion rate is 0 for the original form, no improvement value for the variants will be defined.

A/B testing analytics

The graphs display comparisons between the original form and the variants. You can view comparisons for:

  • The rate of conversion
    This graph displays a comparison of the conversion rate and the improvement rate between the original form and each of the variants.


  • The fields filled in the web form
    This graph displays the rate at which the fields in the original web form and the variants were filled in.

Pause A/B testing

If you temporarily pause the A/B testing, the original web form will be displayed to all visitors while it is paused. When the test is resumed, the variants will be shown again. 

To pause A/B testing

  1. Go to Setup > Developer Space > Webforms > A/B Testing.
  2. Click Pause in the list view in the Status column.

    Alternatively, you can open the A/B test and click the Pause button next to the Status in the A/B Testing Details section.
    You can also filter the list based on Status using the filter button ().
  3. Click Resume to continue testing.

Launch a web form

Once the A/B test is complete and the winner is declared, you can launch the winning web form on your website that will replace the original form. The winner is declared based on the conversion rate, that is, the total number of visits vs. the total number of unique submissions made by the visitors. You can either launch manually or automatically after the period of a/b testing is completed. 

To launch a web form manually
  1. Go to Setup > Developer Space > Web Forms > A/B Testing.
  2. Click Launch in the list view under the Status column.
    The winner will not be launched until you click this button.
    Alternatively, you can open the A/B test, move your mouse pointer over the experiments in the A/B Testing Stats section and click Launch.
To launch a web form automatically 
  1. In the A/B testing Configuration page, select Automatically launch the winner once the test is completed.

  2. Check Based on Condition and mention the conversion rate, if you want to launch only when a certain conversion rate is achieved. 
     

    • Related Articles

    • Webform A/B testing

      Compare and A/B test different versions of your webforms in Zoho CRM Help guide: Webform A/B Testing
    • Webform Analytics

      Analyse, measure and enhance your webforms based on various metrics in Zoho CRM Help guide: Webform Analytics
    • Webform Analytics

      Web forms are the most common tool used by organizations for lead generation. As the lead submits the form their information is captured in CRM from where the reps take it up for the further follow-ups and nurturing processes. Read more about ...
    • Can I use the encrypted field in a webform?

      Yes, you can use an encrypted field in a webform.
    • How to add fields to an existing webform?

      In the form builder, you can drag-and-drop the fields you need and change the font style and size, background color, alignment of the fields, etc. To add fields in an existing web form Click Setup > Extensions & APIs > Webforms. In the Webforms page, ...