AB Testing in SharePoint using Variations: The SharePoint Solution (1)

The solution has the following pre-conditions:
- A site collection should exist with an X number of variation sites. Preferably the variations are named A, B, etc to clearly indicate the variations only exist for testing.
- There is a number of pages on which there is a call to action (e.g. a registration page or a newsletter sign up) and if a user performs the action they are redirected to a target page (e.g. a thank you for registering/signing up page).
- The controls used in the site have been made variation-aware (e.g. they work correctly when variations are turned on) by the developers of the site.

The solution consists of the following components:

1. Variation Switcher
2. Behaviour Recorder
3. Application page for setup
4. (minimal) Masterpage
5. Statistical Results overviews

These components are explained below.

1. Variation Switcher: The Variation switcher component will randomly assign a variation to a new user that visists the site, for existing users the code will redirect it to the variation that was assigned to him/her on their first visit. If possible the code will direct the user to the same page but in the correct variation site, otherwise it redirects to the variation root for the correct variation. A visitor is unique or not depending on the existence of a browser cookie (so if the user deleted cookies, (s)he is a new unique visitor on the next visit).

2. Behaviour Recorder: This component checks (for each page load) if the page is a call to action or a target page and records the visit of a user. It also records some other user information link the user agent string etc. This information is stored in a list in the root web of the site collection.

3. Application page for setup: This page allows a site admin to setup call to action and target page pairs for which page visits need to be recorded. It also allows the site admin to view and update the lists of campaigns etc (via a link to the list or on the page itself, tbd)

4. Masterpage: A minimal masterpage that includes the controls above is deployed by the solution so a site admin can easily customize the branding of the site for AB testing.

5. Statistical Results overviews: The current results of the AB testing should be shown in visual webparts in order to clearly show the performance of the A and B versions of the site. It should show the statistical significance of the site, indicate the power of the test and conversion rates of the versions of the site for each target-call to action page pair. 

Once the variations exist, the target and call to action pages have been set and the controls are added to the masterpage of each variation then (based on the start and end date of the campaign) the AB testing has started.

Next blog will continue about the technical implementation, and have the downloadable solution to try it out.