Strategies

A\B\N Tests have two different available strategies, Random and ML Optimized. While terminology differs somewhat between Multivariate and Comparison tests, in this case a permutation is equivalent to a banner.

Random

This strategy will always pick a random permutation. The use case for the Random strategy is primarly for straight A/B/N testing where a large and even sample size per permutation is of interest.

ML Optimized

This strategy will preferentially pick the best performing permutation with a higher likelihood as the test progresses. This means that a newly started ML Optimized test will behave in much the same way as a Random test while one that has run for a while may pick the best performing permutation most of the time.

The use case for the ML Optimized strategy is primarily for campaigns where you want maximum spend efficiency with limited user input.

Settings

Maximum Exploitation

The percentage chance to pick the best performing permutation that the algorithm will use when the target has been reached. If this is set to 100%, for example, the algorithm will always pick the best performing permutation when the target has been reached.

Note that even if you set this to 100% the algorithm may still change which permutation is displayed if the performance of the current best permutation drops.

Exploration Measurement

This can be set to impressions or hours and determines which measure the Exploitation Target will track.

Exploitation Target

The number of impressions per permutation, or hours that the test has been running, depending on the Exploration Measurement.

Note that the exploitation chance is calculated per creative which means that the target

Exploitation Method

Can be set to logarithmic, linear or exponential to vary how quickly the exploitation chance will approach the Maximum Explotiation as the target completion increases.

tip

Use the default settings if you are feeling unsure! You can always change them later, even after the test has launched.