Skip to content
Stacked Pebbles

13 A/B Testing Platforms and How Their ITP Compatibility Stacks Up

Published by Cory Underwood on October 3, 2019

Safari released Intelligent Tracking Protection (ITP) and has been steadily chipping away at the underlying tech that makes a number of A/B Testing platforms work. This has forced the collection of vendors to adopt workaround methods to ensure their product continues to work as expected. At risk is the fact that the removal of state will prompt a user to be re-segmented as a ‘new’ user on a return visit (as in the same way they would be had they deleted their cookies) affecting sampling and possibly reallocating the user to a new experience.

Below is a list of 13 A/B testing vendors. I’ll name each, provide links to any documentation I found, and tell you how they handle the above scenario. Should anyone know of any platforms or documentation I missed let me know in the comments, I’ll be happy to update the post.

Server Side Solutions

Server Side solutions work due to not be subject to the browser privacy rules by virtue of how they are designed.

  • SiteSpect: Sitting in the flow of traffic, SiteSpec does segmentation on the server. Documentation
  • Conductrics: When deployed via API integration cookies are set by the platform on the server, so not affected to the lower expiry imposed by ITP.
  • Evergage: Can be deployed via server-side. Documentation.
  • Any other server side based platform is likely also unaffected by ITP. If you are not sure you should check with your development team or vendor rep.

Hybrid Solutions

Hybrid solutions are primarily formerly Client Side platforms that have had to adopt a Hybrid model to ensure segmentation is not lost. If the solutions offered are not put in place to cause the system to enter a hybrid state, users will be subject to reassignment upon the cutoff points defined by ITP. If you use one of these platforms, deploying the changes is strongly recommended.

Client Side Platforms with known localStorage fallback

The following platforms have put into place fallback mechanics to leverage localStorage. As such they are still subject to re-segmentation as per the conditions for ITP 2.3 in which localStorage is purged. Depending on how much of your traffic is Safari and coming from sites which are considered tracking domains will determine how big of an issue this will be for you.

Client Side Platforms with no known mechanics for addressing ITP

If the platform has taken no known steps (as in I can’t find documentation one way or the other), it is thus assumed to be subject not only to ITP 2.3, but ITP 2.1 and ITP 2.2 as well. This results in scenarios where re-segmentation will occur upon cookie deletion, which can happen in 24 hours, or 7 Days, depending on conditions. If you have a lot of Safari traffic from social platforms such as Facebook, this could be a large problem for you. It is recommended to reach out to your account reps to determine impact on your optimization program.

  • Google Optimize : Some options exist via the Experiment API. This is a special case as only part of the configuration becomes server based. You are responsible for variant allocation and segmentation persistence as well as passing this information to Google Analytics. This is not turn-key.
  • Unbounce
  • CrazyEgg
  • Nelio A/B Testing

Note: Some of the the above list may be viable for simply rewriting the cookie with a server call. However be sure to check the documentation and discuss with the platform vendor. Some vendors use their cookies for more than just segment assignment and may overwrite the HTTP cookie after it’s set.

Edit: Thanks to Simo Ahava for information on Optimize and Evergage documentation.

Note: This is a mirror of my original LinkedIn post.

Published inA/B TestingBrowser Updates