Google Optimize: A revolution in CRO?

, Posted by Chris Wallis in Blog, Chris's posts, Conversion rate optimisation, Web Analytics

Chris Wallis, Optimisation Executive at Mezzo Labs, explores the implications of Google offering a new and accessible website testing tool.

2017 could be a huge year for the relatively un-sexy field of Conversion Rate Optimisation thanks to one development: Google’s new testing product, Google Optimize 360, and crucially, how much it will cost.

Optimize 360 has been in beta for a few months now, but 2017 is looking like the year that it hits the big time.

What can we expect from Google’s new testing tool? How much of an impact will it have on its competitors, and even the entire practice of CRO?

Optimize 360 is just one part of a bigger picture. Google’s Analytics 360 suite is a septuple threat:

  1. Analytics
  2. Tag Manager
  3. Optimize
  4. Data Studio
  5. Surveys
  6. Attribution
  7. Audience Center


Experiments: Google’s current solution

Optimize 360 isn’t the tech giant’s first attempt to break in to A/B testing territory, but it’s undoubtedly the most direct and dedicated.

Google’s current A/B test offering, Experiments, feels a bit more like an afterthought than a dedicated solution. To name just a few of its shortcomings: there’s no visual editor, no MVT capability and no multi-page testing capability.

It’s not the most sophisticated solution out there, but it’s perfectly adequate as an introduction to the concept of website testing and optimisation. For anyone other than novices with the simplest needs, however, Experiments just won’t cut it.

There are so many independent solutions out there to which Experiments pales in comparison. To stay relevant, Google has had to up its game.

Enter Google Optimize 360

Google Optimize 360 will bring a lot more to the table than Experiments ever did. Here are some of the key features:

  • Native integration with Google Analytics. Anyone running a testing programme will have had trouble marrying up numbers across different tools – the smallest discrepancy in metric tracking logic can be extremely frustrating. GA is very widely used, so assurance that the numbers in tests will make sense will be valuable to many businesses.
  • Visual editing interface to make simple text and image changes, which extends to mobile web pages. This is key – without some sort of WYSIWYG tool, Google can’t hope to compete with the likes of Adobe Target and Optimizely. Raw HTML & JS code editing will also be available to make more complex changes.
  • Easy deployment. As would be expected, the Optimize tag will be a doddle to implement via Google Tag Manager, or as an extension to Google Analytics.
  • Stronger testing capability than Experiments. Where Experiments could only manage small A/B/n tests, Optimize will be able to handle complex MVT structures.
  • Powerful targeting. A staple of any half-decent optimisation tool, Google’s solution allows for use of cookies, Data Layer variables (GTM), geographies and marketing channels (to name a few) to determine who should see what on a site.
  • Preview mode for QA. I personally find this to be a hugely under-rated feature – not all solutions offer a simple way of viewing test content outside of the visual editing tool before it goes live. Maxymiser in particular does this very well – hopefully this feature is well built out. 

How does it compare?

Google Optimize 360 is undoubtedly a welcome improvement on Experiments.

In terms of capabilities, however, there really isn’t a whole lot to get excited about – one competitor or another already provides every feature Google is advertising.

There are also plenty of things the top testing tools can do which Optimize 360 can’t (or at least hasn’t advertised yet):

  • Multi page experiments (e.g. checkout funnel)
  • Mobile App testing
  • Use non-analytics goals
  • Third party integration (heat mapping etc.)
  • Pause & update tests
  • Does not say if a test is inconclusive

But hey, these are all potential additions further down the line.

What will really set Google apart will be the recently announced free version: Optimize.

That’s right. Free.

Things just got interesting.

So what sets the two offerings apart?

Google Optimize vs. Google Optimize 360

It turns out that there aren’t many limitations placed on the free version. Here’s a quick run through of what will be on offer for the paying customer:

  • Targeting: Google Optimize 360 will allow users to target audiences defined in Analytics, where Optimize will not.
  • MVTs: The free version restricts you to 16 experiences per test. Realistically, very few tests will involve more than this.
  • Simultaneous campaigns: only 3 campaigns can run at a time with the free version, so large scale testing roadmaps are out of the question.
  • Support: Anything in-house beyond self-serve support and forums will only be available at an enterprise level.
  • Metrics: Only 3 metrics per test in the free version. That will make it tough to squeeze every insight out. Premium offers 30+. Much better.

Most of the differences resonate with those between the requirements of SMEs and larger businesses. Indeed, by Google’s own admission, Optimize 360 is ‘perfect’ for larger enterprises.

All things considered, the free solution looks like it will provide all the necessary requirements for most businesses to run a decent quality testing programme.

What are the implications?

Consider the reach of Google as a brand: Any person or business with a conversion-focussed website is more than likely to come across Optimize.

That’s great. Everyone should be testing wherever possible, and hopefully very soon nearly everyone will be.

But at the same time, it’s more than likely that we’ll see a spike in poor quality tests and false-positive results. Being relatively inexperienced in the field, Google’s self-serve knowledge base will not be the most comprehensive for best practice around strategy and analysis.

I’m betting that a lot of impressive results will be trumpeted left right and centre, but a worryingly large portion of them will be inaccurate or misinterpreted.

But hey, you have to be doing something to be doing it right.

Being limited by its infancy, Google’s move will by no means render other solutions obsolete. The premium Optimize 360 will enjoy a healthy following of businesses that are already heavily invested in Google for their analytics, but it won’t rock the boat too much for its high-end competition.

There are plenty of affordable tools available (e.g. AB Tasty, VWO) that may better suit many businesses based on their needs, and the high-end tools (e.g. Adobe Target, Oracle Maxymiser) can play on their more comprehensive feature lists and support.

Ultimately though, competitors will no longer get away with simply being enablers of testing and personalisation, except under specific circumstances (e.g. mobile app testing).

With Optimize  looming, the only way for testing providers to stay afloat will be to position themselves as experts. A great deal focus exclusively on CRO (e.g. Optimizely), and can advertise their knowledge and experience gained from years at the forefront of the field.

Google Optimize will bring incredible visibility to Conversion Rate Optimisation, and will be perfectly sufficient for testing novices and knowledge sharing/hit-based sites. I think we’ll see it act as a stepping stone to bigger things – competitors could benefit through increased desire for testing, and the education they would be in a position to offer will be a big part of building up the quality and reputation of CRO.

They’re late to the testing tool party, but when Google struts in, everyone takes notice.

Will Optimize be a revolution in CRO? Probably not, but it will definitely cannon it in to the mainstream.