Blog
9 ways your eCommerce A/B tests are going wrong
learning path
The Complete Guide to Ecommerce Testing and Optimization
What is testing and optimization?
What is testing and optimization?
Your online store is a living, breathing entity, as alive as the shoppers who browse it. It requires constant attention to ensure it’s as user-friendly and conversion-boosting as possible. What worked when you set it up won’t necessarily work for today’s shoppers, and that’s where testing and optimization comes in.
In marketing, testing and optimizing is the process of testing variations of content, design or functionality, monitoring test results and fine-tuning your website, emails and other customer touchpoints.
It’s a constant process, with each test building upon the results and insight from previous tests. When purpose-led and done consistently, it allows you to incrementally improve your website and other marketing channels to become more in-tune with your customers’ needs and behavior, as well as your business goals. This helps you increase revenue and improve the user experience.
Benefits of testing and optimization
Benefits of testing and optimization
The process of testing and optimizing is packed full of benefits for your business:
- It eliminates guesswork and helps you make data-driven decisions
- It enables you to collect useful customer insights
- It helps you refine and optimize your onsite personalization
- It improves your customers’ shopping experience, which in turn means…
- It reduces cart and browse abandonment
- It boosts conversion rates
- It boosts ROI
Methods of testing
Methods of testing
A/B testing
A/B testing, also known as split testing, is a method that compares two versions of a variable (for example a web banner or whole web page), showing each version to equal numbers of users at random to determine which performs better against a business goal.
In an A/B test, A is the ‘control’ or original variable, and B is the new version.
When it use it
A/B testing is useful when you want to test two specific variables against each other.
Multivariate testing
Multivariate testing (MVT testing) is a method used to test multiple variables. Conducting multivariate tests is more complicated than the A/B approach, but can yield rich results as the goal is to assess which combination of variables performs best out of all possible combinations.
An example of this could be testing different variations of your hero banner, headline and CTA text to find out which combination of the variables for the three elements produces the best results for your website.
When to use it
This method is most suited to businesses who receive a significant amount of traffic to their websites or have a significant email database.
Multi-armed bandit
Multi-armed bandit is a ground-breaking approach that produces faster results. It allows you to test several variations and uses machine learning to automatically serve the variation that best meets your pre-defined goal, such as increasing average order value, conversions or identification rate of site visitors. Plus, it will keep monitoring all variants, which means if one variant suddenly starts performing better, it will show that one to more visitors.
When to use it
Multi-armed bandit is a prime method for boosting conversion rate and can be particularly useful for businesses where the opportunity cost of lost conversions is too high, such as luxury goods.
Since multi-armed bandit produces fast results, it’s also an effective method to use during short-lived campaigns, such as during peak season promotions.
Testing and SEO
Testing and SEO
While conducting tests to improve the customer experience is a big tick from Google, there are a few technical considerations to take into account.
SEO issues
The main SEO issues that arise from carrying out tests are due to the following:
1) Cloaking
Cloaking is when you present one version of a web page to search engine bots while showing a different version to the users who visit it. Search engines can interpret this as foul play.
Fresh Relevance treats search engine bots the same as we treat real visitors interacting with your site, so your efforts will not be seen as a manipulation.
2) Page duplication
Duplicate content can negatively impact your search engine ranking and dilute the visibility of your content. This is because search engines rarely show multiple versions of the same content and are forced to choose which version to rank for query results.
If you’re using conducting tests on your website, you could unintentionally be creating duplicate content, since different visitors are served variations of content. If the original content is important for your web page’s search rankings, we recommend keeping this content onsite in the page’s static source code rather than completely replacing it with dynamic content.
3) Performance
Site speed is one of many key factors in Google’s search ranking algorithm. Here are some of the key ways Fresh Relevance works to minimize the effect that testing can have on your site speed:
- Our content and scripts load asynchronously and are appropriately cached, so as not to slow down the page load time.
- The script is hosted on a global CDN to reduce the time to serve and ensure the site won’t have an additional external dependency.
- We configure DNS prefetching for all clients, speeding up the time it takes your site to load.
Server-side vs client-side testing
There are two technical approaches to take into consideration when testing your website: server-side vs client-side.
Client-side
Client-side is a common approach whereby tests and onsite personalization are rendered from the browser via a piece of Javascript on the client-side.
The client-side approach is easy to use, making it suitable for all stakeholders to implement. There’s more flexibility, since you can make changes quickly and easily. Plus, it doesn’t require advanced technical or scripting skills and it’s quicker to implement.
But the downsides of this approach are that the personalized elements load after the main page has loaded, developers have less control and it doesn’t integrate with web deployment frameworks as changes can be deployed at any time.
Server-side
Many businesses are now also making use of the server-side approach to implement tests and onsite personalization without any impact on performance. This approach allows businesses to call tests and personalized experiences on the back-end side before the page is rendered. It means that all changes are made on the server, before it reaches the browser, allowing businesses to reap the benefits of faster page speed.
This approach also integrates well with proprietary frameworks. There’s also increased data security, since sensitive data doesn’t need to be stored client-side, and your personalized content cannot be blocked by script blockers, cookie blockers or other privacy protection approaches.
On the downside, this approach requires much more involvement by technical teams and developers, there’s a longer implementation period and it can be hard or impossible for marketing and commerce teams to make rapid tests and changes.
Best approach
It’s important to look for a testing and personalization platform that offers a hybrid approach, so you can get the best of both worlds.
For example, you could use a client-side approach for initial testing via front-end personalization, because it’s easy to change, quick to iterate and agile. Then once you know which personalized experience performs best, you could switch to a server-side approach for faster loading of personalized content and optimal SEO.
What to test
What to test
Now you know more about the benefits and different methods, it’s time to think about what to test. Here are four ideas to get you started.
1) Wait times for triggered emails
Triggered emails are emails that are sent automatically after an individual takes a certain action, such as abandoning their cart before completing their purchase. Experimenting with different wait times (for example, 20 minutes versus two hours) can be an effective and simple step towards increasing conversions.
Example
Since setting up their browse abandonment emails, Awaze has been testing deployment times to compare the difference in performance between sending the first email 15 minutes, 30 minutes or 60 minutes after abandonment. After one month, the team found that browse abandonment emails sent 15 minutes after abandonment yield the best conversion rate for their cottages.com and Hoseasons brands.
2) Dynamic hero banner content
The brain processes visuals 60,000 times faster than text, which means your hero banner image does a lot of heavy lifting and it’s worth experimenting to see what resonates best with your audience.
Example
Getting Personal tested the effects of dynamic name personalization on their emails with a series of A/B tests. The results showed that with dynamic name personalization, click-through rates increased by 15% and conversion rates increased by 7%. What’s more, they saw a 37% uplift in sales.
3) Types of product recommendations
Product recommendations are a staple of all good eCommerce and travel websites. But with lots of different types of recommendations to choose from, it’s useful to test which ones work best for your website.
For example, try measuring the difference between bestsellers vs new arrivals. Showcasing bestsellers is a great way to give your customers a helping hand with their purchase decision by harnessing wisdom of the crowd, while new arrivals can help shoppers by playing to our built-in expectation that something new must be better than what came before. They are both effective types of product recommendations but one might be a particularly effective conversion booster for your website.
Look Fabulous Forever displays best selling items on their homepage, while Space NK showcases new arrivals.
Tools to help you test and optimize your marketing
Tools to help you test and optimize your marketing
Test hypothesis with just a few clicks and intelligently optimize strategies for maximum ROI with Fresh Relevance’s comprehensive testing and optimization functionality.
With Fresh Relevance, you can:
- Intelligently allocate traffic to variations that are performing well with automatic optimization
- Use A/B tests to try new marketing ideas on a subset of shoppers and refine your personalization
- Dive into up-to-the minute insight with real-time analytics and reporting
- Automatically end tests on a set date or number of impressions and display the variant that generated most revenue
- Determine the influence of the tested variable on results with control groups
Book a demo
Learn more about Fresh Relevance and book a demo to discover how we could help you achieve your business goals.
FAQs
What are the different types of testing in eCommerce?
In eCommerce, there are various types of testing including:
- A/B testing to compare two versions of a webpage or element
- Multivariate testing to analyze multiple variables simultaneously
- Usability testing to evaluate user experience
- Split testing to assess different strategies’ effectiveness in increasing conversions
What is the role of A/B testing in eCommerce optimization?
A/B testing compares two versions of a webpage or element to determine which performs better in terms of conversions and user engagement. It identifies changes that lead to increased sales, improved user experience, and overall website enhancement.
Why is eCommerce optimization important?
eCommerce optimization enhances the online shopping experience, increases conversion rates, boosts revenue, and maximizes the efficiency of digital marketing efforts. It ensures that websites and strategies are finely tuned to meet customer expectations and business goals.