Galaxy Chocolate – Banners – A/B Test

Galaxy chocolate has created vegan chocolate with a few different flavors. They have 2 main product pictures that are used online in the form of banners or ads. The first impression of these images counts a lot, especially online where attention and engagement are the highest currency that lead to conversion.

The question was, which one of these two images is the best one to use for online marketing?

We tested these two images in a standard A/B test with our tool to see how people actually react.

Version A

Version B

Which one do you think is better?

Metrics

30 Testers

1 day turn-around

Results

Let’s go through the results step by step. Each image was shown for 10 seconds, this is plenty of time to measure the first impression. Because these are banners, we are not so much interested in what happens over the course of these 10 seconds. We will look at that anyways to get a better idea of the differences between the two later.

For this type of an A/B test we are interested in seeing:

  • Step 1: The Average Overall Scores
  • Step 2: Maps
  • Step 4: Exact over time development

The first two will already tell us most of what we need to know. To have a more in-depth look, we also double check Step 4.

These specific results can be found on the Summary Overview Page (explanation), which entails the result of each step already. We also need the Quadrants Video (explanation) that shows how the effect of the video was on a direct moment by moment basis. As a final step we also look at the Swarm Video (explanation), which shows us each testers eye movements directly. This is not necessary in this case, but shows how everything is linked together.

Version A

Version B

Overall Scores

The overall scores already paint a very clear picture comparing the two images. The Valence of Version A (-1.55 %) is much worse than the Valence of Version B (10.66 %). This already shows that Version B has a much stronger positive impact than Version A.

The Emotional Impact supports this as well. Version A evoked much less emotions than Version B. The emotion that was most present with Version A was contempt. This was the driving emotion for the negative Valence score. Version B on the other hand had mostly happy as an emotion, which drove the positive Valence score.

The HeatMap of Version A also shows that the main focus remains mostly in the center. Version B, on the other hand, creates much more visual engagement as testers were looking at all chocolate flavors.

All of this already shows a clear winner. Version B is perceived significantly better than Version A. Now, let’s dig a little deeper to see what exactly on these images creates these emotions.

Version B wins

Maps – Valence

To understand WHY Version B is better, we look at the Valence Map.

Here, we combine eye tracking data and the emotion data. This way we know what exactly led to which emotion. Red is a negative Valence Score, and green is a positive Valence score. Of each, we also show the 5 strongest points.

Version A’s Valence Map resembles the negative summary score of Valence. Interestingly, the individual flavors evoke a negative reaction and the surrounding of the chocolate bars also evokes this negative reaction. Version B’s Valence Map also resembles the positive summary score of Valence. But here, the individual flavors evoke a strong positive reaction.

Version A

Version B

The context in which the chocolate is shown makes a huge difference

Quadrants Video

The Quadrants Video shows the timeline coupled with the Heatflow video (top left). This is the best visualization so you have side by side what testers saw and how they felt at exactly that moment.

Just looking at the emotions, it’s clear that Version B creates a constant positive emotion from start to finish. Version A on the other hand does not create any emotional engagement in the first 5 seconds and only then contempt appears.

The Heatflow (top left) also shows the general viewing pattern across all testers. Version A creates a more centered focus, whereas Version B creates a better scanning behavior. Testers look at the different flavors and visually engage rather than being stuck at a few locations like in Version A.

Version A

Version B

Version B creates positive engagement from the start and better viewing behavior

Swarm

The last step to look at viewing behavior is the Swarm Video. The Heatflow video shows the overall pattern across all testers, whereas the Swarm Video shows the same data but now each tester is shown individually.

In both cases you can see that testers read the flavors, but in Version A they also scan the surroundings more than in Version B. This confirms that the context impacts the viewing and the emotions.

In Version B, testers read the flavors and are more structured in their scanning by looking at the brand and the vegan label. The Swarm Videos shows that the structural scan path is highly influenced by the different contexts of Version A and Version B.

Version A

Version B

Version B’s context creates a structured scan path

Conclusion

These results make clear that in this A/B test, Version B is the clear winner. We also know now that it’s the context in which the chocolate is presented that makes this big difference. This can be used to guide future banner creation.

To create and increase a positive brand and product awareness, which also leads to more conversions, use Version B.

Start your tracing study today

Contact Us

MindTrace
+31 (0) 61 53 48 505

contact@mind-trace.com