Improve conversion rate with emotion tracking: the Coach example

Every webshop knows the problem of conversion rate optimization. How to improve conversion rate has been addressed many, many times all across the web with a lot of tips. At the end, they all boil down to one thing though, A/B testing your adjustments. One post had a list of 39 things you can adjust. My question to you is, when was the last time you ran 39 A/B tests?

Instead of just giving the standard, ‘do this and then that’ outline, I’ll tell you how we analysed the webshop of the popular fashion brand Coach. 

Coach reached out to us in July 2018 with the goal of improving the conversion rate of their webshop. Apparently they went through multiple A/B test scenarios and none of that was working well and was actually making them lose money.

The most important aspect for us was to ask the question, ‘what is it exactly that you’re after?’. Is it the conversion rate of one particular product that should be improved or something within the conversion funnel itself?

When it comes to conversion rate optimisation, a lot of variables can be tweaked. To narrow it down, it’s always important to ask yourself or your client these questions first.

The answer came quick: ‘We don’t know, we just want the conversion to go up’.

Ok, that was not very helpful, but our goal was to provide Coach with the biggest bang for their buck. Meaning, which changes would result in the highest return on investment (ROI) for Coach?

What did we do?

It is never a good idea to start testing anything blindly. To make sure we don’t take shots in the dark but rather deliver on their request, we asked Coach for their webshop analytics data.

Analytics data

When it comes to optimization, they understood that it’s good to start with data in the first place. They have provided us with analytics data of 1 week, having a total of 866130 visitors! Luckily that meant we got enough statistical power, even breaking it down into different landing pages would not be a problem. They also added that the percentages of conversions stay roughly constant with slight fluctuations in visitor rates. All these details matter when it comes down to how much we can rely on the data itself.

The bad apple

Going through the data, we started looking at the conversion rates based on landing pages. There was one landing page in particular that performed terribly. It was the landing page of the Dreamer handbag. 

The conversion rate itself that resulted from this landing page was 0.05%. Now you might be thinking that this type of conversion rate is not so bad, especially when you compare it to average search statistics across the web.

Let me add two more details:

  1. Other landing pages had on average 1.5% conversion rate
  2. 10% of the total traffic came in through this landing page

That basically means a lot of potential conversions are lost and advertising money is going down the drain. The next questions is, ‘why is the conversion rate so low compared to other pages?’.

Why?

The first, most obvious reason for this conversion rate could be the product itself. But is it? 

To find an answer to that question, we looked again at the analytics data. The answer was quite clear: the product is not the problem!

The Dreamer handbag itself was responsible for 7% of the total revenue. That is massive for a single product, given that other popular products on the Coach webshop accounted for 7.84% of revenue, combined.

Then what is it on that specific landing page?

As you most likely found on multiple other online articles, now it would be time to go through the list of conversion rate optimisation tips. Oh and don’t forget to finish your guessing with an A/B test.

Luckily, we don’t have to guess blindly. 

Testing

During our test, we exposed 100 testers to the Coach landing page of the Dreamer handbag. While the testers were viewing the landing page, we recorded their eye movements, their emotions and their hear rate through their webcam. We only showed the page for 10 seconds to really get the initial implicit reactions and viewing patterns.

This makes it much easier to understand what people are looking at and also how they process / react to it emotionally. From this, we can infer the ‘why’ much easier than guessing blindly.

Plus, we had all the data collected and analysed within 3 days. Generally quicker than preparing for all the A/B tests.

Results

From the raw data, we create easy to interpret visualisations. For the eye tracking we get a heat map. Red = most attention, blue = least attention, no color = no attention received.

From this is it easy to see that the image within the webshop and the slogan on top receive the most attention.

Now that we know what received attention, let’s have a look at how that was processed. For that we combine all the emotions we measure (7 in total, you can have a look here) into an easy interpretable Valence score, which goes from -100 to +100. Negative score means negative emotion, positive score means positive emotion.

The Average Valence score for this landing page was -0.7. You might be thinking that is not so bad considering the range of the score. But you need to keep in mind that this is just a webshop that people are looking at and nothing controversial, which would much easier stir up emotions.

This is already quite telling but let’s dive in 1 step further. We combined the heat map and the Valence score into a Valence map. This makes it simple to see where exactly what type of emotion was present. This way we know what exactly people felt when they looked at something.

Conclusion and impact

From the Valence map you can easily see that the strongest negative peak is caused by the the slogan on the landing page. Looking back at the timeline of the Valence score, that peak comes at ~6 seconds. The image on the left of the products doesn’t get a lot of positive reactions either.

After we saw this reaction to the image, we looked at the other landing pages and only then we noticed that they don’t have an image nor a slogan.

There are now 2 potential routes you can go:

  1. Get rid of the image and the slogan
  2. Tweak image and slogan

If you’d go down route 1, then with a fair amount of confidence, given the analytics data of other landing pages, the product itself, and the data we collected, the conversion rate should go up. But how much?

It should get close to the average of the other landing pages (1.5%), but let’s be conservative and say it will only go up to 0.8%.

The Dreamer handbag costs ~ $700, with a conversion rate of 0.05% and ~93800 visitors that means around $33k. Coach is a strong brand but they do use online advertising. Because they are such a strong brand they could potentially pay only $0.1 per visitor coming to their website via advertising. That means then $9380 in online advertising costs. That results in revenue minus advertising costs = $23620.

Redoing this with a conversion rate of 0.08% would result in the same costs for advertising but final revenue minus advertising costs = ~ $43k. Oh and remember that this is per week!

As you can see, the impact itself is quite strong and we managed to deliver the biggest bang for their buck.

mindtrace-toolbox-schema

Setup the MindTrace toolbox

If you haven’t already done so, you can download the toolbox with the button above. It’s also important to install all necessary software for it first. You can do that by checking this post (how to install the toolbox).

If you did all of that already then great job on installing everything that is necessary for the toolbox to work. The last part you have to do is to adjust the MT_server.py file. That file is the heart of the toolbox. You need to adjust it so it works with the webcam you want to use.

The most important part that you need to adjust, is to select the webcam you want to use. You can also adjust the resolution and the fps (frames per second), which are set to 1280×720 pixel and 30 fps by default. The resolution and fps settings should work right out of the box. But you can adjust them to your liking. Just make sure all works after you adjust them.

Selecting your webcam

These steps are slightly different from operating system to operating system. But don’t worry, we will take you through it step by step. Within the Python subfolder you can find 2 important scripts that are helping you out:

MT_system_devices.py – This script tells you which devices are available in your system.

MT_device_formats.py – This script helps you to figure out the supported formats of that device. If you want to tinker with it.

You can adjust the webcam, resolution, and the fps (framerate), right at the top of the MT_server.py script.

The default settings for Mac and Linux use the standard integrated webcam. For Windows you have to check what the name of your webcam is (see below). Next we show you how to find out which webcam to use if you want to use another one.

Mac


Open the terminal application on your Mac and go to the correct folder that contains the script. As it shows on the image below, simply type in:

python3 MT_system_devices.py

On the example system it was python3 but on yours it could be python. It all depends on how you’ve set it up. You will now get a list of accessible devices on your system. In this example we are going to use the FaceTime HD Camera, which has the index 0. This is the default setting of MT_server.py. If you want to use the FaceTime Camera, you don’t have to adjust anything. If you want to use another webcam in your list, assign the mac_webcam the index you want to use.

The small issue with Mac is that it does not give you any supported formats of the selected device. So if you use the MT_device_formats.py script like this:

python3 MT_device_formats.py 0

The ‘0’ at the end is the device index. In the end it doesn’t matter though, because you’ll get the following:

Which means you have to check your system settings or if you use an external webcam you have to check its specifications.

Run a quick command routine

Now that you have everything setup and selected the device you want to use, you can run a quick command routine. This is also a script that is available in the Python subfolder and is called MT_connection_example.py. This script does the following:

  1. connects to the MT_server
  2. takes a snapshot through your webcam
  3. starts recording through your webcam
  4. sends information about a calibration point (x, y, duration)
  5. sends information about a fixation cross (x, y, duration)
  6. stops recording
  7. closes the connection to MT_server

You can see an example of this execution in the video below. To do it on your system, open two terminal windows and make sure you are in the correct folder. In one terminal start the MT_server by executing:

python3 MT_server.py

In the other terminal start MT_connection_example by executing:

python3 MT_connection_example.py

You can then find all the recorded data in a subfolder within the Python subfolder.

Linux


Open the terminal and go to the correct folder that contains the script. As it shows on the image below, simply type in:

python3 MT_system_devices.py

On the example system it was python3 but on yours it could be python. It all depends on how you’ve set it up. You will now get a list of accessible devices on your system. In this example we get only the internal webcam to show, which is already the default setting. If you want to use another webcam in your list, assign the linux_webcam the /dev/video# you want to use.

In the line right after, you can see that we want to find out what formats are supported by that device. You can use the MT_device_formats.py script like this:

python3 MT_device_formats.py /dev/video0

Run a quick command routine

Now that you have everything setup and selected the device you want to use, you can run a quick command routine. This is also a script that is available in the Python subfolder and is called MT_connection_example.py. This script does the following:

  1. connects to the MT_server
  2. takes a snapshot through your webcam
  3. starts recording through your webcam
  4. sends information about a calibration point (x, y, duration)
  5. sends information about a fixation cross (x, y, duration)
  6. stops recording
  7. closes the connection to MT_server

You can see an example of this execution in the video below. The video shows it in a Mac terminal but the process is the same. To do it on your system, open two terminal windows and make sure you are in the correct folder. In one terminal start the MT_server by executing:

python3 MT_server.py

In the other terminal start MT_connection_example by executing:

python3 MT_connection_example.py

You can then find all the recorded data in a subfolder within the Python subfolder.

Windows


Open the command line (cmd) and go to the correct folder that contains the script. As it shows on the image below, simply type in:

python MT_system_devices.py

On the example system it was python but on yours it could be python3. It all depends on how you’ve set it up. You will now get a list of accessible devices on your system. In this example we get only the internal webcam to show, which is already the default setting. If you want to use another webcam in your list, assign the windows_webcam the “webcam name” you want to use.

Now you can find out what formats are supported by that device. You can use the MT_device_formats.py script like this:

python3 MT_device_formats.py "webcam name"

Run a quick command routine

Now that you have everything setup and selected the device you want to use, you can run a quick command routine. This is also a script that is available in the Python subfolder and is called MT_connection_example.py. This script does the following:

  1. connects to the MT_server
  2. takes a snapshot through your webcam
  3. starts recording through your webcam
  4. sends information about a calibration point (x, y, duration)
  5. sends information about a fixation cross (x, y, duration)
  6. stops recording
  7. closes the connection to MT_server

You can see an example of this execution in the video below. To do it on your system, open two command line windows and make sure you are in the correct folder. In one command line start the MT_server by executing:

python MT_server.py

In the other command line start MT_connection_example by executing:

python MT_connection_example.py

You can then find all the recorded data in a subfolder within the Python subfolder.

How to install the MindTrace toolbox

The MindTrace toolbox functions as an interface between your experiment code and the webcam. After connecting to the toolbox from within your experiment, you can easily send commands to the toolbox in order to e.g. start recording via the webcam or take snapshots. Sending the appropriate commands for calibration points can also be easily communicated to the toolbox. Additionally, the toolbox ensures that all the data is stored in an appropriate format, so that it can be uploaded seamlessly into our system and analyzed immediately.

mindtrace-toolbox-schema

What do you need on your computer to make use of the MindTrace Toolbox?

First of all, you need Python 3 (up to version 3.6.5). Normally, Python comes pre-installed with all unix systems (Mac OS and Linux). However, you need to make sure that you are indeed using Python 3 (up to version 3.6.5). You can easily check your python version by pasting this line in your terminal:

python -V

If you are already running Python 3, there is no need to install it again. If you execute the commands below in a Mac or Linux environment, the installation command for Python 3 will be ignored, if Python 3 is already installed on your system.

Mac


Mac OS in general has Python 2 pre-installed. Because our Toolbox requires Python 3 and a few additional packages for Python itself and your system, we want to make sure that everything is installed properly. In order to do so, you need to install Homebrew, which is a package manager for Mac OS. Paste the following line into your terminal to install Homebrew:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Now that we have Homebrew installed, we can continue to install Python 3, ffmpeg (package for handling multimedia data), pip (python package manager), and numpy (scientific computing package).

brew install python3 ffmpeg
sudo easy_install pip3
pip3 install numpy

After you’ve installed everything, you can check here how to set the toolbox up to use the device that you want to use – Setup the toolbox

Linux


Linux already has its own package manager. You can directly continue to install Python 3, ffmpeg (package for handling multimedia data) , pip (python package manager), and numpy (scientific computing package) by executing the following commands:

sudo apt-get install python3 ffmpeg
sudo easy_install pip3
pip3 install numpy

After you’ve installed everything, you can check here how to set the toolbox up to use the device that you want to use – Setup the toolbox

Windows


You can download Python 3 via the following URL: https://www.python.org/downloads/. Select Python 3 up to version 3.6.5! Make sure that during the installation you select to add the Python install to your path so that you can easily run commands via the terminal. If the installation process doesn’t offer you that option it is most likely doing it automatically.

You can download ffmpeg via the following URL: https://ffmpeg.zeranoe.com/builds/. Unzip ffmpeg directly to your C:\ drive so that the resulting path and folder is C:\ffmpeg. Finally, you need to add the ffmpeg.exe to your environment variables. That will make it easy to call ffmpeg from within the terminal.

In order to do that, open the Start menu and right click on “Computer” and then click on “Properties”.

add ffmpeg windows path step 1

Select “Advanced system settings”:

add ffmpeg windows path step 2

Click on “Environment variables”:

add ffmpeg windows path step 3

Edit the Path variable:

add ffmpeg windows path step 4

Add C:\ffmpeg\bin at the end. Make sure that this path is separated by a semi-colon (;) from the previous folder.

The package manager pip is generally installed with Python directly, so no need to install it separately. The last step is to now install numpy through the command line:

pip install numpy

After you’ve installed everything, you can check here how to set the toolbox up to use the device that you want to use – Setup the toolbox