Posts

mindtrace-toolbox-schema

Setup the MindTrace toolbox

If you haven’t already done so, you can download the toolbox with the button above. It’s also important to install all necessary software for it first. You can do that by checking this post (how to install the toolbox).

If you did all of that already then great job on installing everything that is necessary for the toolbox to work. The last part you have to do is to adjust the MT_server.py file. That file is the heart of the toolbox. You need to adjust it so it works with the webcam you want to use.

The most important part that you need to adjust, is to select the webcam you want to use. You can also adjust the resolution and the fps (frames per second), which are set to 1280×720 pixel and 30 fps by default. The resolution and fps settings should work right out of the box. But you can adjust them to your liking. Just make sure all works after you adjust them.

Selecting your webcam

These steps are slightly different from operating system to operating system. But don’t worry, we will take you through it step by step. Within the Python subfolder you can find 2 important scripts that are helping you out:

MT_system_devices.py – This script tells you which devices are available in your system.

MT_device_formats.py – This script helps you to figure out the supported formats of that device. If you want to tinker with it.

You can adjust the webcam, resolution, and the fps (framerate), right at the top of the MT_server.py script.

The default settings for Mac and Linux use the standard integrated webcam. For Windows you have to check what the name of your webcam is (see below). Next we show you how to find out which webcam to use if you want to use another one.

Mac


Open the terminal application on your Mac and go to the correct folder that contains the script. As it shows on the image below, simply type in:

python3 MT_system_devices.py

On the example system it was python3 but on yours it could be python. It all depends on how you’ve set it up. You will now get a list of accessible devices on your system. In this example we are going to use the FaceTime HD Camera, which has the index 0. This is the default setting of MT_server.py. If you want to use the FaceTime Camera, you don’t have to adjust anything. If you want to use another webcam in your list, assign the mac_webcam the index you want to use.

The small issue with Mac is that it does not give you any supported formats of the selected device. So if you use the MT_device_formats.py script like this:

python3 MT_device_formats.py 0

The ‘0’ at the end is the device index. In the end it doesn’t matter though, because you’ll get the following:

Which means you have to check your system settings or if you use an external webcam you have to check its specifications.

Run a quick command routine

Now that you have everything setup and selected the device you want to use, you can run a quick command routine. This is also a script that is available in the Python subfolder and is called MT_connection_example.py. This script does the following:

  1. connects to the MT_server
  2. takes a snapshot through your webcam
  3. starts recording through your webcam
  4. sends information about a calibration point (x, y, duration)
  5. sends information about a fixation cross (x, y, duration)
  6. stops recording
  7. closes the connection to MT_server

You can see an example of this execution in the video below. To do it on your system, open two terminal windows and make sure you are in the correct folder. In one terminal start the MT_server by executing:

python3 MT_server.py

In the other terminal start MT_connection_example by executing:

python3 MT_connection_example.py

You can then find all the recorded data in a subfolder within the Python subfolder.

Linux


Open the terminal and go to the correct folder that contains the script. As it shows on the image below, simply type in:

python3 MT_system_devices.py

On the example system it was python3 but on yours it could be python. It all depends on how you’ve set it up. You will now get a list of accessible devices on your system. In this example we get only the internal webcam to show, which is already the default setting. If you want to use another webcam in your list, assign the linux_webcam the /dev/video# you want to use.

In the line right after, you can see that we want to find out what formats are supported by that device. You can use the MT_device_formats.py script like this:

python3 MT_device_formats.py /dev/video0

Run a quick command routine

Now that you have everything setup and selected the device you want to use, you can run a quick command routine. This is also a script that is available in the Python subfolder and is called MT_connection_example.py. This script does the following:

  1. connects to the MT_server
  2. takes a snapshot through your webcam
  3. starts recording through your webcam
  4. sends information about a calibration point (x, y, duration)
  5. sends information about a fixation cross (x, y, duration)
  6. stops recording
  7. closes the connection to MT_server

You can see an example of this execution in the video below. The video shows it in a Mac terminal but the process is the same. To do it on your system, open two terminal windows and make sure you are in the correct folder. In one terminal start the MT_server by executing:

python3 MT_server.py

In the other terminal start MT_connection_example by executing:

python3 MT_connection_example.py

You can then find all the recorded data in a subfolder within the Python subfolder.

Windows


Open the command line (cmd) and go to the correct folder that contains the script. As it shows on the image below, simply type in:

python MT_system_devices.py

On the example system it was python but on yours it could be python3. It all depends on how you’ve set it up. You will now get a list of accessible devices on your system. In this example we get only the internal webcam to show, which is already the default setting. If you want to use another webcam in your list, assign the windows_webcam the “webcam name” you want to use.

Now you can find out what formats are supported by that device. You can use the MT_device_formats.py script like this:

python3 MT_device_formats.py "webcam name"

Run a quick command routine

Now that you have everything setup and selected the device you want to use, you can run a quick command routine. This is also a script that is available in the Python subfolder and is called MT_connection_example.py. This script does the following:

  1. connects to the MT_server
  2. takes a snapshot through your webcam
  3. starts recording through your webcam
  4. sends information about a calibration point (x, y, duration)
  5. sends information about a fixation cross (x, y, duration)
  6. stops recording
  7. closes the connection to MT_server

You can see an example of this execution in the video below. To do it on your system, open two command line windows and make sure you are in the correct folder. In one command line start the MT_server by executing:

python MT_server.py

In the other command line start MT_connection_example by executing:

python MT_connection_example.py

You can then find all the recorded data in a subfolder within the Python subfolder.

How to install the MindTrace toolbox

The MindTrace toolbox functions as an interface between your experiment code and the webcam. After connecting to the toolbox from within your experiment, you can easily send commands to the toolbox in order to e.g. start recording via the webcam or take snapshots. Sending the appropriate commands for calibration points can also be easily communicated to the toolbox. Additionally, the toolbox ensures that all the data is stored in an appropriate format, so that it can be uploaded seamlessly into our system and analyzed immediately.

mindtrace-toolbox-schema

What do you need on your computer to make use of the MindTrace Toolbox?

First of all, you need Python 3 (up to version 3.6.5). Normally, Python comes pre-installed with all unix systems (Mac OS and Linux). However, you need to make sure that you are indeed using Python 3 (up to version 3.6.5). You can easily check your python version by pasting this line in your terminal:

python -V

If you are already running Python 3, there is no need to install it again. If you execute the commands below in a Mac or Linux environment, the installation command for Python 3 will be ignored, if Python 3 is already installed on your system.

Mac


Mac OS in general has Python 2 pre-installed. Because our Toolbox requires Python 3 and a few additional packages for Python itself and your system, we want to make sure that everything is installed properly. In order to do so, you need to install Homebrew, which is a package manager for Mac OS. Paste the following line into your terminal to install Homebrew:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Now that we have Homebrew installed, we can continue to install Python 3, ffmpeg (package for handling multimedia data), pip (python package manager), and numpy (scientific computing package).

brew install python3 ffmpeg
sudo easy_install pip3
pip3 install numpy

After you’ve installed everything, you can check here how to set the toolbox up to use the device that you want to use – Setup the toolbox

Linux


Linux already has its own package manager. You can directly continue to install Python 3, ffmpeg (package for handling multimedia data) , pip (python package manager), and numpy (scientific computing package) by executing the following commands:

sudo apt-get install python3 ffmpeg
sudo easy_install pip3
pip3 install numpy

After you’ve installed everything, you can check here how to set the toolbox up to use the device that you want to use – Setup the toolbox

Windows


You can download Python 3 via the following URL: https://www.python.org/downloads/. Select Python 3 up to version 3.6.5! Make sure that during the installation you select to add the Python install to your path so that you can easily run commands via the terminal. If the installation process doesn’t offer you that option it is most likely doing it automatically.

You can download ffmpeg via the following URL: https://ffmpeg.zeranoe.com/builds/. Unzip ffmpeg directly to your C:\ drive so that the resulting path and folder is C:\ffmpeg. Finally, you need to add the ffmpeg.exe to your environment variables. That will make it easy to call ffmpeg from within the terminal.

In order to do that, open the Start menu and right click on “Computer” and then click on “Properties”.

add ffmpeg windows path step 1

Select “Advanced system settings”:

add ffmpeg windows path step 2

Click on “Environment variables”:

add ffmpeg windows path step 3

Edit the Path variable:

add ffmpeg windows path step 4

Add C:\ffmpeg\bin at the end. Make sure that this path is separated by a semi-colon (;) from the previous folder.

The package manager pip is generally installed with Python directly, so no need to install it separately. The last step is to now install numpy through the command line:

pip install numpy

After you’ve installed everything, you can check here how to set the toolbox up to use the device that you want to use – Setup the toolbox