An Open Source IAT for Online Data Collection

Winter Mason

Stevens Institute of Technology

Implicit Association Test (IAT)

The Implicit Association Test (IAT; Greenwald, McGhee, & Schwartz, 1998) was devised as a way to measure implicit attitudes towards categories of objects or people.

Since then it has been used extensively in the field (McConnell & Leibold, 2001; Conrey, et al., 2003; Nosek, 2005, 2007)

Including online applications (c.f., Project Implicit: http://projectimplicit.com).

However, until now, the means to run the IAT has required commercial software such as e-Prime for laboratory studies and has not been widely and freely available for online research.

Open Source software

Open-source software is software that has been developed by and/or shared with a community of developersFirefox, Linux, R, etc., etc.

Based on the idea that sharing resources reduces unnecessary duplication of effort and increases social efficiency

This idea should be applied to all aspects of scienceSharing methods and materials reduces duplication of

effort and facilitates replicationSharing data allows transparency and reproducibilityAlso see openscienceframework.org

Requirements

Experimenter: A web server that runs

PHP v4.0 or greater

Participant: A web browser that

allows Javascript

Has been successfully tested with Mac OS 10.5, 10.6, Windows XP, Vista Ubuntu 11.

It has also been successfully tested with Mozilla’s Firefox Google Chrome Internet Explorer 8 & 9.

Experimenter View

A list of available IAT templates to modify or make active

The currently active IAT; Participants will complete this version when they go to the main IAT page.

Edit the name of the template

Choose whether participants are presented with the results of their IAT at the end of the test.

Select the category you want to edit

Select the category you want to edit

Select the category you want to edit

This is the name of the category seen by the participants.

This is the name of the category that is stored in the output data file.

Choose whether the items in the category will be text or images.

With images, you can see a preview of the image next to the file name.

Change the file name here…

…and update the thumbnail image with this button.

With text items, the thumbnails and “update images” button are hidden.

Delete an item with the ‘x’ button

Once you have made all of your changes, don’t forget to save!

If you select another template or create a new template, unsaved changed will be lost!

To change the IAT participants see, select the template and press this button.

Create a new template with this button.

Make sure you’ve added items for all 4 categories, and don’t forget to save!

Participant View

Enter a unique identifier for the participant (a random string is pre-filled). The output data file will be associated with this ID. You can also use this to match with a Turker’s submitted HIT.

The name of the active template is shown here.

The category labels for Category A and B are shown in the first trial

The templates are stored in a folder with the same name as the template, and image files are stored in the “img” folder inside the corresponding template folder.

The category labels for Category 1 and 2 are shown in the second trial

If the participant gets it wrong, the X appears until the correct key is pressed.

Output

Trial Round Category

Item Errors RT

0 1 F 4 0 560

0 2 M 3 0 432

0 3 F 0 1 913

… … … … … …

The output is stored in the “output” folder in the template’s folder as a comma-delimited file. The name of the file follows the pattern [Template Name]-[Subject ID]-YYYY-MM-DD-HH-mm. For example, Race-JohnDoe-2013-01-22-18-52.csv.

Output

Trial Round Category

Item Errors RT

0 1 F 4 0 560

0 2 M 3 0 432

0 3 F 0 1 913

… … … … … …

There are 7 Trials in the standard IAT

Output

Trial Round Category

Item Errors RT

0 1 F 4 0 560

0 2 M 3 0 432

0 3 F 0 1 913

… … … … … …

There are 20 or 40 round per trial, depending on whether it is a practice trial or not

Output

Trial Round Category

Item Errors RT

0 1 F 4 0 560

0 2 M 3 0 432

0 3 F 0 1 913

… … … … … …

The character(s) used here to indicate the category are defined in the template.

Output

Trial Round Category

Item Errors RT

0 1 F 4 0 560

0 2 M 3 0 432

0 3 F 0 1 913

… … … … … …

The item number refers to which particular item was presented. The order of item presentation is randomized. The number indicates its order in the template (starting from zero).

Output

Trial Round Category

Item Errors RT

0 1 F 4 0 560

0 2 M 3 0 432

0 3 F 0 1 913

… … … … … …

This is the number of errors the participant made on that round.

Output

Trial Round Category

Item Errors RT

0 1 F 4 0 560

0 2 M 3 0 432

0 3 F 0 1 913

… … … … … …

This is the participant’s response time in milliseconds for that item on that round. The final IAT score is determined by these reaction times.

Validation

The standard deviation of system error was 29.4ms, with fewer than 4% larger than 50ms and the largest error recorded at 167ms.

The typical standard deviation of a user's reaction time is approximately 300ms, as reported in Greenwald, et. al (1998, 2003) and observed in the replications reported here.

The error introduced by the system is an order of magnitude less than user error.

Replicating prior work

63 participants recruited from Amazon’s Mechanical Turk

46.7% female

median age of 29 (mean age 31.8)

median household annual income less than $30k

median education of a Bachelor’s Degree

55.5% from India, 38% from the U.S., and the remainder were from Pakistan, Lithuania, Algeria, and Sweden.

Flowers-Insects IATCongruent first

(Flowers-Good / Insects-Bad)

Incongruent first(Flowers-Bad / Insects-Good)

Results

IATCongruent (ms)

Incongruent (ms)

t-value p d'

Flowers-Insects (Congruent first) 727.52 1003.12 26.342

~0.00

0.65

Flowers-Insects (Incongruent

first)750.88 850.85 10.384

1~0.0

00.26

Race 771.1 903.8812.862

1~0.0

00.32

Future extensions

Allow easy duplication of templates in experimenter interface

Data storage in a database instead of text files

Automatic aggregation and display of results in experimenter interface

Suggestions? Requests?

Thank You!http://github.com/winteram/IAT