Reverse correlation image classification using R

Here I describe how to use the actively maintained (and relatively easy to use) R package rcicr that I wrote to generate reverse correlation stimuli and compute classification images.

You can download the latest version of the package on Github

Download and Installation

Install R (and RStudio to make your life easier). No worries, it’s all free. Then, to get the latest release of the reverse correlation image classification package (rcicr), run the following command in R (unfortunately it is not on CRAN at the moment):

install.packages("devtools") library(devtools) install_github("rdotsch/rcicr", ref = "development")

Generating stimuli

Load the package with:

library(rcicr)

Then generate stimuli with:

generateStimuli2IFC(base_face_files, n_trials = 770)

If you get error messages here, it’s because you haven’t assigned the base_face_files argument yet, see below. This call to generateStimuli2IFC will generate stimuli for 770 trials of a 2 images forced choice reverse correlation image classification task with sinusoid noise. By default the stimuli will have a resolution of 512 x 512 pixels. The stimuli will be saved as jpegs to a folder called stimuli in your current working directory, and an .Rdata file will be saved that contains the stimulus parameters necessary for analysis.

The base_face_files argument is a list of jpegs that should be used as base images for the stimuli. The base_face_files variable might look like this:

base_face_files <- list('male'='male.jpg', 'female'='female.jpg')

For each jpeg a set of stimuli will be created using the same noise patterns as for the other sets. Note that each jpeg should be encoded as grey scale (so not just a grey scale image saved as RGB, make sure it is actually encoded as grey scale). The jpeg should have the resolution that you want the stimuli to have. By default this should be 512 x 512 pixels. If you want a different size, resize your base image to either 128 x 128 or 256 x 256 for smaller stimuli, or 1024 x 1024 for bigger stimuli. In that case, also set the img_size parameter in the generateStimuli2IFC function accordingly.

You are now ready to collect data with the stimuli you just created. The stimuli are named according to their sequence number when generating and whether the original noise is superimposed or the negative/inverted noise. Stimuli with the same sequence number should be presented side by side in the same trial. Record which stimulus a participant selected at any given trial (the original, or the inverted). At the very least be sure that in your data file the connection can be made between the response key of the participant and which stimulus was selected on each trial. Use any presentation software you like.

Data analysis

Analyzing reverse correlation data is all about computing classification images. Use the following function for your data collected using the 2 images forced choice stimuli:

ci <- generateCI2IFC(stimuli, responses, baseimage, rdata)

The stimuli paramater should be a vector containing the sequence numbers of the stimuli that were presented in the trials of the task. The responses parameter contains, in the order of the stimuli vector, the response of a participant to those stimuli (coded 1 if the original stimulus was selected and -1 if the inverted stimulus was selected). The baseimage parameter is a string specifying which base image was used (not the file name, but the name in the list of base_face_files. So for the stimuli generated above, either ‘male’ or ‘female’, depending on which set of stimuli was presented to the participant whose data you’re analyzing). Finally, rdata is a string pointing to the .RData file that was created automatically when you generated the stimuli. It contains the parameters for each stimulus, necessary to create the classification image.

By default jpegs of the classification images will be saved automatically. The returned values can be used later to optimally rescale the noise relative to the base image. For instance, if you have a list of CIs from various participants (i.e., a list of the values returned by several calls to generateCI2IFC, one for each participant), you can use the autoscale function to generate classification images that are scaled identically and therefore straightforward to compare:

ci1 <- generateCI2IFC(…) ci2 <- generateCI2IFC(…) cis <- list(ci1name=ci1, ci2name=ci2) scaled_cis <- autoscale(cis, saveasjpegs = TRUE)

If you have multiple participants (or conditions) in your data frame for which you want to generate separate classification images, you can use a special function that automatizes everything for you. It works like this:

cis <- batchGenerateCI2IFC(dataframe, "columnToSplitDataBy", "columnContainingStimulusNumbers", "columnContainingResponses", baseimage, rdata)

where baseimage and rdata work the same as with generateCI2IFC and all other arguments tell the batch function in which columns of the dataframe to find the right data. Specifically, the second argument tells the function where it can find the index on which to split the data file by. This argument could refer to the name of the column containing subject numbers, or to the column containing conditions. It can only be one column. If you have a complicated design, where you have multiple conditions within subjects, you first want to create a column that contains the interaction between all of these factors, so you have a unique label for each combination of subjects and conditions (R has the handy interaction() function that you can use for this). The cis will be automatically autoscaled and saved as jpegs. You can change that behavior with additional arguments.

Example

If you want to see an example of a reverse correlation analysis implemented in R, check out this repository. The psychopy example includes a script for stimulus generation and analysis script in R. The psychopy script presents reverse correlation stimuli generated with rcicr. You can try it out yourself and then analyse your own data immediately after. You will need to download and install PsychoPy as well.

Running reverse correlation experiments online

Anton Gollwitzer was kind enough to share his code for conducting reverse correlation experiments online here.

Citations

If you end up using the rcicr package, the appropriate citation for the version you are using can be looked up using the following command in R:

citation('rcicr')