Hello!

Logistics

We will meet twice a week

Location

Busch SEC-209

Lecture

Tuesdays and Thursdays, 3:00 P.M.– 4:20 P.M.

Office

RuCCS A121

Research Methods in
Cognitive Science


The aim for this course is to provide students with the necessary foundation to think critically about research in cognitive science. The course reviews the scientific method and considers the strengths and weaknesses of a range of approaches, such as laboratory experimentation, neuroscience methods, and online data collection. Students will be introduced to statistical reasoning in science, including a basic overview of common statistical techniques. We will also discuss principles for the ethical conduct of research both in the laboratory and online. This course will include traditional lectures, in-class activities, and special presentations by cognitive scientists at Rutgers and other universities in the field. Students will also get hands-on experience programming and collecting their own data, which will culminate in a paper and presentation. This course counts for 3 credits.

People

Instructor

Avatar

Dr. Jason Geller

Research Scientist

Avatar

Dr. John McGann

RuCCS Chair

Schedule

I expect students to have read the assigned readings before class. This does not mean just skimming reading, but engaging critically with the scholarship. In particular, look for passages that you disagree with or that seem unclear to you, as these are likely ones that could benefit from further discussion in class. I recommend that, if at all possible, students find a way to mark up the articles or books as they are read to improve reading comprehension.

This schedule represents my current plans and objectives. As we go through the semester, these plans (including exact assignment due dates) may need to change to enhance learning opportunities for the class. (e.g. due to an extension of our discussion of specific topics that arise during lectures). Such changes, communicated clearly, are not unusual and should be expected.

Week Day Date Topic Guest Speaker Readings Due
1 TH 2-Sep Introduction & Course Expectations
2 T 7-Sep Fundamentals of Science and How We Study the Brain CH.1 JHANGIANI et al.
CH.2 JHANGIANI et al.
 September 10th is last day to drop a course without a "W" grade
2 TH 9-Sep Research Ethics CH.3 JHANGIANI et al.

Team Charter Due by Friday, September 10th
3 T 14-Sep Sources of Information Laura Mullen (Rutgers Behavioral Sciences Librarian)
3 TH 16-Sep Ethics Training: CITI TRAINING Topic Approval Due by Friday September 17th
4 T 21-Sep Open Science/Replication - Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632
  (watch the summary video here: https://www.youtube.com/watch?v=bf3GqyBRgzY&t=562s)

- Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600. https://doi.org/10.1073/pnas.1708274114
4 Th 23-Sep Open Science/Replication - Brandt, M. J., IJzerman, H., Dijksterhuis, A., Farach, F. J., Geller, J., Giner-Sorolla, R., Grange, J. A., Perugini, M., Spies, J. R., & van ’t Veer, A. (2014). The Replication Recipe: What makes for a convincing replication? Journal of Experimental Social Psychology, 50, 217–224 https://doi.org/10.1016/j.jesp.2013.10.005l
- Frank, M. C., & Saxe, R. (2012). Teaching Replication.Perspectives on Psychological Science,7(6),600–604.https://doi.org/10.1177/1745691612460686		 Article Summary and CITI training Due By Friday September 24th
5 T 28-Sep Psychological Measurement CH.4 JHANGIANI et al.
5 Th 30-Sep Experiment Basics CH.5 JHANGIANI et al.
6 T 5-Oct Exploring Cognition Through the Eyes: Eye-tracking (Teams 1 and 2) RuCCS eye-tracking lab A120
6 TH 7-Oct Exploring Cognition Through the Eyes: Pupillometry Sarah Colby (University of Iowa): Pupillometry - Torres A and Hout M (2019) Pupils: A Window Into the Mind. Front. Young Minds. 7:3. doi: 10.3389/frym.2019.00003
- Laeng, B., Sirois, S., & Gredebäck, G. (2012). Pupillometry: A Window to the Preconscious? Perspectives on Psychological Science, 7(1), 18–27. https://doi.org/10.1177/1745691611427305		 Preregistration, Presentation Reflection, and Consent Form due by Friday October 8th
7 T 12-Oct Exploring Cognition Through the Eyes: Eye-tracking (Teams 3, 4, and 5) RuCCS eye-tracking lab A120
7 TH 14-Oct Exploring Cognition Through the Brain: EEG McCall Syrett (Villanova University): EEG - Allopenna, P. D. , Magnuson, J. S. , & Tanenhaus, M. K. (1998). Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models. Journal of Memory and Language, 38, 419–439.

Measuring Brain Waves in the Classroom
. (n.d.). Frontiers for Young Minds. Retrieved October 2, 2021, from
https://kids.frontiersin.org/articles/10.3389/frym.2020.00096		 Presentation Reflection due by Friday October 15th
8 T 19-Oct Exploring Cognition Through the Brain : EEG (Teams 1, 2) RuCCS eye-tracking lab A120 - Etienne, A., Laroia, T., Weigle, H., Afelin, A., Kelly, S. K., Krishnan, A., & Grover, P. (2020). Novel Electrodes for Reliable EEG Recordings on Coarse and Curly Hair. BioRxiv, 2020.02.26.965202. https://doi.org/10.1101/2020.02.26.965202
8 TH 21-Oct Exploring Cognition Through the Brain: EEG (Teams 3, 4,5) RuCCS eye-tracking lab A120
9 T 26-Oct Understanding Cognition Through the Brain: fMRI Melissa Thye (University of Edinburgh): fMRI/sEEG - Racine, E., Bar-Ilan, O., & Illes, J. (2005). FMRI in the public eye. Nature Reviews Neuroscience, 6(2), 159–164. https://doi.org/10.1038/nrn1609
- Turner, R. (2016). Uses, misuses, new uses and fundamental limitations of magnetic resonance imaging in cognitive science. Philosophical Transactions of the Royal Society B: Biological Sciences, 371(1705). https://doi.org/10.1098/rstb.2015.0349- Amanamba U., Sojka A., Harris S., Bucknam M.,& Hegdé. J. (2020) A Window Into Your Brain: How fMRI Helps Us Understand What Is Going on Inside Our Heads. Front. Young Minds. 8:484603. doi: 10.3389/frym.2020.484603		 Presentation Reflection Due by Friday October 29th
9 TH 28-Oct Understanding Cognition Through the Brain: fMRI (The Center for Advanced Human Brain Imaging Research (CAHBIR))
10 T 2-Nov Exploring Cognition Through the Web - Crump, McDonnell, Gureckis (2013). Evaluating Amazon's Mechanical Turk as a Tool for Experimental Behavioral Research
- Bridges, D., Pitiot, A., MacAskill, M. R., & Peirce, J. W. (2020). The timing mega-study: Comparing a range of experiment generators, both lab-based and online. PeerJ, 8, e9414. https://doi.org/10.7717/peerj.9414- - Grootswagers, T. A primer on running human behavioral experiments online. Behav Res 52, 2283–2286 (2020).https://doi.org/10.3758/s13428-020-01395-3
- Gagné, N., & Franzen, L., Ph.D. (2021, August 30). How to run behavioural experiments online: best practice suggestions for cognitive psychology and neuroscience. https://doi.org/10.31234/osf.io/nt67j
10 TH 4-Nov No Class - Psychonomics
11 T 9-Nov Introduction to Python/PsychoPy and Pavlovia
11 TH 11-Nov Lucia Cherep (Univeristy of Arizona): VR
12 T 16-Nov Introduction to Python/PsychoPy and Pavlovia & Normalizing Errors
12 TH 18-Nov Programming Experiments
13 T 23-Nov Programming Experiments
13 TH 25-Nov Thanksgiving Break (No Class)
14 T 30-Nov Data Collection Programmed Experiment Due by November 30th.
14 TH 2-Dec Data Collection
15 T 7-Dec Data Analysis
15 TH 9-Dec Data Analysis Methods and Data Analysis Write-Up Due by December 10th
16 T 14-Dec Class Presentations
16 TH 16-Dec Class Presentations
12/20/2021 - 12/23/2021 FINAL EXAMS (DECEMBER 20th-DECEMBER 23rd) Paper Due by December 23rd

Class

01-Week

Course Expectations
Last updated on Jan 19, 2022

02-Week

Fundamentals of science
Last updated on Jan 19, 2022

02-Week

Ethics in research
Last updated on Jan 19, 2022

04-Week

Replication
Last updated on Jan 19, 2022

04-Week

Open science and replication issues
Last updated on Jan 19, 2022

05-Week

Discussion of measurement principles
Last updated on Jan 19, 2022

05-Week

Experimental basics and threats to internal validity
Last updated on Jan 19, 2022

06-Week

This week we did an eye-tracking demo and had Dr. Sarah Colby from UIowa give a guest lecture
Last updated on Jan 19, 2022

07-Week

This week we discussed EEG and had a guest lecture by McCall Sarrett from Villanova.
Last updated on Jan 19, 2022

08-Week

This week I gave a asynchronous introduction to EEG. In additon, I had teams come to the EEG lab and observe how to conduct EEG experiments. Teams also saw what different artifacts look like.
Last updated on Jan 19, 2022

09-Week

This week is all about fMRI. Melissa Thye from The University of Edinbrough is giving a talk on fMRI and how she uses it in her research. We also vist Rutgers new neruoimaging center.
Last updated on Jan 19, 2022

10-Week

This week we talked about the basics of online data collection. What are the ingredients to doing research online? Can cognitive scientists do their research online? What are some of the most frequently asked questions about online research?
Last updated on Jan 19, 2022

11-Week

This week I talk about the basics of PsychoPy and Pavlovia to help prepare groups for working on their projects.
Last updated on Jan 19, 2022

Project

Final Project

I have designed the course to involve a replication study within your area of interest. The replication studies should take some preexisting manipulation and aim to replicate it on a smaller, more manageable scale. There are three primary pedagogical reasons for structuring the final projects in this way:

  1. Replications provide an entry point into research: It is difficult to develop an original research idea, especially when you’re just starting to learn about an area. The problem is only exacerbated by the expansiveness of cognitive science. So, the idea is that by trying to delve into an area where the basic questions have been mapped out already will help ease you into doing the actual research.
  2. Replications are practical exercises: The replication studies provide a way for us to catch problems or issues before the final project. I hope that these studies will familiarize you with some practical elements of basic research.
  3. Replications are part of the scientific process: Much actual research aims to replicate previous claims, either in whole or in part. Ideally, science consists of replicable results. Unfortunately, we sometimes find that the results are due to properties of the stimuli, to a particular subject sample, or to poor experimental design. Thus, replications are integral to the scientific process, helping us discriminate fact from fiction.

The replication studies are group projects (unless you strongly feel the need to work alone).

  1. Form teams: between 2 and 4 people per group (I will form groups based on survey)
  2. Complete team charter
  3. Find candidates for replication
  4. Meet with me in groups to discuss projects: I’ll make times to meet with each group.
  5. Write a short 1-page research proposal
  6. Complete IRB forms
  7. Preregister Study: Draft a preregistration of your replication project, using a suitable template provided at the Open Science Framework, https://osf.io/zab38/wiki/home/?view. Consult preregistrations for previous studies: https://osf.io/e6auq/wiki/Example%20Preregistrations/?view. Provide feedback to the preregistration of another course participant. Go over the checklist in Aguinis, Hill & Bailey (2021): does the plan contain reasonable choices that are detailed enough? Improve your own preregistration after you have received feedback from one of the other course participants. Register the improved version on the Open Science Framework, https://osf.io/registries/osf/new.
  8. Design replication study: try to use similar types of items, but you may need to create your own
  9. Program replication study
  10. Run replication online (Reddit/SampleSize, Facebook, Twitter, family, friends)
  11. Present results to class
  12. Write-up project: Focus the report on analytical and methodological decisions and their consequences. Do not include an extensive theory section in the report. The replication report consists of four parts:
    1. Study Rationale: a brief description of the study that you are attempting to replicate, and reasons why it is important to do so. Draw upon a revised version of assignment 2 for this section.
    2. Data and Methods: an elaborate description of the data and the methods you used in the replication. Update the preregistration with the choices you made eventually, and explain the discrepancies (if any) between the pre-registered plans and the execution in practice in an appendix. Include a link to the replication package on the Open Science Framework.
    3. Results: a clear description of the results of the study, including robustness analyses and extensions (if any). For the assessment it does not matter whether the results of the replication are consistent with the original study or not.
    4. Discussion: discuss the limitations of your study. Go over the differences between the data and methods for the original and the replication: at which points did you make different decisions than the original study? Use the checklist here https://osf.io/zwkqj/. Compare the results of the replication with the results of the original study, and try to explain them from differences in the data and methods. What are the implications of the replication with respect to the generalizability of the original findings, and the robustness of the findings with respect to alternative decisions in the research design, data collection, data pre-processing, and analysis? Limit the report to 4,000 words. Insert the Tables and Figures in the text.

Resources

CITI Training

CITI1

CITI2

Please send me the certificate

IRB Documents

Download the documents here: https://research.rutgers.edu/researcher-support/research-compliance/human-subjects-protection-program-toolkit

You will need to locate:

  • Online Survey/Questionnaire Consent Form
  • (HRP-503b) Non-Interventional Research Protocol

Go here for example documents: https://osf.io/ncxwv/.

PsychoPY