Research Methods in Cognitive Science

class: center, middle, inverse, title-slide

# Research Methods in Cognitive Science
## Week 10: The Onlineification of Cognitive Science
### Jason Geller, Ph.D.
### 2021-11-02

---

# Housekeeping

- No class on Thursday (I am at a virtual conference)

- One more guest lecture (11/11)
  
- Extra credit

- Participate in experiments
  
  - Record a 3-5 minute video (YouTube, TikTok, etc.) on method we have not discussed
  
- Late assignments

- Turn them in when you can (full points)

---
# Outline

- Introduction to online research

- What are the ingredients to doing research online?

- Task builders/experiment builders
  - Hosting/Servers
  - Recruitment

- Frequently asked questions about online research

- Future of online research

---
# Business as Usual

<img src="business.png" width="100%" style="display: block; margin: auto;" />
        
---

background-image: url(covid.png)
background-position: center
background-size: cover

---
background-image: url(onlinemem.png)
background-position: center
background-size: cover

---
background-image: url(bob.png)
background-position: center
background-size: cover
---
background-image: url(booming.png)
background-position: center
background-size: cover
???

The new normal is running experiments online. This will result in folks participating in your experiments from a variety of different places using a variety of different equipment (computers, browsers, headphones, etc). 
---
# Advantages Of Online Data Collection?

- Efficiency

- Faster data collection

- Economical

- Reduced research and equipment costs
  - No one has to be in a physical lab
  
--

- Diversity

- Non-WEIRD samples
--

- Greater power

- More data

???

There are several reasons why you may opt to run experiments online compared to in-lab

1. Efficiency – You can collect many Ps in a matter of horus
2. Economical – No more time sending out recruitment emails scheduling and testing participants. It can also be cheaper!

3. 
It might shock you to know but the undergrads we relie are are
---
# Disadvantages

---
# Disadvantages Of Online Data Collection?

- Data quality

- No person to answer questions

- No experimental control
--

- Task-switching students :D
--

- Money

---

# Million Dollar Question

- How can we take the experiments we are conducting in the lab and adapt them to run online?
---
# Online Recipe

<br>
.pull-left[

]
.pull-right[
1. Experiment/task builders
  - How can I build online experiments that are equivalent to in-lab studies?

2. Hosting
  - How do I get my experiment online?

3. Online recruitment marketplace
  - How do I get Ps?

]
---
# Experiment/Task Builders

- In the lab, we might use a number of different tools to build our experiments

- Experiment Builder (SR)
  - E-Prime
  - Psychtoolbox (Matlab)
  - PsychoPy (Python)

Most of these tools are not portable

---
# Experiment/Task Builders

.pull-left[

<br>
<br>
<br>
- Experiments on-line must be run through a 
web browser

- Gorilla
  - PsychoPy/psychoJS/JavaScript
  - Qualtrics
]

.pull-right[

<img src="onlineplatforms.png" width="100%" style="display: block; margin: auto;" />
]

---
# And Many More...

---
# Gorilla

<img src="gorilla.png" width="20%" style="display: block; margin: auto;" />
---
# PsychoPy

<img src="psychopy3.png" width="40%" style="display: block; margin: auto;" />
---
# Install PsychoPy

- www.psychopy.org

---
# Pavlovia

- Create a Pavlovia account: https://pavlovia.org/

---
# Qualtrics

<img src="qual.jpeg" width="30%" style="display: block; margin: auto;" />
---
class: inverse center middle
# Hosting
---
# Hosting

- Host the source code for your experiment yourself
  - Perform administration, data backup, and security
  - NO ONE HAS TIME FOR THAT!
- Use third-party hosting service
  - Mindprobe 
      - OpenSeasme
      - Free
  - Pavlovia/GitLab
      - PsychoPy
           - .20 per P; 1,500 university license 
  - Microsoft Azure
     - Gorilla
          - 1.18 per P and much more for team/department license
---

class: inverse center middle
# Recruitment and Payment

---

# Recruitment and Payment

> Recruitment and payment platforms do two important things:

1. Get the studies distributed to participants
2. Handle the money/credit

---
<img src="recruitplat.png" width="90%" style="display: block; margin: auto;" />
---

# Online Flow

---
# FAQ

> Q: How good are the data?

A: Some evidence to suggest that you can get good data (e.g., Crump et al., 2013; Zwann et al., 2017)

---

# Zwann et al. (2018)

.pull-left[
- Simon task
- Flanker task
- Motor priming
- Spacing effect
- False memories
- Serial position
- Repetition priming
- Shape simulation
]

.pull-right[

<img src="zwaan.png" width="90%" style="display: block; margin: auto;" />
]

---

# Slote and Strand (2016)

---
# FAQ

> Q: How good are the data?

A: Some evidence to suggest that you can get good data (e.g., Crump et al., 2013; Zwann et al., 2017)

- **Keep experiment short (ideally 20 minutes or less) and entertaining**
---

# FAQ

> Q: How good is the timing?

A: Timing is reasonable (I would probably not present anything for less than 50 ms)

---
# Anwyl-Irvine et al. (2020)
<img src="Picture1.png" width="90%" style="display: block; margin: auto;" />
---
# Anwyl-Irvine et al. (2020)

<img src="Picture2.png" width="90%" style="display: block; margin: auto;" />
---
# Anwyl-Irvine et al. (2020)

---
<iframe src="https://psychopy.org/timing/2020/table3.html" width="90%" height="400px" data-external="1"></iframe>
---

# Group Discussion

- Are there any areas where you think online data collection can be extremely useful? How about impossible or hard?
---
# Future of Online Research

- Online Eye-tracking

- https://www.youtube.com/watch?v=NRLlRh2apA8&feature=youtu.be

---
# Eye-tracking Demo

- https://farm.pcibex.net/r/hQzkGV/

---
# Future of Online Research

- Online eye-tracking

- MouseView

- https://www.youtube.com/watch?v=ozxSla0eNh8

---
# For Next Time

- Assignment: Online Participant For A Day

---