Combining Prolific and Gorilla for both research and teaching

Robb Rutledge is a Cognitive and Computational Neuroscientist and is currently an Assistant Professor of Psychology at Yale University. He uses Prolific and Gorilla in tandem in both his personal research and in the undergraduate classes he teaches at Yale.

The task 

In their research, Rutledge and his team build computational models that explain how humans learn and make decisions. This research poses the questions: how do people make decisions? And what determines the emotions we feel when we experience the outcomes of those decisions? 

Rutledge and his team designed a smartphone-based experiment, Happiness Quest, with multiple games to study decision making. In one game, users have a choice between two options: one safe and one risky. 

For example, one option might be a guaranteed 50 points, whereas the other is the chance to win 100 or get 0. Participants choose an option, and if they choose the risky option they find out whether they win or not. After every choice, they report how happy they feel at that moment. This data then helps develop computational models to try and predict what people might feel as a result of the outcomes of their past decisions. 

When it comes to his students, Rutledge runs a semester-long class at Yale University on the topic of computational models. In the class, he asks each student to design and create a psychological study of their choice and then collect and analyze data.

The challenge 

In the research that his lab conducts, Rutledge needs large samples of people to understand all of the differences between individuals that make us who we are. He wanted to recruit people who may struggle with in-person studies, including the elderly and people with mental health conditions. Rutledge was particularly interested in connecting with people who struggle with anxiety and depression, and who may find the lab setting stressful, meaning that participants may not answer naturally in that setting. 

On the technical side of things, Rutledge wanted to be able to quickly test new games and variations on his original studies that would allow testing of new ideas that could eventually end up in the app. One that didn’t involve messing around with coding and updates to the existing app. 

For his students, Rutledge had to make sure that anything he suggested was easy to use and rapid. This meant that experiments had to be easy to design, with no complicated coding. And data collection would need to be done in under a week. Students need to be able to run multiple experiments and have enough time for their analyses.

The solution 

Rutledge decided to use both Prolific and Gorilla in tandem. And he has continued to do so for the past 5 years! 

Using Prolific in his lab’s research, he was able to gain a large amount of data from big samples in a matter of days. The research taking place online also helped any participants who suffer with anxiety disorders feel comfortable and calm, and able to deliver more ‘true to life’ data. 

The combination of Prolific and Gorilla allowed Rutledge to find connections and conclusions that would have been harder to find in smaller in-lab samples. Mental health is highly heterogeneous, making it hard to understand. There is often some nuance to novel findings. Using these large and broad samples, Rutledge was able to iterate his experiments quickly. 

Rutledge’s lab still does lab-based research (you can’t do a brain scan on someone remotely!), but most projects start out online because, as he says, “when you have a new idea, you can put something together and have results in a matter of days.”

Prolific and Gorilla not only supported his lab’s research but his students also loved these platforms! No one in his class had ever used Prolific or Gorilla before. But students were able to run experiments and analyze the data within a week. Engaging in practical research is an invaluable learning experience that students enjoy - and it enhances their research skills.

The results 

This research influences the development of a new app called Happiness Quest. This app allows users anywhere in the world to play decision-making minigames which evaluate their choices and the emotions they feel as a result of the decisions they make. Results of past studies can be found here. 
 

Previous studies have identified a mathematical equation that can predict how happy people will be from minute to minute. In short, happiness depends not on how well you are doing, but whether you are doing better than expected. You can participate in ongoing research by playing games on your smartphone.