Aligning AI with human values: How Prolific supported groundbreaking research

Challenge

Artificial intelligence now influences important decisions in areas like healthcare, education, and social welfare. But for AI to genuinely help people, it needs to align with what humans actually value. 

Researchers from the University of Washington, Google, Microsoft, and the University of Illinois set out to explore just how well modern AI systems reflect everyday human values. And they used Prolific to connect with participants around the world.

To get meaningful insights, the researchers needed a global group of participants who could respond to complex ethical scenarios. That required finding people from diverse backgrounds who could provide detailed, reliable responses and help the team understand the relationship between human and AI values.

Solution

Prolific helped researchers tackle this head-on. Using the platform, the team quickly connected with 72 participants from different parts of the world, including North America, Europe, Africa, and the Middle East. This global mix gave the researchers confidence they were hearing from people with diverse experiences and viewpoints.

The user-friendly interface and built-in quality checks encouraged participants to actively engage with complex ethical scenarios, leading to thoughtful and reliable responses. These rigorous checks also meant the researchers could fully trust the quality of the data collected.

Additionally, the platform integrated easily with the advanced survey tools used by the researchers, allowing them to spend less time on technical tasks and more time analysing results.

Execution

Participants gave their thoughts on 49 statements about values, using the innovative ValueCompass framework developed for the study. These statements covered real-life scenarios in areas such as collaborative writing, education, public policy, and healthcare.

After gathering responses from participants, researchers compared these human values directly against answers given by advanced AI models, like GPT-4o and Llama3. By doing this, they could pinpoint exactly where human and AI opinions matched or didn't match.

Results

The research found some clear differences between human and AI values:

• People cared most about honesty, truthfulness, and careful decision-making. They strongly expected AI systems to behave ethically and transparently.
  
   
• The AI models, however, tended to prioritize keeping users happy, focusing more on politeness, personalisation, and ease of use rather than ethical issues.
  
   
• AI often showed support for making independent decisions without human oversight, a point that made most human participants deeply uncomfortable.
  
   
• Researchers also found that people's views shifted depending on the scenario. This showed that there's no one-size-fits-all approach to aligning AI values. It must be context-specific.
   

Conclusion

Prolific was essential in helping the team recruit participants from around the world at speed, gather trustworthy responses, and easily integrate the data into their study. The researchers gained clearer insights into how AI aligns – or doesn't – with the values that matter most to people. 

Citation

Shen, H., Knearem, T., Ghosh, R., Yang, Y.-J., Mitra, T., & Huang, Y. (2024). ValueCompass: A Framework of Fundamental Values for Human-AI Alignment. arXiv:2409.09586.

Research institutions

University of Washington, Google, Microsoft, University of Illinois Urbana-Champaign.