The AI agent debate in online research | An open letter by CEO Phelim Bradley

To those who value authentic human data,

For the past year or so, I've spent a lot of time reading about what AI misuse might mean for online research. Some of it is genuinely alarming. Some of it, I think, moves faster than the evidence warrants. In this open letter specifically, I’m going to focus on AI agents, which I feel have been dominating my social media feeds and conversations with peers lately.

The concern around agentic AI is understandable. If automated systems can complete surveys convincingly, the integrity of behavioral research is at stake, and that’s certainly not a small thing. What I would question is whether the problem is already as widespread as the current discussion implies.

Earlier this year, our research team at Prolific ran one of the largest head-to-head comparisons of online survey platforms ever conducted, recruiting 5,200 participants across 10 platforms, applying validated AI detection methods to each response. Outside of Amazon MTurk, agent detection rates were at or below 1% across every platform type. The responses that were flagged also looked far more like traditional scripted bots than LLM agents. When the team ran actual LLM agents through the same survey, those agents scored at or above average human levels on nearly every quality measure. The things being detected looked nothing like that. For now, on the platforms researchers actually use, the threat appears to be more hypothetical than empirical.

That finding doesn't make me comfortable. The capability is real and it will keep improving. Our team has been building toward this for years: bank-grade ID verification, over 50 pre-screening checks, and bot authenticity checks, launched in February 2026, that monitor for non-human behavior across every question in a study with 100% accuracy in internal testing, alongside our LLM authenticity checks that detect AI-generated responses in free-text questions. We even backed bot authenticity checks with our 100% Human Guarantee (if an AI agent is detected in your study, you get twice the cost of that response back). It felt like the right way to demonstrate that we stand behind the infrastructure we've built.

The data quality concern I'm more immediately focused on is one that tends to get less attention: where researchers are actually sourcing their human participants. This comes up a lot in conversations with our own customers. When we ask what worries them most about their data, attentiveness and genuine engagement from the humans in their studies tends to come up far more than contamination from automated systems. The same cross-platform study, conducted by our research team alongside external collaborators, showed that platform type was a much stronger predictor of response quality than anything agent-related. Direct panels consistently outperformed marketplace platforms across every behavioral measure, backing our thesis that end to end control of the user experience is critical for data quality. 

The cheapest platforms per respondent were, without exception, the most expensive once quality thresholds were applied. At a 90% quality threshold, direct panels cost $8.26 per quality respondent. Marketplace platforms cost $74.43. That gap is larger than most people realize, and it has real consequences for research validity. The price per quality data point should be the measure of value against which platforms are compared. 

Both things can be true. The AI agent risk is real and worth building against. The more immediate, practical question is whether the humans researchers recruit are attentive, engaged, and responding genuinely. This is what we’ve always pushed to get right at Prolific, from our first participant in 2014 to the more than 200,000 who are now part of our network in 2026. And this is where I think the most important work still happens.

We’re all navigating the same uncertainty, and the research community is right to demand confidence in the platforms they rely on to push the frontiers of their fields. The more openly we discuss what we’re seeing, the better placed we’ll all be, so please feel free to reach out.

I also want to say a special thank you to our customers who continue to trust us, and our internal teams dedicated to building the systems that earn it. Research powered by the best of humanity is a shared goal, and there is plenty more work ahead.

Sincerely,

Phelim Bradley 
Co-founder and CEO, Prolific