Your agent can now run studies and evaluations on Prolific

Integrate feedback from real, verified humans faster with the Prolific CLI

Access the CLI

Read API docs

Trusted by leading engineering teams

Why Prolific

Built for pipelines

One network for every interface

Reach the same trusted human network and combine with 300+ filters whether you're calling via UI, CLI or REST API. A cohort defined in the dashboard resolves identically when called from your pipeline — same hash, same provenance.

Designed for repeat runs

Idempotency keys on study creation, stable cohort hashes across re-recruits, paginated response export, and JSONL output. Filter provenance is preserved in every response record so downstream training data carries its own cohort description.

Latency that fits a training loop

Studies launch in minutes and return first responses in hours. Webhooks on response.submitted and study.completed let a pipeline ingest incrementally rather than block on a full cohort.

What customers say

"I want to remove any barrier between my agent and the results."

Emerging Products Director: Fortune 500 software company

Watch the CLI demo

How teams do more with the CLI

Four programmatic workflows across one human intelligence network.

Preference collection in the loop

Launch pairwise preferences, Likert ratings or step-level rationale tasks directly from your training pipeline between runs. JSONL export feeds into RLHF, DPO and reward-model workflows.

Human-in-the-loop checkpoints

Agents call Prolific mid-run for human review, escalation or disambiguation — filtered down to the right participants for the task. Responses return as structured JSON.

Embed human evaluation

Run pre-release human evaluation as a gated step in CI. Specify a cohort by filter, aggregate responses against a threshold, and pass or fail the job on the resulting metric. Reproducible across model versions.

On demand cohorts for user simulation

Programmatically recruit targeted participants to role-play end users in multi-turn evaluation. Stable cohort hashes mean the same group can be re-recruited for longitudinal comparison.

Get started

Do more from where you already operate

Start where it matches your stack.

API reference

Studies, cohorts, responses, webhooks, submissions. OpenAPI schema, auth model, rate limits, idempotency semantics.

docs.prolific.com/api

CLI reference

Launch studies, wait on completion, export responses, manage filters from a shell. Scriptable in any pipeline.

docs.prolific.com/cli

Events & webhooks

Subscribe to response.submitted, study.completed, and submission review events. Signed payloads, retry policy.

docs.prolific.com/webhooks

Fast-moving AI teams using Prolific

Trusted by AI/ML developers, researchers, and leading organizations across industries.

Speak to the team

Prolific's CLI in practice

See how agent can launch tasks to real humans on Prolific - from dataset to results

Watch the demo

Building breakthrough AI faster

Ai2 reduced human data collection from weeks to hours with Prolific, building state-of-the-art multimodal AI models faster without sacrificing quality.

Unpacking human preference for LLMs - The HUMAINE framework

Prolific's human-centered leaderboard ranks frontier AI models by how real users actually experience them - not just technical benchmarks. Featured at ICLR 2026.

Read the paper

Gemini 3 Pro: Frontier safety framework

The frontier safety framework report for Google’s latest model.

Start building automated workflows

Get started for free

Read the API docs

FAQ

Questions from engineers integrating Prolific

Does Prolific have an API?

Yes. Prolific exposes a REST API and a CLI over the same 200,000+ identity-verified participant pool. UI, CLI, and API address the same 300+ filter attributes and the same audit trail. Anonymised participant response data is accessible via any of the three.

Can an LLM agent call Prolific directly?

Yes. Any system that can make authenticated HTTPS requests — agent scaffolding, orchestrators, training pipelines, CI runners — can launch studies, subscribe to webhooks, and export responses. Idempotency keys on study creation make retry safe from automated callers.

Can I wire Prolific into an RLHF or DPO training loop?

Yes. Preference-collection studies launched via API return JSONL responses with stable cohort hashes and filter provenance in every record. The format is designed to feed directly into reward-model, DPO, and RLHF training code.

How fast does a Prolific study return data?

Self-serve studies typically launch in minutes and return first responses in hours. Completion time depends on cohort size, filter strictness, and reward. Webhooks on response.submitted allow incremental ingest rather than blocking on full completion.

Can I filter participants programmatically by profession, credentials, or demographics?

Yes. The API accepts the same 300+ filter attributes available in the UI — demographic, professional, behavioural, and credential. A filter set resolves to a stable cohort hash that can be reused across runs for longitudinal comparison between model versions.

Does Prolific support webhooks?

Yes. Subscribe to response.submitted, study.completed, and submission-review events. Payloads are signed and include filter provenance; delivery retries with exponential backoff on non-2xx responses.

Can I use Prolific as a release gate in CI?

Yes. A CI job can launch a study, wait on completion via the CLI or a webhook, aggregate responses against a threshold, and exit pass or fail. Reproducible filter hashes make the same cohort specification repeatable across releases.

How does Prolific authenticate API requests?

API-key authentication over HTTPS. Keys are scoped to a workspace and can be rotated without downtime. Keys are never embedded in participant-facing URLs or exposed to study respondents.