Crowdsourcing Wage Pledge

Frequently Asked Questions

Version 28 July 2021

Q: What are the goals of the Wage Pledge?

A: The goals of the Wage Pledge are to ensure that:

80% or more of the tasks on a given project are compensated at or above the target wage, and
Work is not rejected in a manner inconsistent with the rejection policy posted by the requester for the project in question.

Q: Who can see my pledges?

A: Your pledges are publicly available on the internet. This includes your requester ID and name, the title of your tasks, your target wage(s), and information about resolved or unresolved inquiries relating to your pledges.

Q: Why should I sign up?

A: There are a lot of reasons you might want to create pledges for your projects. Here are a few:

If you care about improving pay on crowdsourcing platforms: by participating in the wage pledge you contribute to normalizing higher pay for crowdworkers.
If you care about attracting higher quality work: by participating in the wage pledge, you signal to workers that you pay tasks well and that you are aware of the issues commonly faced by crowdworkers. This may attract more experienced and skilled workers to your tasks. It may also result in your tasks being completed more quickly.
If you care about maintaining good relationships with the workers doing your tasks, and about maintaining a good reputation as a requester of crowdwork: by participating in the wage pledge, you signal to workers that you are willing to make the effort to resolve issues that workers may encounter while completing your tasks. This may build goodwill with workers.
If you are using crowdsourcing to collect data that will be published: we know that some reviewers recommend rejection for research papers whose data was collected via low-paying crowdsourcing. By participating in the wage pledge, you signal to reviewers of your research that you have taken steps to fairly pay the workers who contributed to your research.

Q: What hourly target wage should I use?

A: We recommend a target wage of $16.54 per hour.

If you can’t afford that much, pay as much as you can.

Why?

$16.54 is the living wage in the United States, calculated in March 2020. We use the US living wage because many Mechanical Turk workers are based in the US, and many research requesters restrict their tasks to US-based workers.

Q: How should I determine how much to pay for each of my tasks?

A: Because workers on Mechanical Turk (and most crowdsourcing platforms) are paid per task, not per hour, and different workers may take different amounts of time to complete the same task, it can be tricky to decide how much to pay for a given task, even if you have a target hourly wage in mind.

We recommend the following process:

Choose a target hourly wage, if you have not already. (If you don’t know what hourly wage to aim for, see “What hourly target wage should I use?”)
Estimate how long your task will take a worker. Workers do some kinds of tasks slower than you might expect and some kinds of tasks faster than you might expect. Do not be surprised if the time it takes you, or one of your students, to do your task is significantly different than the time it takes a worker.
Pilot your task with a group of crowdworkers. If possible, use the same screening criteria you will use for your “real” task.

Piloting your task will help you accurately estimate how long workers will take to do your task.

We recommend you calculate the “task time” as the time taken by the median worker in your pilot group plus 30%.
Set your per-task reward using the following formula:
per-task reward = task time in minutes / 60 * hourly wage target

Q: How can I share my wage pledge commitment with others?

A: You can share your wage pledge commitment with others in your publications, at departmental or ethics board meetings, or via social media using a sentence such as:

Workers in this study were paid $X per hour. This project was registered with the Crowdsourcing Wage Pledge and has no unresolved worker inquiries.

Q: What happens if a worker contacts the Wage Pledge Facilitators about one of my projects?

A: If the worker submits an inquiry about one of your projects and believes that you have either rejected work in a manner inconsistent with your posted rejection policy, or not met your target wage, the Facilitators will contact you and ask you to provide more information about your project. If a legitimate dispute arises, the Facilitators will endeavor to mediate the dispute. A full explanation of the process can be found here.