Page Content
CrowdMAQA
Motivation and Automatic Quality Assessment in Paid Crowdsourcing Online Labor Markets
Description:
Building on the growth in availability and popularity of crowd-sourced data, this project will answer essential questions about data quality and characteristics of crowd-sourced data. It will focus on how to assess, monitor and assure the data quality in the context of crowdsourcing and examine factors influencing the motivation of workers.
Regardless of the growing popularity of crowd sourcing, one essential problem remains untackled. Anonymously paid micro tasks are frequently corrupted by certain proportion of subjects who do not focus enough on the job or who do not work as instructed but give random data. This in turn leads to noisy responses or inaccurate results and thus to a considerable deterioration of the data quality.
In this project, we are intended to answer the following research questions:
- How can we measure and predict the quality of responses to predefined online jobs such as survey, multimedia recording or free-text responses?
- What is the relationship between the motivation of an online worker and performance, and how can a worker’s motivation be influenced?
| Time Frame: | 4/2012 - 3/2015 |
|---|---|
| Team Members: | Babak Naderi |
| Software Campus Partners: | TU-Berlin, Deutsche Telekom AG |
| Funding by: | Bundesministerium für Bildung und Forschung - BMBF |
Pubications
| Citation key | naderi2014b |
|---|---|
| Author | Naderi, Babak and Wechsung, Ina and Polzehl, Tim and Möller, Sebastian |
| Title of Book | Proceedings of the 2014 International ACM Workshop on Crowdsourcing for Multimedia |
| Pages | 31–36 |
| Year | 2014 |
| ISBN | 978-1-4503-3128-9 |
| DOI | 10.1145/2660114.2660122 |
| Location | Orlando, Florida, USA |
| Address | New York, NY, USA |
| Month | nov |
| Publisher | ACM |
| Series | CrowdMM '14 |
| How Published | full |
| Abstract | In this paper, we introduce a scale for measuring the extrinsic motivation of crowd workers. The new questionnaire is strongly based on the Work Extrinsic Intrinsic Motivation Scale (WEIMS) [17] and theoretically follows the Self-Determination Theory (SDT) of motivation. The questionnaire has been applied and validated in a crowdsourcing micro-task platform. This instrument can be used for studying the dynamics of extrinsic motivation by taking into account individual differences and provide meaningful insights which will help to design a proper incentives framework for each crowd worker that eventually leads to a better performance, an increased well-being, and higher overall quality. |