direkt zum Inhalt springen

direkt zum Hauptnavigationsmenü

Sie sind hier

TU Berlin

Page Content

CrowdMAQA

Motivation and Automatic Quality Assessment in Paid Crowdsourcing Online Labor Markets

 

Description:

 

Building on the growth in availability and popularity of crowd-sourced data, this project will answer essential questions about data quality and characteristics of crowd-sourced data.  It will focus on how to assess, monitor and assure the data quality in the context of crowdsourcing and examine factors influencing the motivation of workers.

Regardless of the growing popularity of crowd sourcing, one essential problem remains untackled.  Anonymously paid micro tasks are frequently corrupted by certain proportion of subjects who do not focus enough on the job or who do not work as instructed but give random data. This in turn leads to noisy responses or inaccurate results and thus to a considerable deterioration of the data quality.

In this project, we are intended to answer the following research questions:

  • How can we measure and predict the quality of responses to predefined online jobs such as survey, multimedia recording or free-text responses?
  • What is the relationship between the motivation of an online worker and performance, and how can a worker’s motivation be influenced?

 

 

Time Frame:
4/2012 - 3/2015
Team Members:
Babak Naderi
Software Campus Partners:
TU-Berlin, Deutsche Telekom AG
Funding by:
Bundesministerium für Bildung und Forschung - BMBF

Pubications

Crowdee: Mobile Crowdsourcing Micro-task Platform for Celebrating the Diversity of Languages
Citation key naderi2014a
Author Naderi, Babak and Polzehl, Tim and Beyer, André and Pilz, tibor and Möller, Sebastian
Title of Book Proc. 15th Ann. Conf. of the Int. Speech Comm. Assoc. (Interspeech 2014), Show & Tell Session
Pages 1496–1497
Year 2014
ISSN 1990-9770
Location SG-Singapore
Month sep
Publisher ISCA
How Published full
Abstract This paper introduces a novel crowdsourcing platform provided to the community. The platform operates on mobile devices and makes data generation and labeling scenarios available for many related research tracks potentially covering also small and underrepresented languages. Besides the versatile ways for commencing studies using the platform, also active research on crowdsourcing itself becomes feasible. With special focus on speech- and video recordings, the mobility and scalability of the platform is expected to stimulate and foster data-driven studies and insights throughout the community.
Link to publication Link to original publication Download Bibtex entry

Zusatzinformationen / Extras

Quick Access:

Schnellnavigation zur Seite über Nummerneingabe