TU Berlin

Quality and Usability LabOther Conference Papers

Inhalt des Dokuments

zur Navigation

Other Conference Papers

go back to overview

Environmental Noise Recording as a Quality Control for Crowdsourcing Speech Quality Assessments
Zitatschlüssel zequeirajimenez2018a
Autor Zequeira Jiménez, Rafael and Fernández Gallardo, Laura and Möller, Sebastian
Buchtitel 44. Deutsche Jahrestagung für Akustik (DAGA)
Seiten 303–306
Jahr 2018
ISBN 978-3-939296-13-3
Monat mar
Verlag Deutsche Gesellschaft für Akustik DEGA e.V.
Wie herausgegeben Fullpaper
Zusammenfassung The Crowdsourcing (CS) paradigm offers small tasks to anonymous users on the Internet. Human-centered speech quality assessment studies have been traditionally conducted under controlled laboratory conditions. Nowadays, CS provides an exceptional opportunity to transfer such experiments to the internet and reach a wider and diverse audience. However, data from CS can be corrupted due to users' neglect and hence quality control mechanisms are required to ensure reliable outcomes. While previous works have presented trapping questions or majority voting to ensure good results, this work introduces user-environmental noise recording to discard unreliable users located in noisy places. To this end, a speech quality assessment study was conducted in the clickworker CS platform. The speech stimuli are taken from the database 501 from the ITU-T Rec. P.863 and the results are to be contrasted to the existing lab ratings. This work analyzes whether environmental noise recording can be used to identify unreliable workers. Furthermore, the effects of discarding users deemed untrustworthy on the correlation between the CS and the Lab results is studied. Our outcomes highlight the importance of controlling for users' background noises to ensure reliable results in speech quality assessments conducted via CS.
Link zur Publikation Download Bibtex Eintrag

go back to overview



Schnellnavigation zur Seite über Nummerneingabe