TU Berlin

Quality and Usability LabReviewed Journal Papers

Inhalt des Dokuments

zur Navigation

Reviewed Journal Papers

go back to overview

Towards speech quality assessment using a crowdsourcing approach: evaluation of standardized methods
Zitatschlüssel naderi2020e
Autor Naderi, Babak and Jiménez, Rafael Zequeira and Hirth, Matthias and Möller, Sebastian and Metzger, Florian and Hoßfeld, Tobias
Seiten 2
Jahr 2020
ISSN 2366-0139
DOI 10.1007/s41233-020-00042-1
Adresse Springer Nature
Journal Quality and User Experience
Jahrgang 6
Nummer 1
Monat nov
Notiz online,print
Verlag Springer
Wie herausgegeben Fullpaper
Zusammenfassung Subjective speech quality assessment has traditionally been carried out in laboratory environments under controlled conditions. With the advent of crowdsourcing platforms tasks, which need human intelligence, can be resolved by crowd workers over the Internet. Crowdsourcing also offers a new paradigm for speech quality assessment, promising higher ecological validity of the quality judgments at the expense of potentially lower reliability. This paper compares laboratory-based and crowdsourcing-based speech quality assessments in terms of comparability of results and efficiency. For this purpose, three pairs of listening-only tests have been carried out using three different crowdsourcing platforms and following the ITU-T Recommendation P.808. In each test, listeners judge the overall quality of the speech sample following the Absolute Category Rating procedure. We compare the results of the crowdsourcing approach with the results of standard laboratory tests performed according to the ITU-T Recommendation P.800. Results show that in most cases, both paradigms lead to comparable results. Notable differences are discussed with respect to their sources, and conclusions are drawn that establish practical guidelines for crowdsourcing-based speech quality assessment.
Link zur Publikation Link zur Originalpublikation Download Bibtex Eintrag

go back to overview



Schnellnavigation zur Seite über Nummerneingabe