TU Berlin

Quality and Usability LabReviewed Journal Papers

Page Content

to Navigation

There is no English translation for this web page.

Reviewed Journal Papers by Sebastian Möller

go back to overview

Towards speech quality assessment using a crowdsourcing approach: evaluation of standardized methods
Citation key naderi2020towards
Author Naderi, Babak and Jiménez, Rafael Zequeira and Hirth, Matthias and Möller, Sebastian and Metzger, Florian and Hoßfeld, Tobias
Pages 2
Year 2020
ISSN 2366-0139
DOI 10.1007/s41233-020-00042-1
Address Springer Nature
Journal Quality and User Experience
Volume 6
Number 1
Month nov
Note online,print
Publisher Springer
How Published Fullpaper
Abstract Subjective speech quality assessment has traditionally been carried out in laboratory environments under controlled conditions. With the advent of crowdsourcing platforms tasks, which need human intelligence, can be resolved by crowd workers over the Internet. Crowdsourcing also offers a new paradigm for speech quality assessment, promising higher ecological validity of the quality judgments at the expense of potentially lower reliability. This paper compares laboratory-based and crowdsourcing-based speech quality assessments in terms of comparability of results and efficiency. For this purpose, three pairs of listening-only tests have been carried out using three different crowdsourcing platforms and following the ITU-T Recommendation P.808. In each test, listeners judge the overall quality of the speech sample following the Absolute Category Rating procedure. We compare the results of the crowdsourcing approach with the results of standard laboratory tests performed according to the ITU-T Recommendation P.800. Results show that in most cases, both paradigms lead to comparable results. Notable differences are discussed with respect to their sources, and conclusions are drawn that establish practical guidelines for crowdsourcing-based speech quality assessment.
Link to publication Link to original publication Download Bibtex entry

go back to overview


Quick Access

Schnellnavigation zur Seite über Nummerneingabe