Inhalt des Dokuments
Neslihan Iskender
- © Copyright??
Research Group
Teaching
- Study Project Quality & Usability (Since SS 2018)
- Interdiziplinäres Medienprojekt (Since SS 2018)
- Usability Engineering (Exercise SS 2018)
Biography
Neslihan Iskender received her Bachelor and Master of Science degree in Industrial Engineering and Management at the Karlsruhe Institute of Technology. During her studies, she focused on managing new technologies and innovation management. Since May 2017, she is employed as a research assistant at the Quality and Usability Labs where she is working towards a PhD in the field of crowdsourcing. Her research Topics are:
- Crowd assessments: Usability, UX, QoE, Quality
- Real-time interaction, human computation as a service, (HuaaS)
- Hybrid Worfklows for micro-task crowdsourcing
- Internal Crowdsourcing
Current Projects
Past Projects
- ERICS – European Refugee Information and Communication Service (EIT-Digital, Project Lead)
- OurPuppet: Pflegeunterstützung mit einer interaktiven Puppe für informell Pflegende (BMBF)
- ICU - Internes Crowdsourcing in Unternehmen: Arbeitnehmergerechte Prozessinnovationen durch digitale Beteiligung von Mitarbeiter/innen (BMBF)
Contact
E-Mail: neslihan.iskender@tu-berlin.de
Phone: +49 (30) 8353-58347
Fax: +49 (30) 8353-58409
Address
Quality and Usability Lab
Deutsche Telekom Laboratories
Technische Universität Berlin
Ernst-Reuter-Platz 7
D-10587 Berlin, Germany
Publications
| Zitatschlüssel | iskender2020c |
|---|---|
| Autor | Iskender, Neslihan and Polzehl, Tim and Möller, Sebastian |
| Buchtitel | Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems |
| Seiten | 164–175 |
| Jahr | 2020 |
| Ort | online |
| Adresse | online |
| Monat | nov |
| Verlag | Association for Computational Linguistics (ACL) |
| Serie | EMNLP | Eval4NLP |
| Wie herausgegeben | Fullpaper |
| Zusammenfassung | One of the main challenges in the development of summarization tools is summarization quality evaluation. On the one hand, the human assessment of summarization quality conducted by linguistic experts is slow, expensive, and still not a standardized procedure. On the other hand, the automatic assessment metrics are reported not to correlate high enough with human quality ratings. As a solution, we propose crowdsourcing as a fast, scalable, and cost-effective alternative to expert evaluations to assess the intrinsic and extrinsic quality of summarization by comparing crowd ratings with expert ratings and automatic metrics such as ROUGE, BLEU, or BertScore on a German summarization data set. Our results provide a basis for best practices for crowd-based summarization evaluation regarding major influential factors such as the best annotation aggregation method, the influence of readability and reading effort on summarization evaluation, and the optimal number of crowd workers to achieve comparable results to experts, especially when determining factors such as overall quality, grammaticality, referential clarity, focus, structure & coherence, summary usefulness, and summary informativeness. |
Zusatzinformationen / Extras
Direktzugang
Schnellnavigation zur Seite über Nummerneingabe