Measuring and Visualising Speech Articulation

The digitalisation of second language learning has been developing for decades. With the boom of mobile devices, we see a plethora of applications. The exploration of one sub-field, Computer Assisted Pronunciation Training (CAPT), started just as early. In most cases, however, the application of CAPT has not moved much beyond audio-lingual (“listen and repeat”) exercises and acoustic “scoring”. Can computers do more for learners than a “glorified recorder”? Human experts with knowledge of phonetics are able to listen to a recording and point out specific pronunciation errors and give instructions on production. Can machines perform similar tasks? This master thesis investigates the various aspects related to detecting articulation events, and visualising demonstrative and instructional information for the user in speech production exercises. The exploration focuses on Web Audio APIs, open source feature extraction libraries suitable for web applications, and digital signal processing techniques including Dynamic Time Warping to provide immediate feedback in an interactive environment.