DeepArt

Project Description

People with motor control problems, who are unable to use keyboard and touch driven interfaces to access the digital world, could benefit hugely from automatic speech recognition. Unfortunately, these same people often have co-occurring speech disorders (dysarthria), that make their speech hard to recognise. Sheffield have much experience in developing speech technology for disordered speech, however, the task is made challenging by a shortage of training data and a large inter-speaker variability. Consequently, recognition performance is still well below that for normal speech – bold new approaches are required. This project will investigate new articulatory-based representations in which the mapping from typical to dysarthric speech may be more readily modeled. In particular, three research questions will be addressed: i/ Can deep learning be used to estimate articulatory features from dysarthric acoustics? ii/ Can articulatory synthesis be used to generate dysarthric-style speech to augment existing ASR training data? iii/ Can speaker-adaptive training techniques be employed to model dysarthric variability when supported by articulatory features?

People & Partners

Working with Dr Feifei Xiong, Prof Jon Barker and Dr Rick Rose at Google AI