Analyzing the Correlation and Impact of Speech Evaluation Metrics on Real-World Speaker Verification and Speech Recognition

This study investigates the relationship between speech quality assessment metrics and the performance of downstream tasks such as Automatic Speech Recognition (ASR) and Speaker Verification (SV). We evaluate non-intrusive metrics and estimated versions of traditional intrusive metrics across multiple languages and noise conditions, focusing on their correlation with real-world task performance where clean reference audio is typically unavailable. Our experiments span Vietnamese and English datasets with varying noise types and levels. Results indicate that while estimated intelligibility metrics like STOI show strong correlation with ASR performance, the effectiveness of various non-intrusive metrics varies across languages and noise conditions. We explore enhanced prediction models combining multiple metrics to better estimate downstream performance. This work provides insights for optimizing speech processing systems in real-world applications.