Deep Learning Based Speech Quality Prediction

Preface
Acknowledgments
Contents
Acronyms
1 Introduction
	1.1 Motivation
	1.2 Thesis Objectives and Research Questions
	1.3 Outline
2 Quality Assessment of Transmitted Speech
	2.1 Speech Communication Networks
	2.2 Speech Quality and Speech Quality Dimensions
	2.3 Subjective Assessment
	2.4 Subjective Assessment via Crowdsourcing
	2.5 Traditional Instrumental Methods
		2.5.1 Parametric Models
		2.5.2 Double-Ended Signal-Based Models
		2.5.3 Single-Ended Signal-Based Models
	2.6 Machine Learning Based Instrumental Methods
		2.6.1 Non-Deep Learning Machine Learning Approaches
		2.6.2 Deep Learning Architectures
		2.6.3 Deep Learning Based Speech Quality Models
	2.7 Summary
3 Neural Network Architectures for Speech Quality Prediction
	3.1 Dataset
		3.1.1 Source Files
		3.1.2 Simulated Distortions
		3.1.3 Live Distortions
		3.1.4 Listening Experiment
	3.2 Overview of Neural Network Model
	3.3 Mel-Spec Segmentation
	3.4 Framewise Model
		3.4.1 CNN
		3.4.2 Feedforward Network
	3.5 Time-Dependency Modelling
		3.5.1 LSTM
		3.5.2 Transformer/Self-Attention
	3.6 Time Pooling
		3.6.1 Average-/Max-Pooling
		3.6.2 Last-Step-Pooling
		3.6.3 Attention-Pooling
	3.7 Experiments and Results
		3.7.1 Training and Evaluation Metric
		3.7.2 Framewise Model
		3.7.3 Time-Dependency Model
		3.7.4 Pooling Model
	3.8 Summary
4 Double-Ended Speech Quality Prediction Using Siamese Networks
	4.1 Introduction
	4.2 Method
		4.2.1 Siamese Neural Network
		4.2.2 Reference Alignment
		4.2.3 Feature Fusion
	4.3 Results
		4.3.1 LSTM vs Self-Attention
		4.3.2 Alignment
		4.3.3 Feature Fusion
		4.3.4 Double-Ended vs Single-Ended
	4.4 Summary
5 Prediction of Speech Quality Dimensions with Multi-TaskLearning
	5.1 Introduction
	5.2 Multi-Task Models
		5.2.1 Fully Connected (MTL-FC)
		5.2.2 Fully Connected + Pooling (MTL-POOL)
		5.2.3 Fully Connected + Pooling + Time-Dependency(MTL-TD)
		5.2.4 Fully Connected + Pooling + Time-Dependency + CNN (MTL-CNN)
	5.3 Results
		5.3.1 Per-Task Evaluation
		5.3.2 All-Tasks Evaluation
		5.3.3 Comparing Dimension
		5.3.4 Degradation Decomposition
	5.4 Summary
6 Bias-Aware Loss for Training from Multiple Datasets
	6.1 Method
		6.1.1 Learning with Bias-Aware Loss
		6.1.2 Anchoring Predictions
	6.2 Experiments and Results
		6.2.1 Synthetic Data
		6.2.2 Minimum Accuracy rth
		6.2.3 Training Examples with and Without Anchoring
		6.2.4 Configuration Comparisons
		6.2.5 Speech Quality Dataset
	6.3 Summary
7 NISQA: A Single-Ended Speech Quality Model
	7.1 Datasets
		7.1.1 POLQA Pool
		7.1.2 ITU-T P Suppl. 23
		7.1.3 Other Datasets
		7.1.4 Live-Talking Test Set
	7.2 Model and Training
		7.2.1 Model
		7.2.2 Bias-Aware Loss
		7.2.3 Handling Missing Dimension Ratings
		7.2.4 Training
	7.3 Results
		7.3.1 Evaluation Metrics
		7.3.2 Validation Set Results: Overall Quality
		7.3.3 Validation Set Results: Quality Dimensions
		7.3.4 Test Set Results
		7.3.5 Impairment Level vs Quality Prediction
	7.4 Summary
8 Conclusions
A Dataset Condition Tables
B Train and Validation Dataset Dimension Histograms
References
Index