Human and Machine Hearing: Extracting Meaning from Sound

Contents
Foreword
Preface
Part I Sound Analysis and Representation Overview
	1 Introduction
		1.1 On Vision and Hearing à la David Marr
		1.2 Top-Down versus Bottom-Up Analysis
		1.3 The Neuromimetic Approach
		1.4 Auditory Images
		1.5 The Ear as a Frequency Analyzer?
		1.6 The Third Sound
		1.7 Sound Understanding and Extraction of Meaning
		1.8 Leveraging Techniques from Machine Vision and Machine Learning
		1.9 Machine Hearing Systems “by the Book”
	2 Theories of Hearing
		2.1 A “New” Theory of Hearing
		2.2 Newer Theories of Hearing
		2.3 Active and Nonlinear Theories of Hearing
		2.4 Three Auditory Theories
		2.5 The Auditory Image Theory of Hearing
	3 On Logarithmic and Power-Law Hearing
		3.1 Logarithms and Power Laws
		3.2 Log Frequency
		3.3 Log Power
		3.4 Bode Plots
		3.5 Perceptual Mappings
		3.6 Constant-Q Analysis
		3.7 Use Logarithms with Caution
	4 Human Hearing Overview
		4.1 Human versus Machine
		4.2 Auditory Physiology
		4.3 Key Problems in Hearing
		4.4 Loudness
		4.5 Critical Bands, Masking, and Suppression
		4.6 Pitch Perception
		4.7 Timbre
		4.8 Consonance and Dissonance
		4.9 Speech Perception
		4.10 Binaural Hearing
		4.11 Auditory Streaming
		4.12 Nonlinearity
		4.13 A Way Forward
	5 Acoustic Approaches and Auditory Influence
		5.1 Sound, Speech, and Music Modeling
		5.2 Short-Time Spectral Analysis
		5.3 Smoothing and Transformation of Spectra
		5.4 The Source–Filter Model and Homomorphic Signal Processing
		5.5 Backing Away from Logarithms
		5.6 Auditory Frequency Scales
		5.7 Mel-Frequency Cepstrum
		5.8 Linear Predictive Coding
		5.9 PLP and RASTA
		5.10 Auditory Techniques in Automatic Speech Recognition
		5.11 Improvements Needed
Part II Systems Theory for Hearing
	6 Introduction to Linear Systems
		6.1 Smoothing: A Good Place to Start
		6.2 Linear Time-Invariant Systems
		6.3 Filters and Frequencies
		6.4 Differential Equations and Homogeneous Solutions
		6.5 Impulse Responses
		6.6 Causality and Stability
		6.7 Convolution
		6.8 Eigenfunctions and Transfer Functions
		6.9 Frequency Response
		6.10 Transforms and Operational Methods
		6.11 Rational Functions, and Their Poles and Zeros
		6.12 Graphical Computation of Transfer Function Gain and Phase
		6.13 Convolution Theorem
		6.14 Interconnection of Filters in Cascade, Parallel, and Feedback
		6.15 Summary and Next Steps
	7 Discrete-Time and Digital Systems
		7.1 Simulating Systems in Computers
		7.2 Discrete-Time Linear Shift-Invariant Systems
		7.3 Impulse Response and Convolution
		7.4 Frequency in Discrete-Time Systems
		7.5 Z Transform and Its Inverse
		7.6 Unit Advance and Unit Delay Operators
		7.7 Filters and Transfer Functions
		7.8 Sampling and Aliasing
		7.9 Mappings from Continuous-Time Systems
		7.10 Filter Design
		7.11 Digital Filters
		7.12 Multiple Inputs and Outputs
		7.13 Fourier Analysis and Spectrograms
		7.14 Perspective and Further Reading
	8 Resonators
		8.1 Bandpass Filters
		8.2 Four Resonant Systems
		8.3 Resonator Frequency Responses
		8.4 Resonator Impulse Responses
		8.5 The Complex Resonator and the Universal Resonance Curve
		8.6 Complex Zeros from a Parallel System
		8.7 Keeping It Real
		8.8 Digital Resonators
	9 Gammatone and Related Filters
		9.1 Compound Resonators as Auditory Models
		9.2 Multiple Poles
		9.3 The Complex Gammatone Filter
		9.4 The Real Gammatone Filter
		9.5 All-Pole Gammatone Filters
		9.6 Gammachirp Filters
		9.7 Variable Pole Q
		9.8 Noncoincident Poles
		9.9 Digital Implementations
	10 Nonlinear Systems
		10.1 Volterra Series and Other Descriptions
		10.2 Essential Nonlinearity
		10.3 Hopf Bifurcation
		10.4 Distributed Bandpass Nonlinearity
		10.5 Response Curves of Nonlinear Systems
		10.6 Two-Tone Responses
		10.7 Nonlinearity and Aliasing
		10.8 Cautions
	11 Automatic Gain Control
		11.1 Input–Output Level Compression
		11.2 Nonlinear Feedback Control
		11.3 AGC Compression at Equilibrium
		11.4 Multiple Cascaded Variable-Gain Stages
		11.5 Gain Control via Damping Control in Cascaded Resonators
		11.6 AGC Dynamics
		11.7 AGC Loop Stability
		11.8 Multiple-Loop AGC
	12 Waves in Distributed Systems
		12.1 Waves in Uniform Linear Media
		12.2 Transfer Functions from Wavenumbers
		12.3 Nonuniform Media
		12.4 Nonuniform Media as Filter Cascades
		12.5 Impulse Responses
		12.6 Group Velocity and Group Delay
Part III The Auditory Periphery
	13 Auditory Filter Models
		13.1 What Is an Auditory Filter?
		13.2 From Resonance to Gaussian Filters
		13.3 Ten Good Properties for Auditory Filter Models
		13.4 Representative Auditory Filter Models
		13.5 Complications: Time-Varying and Nonlinear Auditory Filters
		13.6 Fitting Parameters of Filter Models
		13.7 Suppression
		13.8 Impulse Responses from Physiological Data
		13.9 Summary and Application to Cochlear Models
	14 Modeling the Cochlea
		14.1 On the Structure of the Cochlea
		14.2 The Traveling Wave
		14.3 1D, 2D, and 3D Hydrodynamics
		14.4 Long Waves, Short Waves, and 2D Models
		14.5 Active Micromechanics
		14.6 Scaling Symmetry and the Cochlear Map
		14.7 Filter-Cascade Cochlear Models
		14.8 Outer Hair Cells as Active Gain Elements
		14.9 Dispersion Relations from Mechanical Models and Experiments
		14.10 Inner Hair Cells as Detectors
		14.11 Adaptation to Sound via Efferent Control
		14.12 Summary and Further Reading
	15 The CARFAC Digital Cochlear Model
		15.1 Putting the Pieces Together
		15.2 The CARFAC Framework
		15.3 Physiological Elements
		15.4 Analog and Bidirectional Models
		15.5 Open-Source Software
		15.6 Detailing the CARFAC
	16 The Cascade of Asymmetric Resonators
		16.1 The Linear Cochlear Model
		16.2 Coupled-Form Filter Realization
	17 The Outer Hair Cell
		17.1 Multiple Effects in One Mechanism
		17.2 The Nonlinear Function
		17.3 AGC Effect of DOHC
		17.4 Typical Distortion Response Patterns
		17.5 Completing the Loop
	18 The Inner Hair Cell
		18.1 Rectification with a Sigmoid
		18.2 Adaptive Hair-Cell Models
		18.3 A Digital IHC Model
	19 The AGC Loop Filter
		19.1 The CARFAC’s AGC Loop
		19.2 AGC Filter Structure
		19.3 Smoothing Filter Pole–Zero Analysis
		19.4 AGC Filter Temporal Response
		19.5 AGC Filter Spatial Response
		19.6 Time–Space Smoothing with Decimation
		19.7 Adapted Behavior
		19.8 Binaural or Multi-Ear Operation
		19.9 Coupled and Multistage AGC in CARFAC and Other Systems
Part IV The Auditory Nervous System
	20 Auditory Nerve and Cochlear Nucleus
		20.1 From Hair Cells to Nerve Firings
		20.2 Tonotopic Organization
		20.3 Fine Time Structure in Cochleagrams
		20.4 Cell Types in the Cochlear Nucleus
		20.5 Inhibition and Other Computation
		20.6 Spike Timing Codes
	21 The Auditory Image
		21.1 Movies of Sound
		21.2 History
		21.3 Stabilizing the Image
		21.4 Triggered Temporal Integration
		21.5 Conventional Short-Time Autocorrelation
		21.6 Asymmetry
		21.7 Computing the SAI
		21.8 Pitch and Spectrum
		21.9 Auditory Images of Music
		21.10 Auditory Images of Speech
		21.11 Summary SAI Tracks: Pitchograms
		21.12 Cochleagram from SAI
		21.13 The Log-Lag SAI
	22 Binaural Spatial Hearing
		22.1 Rayleigh’s Duplex Theory: Interaural Level and Phase
		22.2 Interaural Time and Level Differences
		22.3 The Head-Related Transfer Function
		22.4 Neural Extraction of Interaural Differences
		22.5 The Role of the Cochlear Nucleus and the Trapezoid Body
		22.6 Binaural Acoustic Reflex and Gain Control
		22.7 The Precedence Effect
		22.8 Completing the Model
		22.9 Interaural Coherence
		22.10 Binaural Applications
	23 The Auditory Brain
		23.1 Scene Analysis: ASA and CASA
		23.2 Attention and Stream Segregation
		23.3 Stages in the Brain
		23.4 Higher Auditory Pathways
		23.5 Prospects
Part V Learning and Applications
	24 Neural Networks for Machine Learning
		24.1 Learning from Data
		24.2 The Perceptron
		24.3 The Training Phase
		24.4 Nonlinearities at the Output
		24.5 Nonlinearities at the Input
		24.6 Multiple Layers
		24.7 Neural Units and Neural Networks
		24.8 Training by Error Back-Propagation
		24.9 Cost Functions and Regularization
		24.10 Multiclass Classifiers
		24.11 Neural Network Successes and Failures
		24.12 Statistical Learning Theory
		24.13 Summary and Perspective
	25 Feature Spaces
		25.1 Feature Engineering
		25.2 Automatic Feature Optimization by Deep Networks
		25.3 Bandpass Power and Quadratic Features
		25.4 Quadratic Features of Cochlear Filterbank Outputs
		25.5 Nonlinearities and Gain Control in Feature Extraction
		25.6 Neurally Inspired Feature Extraction
		25.7 Sparsification and Winner-Take-All Features
		25.8 Which Approach Will Win?
	26 Sound Search
		26.1 Modeling Sounds
		26.2 Ranking Sounds Given Text Queries
		26.3 Experiments
		26.4 Results
		26.5 Conclusions and Followup
	27 Musical Melody Matching
		27.1 Algorithm
		27.2 Experiments
		27.3 Discussion
		27.4 Summary and Conclusions
	28 Other Applications
		28.1 Auditory Physiology and Psychoacoustics
		28.2 Audio Coding and Compression
		28.3 Hearing Aids and Cochlear Implants
		28.4 Visible Sound
		28.5 Diagnosis
		28.6 Speech and Speaker Recognition
		28.7 Music Information Retrieval
		28.8 Security, Surveillance, and Alarms
		28.9 Diarization, Summarization, and Indexing
		28.10 Have Fun
Bibliography
Author Index
Subject Index