Deep Learning for Medical Image Analysis (The MICCAI Society book Series)

Front Cover
Deep Learning for Medical Image Analysis
Copyright
Contents
Contributors
Foreword
1 Deep learning theories and architectures
	1 An introduction to neural networks and deep learning
		1.1 Introduction
		1.2 Feed-forward neural networks
			1.2.1 Perceptron
			1.2.2 Multi-layer perceptron
			1.2.3 Learning in feed-forward neural networks
		1.3 Convolutional neural networks
			1.3.1 Convolution and pooling layer
			1.3.2 Computing gradients
			1.3.3 Deep convolutional neural networks
				1.3.3.1 Skip connection
				1.3.3.2 Inception module
				1.3.3.3 Attention
		1.4 Recurrent neural networks
			1.4.1 Recurrent cell
			1.4.2 Vanishing gradient problem
		1.5 Deep generative models
			1.5.1 Restricted Boltzmann machine
			1.5.2 Deep belief network
			1.5.3 Deep Boltzmann machine
			1.5.4 Variational autoencoder
				1.5.4.1 Autoencoder
				1.5.4.2 Variational autoencoder
			1.5.5 Generative adversarial network
		1.6 Tricks for better learning
			1.6.1 Parameter initialization in autoencoder
			1.6.2 Activation functions
			1.6.3 Optimizers
			1.6.4 Regularizations
			1.6.5 Normalizations
		1.7 Open-source tools for deep learning
		References
	2 Deep reinforcement learning in medical imaging
		2.1 Introduction
		2.2 Basics of reinforcement learning
			2.2.1 Markov decision process
			2.2.2 Model-free methods
				2.2.2.1 Policy gradient methods
				2.2.2.2 Value-based methods
				2.2.2.3 Actor-critic methods
			2.2.3 Model-based methods
				2.2.3.1 Value function
				2.2.3.2 Policy search
		2.3 DRL in medical imaging
			2.3.1 DRL for parametric medical image analysis
				2.3.1.1 Formulation
				2.3.1.2 Landmark detection
				2.3.1.3 Image registration
				2.3.1.4 Object/lesion localization and detection
				2.3.1.5 View plane localization
				2.3.1.6 Plaque tracking
				2.3.1.7 Vessel centerline extraction
			2.3.2 Solving optimization using DRL
				2.3.2.1 Image classification
				2.3.2.2 Image segmentation
				2.3.2.3 Image acquisition and reconstruction
				2.3.2.4 Radiotherapy planning
				2.3.2.5 Video summarization
		2.4 Future perspectives
			2.4.1 Challenges ahead
			2.4.2 The latest DRL advances
		2.5 Conclusions
		References
	3 CapsNet for medical image segmentation
		3.1 Convolutional neural networks: limitations
		3.2 Capsule network: fundamental
		3.3 Capsule network: related work
		3.4 CapsNets in medical image segmentation
			3.4.1 2D-SegCaps
			3.4.2 3D-SegCaps
			3.4.3 3D-UCaps
			3.4.4 SS-3DCapsNet
			3.4.5 Comparison
		3.5 Discussion
		Acknowledgments
		References
	4 Transformer for medical image analysis
		4.1 Introduction
		4.2 Medical image segmentation
			4.2.1 Organ-specific segmentation
				4.2.1.1 2D segmentation
				4.2.1.2 3D medical segmentation
			4.2.2 Multi-organ segmentation
				4.2.2.1 Pure transformers
				4.2.2.2 Hybrid architectures
					4.2.2.2.1 Single-scale architectures
					4.2.2.2.2 Multi-scalearchitectures
		4.3 Medical image classification
			4.3.1 COVID-19 diagnosis
				4.3.1.1 Black-box models
				4.3.1.2 Interpretable models
			4.3.2 Tumor classification
			4.3.3 Retinal disease classification
		4.4 Medical image detection
		4.5 Medical image reconstruction
			4.5.1 Medical image enhancement
				4.5.1.1 LDCT enhancement
				4.5.1.2 LDPET enhancement
			4.5.2 Medical image restoration
				4.5.2.1 Under-sampled MRI reconstruction
				4.5.2.2 Sparse-view CT reconstruction
				4.5.2.3 Endoscopic video reconstruction
		4.6 Medical image synthesis
			4.6.1 Intra-modality approaches
				4.6.1.1 Supervised methods
				4.6.1.2 Semi-supervised methods
				4.6.1.3 Unsupervised methods
			4.6.2 Inter-modality approaches
		4.7 Discussion and conclusion
		References
2 Deep learning methods
	5 An overview of disentangled representation learning for MR image harmonization
		5.1 Introduction
			5.1.1 Domain shift
			5.1.2 Image-to-image translation and harmonization
		5.2 IIT and disentangled representation learning
			5.2.1 Supervised IIT and disentangling
			5.2.2 Unsupervised IIT and disentangling
		5.3 Unsupervised harmonization with supervised IIT
			5.3.1 The disentangling framework of CALAMITI
			5.3.2 Network architecture
			5.3.3 Domain adaptation
			5.3.4 Experiments and results
		5.4 Conclusions
		Acknowledgments
		References
	6 Hyper-graph learning and its applications for medical image analysis
		6.1 Introduction
		6.2 Preliminary of hyper-graph
		6.3 Hyper-graph neural networks
			6.3.1 Hyper-graph structure generation
			6.3.2 General hyper-graph neural networks
			6.3.3 Dynamic hyper-graph neural networks
			6.3.4 Hyper-graph learning toolbox
		6.4 Hyper-graph learning for medical image analysis
		6.5 Application 1: hyper-graph learning for COVID-19 identification using CT images
			6.5.1 Method
			6.5.2 Experiments
		6.6 Application 2: hyper-graph learning for survival prediction on whole slides histopathological images
			6.6.1 Ranking-based survival prediction on histopathological whole-slide images
				6.6.1.1 Method
				6.6.1.2 Experiments
			6.6.2 Big hyper-graph factorization neural network for survival prediction from whole slide image
		6.7 Conclusions
		References
	7 Unsupervised domain adaptation for medical image analysis
		7.1 Introduction
		7.2 Image space alignment
			7.2.1 MI2GAN
				7.2.1.1 X-shape dual auto-encoders
				7.2.1.2 Mutual information discriminator
				7.2.1.3 Objective
			7.2.2 Implementation details
				7.2.2.1 Network architecture
				7.2.2.2 Optimization process
			7.2.3 Experiments
				7.2.3.1 Data sets
				7.2.3.2 Ablation study
				7.2.3.3 Comparison to state of the art
		7.3 Feature space alignment
			7.3.1 Uncertainty-aware feature space domain adaptation
				7.3.1.1 Adversarial learning block for feature space alignment
				7.3.1.2 Uncertainty estimation and segmentation module
				7.3.1.3 Uncertainty-aware cross-entropy loss
				7.3.1.4 Uncertainty-aware self-training
				7.3.1.5 Overall objective
		7.4 Experiments
			7.4.1 Exploration on uncertainty estimation
			7.4.2 Comparison with existing UDA frameworks
		7.5 Output space alignment
			7.5.1 Robust cross-denoising network
				7.5.1.1 Robust cross-denoising learning
				7.5.1.2 Overall training objective
			7.5.2 Experiments
				7.5.2.1 Comparative study
		7.6 Conclusion
		References
3 Medical image reconstruction and synthesis
	8 Medical image synthesis and reconstruction using generative adversarial networks
		8.1 Introduction
		8.2 Types of GAN
			8.2.1 GAN
			8.2.2 Conditional GAN
			8.2.3 AmbientGAN
			8.2.4 Least squares GAN and Wasserstein GAN
			8.2.5 Cycle-consistent GAN
			8.2.6 Optimal transport driven CycleGAN
			8.2.7 StarGAN
			8.2.8 Collaborative GAN
		8.3 Applications of GAN for medical imaging
			8.3.1 Multi-contrast MR image synthesis using cGAN
			8.3.2 MRI reconstruction without fully-sampled data using AmbientGAN
			8.3.3 Low dose CT denoising using CycleGAN
			8.3.4 MRI reconstruction without paired data using OT-CycleGAN
			8.3.5 MR contrast imputation using CollaGAN
		8.4 Summary
		References
	9 Deep learning for medical image reconstruction
		9.1 Introduction
		9.2 Deep learning for MRI reconstruction
			9.2.1 Introduction
			9.2.2 Basic of MR reconstruction
			9.2.3 Deep learning MRI reconstruction with supervised learning
				9.2.3.1 Purely data-driven methods
				9.2.3.2 Unrolling-based methods
			9.2.4 Deep learning MRI reconstruction with unsupervised learning
			9.2.5 Outlook
			9.2.6 Conclusion
		9.3 Deep learning for CT reconstruction
			9.3.1 Image domain post-processing
			9.3.2 Hybrid domain-based processing
			9.3.3 Iterative reconstruction via deep learning
			9.3.4 Direct reconstruction via deep learning
			9.3.5 Conclusion
		9.4 Deep learning for PET reconstruction
			9.4.1 Introduction
			9.4.2 Conventional PET reconstruction
			9.4.3 Deep learning-based algorithms in PET imaging
			9.4.4 Conclusion
		9.5 Discussion and conclusion
		References
4 Medical image segmentation, registration, and applications
	10 Dynamic inference using neural architecture search in medical image segmentation
		10.1 Introduction
		10.2 Related works
			10.2.1 Efficient ConvNet models for medical imaging
			10.2.2 Domain adaptation
			10.2.3 Neural architecture search
		10.3 Data oriented medical image segmentation
			10.3.1 Super-net design and training
			10.3.2 Data adaptation with super-net
		10.4 Experiments
		10.5 Ablation study
			10.5.1 Validation with single path or multiple paths
			10.5.2 Guided search and random search
			10.5.3 Training with single path or multiple paths
		10.6 Additional experiments
		10.7 Discussions
		References
	11 Multi-modality cardiac image analysis with deep learning
		11.1 Introduction
		11.2 Multi-sequence cardiac MRI based myocardial and pathology segmentation
			11.2.1 Introduction
			11.2.2 Methodology summary for challenge events
				11.2.2.1 MS-CMRSeg challenge: cardiac segmentation on late gadolinium enhancement MRI
				11.2.2.2 MyoPS: myocardial pathology segmentation from multi-sequence cardiac MRI
			11.2.3 Data and results
				11.2.3.1 Data
				11.2.3.2 Evaluation metrics
				11.2.3.3 Results from MS-CMRSeg challenge event
				11.2.3.4 Results from MyoPS challenge event
			11.2.4 Discussion and conclusion
		11.3 LGE MRI based left atrial scar segmentation and quantification
			11.3.1 Introduction
			11.3.2 Method
				11.3.2.1 LearnGC: atrial scar segmentation via potential learning in the graph-cut framework
				11.3.2.2 AtrialJSQnet: a new framework for joint segmentation and quantification of left atrium and scars incorporating spat...
			11.3.3 Data and results
				11.3.3.1 Data acquisition
				11.3.3.2 Gold standard and evaluation
				11.3.3.3 Performance of the proposed method
			11.3.4 Conclusion and future work
		11.4 Domain adaptation for cross-modality cardiac image segmentation
			11.4.1 Introduction
			11.4.2 Method
				11.4.2.1 DDFSeg: disentangle domain features for domain adaptation and segmentation
				11.4.2.2 CFDNet: characteristic function distance for unsupervised domain adaptation
				11.4.2.3 VarDA: domain adaptation via variational approximation
			11.4.3 Data and results
				11.4.3.1 Data
				11.4.3.2 Comparison study for DDFSeg
				11.4.3.3 Comparison study for CFDNet
				11.4.3.4 Comparison study for VarDA
			11.4.4 Conclusion
		References
	12 Deep learning-based medical image registration
		12.1 Introduction
		12.2 Deep learning-based medical image registration methods
			12.2.1 Deep learning-based medical image registration: supervised learning
			12.2.2 Deep learning-based medical image registration: unsupervised learning
			12.2.3 Deep learning-based medical image registration: weakly-supervised learning
			12.2.4 Deep learning-based registration: smoothness, consistency and other properties
		12.3 Deep learning-based registration with semantic information
		12.4 Concluding remarks
		References
	13 Data-driven learning strategies for biomarker detection and outcome prediction in Autism from task-based fMRI
		13.1 Introduction
		13.2 BrainGNN
			13.2.1 Notation
			13.2.2 Architecture overview
			13.2.3 ROI-aware graph convolutional layer
			13.2.4 ROI-topK pooling layer
			13.2.5 Readout layer
			13.2.6 Putting layers together
			13.2.7 Loss functions
			13.2.8 Experiments and results
			13.2.9 Brain-GNN implication for dynamic brain states
		13.3 LSTM-based recurrent neural networks for prediction in ASD
			13.3.1 Basic LSTM architecture for task-based fMRI
			13.3.2 Strategies for learning from small data sets
			13.3.3 Prediction of treatment outcome
		13.4 Causality and effective connectivity in ASD
			13.4.1 Dynamic causal modeling
			13.4.2 The effective connectome
			13.4.3 Overcoming long time series and noise with multiple shooting model driven learning (MS-MDL)
			13.4.4 Adjoint state method
			13.4.5 Multiple-shooting adjoint state method (MSA)
			13.4.6 Validation of MSA on toy examples
			13.4.7 Application to large-scale systems
			13.4.8 Apply MDL to identify ASD from fMRI data
			13.4.9 Improved fitting with ACA and AdaBelief
			13.4.10 Estimation of effective connectome and functional connectome
			13.4.11 Classification results for task fMRI
		13.5 Conclusion
		References
	14 Deep learning in functional brain mapping and associated applications
		14.1 Introduction
		14.2 Deep learning models for mapping functional brain networks
			14.2.1 Convolutional auto-encoder (CAE)
			14.2.2 Recurrent neural network (RNN)
			14.2.3 Deep belief network (DBN)
			14.2.4 Variational auto-encoder (VAE)
			14.2.5 Generative adversarial net (GAN)
		14.3 Spatio-temporal models of fMRI
			14.3.1 Deep sparse recurrent auto-encoder (DSRAE)
			14.3.2 Spatio-temporal attention auto-encoder (STAAE)
			14.3.3 Multi-head guided attention graph neural networks (multi-head GAGNNs)
			14.3.4 SCAAE and STCA
		14.4 Neural architecture search (NAS) of deep learning models on fMRI
			14.4.1 Hybrid spatio-temporal neural architecture search net (HS-NASNet)
			14.4.2 Deep belief network with neural architecture search (NAS-DBN)
			14.4.3 eNAS-DSRAE
			14.4.4 ST-DARTS
		14.5 Representing brain function as embedding
			14.5.1 Hierarchical interpretable autoencoder (HIAE)
			14.5.2 Temporally correlated autoencoder (TCAE)
			14.5.3 Potential applications
		14.6 Deep fusion of brain structure-function in brain disorders
			14.6.1 Deep cross-model attention network (DCMAT)
			14.6.2 Deep connectome
		14.7 Conclusion
		References
	15 Detecting, localizing and classifying polyps from colonoscopy videos using deep learning
		15.1 Introduction
		15.2 Literature review
			15.2.1 Polyp detection
			15.2.2 Polyp localization and classification
			15.2.3 Uncertainty and calibration
			15.2.4 Commercial systems
		15.3 Materials and methods
			15.3.1 Data sets
				15.3.1.1 Polyp detection
				15.3.1.2 Polyp localization and classification (with uncertainty and calibration)
			15.3.2 Methods
				15.3.2.1 Detection of frames with water jet sprays and feces
				15.3.2.2 Few-shot polyp detection
				15.3.2.3 Localization and classification of polyps
				15.3.2.4 Polyp classification uncertainty and calibration
		15.4 Results and discussion
			15.4.1 Polyp detection experiments
			15.4.2 Polyp localization and classification experiments
			15.4.3 Uncertainty estimation and calibration experiments
			15.4.4 System running time
		15.5 Conclusion
		References
	16 OCTA segmentation with limited training data using disentangled representation learning
		16.1 Introduction
		16.2 Related work
		16.3 Method
			16.3.1 Overview
			16.3.2 Conditional variational auto-encoder
			16.3.3 Anatomy-contrast disentanglement
			16.3.4 Semi-supervised segmentation
			16.3.5 Data sets and manual delineations
			16.3.6 Foveal avascular zone segmentation
		16.4 Discussion and conclusion
		References
5 Others
	17 Considerations in the assessment of machine learning algorithm performance for medical imaging
		17.1 Introduction
			17.1.1 Medical devices, software as a medical device and intended use
		17.2 Data sets
			17.2.1 General principles
			17.2.2 Independence of training and test data sets
			17.2.3 Reference standard
			17.2.4 Image collection and fairness
			17.2.5 Image and data quality
			17.2.6 Discussion
		17.3 Endpoints
			17.3.1 Metrics
				17.3.1.1 Segmentation
				17.3.1.2 Classification (e.g., computer-aided diagnosis, or CADx, algorithms)
				17.3.1.3 Detection (e.g., computer-aided detection, or CADe, algorithms)
				17.3.1.4 Triage, prioritization and notification (e.g., computer-aided triage, or CADt, algorithms)
				17.3.1.5 Image generation, noise reduction, reconstruction
				17.3.1.6 Quantitative imaging tools
				17.3.1.7 Discussion
		17.4 Study design
			17.4.1 Transportability
			17.4.2 Assessment studies for ML algorithms in medical imaging
				17.4.2.1 Standalone performance assessment
				17.4.2.2 Clinical performance assessment
				17.4.2.3 Prospective vs. retrospective studies
			17.4.3 Discussion
		17.5 Bias
			17.5.1 Bias and precision
			17.5.2 Bias and generalizability
			17.5.3 Types and sources of bias in pre-deployment performance evaluation studies of ML algorithms in medical imaging
				17.5.3.1 Case collection
				17.5.3.2 Over-fitting, train-test contamination and data leakage
				17.5.3.3 Reference standard
				17.5.3.4 Study endpoints and metrics
			17.5.4 Discussion
		17.6 Limitations and future considerations
		17.7 Conclusion
		References
Index
Back Cover