Artificial Intelligence in Medicine: Technical Basis and Clinical Applications

Artificial Intelligence in Medicine
Copyright
Dedication
Contents
List of contributors
Foreword
	References
Preface
Acknowledgments
1 Artificial intelligence in medicine: past, present, and future
	1.1 Introduction
	1.2 A brief history of artificial intelligence and its applications in medicine
	1.3 How intelligent is artificial intelligence?
	1.4 Artificial intelligence, machine learning, and precision medicine
	1.5 Algorithms and models
	1.6 Health data sources and types
	1.7 The promise
	1.8 The challenges
		1.8.1 Quality and completeness of training data
		1.8.2 Trust and performance: the case for model interpretability
		1.8.3 Beyond performance and interpretability: causality
		1.8.4 Defining the question, measuring real-world impact
		1.8.5 Maximizing information gain across modalities, tasks, populations, and time
		1.8.6 Quality assessment and expert supervision
	1.9 Making it a reality: integrating artificial intelligence into the human workforce of a learning health system
	References
2 Artificial intelligence in medicine: Technical basis and clinical applications
	2.1 Introduction
	2.2 Technology used in clinical artificial intelligence tools
		2.2.1 Elements of artificial intelligence algorithms
			2.2.1.1 Activation functions
			2.2.1.2 Fully connected layer
			2.2.1.3 Dropout
			2.2.1.4 Residual blocks
			2.2.1.5 Initialization
			2.2.1.6 Convolution and transposed convolution
			2.2.1.7 Inception layers
		2.2.2 Popular artificial intelligence software architectures
			2.2.2.1 Neural networks and fully connected networks
			2.2.2.2 Convolutional neural networks
			2.2.2.3 U-Nets and V-Nets
			2.2.2.4 DenseNets
			2.2.2.5 Generative adversarial networks
			2.2.2.6 Hybrid generative adversarial network designs
	2.3 Clinical applications
		2.3.1 Applications of regression
			2.3.1.1 Bone age
			2.3.1.2 Brain age
		2.3.2 Applications of segmentation
		2.3.3 Applications of classification
			2.3.3.1 Detection of disease
			2.3.3.2 Diagnosis of disease class
			2.3.3.3 Prediction of molecular markers
			2.3.3.4 Prediction of outcome and survival
		2.3.4 Deep learning for improved image reconstruction
	2.4 Future directions
		2.4.1 Understanding what artificial intelligence “sees”
		2.4.2 Workflow
	2.5 Conclusion
	References
3 Deep learning for biomedical videos: perspective and recommendations
	3.1 Introduction
	3.2 Video datasets
	3.3 Semantic segmentation
	3.4 Object detection and tracking
	3.5 Motion classification
	3.6 Future directions and conclusion
	References
4 Biomedical imaging and analysis through deep learning
	4.1 Introduction
	4.2 Tomographic image reconstruction
		4.2.1 Foundation
		4.2.2 Computed tomography
		4.2.3 Magnetic resonance imaging
		4.2.4 Other imaging modalities
	4.3 Image segmentation
		4.3.1 Introduction
		4.3.2 Localization versus segmentation
		4.3.3 Fully convolutional networks
		4.3.4 Regions with convolutional neural network features
		4.3.5 A priori information
		4.3.6 Manual labeling
		4.3.7 Semisupervised and unsupervised approaches
	4.4 Image registration
		4.4.1 Single-modality image registration
		4.4.2 Multimodality image registration
	4.5 Deep-learning-based radiomics
		4.5.1 Detection
		4.5.2 Characterization and diagnosis
		4.5.3 Prognosis
		4.5.4 Assessment and prediction of response to treatment
		4.5.5 Assessment of risk of future cancer
	4.6 Summary and outlook
	References
5 Expert systems in medicine
	5.1 Introduction
	5.2 A brief history
	5.3 Methods
		5.3.1 Expert system architecture
		5.3.2 Knowledge representation and management
		5.3.3 Uncertainty, probabilistic reasoning, fuzzy logic
			5.3.3.1 Uncertainty
			5.3.3.2 Probabilistic reasoning
			5.3.3.3 Fuzzy logic
	5.4 Applications
		5.4.1 Computer-assisted diagnosis
		5.4.2 Computer-assisted therapy
		5.4.3 Medication alert systems
		5.4.4 Reminder systems
	5.5 Challenges
		5.5.1 Workflow integration
		5.5.2 Clinician acceptance and alert fatigue
		5.5.3 Knowledge maintenance
		5.5.4 Standard, transferability, and interoperability
	5.6 Future directions
	References
6 Privacy-preserving collaborative deep learning methods for multiinstitutional training without sharing patient data
	6.1 Introduction
	6.2 Variants of distributed learning
		6.2.1 Model ensembling
		6.2.2 Cyclical weight transfer
		6.2.3 Federated learning
		6.2.4 Split learning
	6.3 Handling data heterogeneity
	6.4 Protecting patient privacy
	6.5 Publicly available software
	6.6 Conclusion
	References
7 Analytics methods and tools for integration of biomedical data in medicine
	7.1 The rise of multimodal data in biology and medicine
		7.1.1 The emergence of various sequencing techniques
			7.1.1.1 Bulk sequencing
			7.1.1.2 Single-cell sequencing
		7.1.2 The increasing need for combining images and omics in clinical applications
			7.1.2.1 Various modalities of images in clinics
			7.1.2.2 The rise of radiomics: combine medical images with omics
		7.1.3 The availability of large-scale public health data
	7.2 The challenges in multimodal data—problems with learning from multiple sources of data
		7.2.1 The imperfect generation of single-cell data
			7.2.1.1 The complementariness of various sources of data
		7.2.2 The issues of generalizability of machine learning
	7.3 Machine learning algorithms in integrating medical and biological data
		7.3.1 Genome-wide data integration with machine learning
			7.3.1.1 How to integrate various omics for cancer subtyping
			7.3.1.2 How to integrate single-cell multiomics for precision medicine
		7.3.2 Data integration beyond omics—an example with cardiovascular diseases
			7.3.2.1 How to integrate various image modalities such as magnetic resonance imaging computed tomography scans
			7.3.2.2 How to better the diagnosis by linking images with electrocardiograms
		7.3.3 Multimodal decision-making in clinical settings
	7.4 Future directions
	References
8 Electronic health record data mining for artificial intelligence healthcare
	8.1 Introduction
	8.2 Overview of the electronic health record
		8.2.1 History of the electronic health record
		8.2.2 Core functions of an electronic health record
		8.2.3 Electronic health record ontologies and data standards
	8.3 Clinical decision support
		8.3.1 Healthcare primed for clinical decision support
	8.4 Areas of artificial intelligence augmentation for electronic health records
		8.4.1 Artificial intelligence to improve data entry and extraction
		8.4.2 Optimizing care
		8.4.3 Predictions
		8.4.4 Hospital outcomes
		8.4.5 Sepsis and infections
		8.4.6 Oncology
	8.5 Limitations of artificial intelligence and next steps
	References
9 Roles of artificial intelligence in wellness, healthy living, and healthy status sensing
	9.1 Introduction
	9.2 Diet
	9.3 Fitness and physical activity
	9.4 Sleep
	9.5 Sexual and reproductive health
	9.6 Mental health
	9.7 Behavioral factors
	9.8 Environmental and social determinants of health
	9.9 Remote screening tools
	9.10 Conclusion
	References
10 The growing significance of smartphone apps in data-driven clinical decision-making: Challenges and pitfalls
	10.1 Introduction
	10.2 Distribution of apps in the field of medicine
	10.3 Distribution of apps over different locations
	10.4 Reporting applications development approaches
	10.5 Decision-support modalities
	10.6 Camera-based apps
	10.7 Guideline/algorithm applications
	10.8 Predictive modeling applications
	10.9 Sensor-linked apps
	10.10 Discussion
	10.11 Summary
	References
11 Artificial intelligence for pathology
	11.1 Introduction
	11.2 Deep neural networks
		11.2.1 Convolutional neural networks
		11.2.2 Fully convolutional networks
		11.2.3 Generative adversarial networks
		11.2.4 Stacked autoencoders
		11.2.5 Recurrent neural networks
	11.3 Deep learning in pathological image analysis
		11.3.1 Image classification
			11.3.1.1 Image-level classification
			11.3.1.2 Object-level classification
		11.3.2 Object detection
			11.3.2.1 Detection of particular types of objects
			11.3.2.2 Detection of objects without category labeling
			11.3.2.3 Detection of objects with category labeling
		11.3.3 Image segmentation
			11.3.3.1 Nucleus/cell segmentation
			11.3.3.2 Gland segmentation
			11.3.3.3 Segmentation of other biological structures or tissues
		11.3.4 Stain normalization
		11.3.5 Image superresolution
		11.3.6 Computer-aided diagnosis
		11.3.7 Others
	11.4 Summary
		11.4.1 Open challenges and future directions of deep learning in pathology image analysis
			11.4.1.1 Quality control
			11.4.1.2 High image dimension
			11.4.1.3 Object crowding
			11.4.1.4 Data annotation issues
			11.4.1.5 Integration of different types of input data
		11.4.2 Outlook of clinical adoption of artificial intelligence
			11.4.2.1 Potential applications
			11.4.2.2 Barriers to clinical adoption
				11.4.2.2.1 Lagging adoption of digital pathology
				11.4.2.2.2 Lack of standards for interfacing AI to clinical systems
				11.4.2.2.3 Regulatory concerns
				11.4.2.2.4 Computational requirements
				11.4.2.2.5 Algorithm explainability
				11.4.2.2.6 Pathologists’ skepticism
	References
12 The potential of deep learning for gastrointestinal endoscopy—a disruptive new technology
	12.1 Introduction
	12.2 Applications of artificial intelligence in video capsule endoscopy
		12.2.1 Introduction
		12.2.2 Decreasing read time
		12.2.3 Anatomical landmark identification
		12.2.4 Improving sensitivity
		12.2.5 Recent developments
	12.3 Applications of artificial intelligence in upper endoscopy
		12.3.1 Introduction
		12.3.2 Esophageal cancer
		12.3.3 Gastric cancer
		12.3.4 Upper endoscopy quality
		12.3.5 Future directions
	12.4 Applications of artificial intelligence in colonoscopy
		12.4.1 Introduction
		12.4.2 Cecal intubation rate and cecal intubation time
		12.4.3 Withdrawal time
		12.4.4 Boston Bowel Prep Scoring
		12.4.5 Polyp detection
		12.4.6 Polyp size
		12.4.7 Polyp morphology
		12.4.8 Polyp pathology
		12.4.9 Tools
		12.4.10 Mayo endoscopic subscore
	12.5 Conclusion
	12.6 Future directions
	References
13 Lessons learnt from harnessing deep learning for real-world clinical applications in ophthalmology: detecting diabetic r...
	13.1 Introduction
	13.2 Historical artificial intelligence for diabetic retinopathy
	13.3 Deep learning era
	13.4 Lessons from interpreting and evaluating studies
	13.5 Important factors for real-world usage
	13.6 Regulatory approvals and further validation
	13.7 Toward patient impact and beyond
	13.8 Summary
	Conflict of interest
	References
14 Artificial intelligence in radiology
	14.1 Introduction
	14.2 Thoracic applications
		14.2.1 Pulmonary analysis in chest X-ray
		14.2.2 Pulmonary analysis in computerized tomography
			14.2.2.1 Lung, lobe, and airway segmentation
			14.2.2.2 Interstitial lung disease pattern recognition
	14.3 Abdominal applications
		14.3.1 Pancreatic cancer analysis in computerized tomography and magnetic resonance imaging
			14.3.1.1 Pancreas segmentation in computerized tomography and magnetic resonance imaging
			14.3.1.2 Pancreatic tumor segmentation and detection in computerized tomography and magnetic resonance imaging
			14.3.1.3 Prediction and prognosis with pancreatic cancer imaging
		14.3.2 AI in other abdominal imaging
	14.4 Pelvic applications
	14.5 Universal lesion analysis
		14.5.1 DeepLesion dataset
		14.5.2 Lesion detection and classification
		14.5.3 Lesion segmentation and quantification
		14.5.4 Lesion retrieval and mining
	14.6 Conclusion
	References
15 Artificial intelligence and interpretations in breast cancer imaging
	15.1 Introduction
	15.2 Artificial intelligence in decision support
	15.3 Artificial intelligence in breast cancer screening
	15.4 Artificial intelligence in breast cancer risk assessment: density and parenchymal pattern
	15.5 Artificial intelligence in breast cancer diagnosis and prognosis
	15.6 Artificial intelligence for treatment response, risk of recurrence, and cancer discovery
	15.7 Conclusion and discussion
	References
16 Prospect and adversity of artificial intelligence in urology
	16.1 Introduction
	16.2 Basic examinations in urology
		16.2.1 Urinalysis and urine cytology
		16.2.2 Ultrasound examination
	16.3 Urological endoscopy
		16.3.1 Cystoscopy and transurethral resection of the bladder
		16.3.2 Ureterorenoscopy
	16.4 Andrology
	16.5 Diagnostic imaging
		16.5.1 Prostate
		16.5.2 Kidney
		16.5.3 Ureter and bladder
	16.6 Robotic surgery
		16.6.1 Preoperative preparation
		16.6.2 Navigation
		16.6.3 Automated maneuver
	16.7 Risk prediction
	16.8 Future direction
	References
17 Meaningful incorporation of artificial intelligence for personalized patient management during cancer: Quantitative imag...
	17.1 Introduction
		17.1.1 Workflow
			17.1.1.1 Data acquisition
			17.1.1.2 Preprocessing
			17.1.1.3 Model building and evaluation
			17.1.1.4 Inference
		17.1.2 Meaningful incorporation of machine learning
	17.2 Quantitative imaging
		17.2.1 Brief overview of the physics of imaging modalities
		17.2.2 Use of artificial intelligence in different stages of a quantitative imaging workflow
	17.3 Risk assessment in cancer
	17.4 Therapeutic outcome prediction
		17.4.1 Chemotherapy
		17.4.2 Radiation therapy
	17.5 Using artificial intelligence meaningfully
	17.6 Summary
	References
18 Artificial intelligence in oncology
	Abbreviations
	18.1 Introduction
	18.2 Electronic health records and clinical data warehouse
		18.2.1 Data reuse for research purposes
		18.2.2 Data reuse and artificial intelligence
		18.2.3 Data reuse for patient care
	18.3 Artificial intelligence applications for imaging in oncology
		18.3.1 Applications in oncology for diagnosis and prediction
			18.3.1.1 Computer vision and image analysis
			18.3.1.2 Radiomics: data-driven biomarker discovery
			18.3.1.3 Artificial intelligence–assisted diagnosis and monitoring in oncology
			18.3.1.4 Treatment outcome assessment and prediction
		18.3.2 Applications in oncology to improve exam quality and workflow
			18.3.2.1 Improvement of image acquisition
			18.3.2.2 Image segmentation
			18.3.2.3 Improved workflow
			18.3.2.4 Interventional radiology
	18.4 Artificial intelligence applications for radiation oncology
		18.4.1 Treatment planning
			18.4.1.1 Segmentation
				18.4.1.1.1 Brain
				18.4.1.1.2 Head and neck
				18.4.1.1.3 Lung
				18.4.1.1.4 Abdomen
				18.4.1.1.5 Pelvis
			18.4.1.2 Dosimetry
		18.4.2 Outcome prediction
			18.4.2.1 Treatment response
				18.4.2.1.1 Brain
				18.4.2.1.2 Head and neck
				18.4.2.1.3 Lung
				18.4.2.1.4 Esophagus
				18.4.2.1.5 Rectum
			18.4.2.2 Toxicity
	18.5 Future directions
	References
19 Artificial intelligence in cardiovascular imaging
	19.1 Introduction
	19.2 Types of machine learning
	19.3 Deep learning
	19.4 Role of artificial intelligence in echocardiography
	19.5 Role of artificial intelligence computed tomography
	19.6 Role of artificial intelligence in nuclear cardiology
	19.7 Role of artificial intelligence in cardiac magnetic resonance imaging
	19.8 Role of artificial intelligence in electrocardiogram
	19.9 The role of artificial intelligence in large databases
	19.10 Our views on machine learning
	19.11 Conclusion
	References
20 Artificial intelligence as applied to clinical neurological conditions
	20.1 Introduction to artificial intelligence in neurology
	20.2 Integration with clinical workflow
		20.2.1 Diagnosis
		20.2.2 Risk prognostication
		20.2.3 Surgical planning
		20.2.4 Intraoperative guidance and enhancement
		20.2.5 Neurophysiological monitoring
		20.2.6 Clinical decision support
		20.2.7 Theoretical neurological artificial intelligence research
	20.3 Currently adopted methods in clinical use
	20.4 Challenges
		20.4.1 Data volume
		20.4.2 Data quality
		20.4.3 Generalizability
		20.4.4 Interpretability
		20.4.5 Legal
		20.4.6 Ethical
	20.5 Conclusion
	References
21 Harnessing the potential of artificial neural networks for pediatric patient management
	21.1 Introduction
	21.2 Applications of artificial intelligence in diagnosis and prognosis
		21.2.1 Prematurity
		21.2.2 Childhood brain tumors
		21.2.3 Epilepsy and seizure disorders
		21.2.4 Autism spectrum disorder
		21.2.5 Mood disorders and psychoses
		21.2.6 Hydrocephalus
		21.2.7 Traumatic brain injury
		21.2.8 Molecular mechanisms of disease
		21.2.9 Other disease entities
	21.3 Transition to treatment decision-making using artificial intelligence
	21.4 Future directions
	References
22 Artificial intelligence–enabled public health surveillance—from local detection to global epidemic monitoring and control
	22.1 Introduction
	22.2 Artificial intelligence–enhanced data analysis for outbreak detection and early warning
		22.2.1 Analyzing data collected from the physical world
		22.2.2 Analyzing data from the cyberspace
		22.2.3 From syndromic to pre-syndromic disease surveillance: A safety net for public health
	22.3 Artificial intelligence–enhanced prediction in support of public health surveillance
		22.3.1 Time series prediction based on dependent variables
		22.3.2 Time series prediction based on dependent and independent variables
	22.4 Artificial intelligence–based infectious disease transmission modeling and response assessment
		22.4.1 Modeling disease transmission dynamics based on machine learning and complex networks
		22.4.2 Modeling disease transmission dynamics based on multiagent modeling
	22.5 Internet-based surveillance systems for global epidemic monitoring
	22.6 Conclusion
	References
23 Regulatory, social, ethical, and legal issues of artificial intelligence in medicine
	23.1 Introduction
	23.2 Ethical issues in data acquisition
		23.2.1 Ethical issues arising from each type of data source
			23.2.1.1 Ethical issues common to all data sources: Privacy and confidentiality
			23.2.1.2 Ethical issues unique to each data source: Issues of consent
				23.2.1.2.1 Issues of consent with data from research repositories
				23.2.1.2.2 Return of results from research repositories
				23.2.1.2.3 Issues of consent with clinical or public health data
				23.2.1.2.4 Incidental or secondary findings in clinical or public health data
				23.2.1.2.5 Issues of consent with nonclinically collected data
		23.2.2 Future directions: Toward a new model of data stewardship
	23.3 Application problems: Problems with learning from the data
		23.3.1 Values embedded in algorithm design
		23.3.2 Biases in the data themselves
		23.3.3 Biases in the society in which the data occurs
		23.3.4 Issues of implementation
		23.3.5 Summary
	23.4 Issues in regulation
		23.4.1 Challenges to existing regulatory frameworks
		23.4.2 Challenges in oversight and regulation of artificial intelligence used in healthcare
		23.4.3 Regulation of safety and efficacy
		23.4.4 Privacy and data protection
		23.4.5 Transparency, liability, responsibility, and trust
	23.5 Implications for the ethos of medicine
	23.6 Future directions
	References
24 Industry perspectives and commercial opportunities of artificial intelligence in medicine
	24.1 Introduction
	24.2 Exciting growth of artificial intelligence in medicine
	24.3 A framework on development of artificial intelligence in medicine
		24.3.1 The power of public attention and funding
		24.3.2 Technology relies on continuous innovation
		24.3.3 Practical applications bring the innovation to the real world
		24.3.4 Market adoption defines the success
		24.3.5 Apply the framework to the current and future market
		24.3.6 Patient privacy
		24.3.7 Approving a moving target
		24.3.8 Accountability and transparency
	24.4 Business opportunity of artificial intelligence in medicine
	References
25 Outlook of the future landscape of artificial intelligence in medicine and new challenges
	25.1 Overview of artificial intelligence in health care
		25.1.1 Models dealing with input and output data from the same domain
		25.1.2 Deep learning as applied to problems with input and output related by physical/mathematical law
		25.1.3 Models with input and output data domains related by empirical evidence or measurements
		25.1.4 Applications beyond traditional indications
	25.2 Challenges ahead and issues relevant to the practical implementation of artificial intelligence in medicine
		25.2.1 Technical challenges
		25.2.2 Data, data curation, and sharing
		25.2.3 Data and potential bias in artificial intelligence
		25.2.4 Workflow and practical implementation
		25.2.5 Clinical tests
		25.2.6 Economical, political, social, ethical, and legal aspects
		25.2.7 Education and training
	25.3 Future directions and opportunities
	25.4 Summary and outlook
	References
Index