Multimodal AI in Healthcare. A Paradigm Shift in Health Intelligence

Preface
Contents
Contributors
Abbreviations
Multimodal Artificial Intelligence: Next Wave of Innovation in Healthcare and Medicine
	1 Introduction
	2 Clinical and Biomedical Applications of Multimodal AI and Data Science
	3 Advances in AI Technologies and Data Analytics in Healthcare
	References
Unsupervised Numerical Reasoning to Extract Phenotypes from Clinical Text by Leveraging External Knowledge
	1 Introduction
	2 Related Work
	3 Methodology
		3.1 External Knowledge
		3.2 Number and Lexical Candidates Extraction
		3.3 Contextualized Embeddings for Numeric Entities and Lexical Candidates
		3.4 Embedding Similarity and Deterministic HPO Assignment
	4 Experiment Design
		4.1 Datasets
		4.2 Implementation Details
		4.3 Baselines and Evaluation Methods
	5 Results and Discussion
		5.1 Quantitative Analysis
		5.2 Qualitative Analysis
		5.3 Ablation Studies
	6 Conclusions and Future Works
	References
Domain-specific Language Pre-training for Dialogue Comprehension on Clinical Inquiry-Answering Conversations
	1 Introduction
	2 Domain-Specific Language Pre-training
		2.1 Conversation-based Sample Construction
		2.2 Experiment Setup of Pre-training
	3 Dialogue Comprehension on Clinical Inquiry-Answering Conversations
		3.1 Task Definition
		3.2 Clinical Dialogue Corpus
		3.3 Baseline Models
		3.4 Training Configuration
		3.5 Evaluation: Comparison with Baselines
		3.6 Evaluation in Low-Resource Scenarios
		3.7 Evaluation: Pre-training Scheme Comparison
	4 Conclusions
	References
Clinical Dialogue Transcription Error Correction Using Seq2Seq Models
	1 Introduction
	2 Related Work
	3 Clinical Dialogue Transcription
		3.1 Gastrointestinal Clinical Dialogue Dataset
	4 Methods
		4.1 General Purpose Base Language Models
		4.2 PubMed Gastrointestinal Dataset
		4.3 Fine-Tune Using Self-Supervision
	5 Evaluation
		5.1 Performance Metric
		5.2 Comparison of Base Language Models
		5.3 Comparison of Fine-Tuned Language Models
	6 Discussion
	7 Conclusion
	References
Customized Training of Pretrained Language Models to Detect Post Intents in Online Health Support Groups
	1 Introduction
	2 Background and Related Work
	3 Tweet2Quit Dataset
		3.1 Data Collection
		3.2 Identification of the Intents to Annotate
		3.3 Reliability
		3.4 Annotation Process
	4 Models
		4.1 Random Forest (Baseline)
		4.2 Pretrained Language Models
	5 Adapting to the Labels\' Relationships
		5.1 Customized Loss Functions
	6 Experiments
		6.1 Experimental Setup
		6.2 Pretrained Language Models
		6.3 Loss Functions and Adjusted Metrics
	7 Conclusion and Discussion
	References
EXPECT-NLP: An Integrated Pipeline and User Interface for Exploring Patient Preferences Directly from Patient-Generated Text
	1 Introduction
	2 Aspect Based Sentiment Analysis
	3 Data Model
	4 Aspect Extraction Results
	5 EXPECT-NLP Interface
	6 Real-World Use Cases
	7 Conclusion and Future Work
	References
Medication Error Detection Using Contextual Language Models
	1 Introduction
	2 Proposed Methodology
		2.1 Problem Formalization
		2.2 Contextual Language Models
	3 Experimentation
		3.1 Dataset Generation
		3.2 ASR Implementation
		3.3 Results and Analysis
	4 Conclusions
	References
Latent Representation Weights Learning of the Indefinite Length of Views for Conception Diagnosis
	1 Introduction
	2 Integration and Methods
		2.1 Latent Representation Weight Learning
		2.2 Optimization
	3 Results and Discussion
		3.1 Data Collection
		3.2 Experimental Setup
		3.3 Comparison with the Baseline
	4 Conclusion
	References
Phenotyping with Positive Unlabelled Learning for Genome-Wide Association Studies
	1 Introduction
	2 Related Work
	3 Methods
		3.1 Problem Formulation
		3.2 Anchor Learning
		3.3 Using BERT as Anchor Classifier
		3.4 Baselines
		3.5 Anchor Performance Metrics
		3.6 Hyperparameters
	4 Experiments and Results
		4.1 UK Biobank data
		4.2 Anchor Classifier Performance and Robustness to Control Noise
		4.3 Evaluating Phenotypes with GWAS
	5 Discussion
	References
Out-of-Distribution Detection for Medical Applications: Guidelines for Practical Evaluation
	1 Introduction
	2 Related Work
	3 Methods
		3.1 Considerations
		3.2 Designing OOD Tests
		3.3 Dataset
		3.4 Models
		3.5 AUC Score of OOD Detection
	4 Experimental Results
	5 Discussion
	References
A Robust System to Detect and Explain Public Mask Wearing Behavior
	1 Introduction
	2 Background and Related Work
		2.1 Face Mask Detection
		2.2 Explanation
	3 Methodology
		3.1 Detection Architecture
		3.2 Explanation Architecture
		3.3 Transfer Learning
		3.4 Individual and Aggregate Explanation
	4 Experimentation
		4.1 Dataset
		4.2 Augmentation
		4.3 Experiment Setup
	5 Evaluation
		5.1 Qualitative Results
		5.2 Quantitative Results
		5.3 Sanity Check
	6 Conclusion
	References
A Federated Cox Model with Non-proportional Hazards
	1 Introduction
	2 Background and Related Work
	3 Model
	4 Experiments
		4.1 Datasets
		4.2 Setup
		4.3 Results
	5 Conclusion
	Appendix A Additional Figures
	References
A Step Towards Automated Functional Assessment of Activities of Daily Living
	1 Introduction
	2 Related Works
	3 Dataset
	4 Proposed Approach
	5 Training and Evaluation
	6 Experiments and Results
		6.1 Ablation Study
	7 Conclusion
	References
The Interpretation of Deep Learning Based Analysis of Medical Images—An Examination of Methodological and Practical Challenges Using Chest X-ray Data
	1 Introduction
		1.1 Previous Work
	2 Performance Quantification
	3 Model Training
	4 Analysis
		4.1 Understanding Data and Findings Interpretation
	5 Summary and Conclusions
	References
Predicting Drug Functions from Adverse Drug Reactions by Multi-label Deep Neural Network
	1 Introduction
	2 Related Work
	3 Proposed Methodology and Working Architecture
		3.1 Description of the Datasets
		3.2 Problem Statement
		3.3 Proposed Methodology
	4 Performance Evaluation of Experiments
		4.1 Performance Measurement
		4.2 Experimental Results and Discussions
	5 Conclusion
	References
Pattern Discovery in Physiological Data with Byte Pair Encoding
	1 Introduction
	2 Related Work
	3 Method
		3.1 PAA Transformation
		3.2 Discretization
		3.3 Identifying Patterns
		3.4 Handling Consecutive Identical Symbols
		3.5 Post Processing
		3.6 Extension to Multivariate Series
		3.7 Classification/Regression
		3.8 Hyper-Parameter Tuning
	4 Data
	5 Results
		5.1 Computation Speed
		5.2 Interpretability
	6 Conclusion
	References
Predicting ICU Admissions for Hospitalized COVID-19 Patients with a Factor Graph-based Model
	1 Introduction
	2 Model
		2.1 Model Overview
		2.2 Variable Selection
		2.3 Factor Function Construction
		2.4 Inference Algorithms
	3 Experimental Setup
		3.1 Dataset
		3.2 Model Evaluation
	4 Results
		4.1 Predictive Biomarkers
		4.2 Model Validation
	5 Limitations and Future Work
	6 Conclusion
	References
Semantic Network Analysis of COVID-19 Vaccine Related Text from Reddit
	1 Introduction
	2 Data
	3 Methods
	4 Results
	5 Discussion
	6 Conclusion
	References
Towards Providing Clinical Insights on Long Covid from Twitter Data
	1 Introduction
	2 Related Work
		2.1 Social Media Platform for COVID-19
		2.2 Clinical Information Extraction
		2.3 Interpretability
	3 Dataset
		3.1 Data Acquisition and De-identification
		3.2 Filtering Long COVID Self Reports
	4 Methodology
	5 Results and Discussion
	References
Predicting Infections in the Covid-19 Pandemic—Lessons Learned
	1 Introduction
	2 Related Work
	3 Learning-based Models for Predicting the Number of Infections
	4 Experiments
	5 Conclusion
	References
Improving Radiology Report Generation with Adaptive Attention
	1 Introduction
	2 Related Work
	3 Method
		3.1 Encoder-Decoder Framework
		3.2 Multi-Head Adaptive Attention
		3.3 Choice of Pretrained Visual Extractor
	4 Experiments and Results
		4.1 Datasets
		4.2 Model Evaluation
		4.3 Model Development and Hyper Parameter Tuning
		4.4 Ablation Study
		4.5 Quantitative Evaluation
		4.6 Qualitative Analysis
	5 Conclusion
	References
Instantaneous Physiological Estimation Using Video Transformers
	1 Introduction
	2 Related Work
		2.1 Video Based Physiology Extraction
		2.2 Transformers
	3 Methods
		3.1 Optical Basis of Video-Based Bio-Signal Extraction
		3.2 Video Transformer for Physiological Estimation
		3.3 Loss Formulation
	4 Results
		4.1 Implementation Details
		4.2 Datasets and Evaluation Protocol
		4.3 Heart Rate Estimation Results
		4.4 Spatial Attention Mask
		4.5 Respiration Rate Estimation Results
	5 Conclusion
	References
Automated Vision-Based Wellness Analysis for Elderly Care Centers
	1 Introduction
	2 Related Work
	3 Proposed Wellness Analysis System
		3.1 Facial Analysis
		3.2 Activity Analysis
		3.3 Interaction Analysis
		3.4 Analysis of Long-Term Pattern and Trend
	4 Evaluation
		4.1 Data Collection
		4.2 Results
	5 Conclusion
	References
Efficient Extraction of Pathologies from C-Spine Radiology Reports Using Multi-task Learning
	1 Introduction
	2 Datasets
	3 Description of the Workflow
	4 Methods
	5 Results
	6 Empirical Evidence Behind MultiTasking Models
	7 Conclusion
	References
Benchmarking Uncertainty Quantification on Biosignal Classification Tasks Under Dataset Shift
	1 Introduction
	2 Related Work
	3 Uncertainty Quantification Approaches
	4 Bencmark Tasks and Experiemnts
		4.1 Biosignal Classification Tasks
		4.2 Dataset Shift and Evaluation Protocol
		4.3 Metrics
	5 Results and Analysis
	6 Conclusions
	References
Mining Adverse Drug Reactions from Unstructured Mediums at Scale
	1 Introduction
	2 Related Work
	3 Approach
		3.1 Classification
		3.2 Named Entity Recognition
		3.3 Relation Extraction
	4 Experimental Setup
		4.1 Datasets
		4.2 Experiments
		4.3 Results
	5 Conclusion
	References
A Graph-based Imputation Method for Sparse Medical Records
	1 Introduction
	2 Method
	3 Results
	4 Conclusion
	References
Using Nursing Notes to Predict Length of Stay in ICU for Critically Ill Patients
	1 Introduction
	2 Related Works
	3 Dataset
	4 Proposed Architecture of ICU LOS Prediction
		4.1 Transformer Based Document Representation
		4.2 TF-IDF Vector
		4.3 Severity of Illness (SOI) Score
		4.4 Training the Model
	5 Evaluation of the Proposed Predictive Model
		5.1 Results and Discussions
	6 Conclusion
	References
Automatic Classification of Dementia Using Text and Speech Data
	1 Introduction
	2 Related Work
	3 Materials and Methods
		3.1 Dataset: DementiaBank Pitt and WLS Corpora
		3.2 Data Pre-processing
		3.3 Feature Extraction
	4 Ensemble Model
		4.1 Deep Learning Model
		4.2 Model Evaluation
	5 Discussion
	6 Conclusion
	References
Unified Tensor Network for Multimodal Dementia Detection
	1 Introduction
	2 Background
		2.1 Alzheimer\'s Disease and Dementia
		2.2 Multimodal Machine Learning
	3 Unified Tensor Analysis Pipeline
		3.1 Dataset
		3.2 Preprocessing
		3.3 Representation
		3.4 Fusion
		3.5 Evaluation
	4 Experiments and Results
	5 Conclusion
	References