AI Assurance: Towards Trustworthy, Explainable, Safe, and Ethical AI

Front Cover
AI Assurance
Copyright
Contents
Contributors
A note by the editors
A note on the book cover
Foreword 1
Foreword 2
Foreword 3
Part 1 Foundations of AI assurance
	1 An introduction to AI assurance
		1.1 Motivation and overview
			1.1.1 Book content
		1.2 The need for new assurance methods
		1.3 Conclusion
		References
	2 Setting the goals for ethical, unbiased, and fair AI
		2.1 Introduction and background
			2.1.1 Value-loading
				2.1.1.1 The control problem
				2.1.1.2 The value-loading problem
			2.1.2 Human-compatible AI
				2.1.2.1 Cooperative inverse reinforcement learning
			2.1.3 The alignment problem
				2.1.3.1 The role of training data
				2.1.3.2 The objective function
			2.1.4 AI assurance: a formal framework
		2.2 Ethical AI but… how?
			2.2.1 Three normative theories: a brief outline
				2.2.1.1 Deontological ethics: duties
				2.2.1.2 Utilitarianism
				2.2.1.3 Virtue ethics
			2.2.2 The implementation problem
				2.2.2.1 Top-down approach
				2.2.2.2 Bottom-up approach
			2.2.3 Intentional statements and reward functions
				2.2.3.1 The problem of specification
				2.2.3.2 Moral uncertainty
		2.3 Conclusion
		References
	3 An overview of explainable and interpretable AI
		3.1 Introduction
		3.2 Methods and materials
			3.2.1 Statistics and evaluation metrics
				3.2.1.1 Mean
				3.2.1.2 Median
				3.2.1.3 Standard deviation and variance
				3.2.1.4 R2
				3.2.1.5 Accuracy
				3.2.1.6 Precision, recall, and F1
			3.2.2 Shape metrics
				3.2.2.1 Area and perimeter
				3.2.2.2 Shape proportion and encircled image-histograms
				3.2.2.3 FD
				3.2.2.4 Circularity
				3.2.2.5 Eigenvalues and eccentricity
				3.2.2.6 Number of corners
				3.2.2.7 Hu moments
			3.2.3 Modeling algorithms
				3.2.3.1 OLS, GLM, and non-linear models
				3.2.3.2 Knn
				3.2.3.3 Naïve Bayes
				3.2.3.4 Linear and quadratic discriminant analysis
				3.2.3.5 Trees
				3.2.3.6 Random forests
				3.2.3.7 SVM
				3.2.3.8 CNNs
				3.2.3.9 DAMG
				3.2.3.10 Perceived accuracy
			3.2.4 Dimensionality reduction
				3.2.4.1 Subset selection procedures
				3.2.4.2 LASSO, ridge, and elastic net
				3.2.4.3 PCA
				3.2.4.4 FA
				3.2.4.5 Fourier transform
				3.2.4.6 Manifolds
			3.2.5 Model assurance
				3.2.5.1 Resampling methods
				3.2.5.2 Effect comparison
				3.2.5.3 HILT models
				3.2.5.4 Influential observations
				3.2.5.5 Visualization methods
		3.3 Experiments using XAI models
			3.3.1 Satellite imagery
			3.3.2 White blood cell
		3.4 Discussion
			3.4.1 XAI vs. AI in critical applications
			3.4.2 Explainability, interpretability, and model assurance in practice
			3.4.3 XAI models outperform CNN-based solutions
			3.4.4 XAI, deep learning models, and human inputs
			3.4.5 Extending the lessons learned to non-image problems
		3.5 Future work
		3.6 Conclusion
		Acknowledgments
		References
	4 Bias, fairness, and assurance in AI: overview and synthesis
		4.1 Introduction
		4.2 Assurance and ethical AI
			4.2.1 Overview of bias and lack of assurance in AI
			4.2.2 Current assurance methods for bias reduction
		4.3 Validation methods
		4.4 Synthesis of the literature
		4.5 Conclusion
		References
	5 An evaluation of the potential global impacts of AI assurance
		5.1 Introduction
		5.2 Literature review
		5.3 Methodology & modeling
			5.3.1 Scenario 1: full adoption of AI across all regions
			5.3.2 Scenario 2: estimation of gains from AI ethical frameworks across all regions
			5.3.3 Estimation of loss due to strict liabilities across all regions
		5.4 Results and analysis
			5.4.1 Impact of policy shocks on GDP of countries/regions
			5.4.2 Impact of policy shocks on output of countries/regions
			5.4.3 Impact of policy shocks on employment of countries/regions
			5.4.4 Impact of policy shocks on export of countries/regions
			5.4.5 Impact of policy shocks on import of countries/regions
		5.5 Conclusion
		Acknowledgment
		References
Part 2 AI assurance methods
	6 The role of inference in AI: Start S.M.A.L.L. with mindful modeling
		6.1 Real wisdom on artificial intelligence
		6.2 Fundamentals: decision-making, heuristics and cognitive biases
			6.2.1 Dual-process model of decision-making
			6.2.2 Error and bias in medical decision-making
			6.2.3 Implicit and/or explicit: bias in AI practitioners and AI models
		6.3 Fundamentals: yearning to make sense of the world through models and inference
			6.3.1 Mindful modeling approaches: a mark of thoughtful work
			6.3.2 Start S.M.A.L.L. (Specific-Mindful-Attainable-Limited-Lucid)
				6.3.2.1 Conceptual modeling
				6.3.2.2 Group model building process
				6.3.2.3 Causal modeling
			6.3.3 Inference in modeling
				6.3.3.1 Frequentist (Fisherian) inference
				6.3.3.2 Probabilistic (Bayesian) inference: a gateway to causal inference
				6.3.3.3 Causal inference: tempting the trope that ``correlation does not imply causation''
		6.4 Bolstering AI assurance: reducing biases with inferential methods
			6.4.1 What is AI assurance?
				6.4.1.1 Working scenario: mitigating bias in healthcare through AI assurance
			6.4.2 Contemporary AI: mindful modeling before data engineering helps reduce bias
				6.4.2.1 Question 1: what is the basis of ground truth for teaching the machine?
				6.4.2.2 Question 2: who determines when predictive analytics are used in decision-making?
				6.4.2.3 Question 3: when is a problem cognitively complex enough to obscure bias present in decision-making?
			6.4.3 Considering the level of system predictability when designing AI assurance
		6.5 Rest assured: mindful approaches in modeling may help avoid another AI winter
		6.6 Further reading
		Acknowledgments
		References
	7 Outlier detection using AI: a survey
		7.1 Introduction and motivation
		7.2 Outlier detection methods
			7.2.1 Statistical and probabilistic based methods
				7.2.1.1 Parametric distribution models
				7.2.1.2 Non-parametric distribution models
				7.2.1.3 Miscellaneous statistical models
				7.2.1.4 Advantages of statistical and probabilistic based methods
				7.2.1.5 Disadvantages of statistical and probabilistic based methods
				7.2.1.6 Research gaps and suggestions
			7.2.2 Density-based methods
				7.2.2.1 Advantages of density-based methods
				7.2.2.2 Disadvantages of density-based methods
				7.2.2.3 Research gaps and suggestions
			7.2.3 Clustering-based methods
				7.2.3.1 Advantages of clustering-based methods
				7.2.3.2 Disadvantages of clustering based methods
				7.2.3.3 Research gaps and suggestions
			7.2.4 Distance-based methods
				7.2.4.1 K-nearest neighbor models
				7.2.4.2 Pruning techniques
				7.2.4.3 Time series data
				7.2.4.4 Advantages of distance-based methods
				7.2.4.5 Disadvantages of distance-based methods
				7.2.4.6 Research gaps and suggestions
			7.2.5 Ensemble methods
				7.2.5.1 Advantages of ensemble methods
				7.2.5.2 Disadvantages of ensemble methods
				7.2.5.3 Research gaps and suggestions
			7.2.6 Learning-based methods
				7.2.6.1 Subspace learning models
				7.2.6.2 Active learning models
				7.2.6.3 Graph-based learning models
				7.2.6.4 Deep learning models
				7.2.6.5 Advantages of learning-based methods
				7.2.6.6 Disadvantages of learning-based methods
				7.2.6.7 Research gaps and suggestions
		7.3 Tools for outlier detection
		7.4 Datasets for outlier detection
		7.5 AI assurance and outlier detection
		7.6 Conclusions
		References
	8 AI assurance using causal inference: application to public policy
		8.1 Introduction and motivation
		8.2 Causal inference
			8.2.1 An introduction to causal inference
			8.2.2 Overview of causal inference methods
		8.3 AI assurance using causal inference
			8.3.1 AI assurance: goals and methods
			8.3.2 Methods for leveraging causality in assurance
			8.3.3 Application of causality in assurance: economy of technology example
		8.4 Network representations of data
			8.4.1 An introduction to graph theory
			8.4.2 Recurrent graph neural networks (RGNN)
			8.4.3 Economy of technology dataset as a network
		8.5 Conclusion
		Acknowledgments
		References
	9 Data collection, wrangling, and pre-processing for AI assurance
		9.1 Introduction and motivation
		9.2 Relevant data characteristics
		9.3 Data pre-processing: data wrangling and munging
		9.4 Data processing architectures: ETL & ELT
		9.5 DataOps: data operations automation management
		9.6 Data tagging, provenance, and lineage
		References
	10 Coordination-aware assurance for end-to-end machine learning systems: the R3E approach
		10.1 Introduction
		10.2 Background and motivation
			10.2.1 Background – characterizing BDML
			10.2.2 Motivating example: machine learning for classifying building elements
			10.2.3 Research questions
		10.3 Key elements of R3E approach
			10.3.1 QoAChain: chaining diverse types of quality constraints as a contract for optimizing end-to-end BDML
			10.3.2 R3E objects and operations
				10.3.2.1 Conceptualize R3E objects
				10.3.2.2 R3E attributes associated with R3E objects
				10.3.2.3 R3E operations and APIs
			10.3.3 Engineering methods
				10.3.3.1 Coordination for R3E
				10.3.3.2 Monitoring and analytics
				10.3.3.3 Testing, benchmarking, and experimenting for R3E
		10.4 Illustrative examples
		10.5 Discussion
		10.6 Conclusions and future work
		Acknowledgments
		References
Part 3 AI assurance and applications
	11 Assuring AI methods for economic policymaking
		11.1 Introduction to harnessing AI for economics
			11.1.1 ML in economic models
			11.1.2 AI accountability models in economic research
			11.1.3 Adopters of economic forecasting using XAI
		11.2 Commonplace explainability methods
			11.2.1 Local interpretable model-agnostic explanations (LIME) explainer
				11.2.1.1 LIME methodology
				11.2.1.2 LIME implementation
			11.2.2 SHapley Additive exPlanations (SHAP)
				11.2.2.1 SHAP methodology
				11.2.2.2 SHAP implementation
			11.2.3 Partial dependence plots
				11.2.3.1 PDP methodology
				11.2.3.2 PDP implementation
		11.3 Mitigating bias in AI models for economic prediction
			11.3.1 NLP use in central banking
			11.3.2 NLP transformer networks
			11.3.3 LIME for text explanations
			11.3.4 LLMs and the AI central banker
			11.3.5 Data assurance of LLMs
			11.3.6 LLM transparency
			11.3.7 Association rules mining
			11.3.8 Graph neural networks
				11.3.8.1 GNNs for international trade
				11.3.8.2 Explainability methods for GNNs
		11.4 Conclusion
		Acknowledgments
		References
	12 Panopticon implications of ethical AI: equity, disparity, and inequality in healthcare
		12.1 Introduction
		12.2 Ontological perspectives
		12.3 Ethics frameworks
		12.4 Governance in the healthcare domain
		12.5 Societal disparities in wellbeing
		12.6 Conclusion
		References
	13 Recent advances in uncertainty quantification methods for engineering problems
		13.1 Introduction
		13.2 Polynomial chaos method for UQ
		13.3 Gaussian Process or Kriging for UQ
		13.4 Polynomial chaos Kriging for UQ
		13.5 Uncertainty quantification of a supersonic nozzle
			13.5.1 Test case description
			13.5.2 Deterministic results
			13.5.3 Description of uncertainties
			13.5.4 Uncertainty analysis
		13.6 Conclusions
		Acknowledgments
		References
	14 Socially responsible AI assurance in precision agriculture for farmers and policymakers
		14.1 Introduction
			14.1.1 AI in agriculture
			14.1.2 Big data in agriculture
			14.1.3 Political economy of PA
		14.2 Current methods of AI assurance in agriculture
			14.2.1 AI assurance in agricultural policy
			14.2.2 AI assurance in precision agriculture
		14.3 Agricultural policy
		14.4 AI assurance in agriculture recommendations
			14.4.1 Participatory design from the start
			14.4.2 XAI for agricultural end users
		14.5 Conclusion
		CRediT authorship contribution statement
		References
	15 The application of artificial intelligence assurance in precision farming and agricultural economics
		15.1 Introduction
		15.2 AI for smart farms
			15.2.1 Correlation of economic indices and various commodities
			15.2.2 Causation of economic indices and various commodities
			15.2.3 Scoring outlier events for the model and finding anomalies
			15.2.4 Outlier classification and labeling
		15.3 Insight into data driven farming
			15.3.1 Kentland and dairy farm
			15.3.2 Shenandoah Valley Agricultural Research and Extension Center (SVAREC)
			15.3.3 Dairy complex at Virginia Tech's SmartFarms
		15.4 Larger policy implications
		15.5 Conclusion
		Acknowledgments
		References
	16 Bringing dark data to light with AI for evidence-based policymaking
		16.1 Introduction
			16.1.1 Background
			16.1.2 Motivation
			16.1.3 The AIM pipeline
		16.2 The dataset for AIM
			16.2.1 Dataset paradigm
			16.2.2 Metrics of interest
			16.2.3 Legislation data
			16.2.4 Environmental descriptors
		16.3 Feature creation
			16.3.1 Policies as data
			16.3.2 NLP in AIM
			16.3.3 Spectral clustering of laws
			16.3.4 Technology usage as data
		16.4 Learning the trends
			16.4.1 Neural network predicting AIMs
			16.4.2 Training metrics
			16.4.3 Prediction results
		16.5 Discussions and future directions
			16.5.1 Feasible applications
			16.5.2 Future directions
		16.6 Ethics of AI in public policy
			16.6.1 Data in the legislative process
			16.6.2 AI and bias
			16.6.3 AI assurance and the law
		References
Index
Back Cover