Quantum Chemistry in the Age of Machine Learning

Front Cover
Quantum Chemistry in the Age of Machine Learning
Copyright
Contents
Companion website
Contributors
Preface
	Reference
Part 1: Introduction
	Chapter 1: Very brief introduction to quantum chemistry
		Introduction-The foundations of quantum chemistry
			Brief introduction of quantum mechanics
				Basic concepts
				Born-Oppenheimer approximation
				Variational principle
				Perturbation theory
				Comparison of the variation principle and perturbation theory
			Fundamentals of quantum chemistry
				Categories of molecular electronic structure methods
		Methods of molecular electronic structure computations
			Hartree-Fock method
			Post-Hartree-Fock methods
				Configuration interaction
				Møller-Plesset perturbation theory
				Coupled-cluster method
			MCSCF and post-MCSCF methods
			Methods for excited states
			Choice of the method
		Methods of conceptional interpretation based on electronic structure calculations
		Case studies
			Case study 1
			Case study 2
		Conclusions and outlook
		Acknowledgments
		References
	Chapter 2: Density-functional theory
		Introduction
		Theoretical foundations of DFT
			Hohenberg-Kohn theorems
			Kohn-Sham method
			Spin density-functional theory
		Density-functional approximations
			Formal properties of Exc
			Local density approximation
			Generalized gradient approximations
			Meta-GGA
			Hybrid functionals
				Hybrid functionals based on adiabatic connection
				Long-range corrected hybrid functionals
				Screened hybrid functionals
				Hybrid functionals in the general form
			Fifth-rung functionals
				Doubly hybrid functionals
				Exc based on adiabatic-connection fluctuation-dissipation theorem
			Van der Waals dispersion interaction in DFT
			Self-consistent equations for orbital-dependent functionals
		Practical aspects of DFT implementations
			Basis set
				Atomic-type basis
				Plane-wave basis
				Augmented plane-wave (APW)-type basis
			Core-valence interactions
		Case studies
			Basis set convergence
			Molecular structures and vibrational spectra from different DFAs
			Reaction energetics and transition state
		Concluding remarks
		Acknowledgments
		References
	Chapter 3: Semiempirical quantum mechanical methods
		Introduction
		Methods
			Hückel method
			Extended Hückel method
			Approximations neglecting differential overlap
				Zero-differential overlap
				Complete neglect of differential overlap
				Intermediate neglect of differential overlap
				Neglect of diatomic differential overlap
			DFT-based SQM methods
			Non-covalent interactions in SQM methods
				London dispersion correction
				Hydrogen bonding
				Accuracy of SQM methods applied to non-covalent interactions
			Software implementations available
		Case studies
			Case study 1
			Case study 2
			Case study 3
			Case study 4
		Conclusions and outlook
		Acknowledgments
		References
	Chapter 4: From small molecules to solid-state materials: A brief discourse on an example of carbon compounds
		Introduction
		Methods
			Cluster model
			QM/MM model
			Periodic models
			Electronic structures of carbon-based molecules
			Electronic structures of graphene
			Electronic structures of other carbon materials
			Cycloaddition reactions
		Case studies
			Case study 1: Orbital analysis of C60
			Case study 2: Understand the electronic structure of alkali metal-doped C60
			Case study 3: Band structure of single-layer BN sheet
		Conclusions and outlook
		Acknowledgments
		References
	Chapter 5: Basics of dynamics
		Introduction
		Methods
			Quantum dynamics of nuclei under Born-Oppenheimer approximation
			From quantum dynamics to classical MD simulations
			Propagating the MD simulations
		Case studies
			Case study 1: The HCl bond oscillation after photoexcitation from time-dependent quantum dynamics
			Case study 2: Molecular dynamics simulation of naphthalene crystal
		Conclusions and outlook
		Acknowledgments
		References
	Chapter 6: Machine learning: An overview
		Introduction
		Methods
			Supervised learning
				Learning the pKa of substituted phenols
				Supervised ML problems and methods
				Applications in quantum chemistry
			Unsupervised machine learning
			Semi-supervised machine learning
			Reinforcement learning
		Basic concepts of machine learning
			Optimization of the cost function
			Presentation of data
			Hyperparameter tuning and performance evaluation
		Case study
			Case study 1: Single-variable linear regression
			Case study 2: Multivariable linear regression
			Case study 3: Importance of feature selection
			Case study 4: Beyond linear regression
		Conclusions and outlook
		Acknowledgment
		References
		Further reading
	Chapter 7: Unsupervised learning
		Introduction
		Notation guide
		Methods
			Descriptors for encoding chemical information
				What makes a good descriptor?
				Popular descriptors for machine learning
				Kernels
			Dimensionality reduction/mapping
				Dimensionality reduction for data compression
					Principal components analysis
					Multidimensional scaling
					Kernel principal components analysis
					Principal covariates regression
				Dimensionality reduction for visualization and pattern recognition
					t-distributed stochastic neighbor embedding
					UMAP
				Feature selection: Feature-preserving dimensionality reduction
			Clustering
				Motivation
				Theory and examples
					Hierarchical clustering
					Partition-based clustering
					Density-based clustering
				Evaluation of clustering results
		Case studies
			Cyclohexane molecular dynamics simulation
				Construct a PCA
				Construct a kernel PCA
				t-SNE and UMAP
				Performing clustering analysis on the t-SNE embedding
			Exploring surface structural motifs
				Problem description
				Constructing surface atom descriptors
				Reducing dimensionality with PCA
				Advantages of non-linear dimensionality reduction
				Experiment with other clustering algorithms
		Conclusions
		References
	Chapter 8: Neural networks
		Introduction
		Methods
			Feed-forward neural network
			Activation functions
			Training a neural network
			Neural networks for learning time series
				One-dimensional convolutional neural network
				Recurrent neural network
				Long short-term memory
				Gated recurrent unit
				Bidirectional recurrent neural network
		Case study
		Conclusions and outlook
		Acknowledgments
		References
	Chapter 9: Kernel methods
		Introduction
		Methods
			Kernel methods explained for chemists
			From linear regression to kernel methods
			Fitting kernel methods: Kernel ridge regression
			On a choice of the kernel function
			Other kernel methods: Support vector regression
			Other kernel methods: Gaussian process regression
			Including derivative information for training
			Computational resources requirements
				Unsupervised learning
				Kernel principal component analysis-KPCA
				Kernel K-means
		Case studies
			Case study 1: Fitting with kernel ridge regression
			Case study 2: Kernel principal component analysis
		Conclusions and outlook
		IntroductionKernel methods correspond to a learning paradigm that goes beyond simple linear approximations to model or extract
		References
	Chapter 10: Bayesian inference
		Introduction
		Basic concepts of Bayesian statistics
			What is probability
			Bayes theorem
				Prior distribution
				Likelihood function
				Posterior distribution
			Bayesian inference for parameter estimation and confidence interval
		Bayesian regression
			Bayesian linear regression
			Gaussian process regression
				GPR model based on Bayesian linear regression
				Estimation of f(x)
				Bayesian explanation of the GPR model
		Bayesian inference in machine learning: Bayesian neural networks
		Case study
			Gaussian process regression
		Conclusions and outlook
		Acknowledgments
		References
Part 2: Machine learning potentials
	Chapter 11: Potentials based on linear models
		Introduction
		Methods
			From ordinary least squares to sparse linear regression
				Ordinary least square (OLS) method
				Ridge regression
				Lasso regression
					Lasso 1D
					Lasso, general case
					Coordinate descent
					Least angle regression selection
				Geometrical interpretations
					Lasso vs Ridge
					LassoLars vs Lasso coordinate descent
			Different approaches to obtain linear model of machine-learning interaction potentials
				Spectral neighbor analysis potential and developments
					The original approach from A. P. Thompson and coworkers
					A hybrid approach proposed by M-C. Marinica and coworkers
				Generalization of the modified embedded atom method potential and Physical LassoLars Interaction Potential (PLIP)
					The original approach proposed by A. Seko and coworkers
					Physical LassoLars interaction potential
		Case studies
			Case study 1: Learning an arbitrary function using a set of Gaussian functions.
			Case study 2 Learning the potential from Lennard-Jones binary simulations.
		Conclusion and outlook
		Acknowledgments
		References
	Chapter 12: Neural network potentials
		Introduction
		Methods
			Atomic-centered symmetry functions
			SchNet
			DeepPot-SE
			Example of simulation with neural network potentials
		Case studies
			Simulation of the oxidation of methane
				Step 1: Preparing the reference dataset
				Step 2. Training the NN PES
				Step 3: Freeze the model
				Step 4: Running MD simulation based on the NNP
				Step 5: Analysis of the trajectory
		Conclusions and outlook
		Acknowledgment
		References
	Chapter 13: Kernel method potentials
		Introduction
		Methods
			KRR-CM
				Descriptor
				Fitting function
			KREG model
				Descriptor
				pKREG model
			GDML model
				Descriptor
				Fitting function
				sGDML model
			GAP-SOAP model
				Descriptor: SOAP
				Sparsification
				Linear combination of unknown values
				Fitting function: GAP
			Operator learning
			Guidelines for choosing an appropriate kernel method potential
		Case study
			Step 1. Training the KREG potential
			Step 2. Geometry optimization with the trained KREG model
		Conclusions and outlook
		Acknowledgments
		References
	Chapter 14: Constructing machine learning potentials with active learning
		Introduction
		Methods
			Uncertainty sampling
			Query-by-committee
			Failure collection
			Stochastic surface walking method
			Steinhardt-type order parameter
		Case study
		Conclusion and outlook
		Acknowledgments
		References
	Chapter 15: Excited-state dynamics with machine learning
		Introduction
		Methods
			Hierarchical equations of motion
			ML-assisted HEOM
			Trajectory surface hopping
				Fewest switches surface hopping
				Nonadiabatic couplings
			ML-assisted TSH
				Learning potential energy surfaces
				Hopping in ML-assisted TSH: Internal conversion
				Learning spin-orbit couplings
				Training set generation
		Case studies
			Case study 1: Hierarchical equations of motion
			Case study 2: HEOM with machine learning
			Case study 3: Trajectory-surface hopping with machine learning
				Step 1: Create the training set
				Step 2: Model training (MLatom/KREG)
				Step 3: Run the ML-NAMD simulation (Newton-X/MLatom)
		Conclusions and outlook
		Acknowledgments
		References
	Chapter 16: Machine learning for vibrational spectroscopy
		Introduction
			Vibrational spectroscopies: Workhorse characterization technique in many applications
			Case for machine learning-assisted spectroscopy
		Methods
			Introduction to computational vibrational spectroscopy
			Machine learning in vibrational spectroscopy
				Specifics of machine-learned potential energy surfaces for computational vibrational spectroscopy
				Learning the mapping among the structure, potential, and spectrum
				Machine learning to solve the vibrational Schrödinger equation
		Case studies
			Quantum dynamics-friendly neural network potential energy surface
				The code
			Solving the Schrödinger equation with the help of Gaussian process regression
				The code
		Conclusions and outlook
		References
	Chapter 17: Molecular structure optimizations with Gaussian process regression
		Introduction
		Methods
			Established methods for molecular optimizations
				Quasi-Newton methods
				Step restriction
				Approximate Hessian
				Hessian-update methods
				Choice of coordinates
				Constrained geometry optimization
				Geometry optimization in the direct inversion of the iterative subspace
			Machine learning methods for structure prediction
			Machine learning-based surrogate PES
			The restricted variance optimization method
				Hessian approximation
				Coordinates
				The trend function
				The characteristic length scales
				Restricted-variance optimization
		Case studies
			One-dimensional system (H2)
			Two-dimensional system (H2O)
			Transition state optimization (CH3-CH=OCH2=CH-OH)
		Conclusions and outlook
		Acknowledgments
		References
Part 3: Machine learning of quantum chemical properties
	Chapter 18: Learning electron densities
		Introduction
			A property to rule them all
			Topology
		Methods
			Prediction of the electron density
				Numerical methods
				Analytical methods
					Symmetry-adapted Gaussian process regression
					A2MDnet and A3MDnet
				Methods targeting DFT
					KH-maps
					XC functionals
					Wave function prediction
			Electron density methods
				3D space integration
				Electron density
		Case studies
			Case study 1: Using PySCF to obtain reference electron densities
			Case study 2: Training an A3MDnet predictor
			Case study 3: Predicting valence electron densities with DeepDFT
		Conclusions and outlook
		Acknowledgments
		References
	Chapter 19: Learning dipole moments and polarizabilities
		Introduction
		Methods
			Learning permanent and transition dipole moments
			Learning polarizabilities
			Other relevant approaches
			Learning tensorial properties with embedded atom neural networks
		Case studies
		Conclusions and outlook
		Acknowledgments
		References
	Chapter 20: Learning excited-state properties
		Introduction
		Methods
			Absorption spectra
			Case study 1: KREG for learning energy gaps and oscillator strengths of a single compound
			Case study 2: SchNarc for learning transition dipole moments and energy levels of one or more compounds simultaneously
		Case studies
			Case study 1: ML-NEA spectrum for a single molecule
			Case study 2: UV/Vis absorption spectra predicted with SchNarc trained on two different molecules
				UV/visible spectra of learned molecules
					Interpretation of the spectrum
				Transferability in chemical compound space
				Electrostatic potentials
		Conclusions and outlook
			Challenges ahead
		Acknowledgments
		References
Part 4: Machine learning-improved quantum chemical methods
	Chapter 21: Learning from multiple quantum chemical methods: Delta-learning, transfer learning, co-kriging, and
		Introduction
		Methods
			Delta-learning
			LFAF: Low-level QC predictions as features of machine learning models
			Combination of Delta-learning and LFAF
			Transfer learning
			Combination of Delta-learning and transfer learning
			Learning multifidelity and multioutput data with kernel methods: Co-kriging
			Hierarchical machine learning
		Case studies
			Case study 1: Delta-learning vs direct learning
			Case study 2: Hierarchical machine learning
			Case study 3: Transfer learning
		Conclusions and outlook
		IntroductionModern quantum chemistry (QC) offers a vast selection of different methods, each with its advantages and d
		References
	Chapter 22: Data-driven acceleration of coupled-cluster and perturbation theory methods
		Introduction
		Methods
			Data-driven coupled cluster
				Basic theory
				Data-driven coupled-cluster singles and doubles
			DDCCSD and transferability
			Data-driven multiconfigurational methods
				Basic theory
				Data-driven CASPT2
		Case studies
			Water as a case study for DDCCSD
			DDCASPT2: Ozone case study
		Conclusions and outlook
		Acknowledgment
		References
	Chapter 23: Redesigning density functional theory with machine learning
		Introduction
		Methods
			Global electron density formulation of ML-DFTXC
			Quasi-local electron density formulation of ML-DFTXC: The ML XC potential model
				The holographic electron density theorem (HEDT) and its implications on ML-DFTXC
				Pre-calculating XC potential as the target
				Model building, training, and SCF explained with a successful story
			Quasi-local Electron density formulation of ML-DFTXC: The ML XC energy density model
				Theory
				Implementation and illustrative examples
			Quasi-local electron density formulation of ML-DFTXC: The ML XC fragment energy model
				Theory
				Implementation and illustrative examples
			General quasi-local electron density formalism of ML-DFTXC
			Additional ML models: ML For van der Waals interaction
		Case study
			An example for the ML-DFTXC potential model
		Conclusions and outlook
		References
	Chapter 24: Improving semiempirical quantum mechanical methods with machine learning
		Introduction
		Methods
			Correcting predictions by semiempirical quantum mechanical methods with machine learning
				Delta-learning
				General-purpose ML-NDDO method AIQM1
				General-purpose ML-DFTB methods
				ML-improved SQM methods for heats of formation
			Improving semiempirical Hamiltonian parameters with machine learning
				Automatic parameterization technique: ML-OM2
				ML-EHM
					The extended Hückel theory
					ML effective Hamiltonian
					ML-EHM loss function and training
					The ML-EHM learnable parameters
				ML-improved DFTB Hamiltonians
		Case study
		Conclusions and outlook
		Acknowledgments
		References
	Chapter 25: Machine learning wavefunction
		Introduction
		Methods
			Variational Monte Carlo in a nutshell
			Modeling the wavefunction in Fock space
				Neural-network quantum state
				Gaussian process state
			Modeling the wavefunction in real space
				FermiNet
				PauliNet
			Supervised machine learning of the wavefunction
				SchNOrb
		Case studies
			Particle in a box
				Bayesian learning of the wavefunction with a Gaussian process
				Variational optimization of the Gaussian process wavefunction
			Ground state energies from PauliNet
				PauliNet training
				CCSD(T) calculations
				Comparison of PauliNet and CCSD(T)
		Conclusions and outlook
		Acknowledgments
		References
Part 5: Analysis of big data
	Chapter 26: Analysis of nonadiabatic molecular dynamics trajectories
		Introduction
		Theoretical methods
			Descriptor or feature selection
			TSH dynamics
			MM-SQC dynamics
			Dimensionality reduction approaches
				Principal component analysis
				Multidimensional scaling
				Isometric feature mapping (ISOMAP)
				Diffusion map
			Trajectory similarity
		Examples
		Case studies
			Case study 1: Classical MDS analysis of CH2NH2+ dynamics
			Case study 2: Fréchet distance analysis of phytochromobilin
			Case study 3: PCA of site-exciton model dynamics
		Conclusions and outlook
		References
	Chapter 27: Design of organic materials with tailored optical properties: Predicting quantum-chemical polarizabilities an ...
		Introduction
			Rational design strategies for materials
			Data-driven design protocol
		Methods
			Molecular polarizability
			Refractive index
			Computational protocol
		Case studies: Implementing the rational design protocol
			Standard DNNs for α, nr, and N
			Physics infused model
			Transfer learning
		Conclusions and outlook
		References
Index
Back Cover