Low-Rank Models in Visual Analysis: Theories, Algorithms, and Applications

Contents
List of Figures
List of Tables
1 Background and motivation
	1.1 Introduction
	1.2 Objectives and key features of this book
	1.3 PGM introduction
		1.3.1 PGM issues
	1.4 Book outline
	References
2 Foundation and basics
	2.1 Introduction
	2.2 Random variables and probabilities
		2.2.1 Random variable and probability
		2.2.2 Basic probability rules
		2.2.3 Independencies and conditional independencies
		2.2.4 Mean, covariance, correlation, and independence
		2.2.5 Probability inequalities
		2.2.6 Probability distributions
			2.2.6.1 Discrete probability distributions
				2.2.6.1.1 Categorical probability distributions
				2.2.6.1.2 Integer probability distributions
				2.2.6.1.3 Multivariate integer probability distributions
			2.2.6.2 Continuous probability distributions
	2.3 Basic estimation methods
		2.3.1 Maximum likelihood
			2.3.1.1 Maximum joint likelihood
			2.3.1.2 Maximum conditional likelihood estimation
			2.3.1.3 Maximum marginal likelihood estimation
		2.3.2 Bayesian estimation
	2.4 Optimization methods
		2.4.1 Continuous optimization
		2.4.2 Discrete optimization
	2.5 Sampling and sample estimation
		2.5.1 Sampling techniques
		2.5.2 Sample estimation
	References
3 Directed probabilistic graphical models
	3.1 Introduction
	3.2 Bayesian Networks
		3.2.1 BN representation
		3.2.2 Properties of BNs
			3.2.2.1 Markov condition
			3.2.2.2 Markov blanket and moral graph
			3.2.2.3 D-separation
			3.2.2.4 Faithfulness
			3.2.2.5 Explaining-away
			3.2.2.6 Equivalent Bayesian networks
		3.2.3 Types of Bayesian networks
			3.2.3.1 Discrete BNs
			3.2.3.2 Continuous BNs
			3.2.3.3 Hybrid BNs
			3.2.3.4 Naive BNs
			3.2.3.5 Regression BNs
			3.2.3.6 Noisy-OR BN
	3.3 BN inference
		3.3.1 Exact inference methods
			3.3.1.1 Variable elimination
			3.3.1.2 Belief propagation in singly connected Bayesian networks
			3.3.1.3 Belief propagation in multiply connected Bayesian networks
				3.3.1.3.1 Clustering and conditioning methods
				3.3.1.3.2 Junction tree method
		3.3.2 Approximate inference methods
			3.3.2.1 Loopy belief propagation
			3.3.2.2 Monte Carlo sampling
				3.3.2.2.1 Logic sampling
				3.3.2.2.2 MCMC sampling
				3.3.2.2.3 Metropolis Hastings sampling
			3.3.2.3 Variational inference
		3.3.3 Inference for Gaussian BNs
		3.3.4 Bayesian inference
		3.3.5 Inference under uncertain evidences
	3.4 BN learning under complete data
		3.4.1 Parameter learning
			3.4.1.1 Maximum likelihood estimation of BN parameters
			3.4.1.2 Bayesian estimation of BN parameters
			3.4.1.3 Discriminative BN parameter learning
		3.4.2 Structure learning
			3.4.2.1 General BN structure learning
				3.4.2.1.1 Score-based approach
				3.4.2.1.2 Independence-test-based approach
			3.4.2.2 Gaussian BN structure learning
	3.5 BN learning under incomplete data
		3.5.1 Parameter learning
			3.5.1.1 Maximum likelihood estimation
				3.5.1.1.1 Direct method
				3.5.1.1.2 Expectation maximization method
			3.5.1.2 Bayesian parameter estimation
		3.5.2 Structure learning
	3.6 Manual Bayesian Network specification
	3.7 Dynamic Bayesian Networks
		3.7.1 Introduction
		3.7.2 Learning and inference
			3.7.2.1 DBN learning
			3.7.2.2 DBN inference
		3.7.3 Special DBNs
			3.7.3.1 Hidden Markov model (HMM)
				3.7.3.1.1 HMM topology and parameterization
				3.7.3.1.2 HMM inference
				3.7.3.1.3 HMM learning
				3.7.3.1.4 Variants of HMMs
			3.7.3.2 Linear Dynamic System (LDS)
	3.8 Hierarchical Bayesian networks
		3.8.1 Hierarchical Bayes models
		3.8.2 Hierarchical deep models
		3.8.3 Hybrid hierarchical models
	3.9 Appendix
		3.9.1 Proof of Eq. (3.63)
		3.9.2 Proof of Gaussian Bayesian network
		3.9.3 Laplace approximation
	References
4 Undirected probabilistic graphical models
	4.1 Introduction
		4.1.1 Definitions and properties
			4.1.1.1 Definitions
			4.1.1.2 Properties
			4.1.1.3 I-map
	4.2 Pairwise Markov networks
		4.2.1 Discrete pairwise Markov networks
		4.2.2 Label-observation Markov networks
		4.2.3 Gaussian Markov networks
		4.2.4 Restricted Boltzmann machines
	4.3 Conditional random fields
	4.4 High-order and long-range Markov networks
	4.5 Markov network inferences
		4.5.1 Exact inference methods
			4.5.1.1 Variable elimination method
			4.5.1.2 Belief propagation method
			4.5.1.3 Junction tree method
			4.5.1.4 Graph cuts method
			4.5.1.5 Inference for continuous MNs
		4.5.2 Approximate inference methods
			4.5.2.1 Iterated conditional modes
			4.5.2.2 Gibbs sampling
			4.5.2.3 Loopy belief propagation
			4.5.2.4 Variational methods
		4.5.3 Other MN inference methods
	4.6 Markov network learning
		4.6.1 Parameter learning
			4.6.1.1 Parameter learning under complete data
			4.6.1.2 Maximum likelihood estimation
				4.6.1.2.1 Contrastive divergence method
				4.6.1.2.2 Pseudolikelihood method
				4.6.1.2.3 Variational method
			4.6.1.3 Bayesian estimation of MN parameters
			4.6.1.4 Discriminative learning
			4.6.1.5 Parameter learning for CRF
			4.6.1.6 MN parameter learning under incomplete data
		4.6.2 Structure learning
			4.6.2.1 Score-based approach
			4.6.2.2 Structure learning via parameter regularization
	4.7 Markov networks versus Bayesian networks
	References
Index