Image and Video Processing and Analysis and Computer Vision

Academic Press Library in Signal Processing, Volume 6
Copyright
Contributors
About the Editors
Section Editors
Introduction
Multiview video: Acquisition, processing, compression, and virtual view rendering
	Multiview Video
		Multiview Video and 3D Graphic Representation Formats for VR
		Super-Multiview Video for 3D Light Field Displays
		DIBR Smooth View Interpolation
		Basic Principles of DIBR
		DIBR vs. Point Clouds
		DIBR, Multiview Video, and MPEG Standardization
	Multiview Video Acquisition
		Multiview Fundamentals
		Depth in Stereo and Multiview Video
		Multicamera System
		Acquisition System Examples
			Nagoya University multiview camera system
			Fraunhofer HHI camera system
			Poznań University of Technology multiview camera system (linear rig)
			Poznań University of Technology multiview camera system (modular)
			Hasselt University multiview camera system
	Multiview Video Preprocessing
		Geometrical Parameters
			Intrinsic parameters
			Extrinsic parameters
			Lens distortion
			Estimation of camera parameters
			Camera parameters file format
		Video Correction
			Color correction
			Lens distortion removal
	Depth Estimation
		Local Stereo Matching
		Global Stereo Matching
			Graph Cut
			Belief propagation
		Multicamera Depth Estimation
			Plane sweeping
			Epipolar plane images
	View Synthesis and Virtual Navigation
		Warping
		View Blending
		Inpainting
		View Synthesis Reference Software
	Compression
		Introduction
		Monoscopic Video Coding and Simulcast Coding of Multiview Video
		Multiview Video Coding
		3D Video Coding
	Future Trends
	Glossary
	References
	Further Reading
Plenoptic imaging: Representation and processing
	Introduction
	Light Representation: The Plenoptic Function Paradigm
	Empowering the Plenoptic Function: Example Use Cases
		Light Field Communication
			Use case 1.1: Super-multiview home television
			Use case 1.2: Immersive bidirectional communication
		Light Field Editing
			Use case 2.1: Photographic light field editing
			Use case 2.2: Cinematic, mixed reality light field editing
		Free Navigation
			Use case 3.1: Omnidirectional 360 degree viewing of the surrounding environment
			Use case 3.2: Free viewpoint sports event
			Use case 3.3: Free viewpoint home television
		Interactive All-Reality
			Use case 4.1: Surveillance with depth recovery
			Use case 4.2: Remote surgery with glasses-free 3D display
			Use case 4.3: Interactive VR training
			Use case 4.4: Augmented reality surveillance with light field editing
	Plenoptic Acquisition and Representation Models
		Acquisition
		Representation
		Display
	Plenoptic Data Coding
	Plenoptic Data Rendering
		Rendering Textured Meshes and Point Clouds
		Interpolating a Light Field in a Microlens and/or Discrete Camera Array
		View Synthesis in MVV Plus Depth
		Refocusing With Microlens Light Field
	Plenoptic Representations Relationships
	Related Standardization Initiatives
		MPEG FTV
		JPEG PLENO
	Future Trends and Challenges
	Acknowledgments
	Glossary
	References
	Further Reading
Visual attention, visual salience, and perceived interest in multimedia applications
	Introduction
		Visual Attention in the Field of Multimedia: A Rising Story
		From Vision Science to Engineering: Concepts Mash Up and Confusion
	Classification of Attention Mechanisms
		Overt and Covert Attention
		Types of Overt Visual Attention Mechanisms
			Endogenous and exogenous visual attention
			Top-down and bottom-up attention
			Interaction between the top-down and bottom-up attention mechanisms
			Concept of perceived importance: Top-down attention is not equal to object of interest
			Importance maps and salience maps: Which ground truth for perceived interest?
	Computational Models of Visual Attention
		Top-Down Computational Attention Models
			Visual search task
			Object recognition task
			Driving, gaming, and sports
		Information-Theory and Decision-Theory Models
			Entropy/information maximization
			Action-reward based
		Spatio-Temporal Computational Models
			Center-surround in the temporal domain
			Detection of irregular actions/behavior
		Graph-Based Methods
			Graph flow techniques
			Foreground-background segmentation
			Random walk based
			Salient boundary and object identification
			Graph spectral methods
		Scan-Path (Saccadic) Models
			Memory-based modeling
			Semantic region based
			Residual information based
			Oculomotor bias and memory based
	Acquiring Ground Truth Visual Attention Data for Model Verification
		Eye-Tracking
			Conducting an eye-tracking experiment
			Existing eye and video tracking datasets for model validation
		Processing the Eye-Tracking Data
			Saccades and fixations
			Saliency maps for images and videos
			Scan-path generation
			Analysis of disruptions
		Testing the Computational Models
			Statistical analysis of fixation and saccades
			Similarity in saliency maps
			Scan-path similarity metrics
			Hybrid approaches
	Applications of Visual Attention
		Quality Assessment
			Using saliency as a weighting factor of local distortions
			Purely attention-based image quality measures: Visual attention deployment as a proxy for quality
		Visual Attention in Multimedia Delivery
			Interactive streaming
			Dealing with packet loss
			Image re-targeting
		Applications in Medicine
			Eye-tracking in disease detection
			Eye-tracking in the training of medical personnel
			Tele-surgery
		Visual Attention and Immersive Media: A Rising Love Story
			Stereoscopy and 3D displays
			Virtual reality (VR)
	References
Emerging science of QoE in multimedia applications: Concepts, experimental guidelines, and validation of models
	QoE Definition and Influencing Factors
		Factors Influencing QoE
			System influence factors
			Context influence factors
			Human influence factors
	QoE Measurement
		Including System Influence Factors in QoE Measurement
		Including Context Influence Factors in QoE Measurement
		Including Human Influence Factors in QoE Measurement
		Multidimensional Perceptual Scales for QoE Measurement
			Scales and scaling methods
		Direct Scaling Methods
			Single Stimulus/Absolute Category Rating
				ACR with hidden reference
			Double Stimulus Impairment Scale/Degradation Category Rating
			Double Stimulus Continuous Quality Scale
			SAMVIQ and MUSHRA
		Processing of Results of Direct Scaling Methods
			Mean scores calculation
			Confidence intervals calculation
			Screening of the subjects
		Indirect Scaling Methods
			Ranking
			Paired Comparison
				Square design PC
				Adaptive square design PC
		Processing of Results of Indirect Screening Methods
			Thurston-Moesteller model
			Bradley-Terry-Luce model
			Direct processing of pair comparison matrix
		Influence Factors Significance Calculation
			Calculating significance ratio in direct scaling experiments
			Calculating significance ratio in indirect scaling experiments
				Calculating SR in indirect scaling partial design experiments
	Performance Evaluation of Objective QoE Estimators
		Pearsons Linear Correlation Coefficient
		Root-Mean-Squared Error
		Epsilon-Insensitive Root-Mean-Squared Error
		Outlier Ratio
		Spearman's Rank Order Correlation Coefficient
		Kendall's Rank Order Correlation Coefficient
		Resolving Power Measures
			RP accuracy
			Classification plots
		ROC-Based Performance Evaluation
			Different vs. similar analysis
			Better vs. worse analysis
			Statistical comparison of objective algorithms
		Compensation for Multiple Comparisons
			Bonferroni correction procedure
			Holm-Bonferroni correction procedure
			Benjamini-Hochberg correction procedure
	Conclusion
	References
Computational photography
	Introduction
	Breaking Precepts Underlying Photography
		Sensor Resolution ≠ Image Resolution
			Spatial multiplexing
			Spatial multiplexing designs
		Space-Time Bandwidth Product Can Be Greater Than the ADC Rate
			Image models and coded spatiotemporal imaging
		Depth of Field Can Be Changed Independent of Exposure Time
			Coded apertures
			Extended DoF via depth-invariant defocus blur
			Light field cameras
	Cameras With Novel Form Factors and Capabilities
		Lensless Imaging
		Ptychography
			Fourier ptychography
		Subdiffraction Limited Microscopy
	Solving Inverse Problems
		Time-of-Flight-Based Range Imaging
			Principles of CWAM ToF
				Four-bucket technique
			Principles of discrete ToF
			Applications of ToF cameras
		Direct-Global Separation
		Scattering
	Conclusions
	References
Face detection with a 3D model
	Introduction
		Contributions
		Related Work
			Multiview models
			3D view-based models
			3D models
			Cascade approaches
			Face alignment
			Parameter sensitive classifiers
			Face detection with pose estimation
	Face Detection Using a 3D Model
		Face representation
		Face 3D model
		Energy Model
		Inference Algorithm
		Detecting Face Keypoints
		Generating 3D Pose Candidates
			Image-based regression
			Ground truth 3D pose
			Training details
		Generating Face Candidates
			Keypoint support
		Scoring the Face Candidates
			Local difference features
			Modified LBF features
			Local selected features (LSF)
			Special features
			Score function
		Nonmaximal Suppression
	Parameter Sensitive Model
		Parameter sensitive linear model
		Nonlinear model
		Training the Parameter Sensitive Model
			Training cost function
			Optimization
	Fitting 3D Models
		Fitting a Rigid Projection Transformation
		Learning a 3D Model From 2D Annotations
	Experiments
		Training dataset
		Method nomenclature
		Evaluation of Face Candidates
		Face Detection Results
			Evaluation of design decisions
			Failure modes
			Detection time
	Conclusions and Future Trends
	References
A survey on nonrigid 3D shape analysis
	Introduction
	General Formulation
		Representations
		Invariance Requirements
		Problem Statement and Taxonomy
	Shape Spaces and Metrics
		Kendall's Shape Space
			Morphable models
			The nonlinear nature of Kendalls shape space
		Metrics That Capture Physical Deformations
			The shape space of thin shells
			The shape space of square-root representations
				Properties
		Transformation-Based Representations
			Choice of the template T
			Deformation models
			Metrics on the space of deformations
	Registration and Geodesics
		Registration
			Landmark-based elastic registration
			Elastic registration as a re-parameterization problem
		Geodesics
			Geodesics using pullback metrics
			Geodesics in the space of SRNFs
			Comparison and discussion
	Statistical Analysis Under Elastic Metrics
		Statistical Analysis Using Non-Euclidean Metrics
		Statistical Analysis by SRNF Inversion
	Examples and Applications
		Registration and Geodesic Deformations
		Elastic Coregistration of 3D Shapes
		Classification
		Random 3D Model Synthesis
		Other Applications
	Summary and Perspectives
		Topological and structural variabilities
		Multiply interacting shapes
		Correspondence
	Acknowledgments
	References
Markov models and MCMC algorithms in image processing
	Introduction: The Probabilistic Approach in Image Analysis
	Lattice-based Models and the Bayesian Paradigm
		Modeling
		Optimization
		Parameter Estimation
	Some Inverse Problems
		Denoising and Deconvolution: The Restoration Problem
		Segmentation Problem
		Texture Modeling
	Spatial Point Processes
		Modeling
		Optimizing
	Multiple Objects Detection
		Population Evaluation
		Road Network Detection
	Conclusion
	References
	Further Reading
Image and video-based analytics
Scalable image informatics
	Introduction
		Core Requirements
	Core Concepts
		Metadata Graph
		Versioning, Provenance, and Queries
	Basic micro-services
		Uniform Metadata Representation and Query Orchestration: Data Service
		Scalability of Micro-Services and Analysis
		Analysis Extensions: Module Service
		Uniform Representation of Heterogeneous Storage Subsystems: Blob Service
		Uniform Access and Operations Over Data Files: Image Service and Table Service
			Image service
			Table service
	Analysis Modules
		Python and Matlab Scripting
		Pipeline Support
		Complex Module Execution Descriptors
	Building on the Concepts: Sparse Images
	Feature Services and Machine Leaning
		Feature Service
		Connoisseur Service for Deep Learning
		Connoisseur Module for Domain Experts
	Application Example: Annotation and Classification of Underwater Images
	Summary
	References
Person re-identification
	Introduction
	The re-identification Problem: Scenarios, Taxonomies, and Related Work
		The Scenarios and Taxonomy
		Related Work
		Feature Extraction
		Model Learning
	Experimental Evaluation of re-id Datasets and Their Characteristics
	The SDALF Approach
		Object Segmentation
		Symmetry-Based Silhouette Partition
		Symmetry-Driven Accumulation of Local Features
			Weighted color histograms
			MSCRs
			Recurrent high structured patches (RHSPs)
		The Matching Phase
	Metric Learning
		Mahalanobis Metric Learning
		Large Margin Nearest Neighbor
		Efficient Impostor-Based Metric Learning
		KISSME
	Conclusions and New Challenges
	References
Social network inference in videos
	Introduction
	Related Work
	Video Shot Segmentation
	Actor Recognition
	Learning to Group Actors
		Visual Features
		Auditory Features
		Grouping Criteria
	Inferring Social Communities
		Social Network Graph
		Actor Interaction Model
	Social Network Analysis
		Assignment to Communities
		Estimating Community Leader
	Experiments
		The Dataset
		Audiovisual Alignment
		Social Affinity
		Community Assignment
		Actor Affinity
		Community Leaders
	Latent Features
	Summary
	References
	Further Reading
Index
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
V
W
X
Z