3-D Computer Vision: Principles, Algorithms and Applications

Preface
Contents
Chapter 1: Computer Vision Overview
	1.1 Human Vision and Characteristics
		1.1.1 Visual Characteristics
			1.1.1.1 Vision and Other Sensations
			1.1.1.2 Vision and Computer Vision
			1.1.1.3 Vision and Machine Vision
			1.1.1.4 Vision and Image Generation
		1.1.2 Brightness Properties of Vision
			1.1.2.1 Simultaneous Contrast
			1.1.2.2 Mach Band Effect
			1.1.2.3 Contrast Sensitivity
		1.1.3 Spatial Properties of Vision
			1.1.3.1 Spatial Cumulative Effect
			1.1.3.2 Spatial Frequency
			1.1.3.3 Visual Acuity
		1.1.4 Temporal Properties of Vision
			1.1.4.1 Visual Phenomena That Change Over Time
				Brightness Adaptation
				Time Resolution of the Eyes
			1.1.4.2 Time Cumulative Effect
			1.1.4.3 Time Frequency
		1.1.5 Visual Perception
			1.1.5.1 Visual Sensation and Visual Perception
			1.1.5.2 The Complexity of Visual Perception
				Perception of the Visual Edge
				Perception of Brightness Contrast
	1.2 Computer Vision Theory and Framework
		1.2.1 Reaserch Goals, Tasks, and Methods of Computer Vision
		1.2.2 Visual Computational Theory
			1.2.2.1 Vision Is a Complex Information Processing Process
			1.2.2.2 Three Essential Factors of Visual Information Processing
			1.2.2.3 Three-Level Internal Expression of Visual Information
			1.2.2.4 Visual Information Understanding Is Organized in the Form of Functional Modules
			1.2.2.5 The Formal Representation of Computational Theory Must Consider Constraints
		1.2.3 Framework Problems and Improvements
	1.3 Three-Dimensional Vision System and Image Technology
		1.3.1 Three-Dimensional Vision System Process
		1.3.2 Computer Vision and Image Technology Levels
		1.3.3 Image Technology Category
	1.4 Overview of the Structure and Content of This Book
		1.4.1 Structural Framework and Content of This Book
		1.4.2 Chapter Overview
	1.5 Key Points and References for Each Section
	Self-Test Questions
	References
Chapter 2: Camera Calibration
	2.1 Linear Camera Model
		2.1.1 Complete Imaging Model
		2.1.2 Basic Calibration Procedure
		2.1.3 Internal and External Parameters
			2.1.3.1 External Parameters
			2.1.3.2 Internal Parameters
	2.2 Non-Linear Camera Model
		2.2.1 Type of Distortion
			2.2.1.1 Radial Distortion
			2.2.1.2 Tangential Distortion
			2.2.1.3 Eccentric Distortion
			2.2.1.4 Thin Prism Distortion
		2.2.2 Calibration Steps
		2.2.3 Classification of Calibration Methods
	2.3 Traditional Calibration Methods
		2.3.1 Basic Steps and Parameters
		2.3.2 Two-Stage Calibration Method
		2.3.3 Precision Improvement
	2.4 Self-Calibration Methods
	2.5 Key Points and References for Each Section
	Self-Test Questions
	References
Chapter 3: Three-Dimensional Image Acquisition
	3.1 High-Dimensional Image
	3.2 Depth Image
		3.2.1 Depth Image and Grayscale Image
		3.2.2 Intrinsic Image and Non-Intrinsic Image
		3.2.3 Depth Imaging Modes
	3.3 Direct Depth Imaging
		3.3.1 Time-of-Flight Method
			3.3.1.1 Pulse Time Interval Measurement Method
			3.3.1.2 Phase Measurement Method of Amplitude Modulation
			3.3.1.3 Coherent Measurement Method of Frequency Modulation
		3.3.2 Structured Light Method
			3.3.2.1 Structured Light Imaging
			3.3.2.2 Imaging Width
		3.3.3 Moiré Contour Stripes Method
			3.3.3.1 Basic Principles
			3.3.3.2 Basic Method
			3.3.3.3 Improvement Methods
		3.3.4 Simultaneous Acquisition of Depth and Brightness Images
	3.4 Stereo Vision Imaging
		3.4.1 Binocular Horizontal Mode
			3.4.1.1 Disparity and Depth
			3.4.1.2 Angular Scanning Imaging
		3.4.2 Binocular Convergence Horizontal Mode
			3.4.2.1 Disparity and Depth
			3.4.2.2 Image Rectification
		3.4.3 Binocular Axial Mode
	3.5 Key Points and References for Each Section
	Self-Test Questions
	References
Chapter 4: Video Image and Motion Information
	4.1 Video Basic
		4.1.1 Video Expression and Model
			4.1.1.1 Video Representation Function
			4.1.1.2 Video Color Model
			4.1.1.3 Video Space Sampling Rate
		4.1.2 Video Display and Format
			4.1.2.1 Video Display
			4.1.2.2 Video Bit Rate
			4.1.2.3 Video Format
		4.1.3 Color TV System
	4.2 Motion Classification and Representation
		4.2.1 Motion Classification
		4.2.2 Motion Vector Field Representation
		4.2.3 Motion Histogram Representation
			4.2.3.1 Histogram of Motion Vector Direction
			4.2.3.2 Histogram of Movement Area Types
		4.2.4 Motion Track Description
	4.3 Motion Information Detection
		4.3.1 Motion Detection Based on Camera Model
			4.3.1.1 Camera Motion Type
			4.3.1.2 Motion Camera
		4.3.2 Frequency Domain Motion Detection
			4.3.2.1 Detection of Translation
			4.3.2.2 Detection of Rotation
			4.3.2.3 Detection of Scale Changes
		4.3.3 Detection of Movement Direction
	4.4 Motion-Based Filtering
		4.4.1 Motion Detection-Based Filtering
			4.4.1.1 Direct Filtering
			4.4.1.2 Using Motion Detection Information
		4.4.2 Motion Compensation-Based Filtering
			4.4.2.1 Motion Trajectory and Time-Space Spectrum
			4.4.2.2 Filtering Along the Motion Trajectory
			4.4.2.3 Motion Compensation Filter
			4.4.2.4 Spatial-Temporal Adaptive Linear Minimum Mean Square Error Filtering
			4.4.2.5 Adaptive Weighted Average Filtering
	4.5 Key Points and References for Each Section
	Self-Test Questions
	References
Chapter 5: Moving Object Detection and Tracking
	5.1 Differential Image
		5.1.1 Calculation of Difference Image
		5.1.2 Calculation of Accumulative Difference Image
	5.2 Background Modeling
		5.2.1 Basic Principle
		5.2.2 Typical Practical Methods
			5.2.2.1 Method Based on Single Gaussian Model
			5.2.2.2 Method Based on Video Initialization
			5.2.2.3 Method Based on Gaussian Mixture Model
			5.2.2.4 Method Based on Codebook
		5.2.3 Effect Examples
			5.2.3.1 No Moving Foreground in Static Background
			5.2.3.2 There Is a Moving Foreground in a Static Background
			5.2.3.3 There Is a Moving Foreground in the Moving Background
	5.3 Optical Flow Field and Motion
		5.3.1 Optical Flow Equation
		5.3.2 Optical Flow Estimation with Least Square Method
		5.3.3 Optical Flow in Motion Analysis
			5.3.3.1 Mutual Velocity
			5.3.3.2 Focus of Expansion
			5.3.3.3 Collision Distance
		5.3.4 Dense Optical Flow Algorithm
			5.3.4.1 Solving the Optical Flow Equation
			5.3.4.2 Global Motion Compensation
	5.4 Moving Object Tracking
		5.4.1 Kalman Filter
		5.4.2 Particle Filter
		5.4.3 Mean Shift and Kernel Tracking
	5.5 Key Points and References for Each Section
	Self-Test Questions
	References
Chapter 6: Binocular Stereo Vision
	6.1 Stereo Vision Process and Modules
		6.1.1 Camera Calibration
		6.1.2 Image Acquisition
		6.1.3 Feature Extraction
		6.1.4 Stereo Matching
		6.1.5 3-D Information Recovery
		6.1.6 Post-Processing
			6.1.6.1 Depth Interpolation
			6.1.6.2 Error Correction
			6.1.6.3 Precision Improvement
	6.2 Region-Based Stereo Matching
		6.2.1 Template Matching
		6.2.2 Stereo Matching
			6.2.2.1 Epipolar Line Constraint
			6.2.2.2 Essential Matrix and Fundamental Matrix
			6.2.2.3 Influencing Factors in Matching
			6.2.2.4 Calculation of Surface Optical Properties
	6.3 Feature-Based Stereo Matching
		6.3.1 Basic Steps and Methods
			6.3.1.1 Matching with Edge Points
			6.3.1.2 Matching with Zero-Crossing Points
			6.3.1.3 Depth of Feature Points
			6.3.1.4 Sparse Matching Points
		6.3.2 Scale Invariant Feature Transformation
		6.3.3 Dynamic Programming Matching
	6.4 Error Detection and Correction of Parallax Map
		6.4.1 Error Detection
		6.4.2 Error Correction
	6.5 Key Points and References for Each Section
	Self-Test Questions
	References
Chapter 7: Monocular Multiple Image Recovery
	7.1 Photometric Stereo
		7.1.1 Light Source, Scenery, Lens
		7.1.2 Scene Brightness and Image Brightness
			7.1.2.1 The Relationship Between Scene Brightness and Image Brightness
			7.1.2.2 Bidirectional Reflectance Distribution Function
		7.1.3 Surface Reflection Characteristics and Brightness
			7.1.3.1 Ideal Scattering Surface
			7.1.3.2 Ideal Specular Reflecting Surface
	7.2 Shape from Illumination
		7.2.1 Representation of the Surface Orientation of a Scene
		7.2.2 Reflectance Map and Brightness Constraint Equation
			7.2.2.1 Reflection Map
			7.2.2.2 Image Brightness Constraint Equation
		7.2.3 Solution of Photometric Stereo
	7.3 Optical Flow Equation
		7.3.1 Optical Flow and Motion Field
		7.3.2 Solving Optical Flow Equation
			7.3.2.1 Optical Flow Calculation: Rigid Body Motion
			7.3.2.2 Optical Flow Calculation: Smooth Motion
			7.3.2.3 Optical Flow Calculation: Gray Level Mutation
			7.3.2.4 Optical Flow Calculation: Based on High-Order Gradient
	7.4 Shape from Motion
	7.5 Key Points and References for Each Section
	Self-Test Questions
	References
Chapter 8: Monocular Single Image Recovery
	8.1 Shape from Shading
		8.1.1 Shading and Orientation
		8.1.2 Gradient Space Method
	8.2 Solving Brightness Equation
		8.2.1 Linearity Case
		8.2.2 Rotational Symmetry Case
		8.2.3 The General Case of Smoothness Constraints
	8.3 Shape from Texture
		8.3.1 Monocular Imaging and Distortion
		8.3.2 Orientation Restoration from the Change of Texture
			8.3.2.1 Three Typical Methods
			8.3.2.2 Shape from Texture
				Isotropic Assumption
				Homogeneity Assumption
			8.3.2.3 Texture Stereo Technology
	8.4 Detection of Texture Vanishing Points
		8.4.1 Detecting the Vanishing Point of Line Segment Texture
		8.4.2 Determine the Vanishing Point Outside the Image
	8.5 Key Points and References for Each Section
	Self-Test Questions
	References
Chapter 9: Three-Dimensional Scenery Representation
	9.1 Local Features of the Surface
		9.1.1 Surface Normal Section
		9.1.2 Surface Principal Curvature
		9.1.3 Mean Curvature and Gaussian Curvature
	9.2 Three-Dimensional Surface Representation
		9.2.1 Parameter Representation
			9.2.1.1 The Parameter Representation of the Curve
			9.2.1.2 Parameter Representation of Curved Surface
		9.2.2 Surface Orientation Representation
			9.2.2.1 Extended Gaussian Image
			9.2.2.2 Spherical Projection and Stereographic Projection
	9.3 Construction and Representation of Iso-surfaces
		9.3.1 Marching Cube Algorithm
		9.3.2 Wrapper Algorithm
	9.4 Interpolating Three-Dimensional Surfaces from Parallel Contours
		9.4.1 Contour Interpolation and Tiling
		9.4.2 Problems That May Be Encountered
			9.4.2.1 Corresponding Problems
			9.4.2.2 Tiling Problem
			9.4.2.3 Branching Problem
		9.4.3 Delaunay Triangulation and Neighborhood Voronoï Diagram
	9.5 Three-Dimensional Entity Representation
		9.5.1 Basic Representation Scheme
			9.5.1.1 Spatial Occupancy Array
			9.5.1.2 Cell Decomposition
			9.5.1.3 Geometric Model Method
		9.5.2 Generalized Cylinder Representation
	9.6 Key Points and References for Each Section
	Self-Test Questions
	References
Chapter 10: Generalized Matching
	10.1 Matching Overview
		10.1.1 Matching Strategies and Categories
			10.1.1.1 Matching in Image Space
			10.1.1.2 Matching in Object Space
			10.1.1.3 Matching Based on Raster
			10.1.1.4 Feature-Based Matching
			10.1.1.5 Matching Based on Relationship
		10.1.2 Matching and Registration
		10.1.3 Matching Evaluation
	10.2 Object Matching
		10.2.1 Measure of Matching
			10.2.1.1 Hausdorff Distance
			10.2.1.2 Structural Matching Measure
		10.2.2 Corresponding Point Matching
		10.2.3 String Matching
		10.2.4 Matching of Inertia Equivalent Ellipses
		10.2.5 Shape Matrix Matching
	10.3 Dynamic Pattern Matching
		10.3.1 Matching Process
		10.3.2 Absolute Pattern and Relative Pattern
	10.4 Graph Theory and Graph Matching
		10.4.1 Introduction to Graph Theory
			10.4.1.1 Basic Definition
			10.4.1.2 The Geometric Representation of the Graph
			10.4.1.3 Colored Graph
			10.4.1.4 Sub-Graph
		10.4.2 Graph Isomorphism and Matching
			10.4.2.1 The Identity and Isomorphism of Graph
			10.4.2.2 Determination of Isomorphism
	10.5 Line Drawing Signature and Matching
		10.5.1 Contour Marking
			10.5.1.1 Blade
			10.5.1.2 Limb
			10.5.1.3 Crease
			10.5.1.4 Mark
			10.5.1.5 Shade/Shadow
		10.5.2 Structural Reasoning
		10.5.3 Labeling via Backtracking
	10.6 Key Points and References for Each Section
	Self-Test Questions
	References
Chapter 11: Knowledge and Scene Interpretation
	11.1 Scene Knowledge
		11.1.1 Model
		11.1.2 Attribute Hypergraph
		11.1.3 Knowledge-Based Modeling
	11.2 Logic System
		11.2.1 Predicate Calculation Rules
		11.2.2 Inference by Theorem Proving
	11.3 Fuzzy Reasoning
		11.3.1 Fuzzy Sets and Fuzzy Operations
		11.3.2 Fuzzy Reasoning Method
			11.3.2.1 Basic Model
			11.3.2.2 Fuzzy Combination
			11.3.2.3 De-fuzzification
	11.4 Scene Classification
		11.4.1 Bag-of-Words/Feature Model
		11.4.2 pLSA Model
			11.4.2.1 Model Description
			11.4.2.2 Model Calculation
			11.4.2.3 Model Application Example
	11.5 Key Points and References for Each Section
	Self-Test Questions
	References
Chapter 12: Spatial-Temporal Behavior Understanding
	12.1 Spatial-Temporal Technology
		12.1.1 New Research Field
		12.1.2 Multiple Levels
	12.2 Spatial-Temporal Interest Point Detection
		12.2.1 Detection of Points of Interest in Space
		12.2.2 Detection of Points of Interest in Space and Time
	12.3 Spatial-Temporal Dynamic Trajectory Learning and Analysis
		12.3.1 Automatic Scene Modeling
			12.3.1.1 Object Tracking
			12.3.1.2 Point of Interest Detection
			12.3.1.3 Activity Path Learning
		12.3.2 Path Learning
			12.3.2.1 Trajectory Preprocessing
			12.3.2.2 Trajectory Clustering
			12.3.2.3 Path Modeling
		12.3.3 Automatic Activity Analysis
	12.4 Spatial-Temporal Action Classification and Recognition
		12.4.1 Motion Classification
			12.4.1.1 Direct Classification
			12.4.1.2 Time State Model
			12.4.1.3 Motion Detection
		12.4.2 Action Recognition
			12.4.2.1 Holistic Recognition
			12.4.2.2 Posture Modeling
			12.4.2.3 Activity Reconstruction
			12.4.2.4 Interactive Activities
			12.4.2.5 Group Activities
			12.4.2.6 Scene Interpretation
	12.5 Key Points and References for Each Section
	Self-Test Questions
	References
Appendix A: Visual Perception
	A.1 Shape Perception
	A.2 Spatial Perception
		A.2.1 Nonvisual Indices of Depth
		A.2.2 Binocular Indices of Depth
		A.2.3 Monocular Indices of Depth
	A.3 Motion Perception
		A.3.1 The Condition of Motion Perception
		A.3.2 Detection of Moving Objects
		A.3.3 Depth Motion Detection
		A.3.4 Real Motion and Apparent Motion
		A.3.5 Correspondence Matching of Apparent Motion
		A.3.6 Aperture Problem
		A.3.7 Dynamic Indices of Depth
	A.4 Key Points and References for Each Section
	References
Answers to Self-Test Questions
	Chapter 1 Computer Vision Overview
	Chapter 2 Camera Calibration
	Chapter 3 Three-Dimensional Image Acquisition
	Chapter 4 Video Image and Motion Information
	Chapter 5 Moving Object Detection and Tracking
	Chapter 6 Binocular Stereo Vision
	Chapter 7 Monocular Multiple Image Recovery
	Chapter 8 Monocular Single Image Recovery
	Chapter 9 Three-Dimensional Scene Representation
	Chapter 10 Scene Matching
	Chapter 11 Knowledge and Scene Interpretation
	Chapter 12 Spatial-Temporal Behavior Understanding
Index