Multi-Camera Networks: Principles and Applications

Cover Page
......Page 1
Copyright......Page 2
Foreword......Page 3
Distributed Processing in Multi-Camera Networks......Page 6
Multi-Camera Calibration and Topology......Page 7
Active and Heterogeneous Camera Networks......Page 8
Multi-Camera Human Detection, Tracking, Pose, and Behavior Analysis......Page 9
Smart Camera Networks: Architecture, Middleware, and Applications......Page 10
Acknowledgments......Page 12
Introduction......Page 13
Perspective Projection......Page 14
Camera Matrices......Page 15
Estimating the Camera Matrix......Page 17
Two-Camera Geometry......Page 18
Epipolar Geometry and Its Estimation......Page 20
Relating the Fundamental Matrix to the Camera Matrices......Page 21
Estimating the Fundamental Matrix......Page 22
Projective Transformations......Page 24
Estimating Projective Transformations......Page 26
Rectifying Projective Transformations......Page 27
Feature Detection and Matching......Page 28
Affine Reconstruction......Page 30
Metric Reconstruction......Page 32
Bundle Adjustment......Page 34
Resources......Page 35
References......Page 36
Introduction......Page 38
Camera Network Calibration and Synchronization......Page 40
Related Work......Page 43
Camera Network Calibration......Page 48
Camera Network Synchronization......Page 51
Results......Page 53
Dynamic Scene Reconstruction from Silhouette Cues......Page 58
Related Work......Page 59
Probabilistic Framework......Page 61
Automatic Learning and Tracking......Page 70
Results and Evaluation......Page 73
Conclusions......Page 80
References......Page 81
Introduction......Page 85
Base Triangle......Page 88
Large-Scale Networks......Page 89
Bundle Adjustment Refinement......Page 91
Actuation Strategies......Page 92
System Description......Page 93
Actuated Camera Platform......Page 94
Network Architecture......Page 95
Localization Accuracy......Page 96
Node Density......Page 98
Latency......Page 100
References......Page 101
Introduction......Page 103
Simplicial Homology......Page 105
Example......Page 106
Čech Theorem......Page 107
The Camera and the Environment Models......Page 108
The CN-Complex......Page 109
Recovering Topology: 2D Case......Page 111
Algorithms......Page 112
Simulation in 2D......Page 114
Recovering Topology: 2.5D Case......Page 116
Building the CN-Complex......Page 117
Experimentation......Page 118
References......Page 122
Introduction......Page 124
Related Work......Page 125
Definitions......Page 127
Modeling a Camera's Field of View......Page 128
Modeling Space......Page 130
Exact Algorithms......Page 131
Heuristics......Page 135
Random Selection and Placement......Page 137
Experiments......Page 138
Comparison of Approaches......Page 139
Complex Space Examples......Page 141
Possible Extensions......Page 143
References......Page 144
6 Optimal Visual Sensor Network Configuration......Page 146
Introduction......Page 147
Related Work......Page 148
General Visibility Model......Page 149
Visibility Model for Visual Tagging......Page 151
Discretization of Camera and Tag Spaces......Page 154
MIN_CAM: Minimizing the Number of Cameras for Target Visibility......Page 155
FIX_CAM: Maximizing Visibility for a Given Number of Cameras......Page 156
GREEDY: An Algorithm to Speed Up BIP......Page 158
Optimal Camera Placement Simulation Experiments......Page 159
Comparison with Other Camera Placement Strategies......Page 165
Conclusions and Future Work......Page 167
References......Page 168
Introduction......Page 170
Related Work......Page 171
Tracking......Page 173
Objective Function for PTZ Scheduling......Page 175
Asynchronous Optimization......Page 176
View Angle......Page 178
Target--Zone Boundary Distance......Page 181
Combined Quality Measure......Page 182
Experiments......Page 183
References......Page 191
Introduction......Page 194
Related Work......Page 195
Pan-Tilt-Zoom Camera Geometry......Page 197
PTZ Camera Networks with Master--Slave Configuration......Page 198
Minimal PTZ Camera Model Parameterization......Page 199
Cooperative Target Tracking......Page 200
Tracking Using SIFT Visual Landmarks......Page 201
Extension to Wider Areas......Page 203
The Vanishing Line for Zoomed Head Localization......Page 205
Experimental Results......Page 208
Conclusions......Page 213
References......Page 214
Introduction......Page 217
Architecture Design in Multi-Modal Systems......Page 218
Logical Architecture Design......Page 219
Physical Architecture Design......Page 221
Data Alignment......Page 225
Multi-Modal Techniques for State Estimation and Localization......Page 227
Fusion of Multi-Modal Cues for Event Analysis......Page 233
Applications......Page 234
Ambient Intelligence Applications......Page 235
Conclusions......Page 238
References......Page 239
Introduction......Page 242
Cameras......Page 243
Projective Geometry for Catadioptric Systems......Page 244
Spherical Camera Model......Page 246
Image Processing on the Sphere......Page 248
Intrinsic Parameters......Page 250
Extrinsic Parameters......Page 252
Epipolar Geometry for Paracatadioptric Cameras......Page 253
Disparity Estimation......Page 255
Correlation Estimation with Sparse Approximations......Page 259
Distributed Coding of 3D Scenes......Page 261
Conclusions......Page 264
References......Page 265
Introduction......Page 268
Classic Approach to Video Coding......Page 269
Slepian-Wolf Theorem......Page 273
A Simple Example......Page 275
Channel Codes for Binary Source DSC......Page 276
Wyner-Ziv Theorem......Page 278
Applying DSC to Video Coding......Page 279
PRISM Codec......Page 281
Stanford Approach......Page 283
Remarks......Page 286
Applying DVC to Multi-View Systems......Page 289
Extending Mono-View Codecs......Page 290
Remarks on Multi-View Problems......Page 292
References......Page 293
Introduction......Page 296
Foundations of Distributed Source Coding......Page 297
Structure and Properties of the Plenoptic Data......Page 300
Distributed Compression of Multi-View Images......Page 302
Multi-Terminal Distributed Video Coding......Page 307
Conclusions......Page 308
References......Page 309
Introduction......Page 311
Co-Training......Page 314
Boosting for Feature Selection......Page 315
Co-Training System......Page 317
Scene Calibration......Page 318
Online Co-Training......Page 319
Experimental Results......Page 322
Indoor Scenario......Page 323
Outdoor Scenario......Page 326
Resources......Page 327
References......Page 330
Introduction......Page 333
Background......Page 334
Tracking......Page 335
Example-Based Methods......Page 336
Segmentation......Page 337
Reconstruction......Page 339
Linear Discriminant Analysis......Page 342
Average Neighborhood Margin Maximization......Page 343
3D Haarlets......Page 345
Training......Page 346
Experiments......Page 349
Rotation Invariance......Page 350
Overhead Tracker......Page 351
Experiments......Page 354
Results and Conclusions......Page 355
References......Page 357
Introduction......Page 360
Key Factors and Related Work......Page 361
Approach and Chapter Organization......Page 365
Bayesian Tracking Problem Formulation......Page 366
Single-Object 3D State and Model Representation......Page 367
Joint Dynamic Model......Page 368
Single-Object Dynamic Model......Page 370
Color Likelihood......Page 372
Reversible-Jump MCMC......Page 375
Move Proposals......Page 376
Calibration and Slant Removal......Page 379
Results......Page 380
Conclusions......Page 382
References......Page 384
Introduction......Page 386
Background Modeling......Page 388
Single-Camera Person Tracking......Page 390
The Tracking Algorithm......Page 391
Occlusion Detection and Classification......Page 395
Bayesian-Competitive Consistent Labeling......Page 397
Trajectory Shape Analysis for Abnormal Path Detection......Page 401
Trajectory Shape Classification......Page 404
Experimental Results......Page 406
References......Page 409
Introduction......Page 411
Related Work......Page 413
Multiple Stationary Cameras with Overlapping Fields of View......Page 414
Multiple Pan-Tilt-Zoom Cameras......Page 415
Evaluating an Association Using Appearance Information......Page 416
Estimating the Subspace of BTFs Between Cameras......Page 417
Data Model......Page 418
Maximum Likelihood Estimation......Page 420
Simulations......Page 422
Real Sequences......Page 424
Conclusions......Page 426
References......Page 427
Introduction......Page 430
Single-Camera Surveillance System Architecture......Page 434
System Design......Page 435
Cross-Camera Calibration......Page 437
Data Fusion......Page 441
Critical Infrastructure Protection......Page 444
Hazardous Lab Safety Verification......Page 446
Testing and Results......Page 447
Future Work......Page 448
References......Page 449
19 Composite Event Detection in Multi-Camera and Multi-Sensor Surveillance Networks......Page 452
Introduction......Page 453
Related Work......Page 454
System Infrastructure......Page 456
Event Representation and Detection......Page 458
Primitive Events and User Interfaces......Page 460
Composite Event Search......Page 463
Query-Based Search and Browsing......Page 464
Application: Retail Loss Prevention......Page 467
Application: Tailgating Detection......Page 469
Application: False Positive Reduction......Page 471
References......Page 472
Introduction......Page 476
The Evolution of Smart Camera Systems......Page 478
Single Smart Cameras......Page 479
Distributed Smart Cameras......Page 480
Smart Cameras in Sensor Networks......Page 481
Future and Challenges......Page 483
Distributed Algorithms......Page 484
Privacy and Security......Page 485
Conclusions......Page 486
References......Page 487
Introduction......Page 490
Centralized Processing......Page 491
Distributed Processing......Page 493
Sensor Modules......Page 494
Processing Module......Page 495
Communication Modules......Page 497
MeshEye......Page 498
CMUcam3......Page 499
Conclusions......Page 500
References......Page 501
Introduction......Page 504
Smart Cameras......Page 505
Distributed Smart Cameras......Page 506
Challenges of Distributed Smart Cameras......Page 507
Middleware Architecture......Page 508
Middleware for Embedded Systems......Page 510
Specific Requirements of Distributed Smart Cameras......Page 511
From Objects to Agents......Page 512
Code Mobility and Programming Languages......Page 513
Mobile Agents for Embedded Smart Cameras......Page 514
DSCAgents......Page 515
Decentralized Multi-Camera Tracking......Page 519
Sensor Fusion......Page 523
Conclusions......Page 526
References......Page 527
Introduction......Page 531
Event-Driven Clustering Protocols......Page 533
Distributed Kalman Filtering......Page 536
Object Tracking with Wireless Camera Networks......Page 538
Clustering Protocol......Page 540
Kalman Filter Equations......Page 546
State Estimation......Page 550
Experimental Results......Page 552
Simulator Environment......Page 553
Testbed Implementation......Page 558
Conclusions and Future Work......Page 560
References......Page 561
Systems and Algorithms in Realistic Scenarios......Page 565
Interfacing Vision Processing and Reasoning......Page 566
Performance Evaluation......Page 568
User and Social Acceptance......Page 569
Conclusions......Page 570
A......Page 571
C......Page 572
D......Page 575
E......Page 576
F......Page 577
I......Page 578
M......Page 579
O......Page 581
P......Page 582
R......Page 583
S......Page 584
T......Page 586
W......Page 587
Z......Page 588