Multi-sensor image fusion and its applications

Multi-Sensor Image Fusion and Its Applications......Page 1
Preface......Page 7
Editors......Page 11
Contributors......Page 12
Table of contents......Page 15
Dedication......Page 17
CONTENTS......Page 18
I. INTRODUCTION TO IMAGE FUSION......Page 19
B. APPLICATIONS OF IMAGE FUSION......Page 21
A. MULTISCALE-DECOMPOSITION-BASED FUSION METHODS......Page 24
1. Multiscale Decomposition......Page 26
3. Coefficient Grouping Method......Page 33
B. NONMULTISCALE-DECOMPOSITION-BASED METHODS......Page 34
3. Estimation Theory Based Methods......Page 35
5. Artificial Neural Networks......Page 36
A. PERFORMANCE EVALUATION......Page 37
1. Objective Evaluation Measures Requiring a Reference Image......Page 38
2. Objective Evaluation Measures Not Requiring a Reference Image......Page 41
B. IMAGE REGISTRATION......Page 43
REFERENCES......Page 46
I. INTRODUCTION......Page 53
II. MUTUAL INFORMATION AS A GENERIC SIMILARITY MEASURE......Page 56
III. INTERPOLATION INDUCED ARTIFACTS......Page 59
IV. GENERALIZED PARTIAL VOLUME ESTIMATION OF JOINT HISTOGRAM......Page 61
V. OPTIMIZATION......Page 63
A. SIMPLEX SEARCH ALGORITHM......Page 65
B. MULTIRESOLUTION OPTIMIZATION......Page 66
VI. APPLICATION TO 3D BRAIN IMAGE REGISTRATION......Page 67
VII. SUMMARY......Page 68
REFERENCES......Page 71
CONTENTS......Page 73
II. IMAGING MODALITIES......Page 74
A. TRANSFORMATION TYPES......Page 78
a. Correlation Methods......Page 83
b. Sequential Methods......Page 84
2. Fourier Methods......Page 86
3. Feature-Based Methods......Page 88
a. Vessel Segment Ends......Page 91
c. Blood Vessel Bifurcations......Page 92
6. Mutual Information Methods......Page 97
2. Quantitative......Page 99
IV. FUSION......Page 100
A. COMBINATION BY GRAPHICAL SUPERPOSITION......Page 101
c. Multiresolution Methods......Page 103
c. MIT False-Color Method......Page 107
1. Qualitative......Page 111
c. Cross-Entropy......Page 113
d. Image Noise Index......Page 114
e. Spatial Frequency......Page 115
V. CONCLUSION......Page 116
REFERENCES......Page 118
FURTHER READING......Page 121
CONTENTS......Page 123
A. CONTEXT......Page 124
B. MR/US REGISTRATION......Page 125
D. INTENSITY BASED NONRIGID REGISTRATION ALGORITHMS......Page 126
E. OVERVIEW OF THE ARTICLE’S ORGANIZATION......Page 127
2. Correlation Coefficient (CC)......Page 128
B. BIVARIATE CORRELATION RATIO......Page 129
C. PARAMETRIC INTENSITY FIT......Page 130
D. ROBUST INTENSITY DISTANCE......Page 131
A. PARAMETERIZATION OF THE TRANSFORMATION......Page 132
C. MINIMIZING THE SSD FOR A FREE-FORM DEFORMATION......Page 133
E. REGULARIZATION ENERGY......Page 135
F. FROM REGISTRATION TO TRACKING......Page 136
IV. EXPERIMENTS......Page 137
2. Patient Images During Tumor Resection......Page 138
3. A Phantom Study......Page 141
B. MR/US RIGID REGISTRATION CONSISTENCY EVALUATION......Page 146
1. Registration Loops......Page 149
2. Bronze Standard Registration......Page 150
3. Consistency Results......Page 151
C. 3D US TRACKING PERFORMANCES......Page 152
V. DISCUSSION......Page 154
ACKNOWLEDGMENTS......Page 155
REFERENCES......Page 156
CONTENTS......Page 160
I. INTRODUCTION......Page 161
II. BACKGROUND AND PROBLEM STATEMENT......Page 162
A. PROBLEM STATEMENT......Page 164
1. Reconstruction from Projection Data......Page 165
2. Deblurring of Locally Sensed Data......Page 167
B. RELATED WORK IN IMAGE-BASED FUSION......Page 168
1. The Mumford–Shah Variational Approach to Image Processing......Page 169
2. Single Parameter Image Fusion......Page 170
3. Multiparameter Image Fusion......Page 171
III. SHARED BOUNDARY FUSION FORMULATION......Page 173
A. SENSOR OBSERVATION MODEL TERM......Page 175
B. NOISE SUPPRESSION TERM......Page 176
C. ALIGNMENT TERM......Page 177
D. BOUNDARY TERM......Page 178
IV. OPTIMIZATION APPROACH......Page 179
A. SHARED BOUNDARY ESTIMATION......Page 181
B. BOUNDARY AWARE IMAGE FORMATION......Page 182
C. MULTIMODAL ALIGNMENT......Page 184
1. Observation and Inversion Model......Page 188
2. Fusion Results......Page 191
1. Data Acquisition......Page 193
2. Fusion Results......Page 194
REFERENCES......Page 196
CONTENTS......Page 200
I. INTRODUCTION......Page 201
A. RÉNYI ENTROPY AND DIVERGENCE......Page 206
B. MUTUAL INFORMATION AND alpha-MUTUAL INFORMATION......Page 207
D. alpha-GEOMETRIC-ARITHMETIC MEAN DIVERGENCE......Page 210
E. HENZE–PENROSE AFFINITY......Page 211
III. CONTINUOUS QUASIADDITIVE EUCLIDEAN FUNCTIONALS......Page 212
A. A MINIMAL SPANNING TREE FOR ENTROPY ESTIMATION......Page 213
B. NEAREST NEIGHBOR GRAPH ENTROPY ESTIMATOR......Page 218
IV. ENTROPIC GRAPH ESTIMATE OF HENZE–PENROSE AFFINITY......Page 221
V. ENTROPIC GRAPH ESTIMATORS OF alpha-GA AND alpha-MI......Page 222
A. ICA BASIS PROJECTION FEATURES......Page 226
B. MULTIRESOLUTION WAVELET BASIS FEATURES......Page 227
A. REDUCING TIME-MEMORY COMPLEXITY OF THE MST......Page 228
B. REDUCING TIME-MEMORY COMPLEXITY OF THE KNNG......Page 231
VIII. APPLICATIONS: MULTISENSOR SATELLITE IMAGE FUSION......Page 234
A. DEFORMATION AND FEATURE DEFINITION......Page 235
A. DEFORMATION LOCALIZATION......Page 237
B. LOCAL FEATURE MATCHING RESULTS......Page 241
REFERENCES......Page 243
A1. APPENDIX......Page 248
I. INTRODUCTION......Page 279
A. IMAGERY......Page 281
B. FUSION METHODS......Page 282
C. TEST METHODS......Page 284
D. RESULTS......Page 287
E. DISCUSSION......Page 288
A. IMAGERY......Page 290
C. TEST METHODS......Page 294
D. RESULTS......Page 297
1. Perception of Global Structure......Page 298
2. Perception of Detail......Page 299
3. Summary......Page 300
E. DISCUSSION......Page 301
IV. CONCLUSIONS......Page 302
REFERENCES......Page 303
CONTENTS......Page 307
I. INTRODUCTION......Page 308
A. TREE STRUCTURE OF THE WAVELET COEFFICIENTS......Page 309
C. IMAGE FORMATION MODEL......Page 310
III. FUSION WITH THE EM ALGORITHM......Page 312
B. UPDATING PARAMETERS USING THE EM ALGORITHM......Page 314
C. INITIALIZATION OF THE FUSION ALGORITHM......Page 315
A. CWD WITH VISUAL AND MMW IMAGES......Page 317
V. CONCLUSIONS......Page 319
REFERENCES......Page 322
A.1. DERIVATION OF CONDITIONAL PROBABILITIES......Page 324
A.2.1. Upward Step......Page 325
APPENDIX B OUTLINE OF THE DERIVATION OF THE UPDATE EQUATIONS......Page 326
I. INTRODUCTION......Page 330
II. LEVELS OF FUSION......Page 334
III. FUSION SCENARIOS......Page 336
V. INTEGRATION STRATEGIES......Page 337
VI. DESIGN ISSUES......Page 338
VII. SUMMARY AND CONCLUSIONS......Page 339
REFERENCES......Page 340
CONTENTS......Page 343
II. A PRIORI INFORMATION......Page 344
1. Methodology......Page 345
2. Choosing the Clique Topology and the Optimization Algorithm......Page 348
B. RESULTS ON RADARSAT-1 IMAGERY......Page 350
A. HIERARCHICAL CLASSIFIER......Page 354
B. SELECTION OF GLCM FEATURES USING GENETIC ALGORITHMS......Page 355
3. Results......Page 356
IV. RESULTS......Page 357
REFERENCES......Page 360
CONTENTS......Page 362
A. DEFINITION AND IMPORTANCE OF REMOTELY SENSED IMAGE REGISTRATION......Page 363
1. Data Acquisition Issues......Page 365
b. Temporal Changes......Page 366
c. Terrain Relief......Page 367
d. Multisensor Issues......Page 368
b. Lack of Ground Truth......Page 370
A. CHARACTERISTICS OF IMAGE REGISTRATION METHODS FOR REMOTE SENSING......Page 371
1. Intensity, Area-Based Algorithms......Page 373
4. Mutual Information Algorithms......Page 374
A. CORRELATION-BASED EXPERIMENTS......Page 375
C. SIMILARITY MEASURES EXPERIMENTS......Page 376
D. COMBINATION ALGORITHMS EXPERIMENTS......Page 377
2. Multitemporal Dataset......Page 378
3. Multisensor Dataset......Page 379
IV. CONCLUSION AND FUTURE WORK......Page 380
ACKNOWLEDGMENTS......Page 381
REFERENCES......Page 382
I. INTRODUCTION......Page 385
A. LINEAR MINIMUM MEAN SQUARE ERROR FILTER......Page 387
1. LMMSE Filter — System with N Inputs without Degradation......Page 388
2. LMMSE Filter — System with N Inputs Degraded by Additive Noise......Page 391
B. MORPHOLOGICAL PROCESSING APPROACH TO FUSION......Page 397
III. MODEL-BASED DATA FUSION......Page 401
A. Q-TRANSFORM......Page 402
B. DEFINITION AND MAPPING PROPERTY......Page 405
C. NUMERICAL COMPUTATION OF THE Q-TRANSFORM......Page 406
1. Signal Level Data Fusion......Page 408
2. Feature Level Data Fusion......Page 410
REFERENCES......Page 411
I. INTRODUCTION......Page 413
B. NDI TECHNIQUES FOR CORROSION DETECTION......Page 417
C. TEST COMPONENT......Page 418
D. QUANTIFICATION OF NDI RESULTS......Page 421
A. DATA ALIGNMENT AND REGISTRATION......Page 422
B. VERIFICATION AND EVALUATION......Page 423
1. Pixel-Level Fusion......Page 424
2. Classification-Based Approach......Page 426
3. Estimation with a General Additive Model......Page 431
IV. DISCUSSION......Page 436
ACKNOWLEDGMENTS......Page 439
REFERENCES......Page 440
I. INTRODUCTION......Page 443
II. MULTICHANNEL IMAGE ACQUISITION MODELS......Page 445
III. PIECEWISE IDEAL IMAGING......Page 446
A. APPLICATION IN CONFOCAL MICROSCOPY......Page 447
IV. UNIFORMLY BLURRED CHANNELS......Page 449
A. ALTERNATING MINIMIZATION ALGORITHM......Page 451
2. Regularization of the Blurs R(h)......Page 452
3. Iterative Minimization Algorithm......Page 453
B. EXPERIMENT WITH ARTIFICIAL DATA......Page 454
C. EXPERIMENT WITH REAL DATA......Page 455
V. SLIGHTLY MISREGISTERED BLURRED CHANNELS......Page 458
A. MAXIMUM A POSTERIORI PROBABILITY ALGORITHM......Page 459
VI. HEAVILY MISREGISTERED BLURRED CHANNELS......Page 460
VII. CHANNELS WITH SPACE-VARIANT BLURRING......Page 463
VIII. CONCLUSION......Page 465
REFERENCES......Page 466
CONTENTS......Page 469
I. INTRODUCTION......Page 470
II. MULTIMODALITY DISPLAYS......Page 471
III. FOCUS + CONTEXT DISPLAYS......Page 472
V. GAZE-CONTINGENT DISPLAYS......Page 477
VII. GAZE-CONTINGENT MULTIMODALITY DISPLAYS......Page 478
A. TWO-DIMENSIONAL GCMMD FOR IMAGE FUSION......Page 480
2. Surveillance Images......Page 481
3. Remote Sensing Images......Page 482
1. Challenges of Three-Dimensional Gaze-Tracking......Page 485
2. Three-Dimensional Spatially Variant Rendering......Page 490
3. Three-Dimensional GCMMDs of Medical Images......Page 491
1. Implementation......Page 493
2. Performance......Page 495
1. Implementation......Page 496
2. Performance......Page 499
C. INTEGRATION WITH EYE-TRACKERS......Page 501
3. Three-Dimensional Gaze-Tracker......Page 502
ACKNOWLEDGMENTS......Page 505
REFERENCES......Page 506
I. INTRODUCTION......Page 510
II. THE STRUCTURAL SIMILARITY PARADIGM......Page 514
A. THE STRUCTURAL SIMILARITY INDEX......Page 515
B. SSIM INDEX IN IMAGE QUALITY ASSESSMENT......Page 519
III. THE INFORMATION THEORETIC PARADIGM......Page 520
A. NATURAL SCENE MODEL......Page 523
C. HVS MODEL......Page 524
D. THE VISUAL INFORMATION FIDELITY MEASURE......Page 525
IV. PERFORMANCE OF SSIM AND VIF......Page 527
REFERENCES......Page 534