Visual Sensing and its Applications: Integration of Laser Sensors to Industrial Robots

A.3 Nonlinear Optimization Algorithms......Page 3
Title Page......Page 4
Copyright Page......Page 5
Index......Page 7
  7.1.5 Experimental Results and Conclusion......Page 10
Cover......Page 1
 A.2 Singular Value Decomposition (SVD)......Page 2
Foreword......Page 6
  4.1.1 Calibration Target with Geometry Constraint: Point......Page 9
Table of Contents......Page 11
  7.2.1 Introduction......Page 12
  7.2.3 Workpiece Calibration......Page 14
1 Introduction......Page 15
  1.1.2 Stereo Vision......Page 18
Preface......Page 8
  7.2.2 Offline Programming......Page 13
  1.1.1 2D Vision......Page 17
  1.1.3 Time of Flight......Page 19
 1.1 3D Acquisition Techniques......Page 16
  7.3.2 Profile Modeling Based Grinding......Page 22
  1.2.2 Selection of Industrial Robots......Page 23
  7.3.3 Experimental Setup......Page 24
  1.2.3 Applications of Robot Visual Systems......Page 25
  7.4.2 Approximately Locating Workpieces......Page 27
  1.2.5 Laser Sensor Based Commercial Robot Visual Systems......Page 28
 1.3 Outline of Chapters......Page 29
  7.4.4 Another Example......Page 30
 References......Page 32
 2.1 Formation of Laser Structured-Light Sensors......Page 34
  7.5.5 Other Examples......Page 35
  2.1.1 Light Source......Page 36
  7.6.1 System Overview......Page 37
  7.6.2 Pick-up Error Compensation......Page 38
   2.1.2.1 PSD Sensor......Page 39
   2.1.2.2 CCD Sensor......Page 40
  2.1.3 Triangulation Measurement Principle......Page 42
   7.8.1.3 Log Centering......Page 49
  1.1.4 Laser Triangulation Sensor......Page 20
   7.3.1.3 Turbine Blade/Vane Refurbishment......Page 21
   2.1.2.3 CMOS Sensor......Page 41
  7.7.3 Path Generation......Page 43
 2.2 Accuracy Analysis......Page 44
  7.8.1 Robot Logs Pick and Center System......Page 47
  1.2.4 Calibration of Robot Visual Systems......Page 26
  7.5.1 Introduction......Page 31
  7.5.3 Scanning and Edge Detection......Page 33
  7.7.5 A Robotic Tube Panel Weld System......Page 45
  2.2.2 Effect of the Environmental Factors on the Measurement Accuracy......Page 48
 2.3 Commercial Systems......Page 50
  7.8.2 Robot Solar Panel Installation System......Page 51
 References......Page 52
  7.8.3 Summary and Remarks......Page 53
 3.1 Modeling of Laser Stripe Sensor and Calibration Strategy......Page 54
  2.2.1 Effect of Laser Speckle Noise on the Measurement Accuracy......Page 46
 3.2 Camera Modeling......Page 55
  3.2.1 Pinhole Model of the Camera......Page 56
  3.2.2 Nonlinear Modeling with Lens Distortion......Page 59
  3.3.1 Calibration with Direct Linear Transform Method......Page 60
   3.3.1.1 Solution of DLT Parameters......Page 62
   3.3.1.2 Solution of DLT Parameters with Lens Distortion......Page 63
  3.3.2 Calibration with Tsai’s RAC Based Algorithm......Page 64
  3.3.3 Calibration with Multiple View Algorithms......Page 67
   3.3.3.1 Mapping between 3D Calibration Planar Points and 2D Image Points......Page 68
   3.3.3.2 Closed Form Solution......Page 69
   3.3.3.3 Nonlinear Solutions......Page 70
 3.4 Calibration of Laser Stripe Sensor......Page 71
  3.4.1 Laser Stripe Plane Calibration with Two Known Planes......Page 75
  3.4.2 Laser Stripe Plane Calibration Based on Invariance of Cross Ratios......Page 76
   3.4.2.2 Invariance of Double Cross Ratio......Page 77
   3.4.2.3 Selection of Calibration Target......Page 78
  3.4.3 Laser Plane Calibration with a Planar Target......Page 80
   3.4.3.1 Determination of the Control Points Based on the Invariance of the Cross Ratio......Page 81
   3.4.3.2 Determination of the Control Points based on Ray Tracing......Page 82
  3.4.5 Calibration of the Rotation Table......Page 83
  3.4.6 Calibration of the Laser Stripe Sensor with Robot Alignment......Page 85
   3.4.7.1 Linear mapping......Page 87
   3.4.7.2 Polynomial Mapping Function......Page 88
   3.4.7.3 Measurement of the Calibration Points on the Laser Plane......Page 89
  3.4.8 Calibration of Laser Stripe Sensor with Scheimpflug Configuration......Page 90
   3.4.8.1 The Extension of Depth-of-View based on the Scheimpflug Condition......Page 91
   3.4.8.2 Camera and Scanner Calibration......Page 92
   3.4.8.3 Calibration of Camera......Page 93
   3.4.8.4 Experimental Results and Discussion......Page 95
 References......Page 102
4 Calibration of a Robot Visual System......Page 105
 4.1 General Solution of Robot Tool Calibration......Page 106
  4.1.1 Calibration Target with Geometry Constraint: Point......Page 113
  4.1.2 Calibration Target with Geometry Constraint: Line......Page 114
  4.1.3 Calibration Target with Geometry Constraint: Sphere......Page 116
  4.1.4 Calibration Target with Geometry Constraint: Plane......Page 117
   4.1.4.1 Measurement of Surface Plane Normal......Page 118
   4.1.4.2 Calibration of tx, ty, and tz......Page 119
  4.2.1 Algorithms......Page 121
  4.2.2 Calibration of Laser Beam Orientation (nx , ny , nz)......Page 122
  4.2.3 Calibration of Laser Sensor Position (x0 , y0 , z0)......Page 123
  4.2.4 Experimental Results......Page 124
 4.3 TCP Calibration for Cameras......Page 125
  4.3.1 Camera Pose Calibration with Linear Equations......Page 126
  4.3.2 Camera Pose Calibration with Nonlinear Optimizations......Page 128
   4.4.1.1 Calibration Algorithms......Page 129
   4.4.1.2 Experimental Verification......Page 132
  4.4.2 TCP Calibration with a Plane......Page 133
  4.4.3 TCP Calibration with a Structured Pattern......Page 136
 4.5 TCP Calibration with Direct Measurement......Page 140
  4.5.1 Calibration of Spindle......Page 141
  4.5.2 Calibration of Tools with Different Length......Page 142
  4.6.1 Robot Workcell Calibration......Page 143
   4.6.1.1 TCP Calibration......Page 144
   4.6.1.2 Zero Reference Path Generation......Page 145
   4.6.1.3 Workpiece Calibration......Page 147
  4.6.2 Robot Error Compensation with Relative Measurement......Page 148
 4.7 Summary......Page 151
 5.1 Control Point Extraction from Pattern Images......Page 154
  5.1.1 Feature Extraction from Squared Control Points......Page 155
   5.1.1.1 Edge Extraction by Using the Gradient Operator......Page 156
   5.1.1.2 Line Intersection to Calculate the Corner Positions......Page 157
  5.1.2 Feature Extraction from Circle Control Points......Page 158
   5.1.2.1 Median Filtering......Page 159
   5.1.2.3 Binary Image Labeling......Page 160
   5.1.2.4 Calculation of the Circle Position......Page 161
  5.2.1 Thinning and Pruning Algorithm......Page 163
   5.2.2.1 Calculation of Normal Direction with Directional Template......Page 167
   5.2.2.2 Calculation of Normal Direction with Sobel Operator......Page 168
 5.3 Range Image Registration with the ICP Algorithm......Page 169
  5.3.1 Determination of Corresponding Points......Page 170
  5.3.2 Calculation of Transformation Matrix......Page 171
 References......Page 174
 6.1 Background......Page 176
 6.2 Model Function of Robots......Page 178
 6.3 Determination of Independent Error Parameters Using SVD Method......Page 180
 6.4 Error Budget Analysis......Page 181
 6.5 Solving the Error Parameters......Page 182
 6.7 TAU Parallel Robot Calibration......Page 185
  6.7.1 Kinematic Modeling......Page 186
  6.7.2 Jacobian Matrix of TAU Robot with All Error Parameters......Page 189
   6.7.2.1 Formulation of Forward Jacobian Matrix......Page 190
   6.7.2.2 Jacobian Matrix in Case a Tool is Attached......Page 191
   6.7.2.3 Formulation of Inverse Jacobian Matrix......Page 192
  6.7.3 Kinematic Modeling with all Error Parameters......Page 195
  6.7.4 Determination of Independent Design Variables......Page 196
  6.7.5 Error Budget Analysis......Page 197
  6.7.6 Simulation Results......Page 199
 References......Page 201
7 Visual Sensing and Control-Laser Sensor Based Robot Applications......Page 204
  7.1.1 Introduction......Page 205
  7.1.2 System Overview......Page 206
   7.1.3.1 Calibration of Vision System......Page 207
   7.1.3.2 Calibration of Work Object......Page 210
   7.1.4.1 Determine the Hole Orientation......Page 211
   7.1.4.2 Determination of the Hole Position......Page 213
 7.2 Robotic Grinding System of Free-Form Workpieces......Page 214
  7.2.1 Introduction......Page 215
  7.2.2 Offline Programming......Page 216
  7.2.3 Workpiece Calibration......Page 217
  7.2.4 Robotic System Error Compensation......Page 218
  7.2.5 Experimental System......Page 221
   7.3.1.1 Traditional Grinding Process......Page 223
   7.3.1.3 Turbine Blade/Vane Refurbishment......Page 224
  7.3.2 Profile Modeling Based Grinding......Page 225
  7.3.3 Experimental Setup......Page 227
  7.4.1 System Overview......Page 229
  7.4.2 Approximately Locating Workpieces......Page 230
  7.4.3 Precisely Locating Workpieces......Page 231
  7.4.4 Another Example......Page 233
  7.5.1 Introduction......Page 234
  7.5.2 System Overview......Page 235
  7.5.3 Scanning and Edge Detection......Page 236
  7.5.4 Path Smoothing Based on the B-Spline......Page 237
  7.5.5 Other Examples......Page 238
  7.5.6 Summary and Remarks......Page 240
  7.6.2 Pick-up Error Compensation......Page 241
  7.6.4 Point Cloud Comparison......Page 243
  7.6.5 Summary and Remarks......Page 244
  7.7.2 Welding Joint Detection......Page 245
  7.7.3 Path Generation......Page 246
  7.7.4 Computer-Robot Communication......Page 247
  7.7.5 A Robotic Tube Panel Weld System......Page 248
  7.7.6 Summary and Remarks......Page 250
   7.8.1.1 Laser Sensor......Page 251
   7.8.1.3 Log Centering......Page 252
   7.8.1.4 Locate the Object with Laser Sensors......Page 253
  7.8.2 Robot Solar Panel Installation System......Page 254
   7.8.2.1 System Layout and Positioning Process......Page 255
  7.8.3 Summary and Remarks......Page 256
 References......Page 257
 A.1 Linear Equations and Its Solution......Page 259
 A.2 Singular Value Decomposition (SVD)......Page 260
 A.3 Nonlinear Optimization Algorithms......Page 261
 A.4 B-Spline Data Fitting......Page 262
Index......Page 265