CUDA application design and development: Machine generated contents note: 1. How to think in CUDA 2. Tools to build, debug and profile 3. The GPU performance envelope 4. The CUDA memory subsystems 5. Exploiting the CUDA execution grid 6. MultiGPU applications and scaling 7. Numerical CUDA, libraries and high-level language bindings 8. Mixing CUDA with rendering 9. High Performance Machine Learning 10. Scientific Visualization 11. Multimedia with OpenCV 12. Ultra Low-power Devices: Tegra

01-fm-i-iii-9780123884268......Page 3
02-cpyrt-iv-iv-9780123884268......Page 6
03-ded-v-vi-9780123884268......Page 7
04-toc-vii-x-9780123884268......Page 9
05-fore-xi-xii-9780123884268......Page 13
06-pref-xiii-xviii-9780123884268......Page 15
1 First Programs and How to Think in CUDA......Page 20
Distinguishing CUDA from Conventional Programming with a Simple Example......Page 21
Choosing a CUDA API......Page 24
Some Basic CUDA Concepts......Page 27
Understanding Our First Runtime Kernel......Page 30
Rule 1: Get the Data on the GPU and Keep It There......Page 32
Rule 3: Focus on Data Reuse within the GPGPU to Avoid Memory Bandwidth Limitations......Page 33
Big-O Considerations and Data Transfers......Page 34
CUDA and Amdahl’s Law......Page 36
Data and Task Parallelism......Page 37
Hybrid Execution: Using Both CPU and GPU Resources......Page 38
Regression Testing and Accuracy......Page 40
Silent Errors......Page 41
Introduction to Debugging......Page 42
NVIDIA's cuda-gdb Debugger......Page 43
The CUDA Memory Checker......Page 45
Use cuda-gdb with the UNIX ddd Interface......Page 46
Windows Debugging with Parallel Nsight......Page 48
Summary......Page 49
2 CUDA for Machine Learning and Optimization......Page 51
Modeling and Simulation......Page 52
Fitting Parameterized Models......Page 53
Levenberg-Marquardt Method......Page 54
Algorithmic Speedups......Page 55
Machine Learning and Neural Networks......Page 56
XOR: An Important Nonlinear Machine-Learning Problem......Page 57
An Example Objective Function......Page 59
A Complete Functor for Multiple GPU Devices and the Host Processors......Page 60
Brief Discussion of a Complete Nelder-Mead Optimization Code......Page 62
Performance Discussion......Page 71
Summary......Page 74
The C++ Nelder-Mead Template......Page 75
3 The CUDA Tool Suite: Profiling a PCA/NLPCA Functor......Page 80
PCA and NLPCA......Page 81
Autoencoders......Page 82
An Example Functor for PCA Analysis......Page 83
An Example Functor for NLPCA Analysis......Page 85
Obtaining Basic Profile Information......Page 88
Gprof: A Common UNIX Profiler......Page 90
The NVIDIA Visual Profiler: Computeprof......Page 91
The Nsight Timeline Analysis......Page 94
The NVTX Tracing Library......Page 96
Scaling Behavior of the CUDA API......Page 97
Tuning and Analysis Utilities (TAU)......Page 99
Summary......Page 100
4 The CUDA Execution Model......Page 101
GPU Architecture Overview......Page 102
Thread Scheduling: Orchestrating Performance and Parallelism via the Execution Configuration......Page 103
Warp Divergence......Page 106
Guidelines for Warp Divergence......Page 107
Warp Scheduling and TLP......Page 108
ILP: Higher Performance at Lower Occupancy......Page 110
ILP Hides Arithmetic Latency......Page 111
ILP in the Future......Page 114
Little’s Law......Page 116
CUDA Tools to Identify Limiting Factors......Page 118
The nvcc Compiler......Page 119
Launch Bounds......Page 120
The Disassembler......Page 121
PTX Kernels......Page 122
GPU Emulators......Page 123
Summary......Page 124
The CUDA Memory Hierarchy......Page 125
GPU Memory......Page 127
L2 Cache......Page 128
Relevant computeprof Values for the L2 Cache......Page 129
L1 Cache......Page 130
Relevant computeprof Values for the L1 Cache......Page 131
Local memory......Page 132
Shared Memory......Page 133
Constant Memory......Page 136
Texture Memory......Page 137
Global Memory......Page 140
Common Coalescing Use Cases......Page 142
Allocation of Global Memory......Page 143
Limiting Factors in the Design of Global Memory......Page 144
Relevant computeprof Values for Global Memory......Page 146
Summary......Page 147
6 Efficiently Using GPU Memory......Page 148
The Reduction Template......Page 149
A Test Program for functionReduce.h......Page 155
Results......Page 159
Utilizing Irregular Data Structures......Page 161
Sparse Matrices and the CUSP Library......Page 164
Graph Algorithms......Page 166
Tiles and Stencils......Page 169
Summary......Page 170
7 Techniques to Increase Parallelism......Page 172
CUDA Contexts Extend Parallelism......Page 173
Multiple GPUs......Page 174
Explicit Synchronization......Page 175
Implicit Synchronization......Page 176
A Simple Example......Page 177
Profiling Results......Page 180
Out-of-Order Execution with Multiple Streams......Page 181
Atomic Operations for Implicitly Concurrent Kernels......Page 184
Manually Partitioning Data......Page 187
Mapped Memory......Page 188
How Mapped Memory Works......Page 190
Summary......Page 191
8 CUDA for All GPU and CPU Applications......Page 193
Pathways from CUDA to Multiple Hardware Backends......Page 194
The PGI CUDA x86 Compiler......Page 195
The PGI CUDA x86 Compiler......Page 197
An x86 core as an SM......Page 199
The NVIDIA NVCC Compiler......Page 200
Ocelot......Page 201
Accessing CUDA from Other Languages......Page 202
Copperhead......Page 203
MATLAB......Page 204
CUFFT......Page 205
MAGMA......Page 216
CURAND......Page 217
Summary......Page 219
9 Mixing CUDA and Rendering......Page 220
GLUT......Page 221
Mapping GPU Memory with OpenGL......Page 222
Using Primitive Restart for 3D Performance......Page 223
The Demo and Perlin Example Kernels......Page 226
The Demo Kernel to Generate a Colored Sinusoidal Surface......Page 227
Perlin Noise......Page 230
Using the Perlin Noise Kernel to Generate Artificial Terrain......Page 232
The simpleGLmain.cpp File......Page 237
The simpleVBO.cpp File......Page 241
The callbacksVBO.cpp File......Page 246
Summary......Page 251
10 CUDA in a Cloud and Cluster Environments......Page 253
The MPI Programming Model......Page 254
MPI Rank......Page 255
Point-to-Point Basics......Page 257
How MPI Communicates......Page 258
Bandwidth......Page 260
Balance Ratios......Page 261
Scalability of the Initial Data Load......Page 264
Using MPI to Perform a Calculation......Page 265
Check Scalability......Page 266
Cloud Computing......Page 267
Data Generation......Page 268
Summary......Page 276
11 CUDA for Real Problems......Page 277
Working with High-Dimensional Data......Page 278
Multidimensional Scaling......Page 279
Expectation-Maximization......Page 280
Bayesian Networks......Page 281
Mutual information......Page 282
Force-Directed Graphs......Page 283
Monte Carlo Methods......Page 284
Quantum Chemistry......Page 285
A Plethora of Projects......Page 286
Summary......Page 287
12 Application Focus on Live Streaming Video......Page 288
Topics in Machine Vision......Page 289
Segmentation of Flesh-colored Regions......Page 290
Edge Detection......Page 291
FFmpeg......Page 292
TCP Server......Page 294
kernelWave(): An Animated Kernel......Page 298
kernelSkin(): Keep Only Flesh-colored Regions......Page 299
kernelSobel(): A Simple Sobel Edge Detection Filter......Page 300
The launch_kernel() Method......Page 301
The callbacksVBO.cpp File......Page 302
Machine Learning......Page 306
The Connectome......Page 307
Listing for simpleVBO.cpp......Page 308
Works Cited......Page 313
D......Page 320
K......Page 321
O......Page 322
T......Page 323
X......Page 324