Intel Xeon Phi Coprocessor High Performance Programming

Front Cover......Page 1
Intel® Xeon Phi™ Coprocessor High-Performance Programming......Page 4
Copyright Page......Page 5
Contents......Page 6
Foreword......Page 14
Organization......Page 18
Lots-of-cores.com......Page 19
Acknowledgements......Page 20
Trend: more parallelism......Page 22
Why Intel® Xeon Phi™ coprocessors are needed......Page 23
Platforms with coprocessors......Page 26
The first Intel® Xeon Phi™ coprocessor......Page 27
Keeping the “Ninja Gap” under control......Page 30
Transforming-and-tuning double advantage......Page 31
Maximizing performance on processors first......Page 32
Why scaling past one hundred threads is so important......Page 33
Measuring readiness for highly parallel execution......Page 36
Beyond the ease of porting to increased performance......Page 37
Hyper-threading versus multithreading......Page 38
Coprocessor major usage model: MPI versus offload......Page 39
Compiler and programming models......Page 40
Cache optimizations......Page 41
For more information......Page 42
2 High Performance Closed Track Test Drive!......Page 44
Looking under the hood: coprocessor specifications......Page 45
Starting the car: communicating with the coprocessor......Page 47
Taking it out easy: running our first code......Page 49
Starting to accelerate: running more than one thread......Page 53
Petal to the metal: hitting full speed using all cores......Page 59
Easing in to the first curve: accessing memory bandwidth......Page 70
High speed banked curved: maximizing memory bandwidth......Page 75
Back to the pit: a summary......Page 78
Preparing for our country road trip: chapter focus......Page 80
Getting a feel for the road: the 9-point stencil algorithm......Page 81
At the starting line: the baseline 9-point stencil implementation......Page 82
Rough road ahead: running the baseline stencil code......Page 89
Cobblestone street ride: vectors but not yet scaling......Page 91
Open road all-out race: vectors plus scaling......Page 93
Some grease and wrenches!: a bit of tuning......Page 96
Adjusting the “Alignment”......Page 97
Using streaming stores......Page 98
Using huge 2-MB memory pages......Page 100
For more information......Page 102
4 Driving Around Town: Optimizing A Real-World Code Example......Page 104
Turn ahead: accounting for boundary effects......Page 105
Finding a wide boulevard: scaling the code......Page 112
Thunder road: ensuring vectorization......Page 114
Peeling out: peeling code from the inner loop......Page 118
Trying higher octane fuel: improving speed using data locality and tiling......Page 121
High speed driver certificate: summary of our high speed tour......Page 126
Why vectorize?......Page 128
Five approaches to achieving vectorization......Page 129
Six step vectorization methodology......Page 131
Step 3. Determine loop candidates using Intel Compiler vec-report......Page 132
Streaming through caches: data layout, alignment, prefetching, and so on......Page 133
Why data layout affects vectorization performance......Page 134
Data alignment......Page 135
Prefetching......Page 137
Compiler prefetches......Page 139
Manual prefetches (using vprefetch0 and vprefetch1)......Page 140
Streaming stores......Page 142
Compiler generation of clevicts......Page 143
Avoid manual loop unrolling......Page 144
Requirements for a loop to vectorize (Intel® Compiler)......Page 145
Compiler options......Page 147
Memory disambiguation inside vector-loops......Page 148
Compiler directives......Page 149
SIMD directives......Page 150
Requirements to vectorize with SIMD directives (Intel® Compiler)......Page 151
SIMD directive clauses......Page 152
Use SIMD directives with care......Page 154
The VECTOR and NOVECTOR directives......Page 155
The IVDEP directive......Page 156
IVDEP examples in C......Page 157
Random number function vectorization......Page 158
Utilizing full vectors, -opt-assume-safe-padding......Page 159
Remainder loop......Page 160
Peel loop......Page 161
Data alignment to assist vectorization......Page 163
How to define aligned STATIC arrays......Page 164
Step 2: Inform the compiler of the alignment......Page 165
Mixing aligned and unaligned accesses: how to tell the vectorizer all RHS memory references are 64-Byte aligned......Page 166
Tradeoffs in array notations due to vector lengths......Page 167
Fortran array sections......Page 171
Implications for array copies, efficiency issues......Page 172
Cilk Plus array sections and elemental functions......Page 173
Operations on array sections......Page 174
Avoid partial overlap of array sections......Page 175
Specifying unit-stride accesses inside elemental functions......Page 176
Look at what the compiler created: assembly code inspection......Page 177
How to find the assembly code......Page 178
Symptom 1: usage of unaligned loads and stores......Page 179
Symptom 2: usage of %k1 mask......Page 180
Symptom 3: usage of scatters or gathers......Page 182
Symptoms: quick inspections......Page 183
For more information......Page 184
6 Lots of Tasks (not Threads)......Page 186
Task creation needs to happen on the coprocessor......Page 187
Parallel processing model......Page 189
Significant controls over OpenMP......Page 190
Nesting......Page 191
DO CONCURRENT and DATA RACES......Page 192
DO CONCURRENT definition......Page 193
DO CONCURRENT vs. OpenMP “Parallel”......Page 194
Intel® TBB......Page 195
History......Page 196
blocked_range......Page 198
Partitioners......Page 199
parallel_reduce......Page 200
Notes on C++11......Page 201
Cilk Plus......Page 202
Borrowing components from TBB......Page 204
cilk_for......Page 205
cilk_spawn and cilk_sync......Page 206
Summary......Page 208
For more information......Page 209
7 Offload......Page 210
Two offload models......Page 211
Shared virtual memory model: using offload with shared VM......Page 212
Language extensions for offload......Page 213
Compiler options and environment variables for offload......Page 214
Using pragma/directive offload......Page 216
Placing variables and functions on the coprocessor......Page 219
Managing memory allocation for pointer variables......Page 221
Coprocessor memory management for input pointer variables......Page 222
Transferring data into pre-allocated memory on the target......Page 223
C/C++ offload pragma examples......Page 224
Fortran offload pragma examples......Page 225
Target-specific code using a pragma in C/C++......Page 227
Code that should not be built for processor-only execution......Page 230
Fortran arrays......Page 232
Allocating memory for parts of C/C++ arrays......Page 233
Allocating memory for parts of fortran arrays......Page 234
Moving data from one variable to another......Page 235
Restrictions on offloaded code using a pragma......Page 236
Using shared memory and shared variables......Page 238
Synchronous and asynchronous function execution: _cilk_offload......Page 240
Sharing variables and functions: _cilk_shared......Page 241
Synchronization between the processor and the target......Page 243
Writing target-specific code with _cilk_offload......Page 244
Restrictions on offloaded code using shared virtual memory......Page 245
Persistent data when using shared virtual memory......Page 246
C++ declarations of persistent data with shared virtual memory......Page 248
About asynchronous computation......Page 249
C/C++ Asynchronous data transfer from the processor to the coprocessor......Page 250
C/C++ asynchronous data transfer from the coprocessor to the processor......Page 251
Fortran asynchronous data transfer from the processor to the coprocessor......Page 253
Fortran asynchronous data transfer from the coprocessor to the processor......Page 254
Applying the target attribute to multiple declarations......Page 255
Vec-report option used with offloads......Page 256
__Offload_report......Page 257
About creating offload libraries with xiar and xild......Page 258
Performing file I/O on the coprocessor......Page 259
Logging stdout and stderr from offloaded code......Page 261
For more information......Page 262
8 Coprocessor Architecture......Page 264
The Intel® Xeon Phi™ coprocessor family......Page 265
Coprocessor card design......Page 266
Intel® Xeon Phi™ coprocessor silicon overview......Page 267
Individual coprocessor core architecture......Page 268
Instruction and multithread processing......Page 270
Cache organization and memory access considerations......Page 272
Prefetching......Page 273
Vector processing unit architecture......Page 274
Vector instructions......Page 275
Coprocessor PCIe system interface and DMA......Page 278
DMA capabilities......Page 279
Descriptor ring overview......Page 280
Coprocessor power management capabilities......Page 281
Reliability, availability, and serviceability (RAS)......Page 284
Machine check architecture (MCA)......Page 285
Sensors......Page 286
Potential application impact......Page 287
For more information......Page 288
Coprocessor software architecture overview......Page 290
Coprocessor programming models and options......Page 292
Breadth and depth......Page 294
Offload model......Page 295
Symmetric model......Page 296
Development tools and application layer......Page 297
MYO: mine yours ours......Page 298
Virtual networking (NetDev), TCP/IP, and sockets......Page 299
Sysfs......Page 300
Board tools, control panel, MIC Management SDK......Page 301
Coprocessor communication link (Intel CCL)......Page 303
IB proxy client......Page 307
Linux support for Intel® Xeon Phi™ coprocessors......Page 308
Monitoring the number of 2MB pages on the coprocessor......Page 309
A sample method for allocating 2MB pages......Page 310
Summary......Page 311
For more information......Page 312
Coprocessor Linux baseline......Page 314
Introduction to coprocessor Linux bootstrap and configuration......Page 315
Default coprocessor Linux configuration......Page 316
Step 4: Start the Intel® MPSS service......Page 317
Configurable components......Page 318
Configuring boot parameters......Page 319
PowerManagement kernel command line parameter......Page 320
Coprocessor root file system......Page 321
User access......Page 322
Network access......Page 323
The micctrl utility......Page 326
Rebooting the coprocessors......Page 327
Coprocessor status......Page 328
Resetting configuration parameters......Page 329
Change the UserAuthentication configuration parameter......Page 330
Setting the root device......Page 331
Updating the compressed CPIO image......Page 332
The file filelist directive......Page 333
Adding files to the root file system......Page 334
Example: Adding a new global file set......Page 335
Set the kernel command line......Page 336
Root is a RAM disk image......Page 338
Coprocessors in a Linux cluster......Page 339
How Intel® Cluster Checker works......Page 340
Intel® Cluster Checker support for coprocessors......Page 341
Summary......Page 343
For more information......Page 344
11 Math Library......Page 346
Intel Math Kernel Library overview......Page 347
Coprocessor support overview......Page 348
Control functions for automatic offload......Page 349
Using the coprocessor in native mode......Page 351
Using automatic offload mode......Page 353
Tips for effective use of automatic offload......Page 354
Automatic offload works better when matrix size is right......Page 355
Use automatic and compiler-assisted offload together......Page 356
Favor LAPACK unpacked routines......Page 357
Using compiler-assisted offload......Page 358
Tips for using compiler assisted offload......Page 359
Basics......Page 360
Fused multiply-add......Page 361
Comparing floating-point results between Intel Xeon Phi coprocessors and Intel Xeon processors......Page 362
For more information......Page 363
MPI overview......Page 364
Heterogeneity (and why it matters)......Page 366
Prerequisites (batteries not included)......Page 369
Offload from an MPI rank......Page 370
Trapezoidal rule......Page 371
Hello world (again)......Page 375
Method 2: wrapper script......Page 376
Trapezoidal rule (revisited)......Page 377
Manual load balance of hybrid workloads......Page 378
Dynamic load balance......Page 379
Summary......Page 382
For more information......Page 383
13 Profiling and Timing......Page 384
Efficiency metrics......Page 385
CPI – events used......Page 386
CPI – description and usage......Page 387
CPI – tuning suggestions......Page 389
Compute to data access ratio – description and usage......Page 390
Potential performance issues......Page 391
General cache usage – formulas and thresholds......Page 392
General cache usage – description and usage......Page 393
TLB misses – formulas and thresholds......Page 394
VPU usage – formula and threshold......Page 395
VPU usage – tuning suggestions......Page 396
Memory bandwidth – description and usage......Page 397
Intel® VTune™ Amplifier XE product......Page 398
MPI analysis: Intel Trace Analyzer and Collector......Page 399
Generating a trace file: processor+coprocessor application......Page 400
Time stamp counter (tsc)......Page 401
Time structures......Page 402
Time penalty......Page 403
For more information......Page 404
Advice......Page 406
Feedback appreciated......Page 407
Glossary......Page 408
Index......Page 422