Principles of High-Performance Processor Design: For High Performance Computing, Deep Neural Networks and Data Science

Preface
Contents
Acronyms
1 Introduction
	References
2 Traditional Approaches and Their Limitations
	2.1 History
		2.1.1 CDC 6600 and 7600
		2.1.2 Cray-1
		2.1.3 The Evolution of Vector Processors
		2.1.4 Lessons from the History of Vector-Parallel Architecture
		2.1.5 The Evolution of Single-Chip Processors
		2.1.6 The Impact of Hierarchical Cache
		2.1.7 Alternatives to Cache Memories
	2.2 The Need for Quantitative Approach
	2.3 What Is Measured and What Is Not
	References
3 The Lower Limit of Energy Consumption
	3.1 Range of Applications We Consider
		3.1.1 Structured Mesh
		3.1.2 Unstructured Mesh
		3.1.3 Particles
		3.1.4  Random Graphs
		3.1.5  Dense Matrices
		3.1.6  Miscellanies
		3.1.7 Distribution of Application Types
	3.2 Definition of Efficiency
	3.3 Structured Mesh
		3.3.1 Choice of the Numerical Methods
		3.3.2 The Design of An Ideal Processor Architecture for Structured Mesh Calculations
	3.4 Unstructured Mesh
	3.5 Particles
		3.5.1 The Overview of Particle-Based Methods
		3.5.2 Short-Range Interactions
		3.5.3 Long-Range Interactions
	3.6  Random Graphs
	3.7  Dense Matrices
	3.8 Summary
	References
4 Analysis of Past and Present Processors
	4.1 CDC 6600
	4.2 Cray-1 and Its Successors
	4.3 x86 Processors
		4.3.1 i860
		4.3.2 From Pentium to Skylake
	4.4 NEC SX-Aurora and Fujitsu A64fx
	4.5 SIMD Supercomputers—Illiac IV and TMC CM-2
		4.5.1 Illiac IV
		4.5.2 CM-2
		4.5.3 Problems with Large-Scale SIMD Processors
			4.5.3.1 Synchronous System Clock
			4.5.3.2 Memory Bandwidth
	4.6 GPGPUs
	4.7 PEZY Processors and Sunway SW26010
		4.7.1 PEZY Processors
		4.7.2 Sunway SW26010
	4.8 Conclusion
	References
5 ``Near-Optimal\'\' Designs
	5.1 The Special-Purpose Designs: GRAPE Processors
	5.2 The Baseline Design: GRAPE-DR
		5.2.1 Design Concept and Architecture
		5.2.2 The Efficiency
		5.2.3 Software
	5.3 Functions Necessary to Widen Application Area
		5.3.1 Particles
		5.3.2 Dense Matrices
		5.3.3 Other Applications
			5.3.3.1 Structured Mesh
			5.3.3.2 Unstructured Mesh
			5.3.3.3 Summary
	5.4 An Extreme for Deep Learning: MN-Core/GRAPE-PFN
	5.5 A ``General-Purpose\'\' Design
		5.5.1 On-Chip Network for Sorting
		5.5.2 Off-Chip DRAM Access
		5.5.3 Chip-to-Chip Communications Network for Deep Learning and Unstructured-Mesh Calculations
		5.5.4 Support for FP64, FP32 and FP16 or Other Mixed-Precision Operations
	5.6 The Reference SIMD Processor
		5.6.1 PE
		5.6.2 BM
		5.6.3 TBM
		5.6.4 DRAM Interface
		5.6.5 Host Data Interface
		5.6.6 Instruction Fetch/Issue
	References
6 Software
	6.1 Traditional Approaches
	6.2 How Do We Want to Describe Applications?
		6.2.1 Structured Mesh
		6.2.2 Unstructured Mesh
		6.2.3 Particles
		6.2.4 Dense Matrices
	6.3 Summary
	References
7 Present, Past and Future
	7.1 Principles of High-Performance Processor Design
	7.2 The Current Practice
	7.3 Our Past
	7.4 GPGPUs and Deep Learning Processors
	7.5 The Future
	References
Index