Processing-in-Memory for AI: From Circuits to Systems

Contents
1 Introduction
	1.1 Hardware Acceleration for Artificial Intelligence and Machine Learning
	1.2 Machine Learning Computations
		1.2.1 Fully Connected Layer
		1.2.2 Convolutional Layer
		1.2.3 Recurrent Layer
	1.3 von Neumann Bottleneck
		1.3.1 Memory Wall Problem
		1.3.2 Latest AI Accelerators with High-Bandwidth Memories
	1.4 Processing-in-Memory Architecture
		1.4.1 Paradigm Shift from Compute to Memory
		1.4.2 Challenges
	1.5 Book Organization
	References
2 Backgrounds
	2.1 Basic Memory Operations
		2.1.1 SRAM Basics
		2.1.2 DRAM Basics
		2.1.3 ReRAM Basics
	2.2 PIM Fundamentals
	2.3 PIM Output Read-out
	2.4 PIM Design Challenges
	References
3 SRAM-Based Processing-in-Memory (PIM)
	3.1 Introduction
	3.2 SRAM-Based PIM Cell Designs
		3.2.1 Standard 6T SRAM-Based PIM
		3.2.2 Custom SRAM Cells for PIM
	3.3 SRAM-Based PIM Macro Designs
	3.4 Summary
	References
4 DRAM-Based Processing-in-Memory
	4.1 Introduction
	4.2 Basic DRAM Operation
	4.3 Bulk Bitwise Processing-in-Memory
		4.3.1 AMBIT
			4.3.1.1 Triple Row Activation
			4.3.1.2 AMBIT DRAM Organization
			4.3.1.3 Fast Row Copy
			4.3.1.4 Bulk Bitwise NOT
			4.3.1.5 Row Addressing
			4.3.1.6 AMBIT Command Execution
			4.3.1.7 Evaluation
		4.3.2 DRISA
			4.3.2.1 Motivation
			4.3.2.2 Cell Microarchitectures
			4.3.2.3 Computing Using NOR Operation
			4.3.2.4 Evaluation
	4.4 Bank-Level Processing-in-Memory
		4.4.1 Newton
			4.4.1.1 Motivation
			4.4.1.2 Architecture
			4.4.1.3 Newton's Operation
			4.4.1.4 Evaluation
		4.4.2 HBM-PIM
			4.4.2.1 Motivation
			4.4.2.2 HBM-PIM Architecture
			4.4.2.3 HBM-PIM Controller
			4.4.2.4 Programmable Computing Unit
			4.4.2.5 Operation Flow
			4.4.2.6 Data Movements
			4.4.2.7 Implementation Results
	4.5 3-D Processing-in-Memory
		4.5.1 Neurocube
		4.5.2 Tetris
		4.5.3 iPIM
	References
5 ReRAM-Based Processing-in-Memory (PIM)
	5.1 Introduction
	5.2 Basic ReRAM PIM Operation
	5.3 Multiplication in ReRAM PIMs
		5.3.1 Binary Multiply
		5.3.2 Multiplication with Ternary Weight
		5.3.3 Multi-bit Multiplication
			5.3.3.1 Multiplication Using One Cycle and One Column
			5.3.3.2 Parallel-Input Parallel-Weight (PIPW)
			5.3.3.3 Serial-Input Parallel-Weight (SIPW)
	5.4 ReRAM PIM Architecture
		5.4.1 Introduction
		5.4.2 Non-volatile PIM Processor
		5.4.3 ReRAM PIM Architecture
		5.4.4 ADCs and DACs in ReRAM PIM
	5.5 ReRAM Co-processor
		5.5.1 Architecture
		5.5.2 Mixed-Signal Interface
		5.5.3 ADCs and DACs Operation
	5.6 Transposable ReRAM for Inference and Training
	5.7 Bitline Sensing for MAC Accuracy Improvement
		5.7.1 Variations in Bitline Current
		5.7.2 Input-Aware Dynamic Reference Generation
		5.7.3 Weighted Current Generation
			5.7.3.1 PIM Macro Architecture
			5.7.3.2 Serial-Input Non-weighted Product (SINWP)
			5.7.3.3 Down-scaling Weighted Current Translator (DSWCT)
	5.8 Versatile ReRAM-Based PIM Functions
		5.8.1 Versatile PIM Architecture
		5.8.2 2T2R ReRAM Bit Cell for Versatile Functions
			5.8.2.1 Basic Memory Operation
			5.8.2.2 TCAM Operation
			5.8.2.3 Logic-in-Memory Operation
			5.8.2.4 Dot Product Operation
	5.9 Summary
	References
6 PIM for ML Training
	6.1 Introduction
	6.2 Training Computations
		6.2.1 Feed-Forward Propagation
		6.2.2 Backward Propagation
		6.2.3 Gradient Calculation and Weight Update
	6.3 SRAM-Based PIM for Training
		6.3.1 Two-Way Transpose SRAM PIM
			6.3.1.1 SRAM Compute-in-Memory Macro Design
			6.3.1.2 In-memory Multiplication for Forward and Backward Propagation
		6.3.2 CIMAT
			6.3.2.1 7T and 8T Transpose SRAM Cell Design
			6.3.2.2 Weight Mapping Strategies and Data Flow
			6.3.2.3 Pipeline Design
		6.3.3 HFP-CIM
			6.3.3.1 Heterogeneous Floating-Point Computing Architecture
			6.3.3.2 Overall Processor Design and Sparsity Handling
	6.4 ReRAM-Based PIM for Training
		6.4.1 PipeLayer
			6.4.1.1 Architecture of PipeLayer
			6.4.1.2 Data Mapping and Parallelism of PipeLayer
		6.4.2 FloatPIM
			6.4.2.1 FloatPIM's Digital Operation
			6.4.2.2 Hardware Architecture
			6.4.2.3 Training of FloatPIM
	References
7 PIM Software Stack
	7.1 PIM Software Stack Overview
		7.1.1 PIM Software Stack Challenges
	7.2 PIM Offloading Execution
	7.3 PIM Data Mapping
	7.4 PIM Execution Scheduling
	7.5 Cache Coherence
	References
8 Conclusion
Correction to: Processing-in-Memory for AI