Ascend AI Processor Architecture and Programming: Principles and Applications of CANN

Front matter
Copyright
About the Author
Preface
Theoretical basis
	Brief history of artificial intelligenceArtificial Intelligence
		Birth of AI
		Set sail
		Encounter bottlenecks
		Moving on again
		Dawn rise
	Introduction to deep learningDeep learninghistory
		History
		ApplicationsDeep learningapplications
		Future challengesDeep learningFuture Challenges
	Neural network theoryNeural networktheory
		Neuron model
		Perceptron
		Multilayer perceptron
			Working principle
			Computational complexity
		Convolutional neural networkConvolutional Neural Network
			Introduction to network architecture
			Convolutional layer
				Properties of the convolution layer
				Implementation of the convolutional layer
			Pooling layer
			Fully connected layer
			Optimization and acceleration
				Synaptic parallelism
				Neuron parallelism
				Input feature map parallelism
				Output feature map parallelism
				Batch parallelism
		Application example
			Initial setup
			Training
			Test
			References
Industry background
	Current status of the neural network processorsNeural networkprocessors
		CPUNeural networkprocessorsCPU
		GPUNeural networkprocessorsGPU
		TPUNeural networkprocessorsTPU
		FPGANeural networkprocessorsFPGA
		Ascend AI processorAscend AI processorintroductionNeural networkprocessorsAscend AI processor
	Neural network processor acceleration theoryNeural networkacceleration theory
		GPU acceleration theory
			GPU computing neural networks principles
			Modern GPU architecture
		TPU acceleration theory
			Systolic array for neural networks computing
			Google TPU architecture
	Deep learning frameworkDeep learningframework
		MindSporeDeep learningframeworkMindSpore
		CaffeDeep learningframeworkCaffe
		TensorFlowDeep learningframeworkTensorFlow
		PyTorchDeep learningframeworkPyTorch
	Deep learning compilation frameworkDeep learningcompilation framework-TVM
Hardware architecture
	Hardware architecture overview
	DaVinci architectureAscend AI processorDaVinci Architecture
		Computing unit
			Cube unit
			Vector unit
			Scalar unit
		Memory system
			Memory unit
			Data flow
		Control units
		Instruction set design
			Scalar instruction set
			Vector instruction set
			Matrix instruction set
	Convolution acceleration principleAscend AI processoracceleration principle
		Convolution acceleration
		Architecture comparison
Software architecture
	Ascend AI software stack overview
		L3 application enabling layer
		L2 execution framework layer
		L1 processor enabling layer
		L0 computing resource layer
		Tool chain
	Neural network software flowAscend AI processorsoftware flow
		Process orchestratorAscend AI processorsoftware flow Process orchestrator (matrix)
			Functions
			Application scenarios
				Acceleration card form
				Developer board form
		Digital vision pre-processingAscend AI processorsoftware flow DVPP module
			Functional architecture
			Preprocessing mechanisms
		Tensor boost engineAscend AI processorsoftware flow TBE
			Functional framework
			Application scenarios
		RuntimeAscend AI processorsoftware flow Runtime
		Task schedulerAscend AI processorsoftware flow Task scheduler
			Functions
			Scheduling process
		Framework managerAscend AI processorsoftware flow Framework manager
			Functional framework
			Offline model generation
				Parsing
				Quantification
				Compilation
				Serialization
			Loading offline model
			Offline model inference
		Application of neural network software flow
	Development tool chainAscend AI processordevelopment tool
		Introduction to functions
		Functional framework
		Tool functions
Programming methods
	Basics of deep learning development
		Deep learning programming theoryDeep learningprogramming theory
			Declarative programming and imperative programming
			Caffe
			TensorFlow
			Metaprogramming
			Domain-specific language
			TensorFlow and domain-specific language
			Turing completeness
			Branch and jump in TensorFlow
			Collaboration between TensorFlow and python
			Imperative programming and dynamic graph mechanism of PyTorch
			Difference between training and inference computing graph
			Model saving and loading
			Model conversion
			Deep learning inference engine and computing graph
		Deep learning inference optimization principle
			Inference process in the traditional deep learning framework
			Computing graph optimization
				Optimize the memory allocation mechanism
				Specify appropriate batch size
				Reuse memory by fine memory management
				Eliminate the control logic in the model as much as possible
				Use parallel computing in the model properly
				A magic optimization tool-Kernel fusion
			Kernel function optimization
				Loop expansion and partitioned matrix
				Optimization of the order of data storage
				Dedicated kernel
		Deep learning inference engineDeep learninginference engine
			The first generation of computing graph intermediate representation of TVM-NNVM
			The TVM second generation of computing graph intermediate representation-Relay
			Graph optimization in TVM
			Low-level intermediate representation of TVM
			Basis of the low-level intermediate representation of TVM-Halide
			Features of low-level intermediate representation of TVM
			TVM operator implementation
			TVM operator scheduling
			Optimization method for TVM low-level intermediate representation
			TOPI mechanism
			Code generation in TVM
	Techniques of Ascend AI software stackAscend AI processordevelopment phases
		Model generation phase
			Model parsing
			Intermediate representation in the computational graph
			Memory allocation
			Memory layout
			Kernel fusion
			Operator support
			Offline model file
		Application compilation and deployment phase
			Computing engine
			Application development based on multiple devices and multiple processors
			Operator scheduling
			Heterogeneous computing system
			Operator implementation-common kernel and customized kernel
	Customized operator developmentAscend AI processoroperator development
		Development procedure
			Development motivation
			Development process
			Customized operator development
				Development using the TVM primitives
				Domain-specific language development
			Customized plug-in development
			Load the plug-in for model conversion
		AI CPU operator development
			Features of the reduction operator
				Parameter parsing
				Input and output parameters
			Create a customized operator project
			Operator logic development
				operator.cce
				op_attr.h
				Reduction.h
				Reduction.cce
			Operator plug-in development
				Operator registration
				ReductionParseParams: This function is used for parsing operator parameters
				InferShapeAndTypeFn: This function is used for inferring output shapes and types
				UpdateOpDescFn: This function is used to update the operator description
		AI Core operator development
			Reduction operator features
			Create an operator project
			Operator logic development
				Import the Python module
				Definition of the operator function
				Operator logic implementation
				Operator scheduling and compilation
			Operator plug-in development
				Operator registration
				TEBinBuildFn: This function is used to compile operators
	Customized application development
		Development motivations
		Development process
		Serial configuration of the computing engine
		Computing engine development
		The computing engine is connected in series
Case studies
	Evaluation criteria
		Accuracy
		IoU
		Mean average precision
		Throughput and latency
		Energy efficiency ratio
	Image classification
		Dataset: ImageNet
		Algorithm: ResNet18
		Model migration practice
			Model preparation
			Image processing
			Project compilation
			Analysis of results
			Quantification methodology
	Object detection
		Dataset: COCO
		Algorithm: YoloV3
		Customized operator practice
			Development of plugin
			Implementation of the operator
				Matrix operation based on Cube Unit
				Definition of convolution operator parameters and data arrangement
				Example of the Convolutional operator
				Format conversion of the input feature map
				Format conversion of the weight data
				Format conversion of the output feature map
			Performance analysis
			Performance improvement tips
Index