Pro .NET Memory Management : For Better Code, Performance, and Scalability

Table of Contents
About the Authors
About the Technical Reviewer
Acknowledgments
Foreword
Introduction
Chapter 1: Basic Concepts
	Memory-Related Terms
		Static Allocation
		The Register Machine
		The Stack
		The Stack Machine
		The Pointer
		The Heap
	Manual Memory Management
	Automatic Memory Management
		Allocator, Mutator, and Collector
			The Mutator
			The Allocator
			The Collector
	Reference Counting
	Tracking Collector
		Mark Phase
			Conservative Garbage Collector
			Precise Garbage Collector
			Collect Phase
			Sweep
			Compact
	Summary
		Rule 1 – Educate Yourself
Chapter 2: Low-Level Memory Management
	Hardware
		Memory
		CPU
			CPU Cache
			Cache Hit and Miss
			Data Locality
			Cache Implementation
			Data Alignment
			Non-temporal Access
			Prefetching
			Hierarchical Cache
			Multicore Hierarchical Cache
	Operating System
		Virtual Memory
		Large Pages
		Virtual Memory Fragmentation
		General Memory Layout
		Windows Memory Management
		Windows Memory Layout
		Linux Memory Management
		Linux Memory Layout
		Operating System Influence
	NUMA and CPU Groups
	Summary
		Rule 2 – Random Access Should Be Avoided, Sequential Access Should Be Encouraged
		Rule 3 – Improve Spatial and Temporal Data Locality
		Rule 4 – Consider More Advanced Possibilities
Chapter 3: Memory Measurements
	Measure Early
		Overhead and Invasiveness
		Sampling vs. Tracing
		Call Tree
		Object Graphs
		Statistics
		Latency vs. Throughput
		Memory Dumps, Tracing, Live Debugging
	Windows and Linux Environments
		Overview
		VMMap
		.NET Framework Performance Counters
		.NET Core Counters
		Event Tracing for Windows
		PerfView
			Data Collection
			Data Analysis
			Memory Snapshots
		dotnet-trace CLI Tool
		dotnet-gcmon CLI tool
		ProcDump, dotnet-dump
		dotnet-gcdump CLI tool
		WinDbg
		dotnet-dump for Analysis
		Disassemblers and Decompilers
		BenchmarkDotNet
		Tools from the Authors
			GummyCat
			dotnet-counters-ui
			dotnet-gcstats
			dotnet-fullgc
		Commercial Tools
			Visual Studio
			Scitech .NET Memory Profiler
			JetBrains DotMemory
			RedGate ANTS Memory Profiler
			Intel VTune Amplifier and AMD CodeAnalyst Performance Analyzer
			Datadog, Dynatrace, and AppDynamics
	Summary
		Rule 5 – Measure GC Early
Chapter 4: .NET Fundamentals
	.NET Versions
	.NET Internals
		Sample Program in Depth
	Assemblies and Application Domains
		Collectible Assemblies
	Process Memory Regions
		Scenario 4-1 – How Big Is My Program in Memory?
		Scenario 4-2 – My Program’s Native Memory Usage Keeps Growing
		Scenario 4-3 – My Program’s Virtual Memory Usage Keeps Growing
		Scenario 4-4 – My Program’s Managed Memory Usage Keeps Growing with Assemblies Count
		Scenario 4-5 – My Program Is Unable to Unload Plugins
	Type System
		Type Categories
		Type Storage
		Value Types
			Value Type Storage
			Structs
			Structs in General
			Struct Storage
		Reference Types
			Classes
	Strings
		String Interning
		Scenario 4-6 – My Program’s Memory Usage Is Too Big
	Boxing and Unboxing
	Passing by Reference
		Pass-by-Reference Value Type Instance
		Pass-by-Reference Reference Type Instance
	Null
	Type Data Locality
	Static Data
		Static Fields
		Static Data Internals
	Summary
		Structs
		Classes
			Rule 6 – Measure Your Program
			Rule 7 – Do Not Assume There Is (No) Memory Leak
			Rule 8 – Consider Using Struct
			Rule 9 – Consider Using String Interning
			Rule 10 – Avoid Boxing
Chapter 5: Memory Partitioning
	Partitioning Strategies
	Size Partitioning
		Small Object Heap
		Large Object Heap
			Large Object Heap – Arrays of Doubles
	Lifetime Partitioning
		Scenario 5-1 – Is My Program Healthy? Generation Sizes in Time
		Remembered Sets
		Card Tables
		Card Bundles
	Kind Partitioning
		The NonGC Heap
		The Pinned Object Heap
		Pinned Object Heap and Internal CLR Data
			PinnedHeapHandleTable
	Physical Partitioning
		Segment Implementation (Pre-.NET 7)
		Region Implementation (.NET 7+)
		NonGC Heap
		Scenario 5-2 – nopCommerce Memory Leak?
		Scenario 5-3 – Large Object Heap Waste?
		Segments, Regions, and Heap Anatomy
		Segment Reuse
		Region Reuse
	Summary
		Rule 11 – Monitor Generation Sizes
		Rule 12 – Avoid Unnecessary Heap References
		Rule 13 – Monitor Segment Usage
Chapter 6: Memory Allocation
	Allocation Introduction
	Bump Pointer Allocation
	Free-List Allocation
	Creating a New Object
		Small Object Heap Allocation
		Large Object Heap and Pinned Object Heap Allocation
	Heap Balancing
		Pinned Object Heap Allocation API
	OutOfMemoryException
		Scenario 6-1 – Out of Memory
	Stack Allocation
	Avoiding Allocations
		Explicit Allocations of Reference Types
			General Case – Consider Using Struct
			Tuples – Use ValueTuple Instead
			Small Temporary Local Data – Consider Using stackalloc
			Creating Arrays – Use ArrayPool
	Creating Streams – Use RecyclableMemoryStream
		Creating a Lot of Objects – Use Object Pool
		Async Methods Returning Task – Use ValueTask
		Hidden Allocations
			Delegate Allocation
			Boxing
			Closures
			Yield Return
			Parameter Array
			String Concatenation and Formatting
		String Encoding
		Various Hidden Allocations Inside Libraries
			System.Generics Collections
			LINQ – Delegates
			LINQ – Anonymous Type Creation
			LINQ – Enumerables
		Scenario 6-2 – Investigating Allocations
		Scenario 6-3 – Azure Functions
	Summary
		Rule 14 – Avoid Allocations on the Heap in Hot Paths
		Rule 15 – Avoid Excessive UOH Allocations
		Rule 16 – Allocate on the Stack When Appropriate
Chapter 7: Garbage Collection – Introduction
	High-Level View
	GC Work by Examples – Segments
	GC Work by Examples – Regions
	GC Step by Step
		Scenario 7-1 – Analyzing the GC Usage
	Profiling the GC
	Garbage Collection Performance Tuning Data
		Static Data
		Dynamic Data
		Scenario 7-2 – Understanding the Allocation Budget
			GC #1 – Triggered by Explicit GC.Collect() Call
			GC #2 – Triggered by Allocation
			GC #3 – Triggered by Allocation
			GC #4 – Triggered by Allocation
			GC #5 – Triggered by Allocation
			Subsequent GCs
	Collection Triggers
		Allocation Trigger
		Explicit Trigger
		Scenario 7-3 – Analyzing the Explicit GC Calls
		Low Memory Level System Trigger
		Various Internal Triggers
		EE Suspension
		Scenario 7-4 – Analyzing GC Suspension Times
	Collection Tuning
		Hard Memory Limit
		Provisional Mode
		“Servo Tuning”
		Scenario 7-5 – Condemned Generation Analysis
		Scenario 7-6 – Provisional Mode
	Summary
Chapter 8: Garbage Collection – Mark Phase
	Object Traversal and Marking
	Local Variable Roots
		Local Variable Storage
		Stack Roots
		Lexical Scope
		Live Stack Roots vs. Lexical Scope
		Live Stack Roots with Eager Root Collection
		GC Info
		Pinned Local Variables
		Stack Root Scanning
	Finalization Roots
	GC Internal Roots
	GC Handle Roots
	Handling Memory Leaks
		Scenario 8-1 – nopCommerce Memory Leak?
			Approach 1 – Analyzing a Single Memory Snapshot
			Approach 2 – Comparing Memory Snapshots
		Scenario 8-2 – Identifying the Most Popular Roots
		Scenario 8-3 – Generational Aware Analysis
	Summary
Chapter 9: Garbage Collection – Plan Phase
	Small Object Heap
		Plugs and Gaps
		Scenario 9-1 – Memory Dump with Invalid Structures
		Brick Table
		Pinning
		Scenario 9-2 – Investigating Pinning
		Generation Boundaries
		Demotion
	Large Object Heap
		Plugs and Gaps
	Decide on Compaction
	Special Regions
	Summary
Chapter 10: Garbage Collection – Sweep and Compact
	Sweep Phase
		Small Object Heap
		LOH and POH
	Compact Phase
		Small Object Heap
			Getting a New Ephemeral Segment If Necessary
			Relocate References
			Compact Objects
			Fix Generation Boundaries
			Delete/Decommit Segments If Necessary
			Creating Free-List Items
			Age Roots
		Large Object Heap
		Scenario 10-1 – Large Object Heap Fragmentation
	Summary
		Rule 17 – Watch Runtime Suspensions
		Rule 18 – Avoid Mid-Life Crisis
		Rule 19 – Avoid Old Generation and LOH Fragmentation
		Rule 20 – Avoid Explicit GC
		Rule 21 – Avoid Memory Leaks
		Rule 22 – Use Pinning Carefully
Chapter 11: GC Flavors and Settings
	Mode Overview
		Workstation vs. Server Mode
			Workstation Mode
			Server Mode
		Non-concurrent vs. Concurrent Mode
			Non-concurrent Mode
			Concurrent Mode
	Mode Configuration
		.NET Framework
		.NET Core
	GC Pause and Overhead
	Mode Descriptions
		Workstation Non-concurrent
		Workstation Concurrent (Before 4.0)
		Background Workstation
			Concurrent Mark
			Concurrent Sweep
		Server Non-concurrent
		Background Server
	Latency Modes
		Batch Mode
		Interactive
		Low Latency
		Sustained Low Latency
		No GC Region
		Latency Optimization Goals
	Choosing the GC Flavor
	Other GC Configuration Knobs
		Changing the Heap Limits
		Changing the Number of Heaps
		Changing the GC Thread Affinity
		Memory Load Threshold
		GC Conservative Mode
		Dynamic Adaptation Mode
		Large Pages
		Scenario 11-1. Checking GC Settings
		Scenario 11-2. Benchmarking Different GC Modes
	Summary
		Rule 23 – Choose GC Mode Consciously
		Rule 24 – Remember About Latency Modes
Chapter 12: Object Lifetime
	Object vs. Resource Life Cycle
	Finalization
		Introduction
		Eager Root Collection Problem
		Critical Finalizers
		Finalization Internals
			Finalization Overhead
		Scenario 12-1 – Finalization Memory Leak
		Resurrection
	Disposable Objects
	Safe Handles
	Weak References
		Caching
		Weak Event Pattern
		Scenario 12-2 – Memory Leak Because of Events
	Summary
		Rule 25 – Avoid Finalizers
		Rule 26 – Prefer Explicit Cleanup
Chapter 13: Miscellaneous Topics
	Dependent Handles
	Thread Local Storage
		Thread Static Fields
		Thread Data Slots
		Thread Local Storage Internals
		Usage Scenarios
	Managed Pointers
		Ref Locals
		Ref Returns
		Readonly Ref and in Parameters
		Ref Type Internals
			Managed Pointer into Stack-Allocated Object
			Managed Pointer into Heap-Allocated Object
		Managed Pointers in C# – Ref Variables
	More on Structs…
		Readonly Structs
		Ref Structs and Ref Fields
		Fixed-Size Buffers
		Inline Arrays
	Object/Struct Layout
	Unmanaged Constraint
		Blittable Types
	Summary
Chapter 14: Advanced Techniques
	Span and Memory
		Span
			Usage Examples
			Span Internals
			“Slow Span”
			“Fast Span”
		Memory
		IMemoryOwner
		Memory Internals
		Span and Memory Guidelines
	Unsafe
	Data-Oriented Design
		Tactical Design
			Design Types to Fit As Much Relevant Data As Possible in the First Cache Line
			Design Data to Fit into Lower Cache Levels
			Design Data That Allows Easy Parallelization
				Avoid Nonsequential Memory Access, Especially Random
		Strategic Design
			Moving from Array-of-Structures to Structure-of-Arrays
			Entity Component System
	Pipelines
	Summary
Chapter 15: Programmatical APIs
	GC API
		Collection Data and Statistics
			GC.GetConfigurationVariables()
			GC.MaxGeneration
			GC.CollectionCount(Int32)
			GC.GetGeneration
			GC.GetTotalMemory
			GC.GetTotalAllocatedBytes
			GC.GetAllocatedBytesForCurrentThread
			GC.GetGCMemoryInfo
			GC.KeepAlive
			GCSettings.LargeObjectHeapCompactionMode
			GCSettings.LatencyMode
			GCSettings.IsServerGC
		GC Notifications
		Controlling Unmanaged Memory Pressure
		Explicit Collection
		No-GC Regions
		Finalization Management
		Memory Usage
		Memory Limits
		Internal Calls in the GC Class
		Frozen Segments
	ClrMD
	TraceEvent Library
	Custom GC
	Summary
Index