x86 Software Reverse-Engineering, Cracking, and Counter-Measures

Cover
Title Page
Copyright Page
About the Authors
About the Technical Writer
About the Technical Editor
Contents at a Glance
Contents
Introduction
	Who Should Read This Book
	What to Expect from This Book
	History
	Legal
Chapter 1 Decompilation and Architecture
	Decompilation
		When Is Decompilation Useful?
		Decompiling JIT Programming Languages
		Defending JIT Languages
	Lab 1: Decompiling
		Skills to Practice
		Takeaways
	Architecture
		Computer Architecture
			The Central Processing Unit
			Bridges and Peripherals
			Memory and Registers
		Assembly
			Introduction to Machine Code
			From Machine Code to Assembly
			Instruction Set Architectures and Microarchitectures
			RISC vs. CISC Computer Architectures
	Summary
Chapter 2 x86 Assembly: Data, Modes, Registers, and Memory Access
	Introduction to x86
	Assembly Syntax
	Data Representation
		Number System Bases
		Bits, Bytes, and Words
		Working with Binary Values
			Zero-Extension and Readability
			Bit and Byte Significance
			Endianness
	Registers
		Registers in x86
			x86 General-Purpose Registers
			Special-Purpose Registers
		Working with Registers
			64-Bit Registers
	Memory Access
		Specifying Data Lengths
	Addressing Modes
		Absolute Addressing
			Example: Global Variables
		Indirect Addressing
			Example: Pointers
		Base + Displacement Addressing
		Indexed Addressing
			Example: Arrays
		Based-Index Addressing
			Example: Structs
	Summary
Chapter 3 x86 Assembly: Instructions
	x86 Instruction Format
	x86 Instructions
		mov
			Hands-on Example
		inc, dec
		add, sub
		mul
		div
			Hands-on Example
		and, or, xor
		not
		shr, shl
		sar, sal
		nop
		lea
			Hands-on Example
	Putting It All Together
	Common x86 Instruction Mistakes
		When In Doubt, Look It Up
	Summary
Chapter 4 Building and Running Assembly Programs
	Output
		Controlling Pins
		Tedium
	System Calls
		sys_write
		sys_exit
		Printing a String
	Building and Linking
		Building and Linking in Linux
		Writing an Assembly Program
			Sections and Stat
			Labels
			Constants
			Global Data
			Strings
			times
			$
	objdump
	Lab: Hello World
		Skills
		Takeaways
	ASCII
		Identifying ASCII Strings
		ASCII Manipulation Tip
	Summary
Chapter 5 Understanding Condition Codes
	Condition Codes
		eflags
			Carry Flag
			Zero Flag
			Sign Flag
			Overflow Flag
			Other Status Flags
		Operations Affecting Status Flags
			add
			sub
			cmp
			test
	Summary
Chapter 6 Analyzing and Debugging Assembly Code
	Binary Analysis
		Static and Dynamic Analysis
		Debugging
	Breakpoints
		Software Breakpoints
		Hardware Breakpoints
	gdb
		Debugging with gdb
			Launching gdb
			Disassembly with gdb
			Starting and Stopping Code in gdb
			gdb Breakpoints
			gdb info Commands
			Stepping Through Instructions
			Examining Memory
	Segmentation Faults
	Lab: Shark Sim 3000
		Skills
		Takeaways
	Tuning Out the Noise
	Summary
Chapter 7 Functions and Control Flow
	Control Flow
		The Instruction Pointer
		Control Flow Instructions
			jmp
			Conditional Jumps
			Pitfalls of Conditional Jumps
			Example
	Logic Constructs in x86
		if (. . .) {. . .}
		if (. . .) { . . . } else { . . . }
			if (. . .) { . . . } else if { . . . } else { . . . }
		do { . . . } while (. . .);
		while (. . .) { . . . }
		for (. . .; . . .; . . .) { . . . }
		switch (. . .) { . . . }
			Building a Jump Table
		Continue
		break
		&&
		||
	Stack
		How the Stack Works
		The x86 Stack
			Push and Pop
			The Stack as a Scratch Pad
			Using Pop Cautiously
	Function Calls and Stack Frames
		Functions in x86
			call
			ret
		Stack Analysis
		Calling Conventions
			Why Conventions Are Necessary
			Introduction to Calling Conventions
		cdecl
			Saving Registers
			Return Values
			Accessing Parameters
		Stack Frames
			Prologues and Epilogues
			Accessing Parameters
			Local Variables
			Shortcuts
			Stack Alignment
		The Big Picture
		Things to Memorize
	Summary
Chapter 8 Compilers and Optimizers
	Finding Starting Code
	Compilers
		Optimization
		Stripping
		Linking
			Static Linking
			Dynamic Linking
			Security Impacts of Linking
	Summary
Chapter 9 Reverse Engineering: Tools and Strategies
	Lab: RE Bingo
		Skills
		Takeaways
	Basic REconnaissance
		objdump
		strace and ltrace
			ltrace
			strace
			strace Example: echo
			strace Example: Malicious Kittens
		strings
		Dependency Walker
	Reverse Engineering Strategy
		Find Areas of Interest
		Iteratively Annotate Code
	Summary
Chapter 10 Cracking: Tools and Strategies
	Key Checkers
		The Bad Way
		A Reasonable Way
		A Better Way
			Digitally Signed Keys
		The Best Way
		Other Suggestions
			Prefer Offline Activation
			Perform Partial Key Verification
			Encode Useful Data in the Key
	Key Generators
		Why Build Key Generators?
		The Philosophy of Key Generation
		Cracking Different Types of Key Checks
			Key Check Type I: Transform Just the Username
			Key Check Type II: Transform Both
			Key Check Type III: Brute Forceable
		Defending Against Keygens
	Lab: Introductory Keygen
		Skills
		Takeaways
	Procmon
		Example: Notepad.exe
		How Procmon Aids RE and Cracking
			Call Stacks
			File Operations
			Registry Queries
	Resource Hacker
		Example
		Mini-Lab: Windows Calculator
	Patching
		Patching vs. Key-Genning
		Where to Patch
		NOPs
	Other Debuggers
		OllyDbg
		Immunity
		x86dbg
		WinDbg
	Debugging with Immunity
		Immunity: Assembly
		Immunity: Modules
		Immunity: Strings
		Immunity: Running the Program
		Immunity: Exceptions
		Immunity: REwriting the Program
	Lab: Cracking with Immunity
		Skills
		Takeaways
	Summary
Chapter 11 Patching and Advanced Tooling
	Patching in 010 Editor
	CodeFusion Patching
	Cheat Engine
		Cheat Engine: Open a Process
		Cheat Engine: View Memory
		Cheat Engine: String References
		Cheat Engine: REwriting Programs
		Cheat Engine: Copying Bytes
		Cheat Engine: Getting Addresses
	Lab: Cracking LaFarge
		Skills
		Takeaways
	IDA Introduction
		IDA: Strings
		IDA: Basic Blocks
		IDA: Functions and Variables
		IDA: Comments
		IDA: Paths
	IDA Patching
	Lab: IDA Logic Flows
		Skills
		Takeaways
	Ghidra
	Lab: Cracking with IDA
		Skills
		Takeaways
	Summary
Chapter 12 Defense
	Obfuscation
		Evaluating Obfuscation
		Automated Obfuscation
			Name Mangling
			String Encryption
			Control Flow Flattening
			Opaque Predicates
			Instruction Substitution
		Obfuscators
		Defeating Obfuscators
	Lab: Obfuscation
		Skills
		Takeaways
	Anti-Debugging
		IsDebuggerPresent()
		Debug Registers
		RDTSC
		Invalid CloseHandle()
		Directory Scanning
		Offensive Anti-Debugging
		Defeating Anti-Debugging
	Lab: Anti-Debugging
		Skills
		Takeaways
	Summary
Chapter 13 Advanced Defensive Techniques
	Tamper-Proofing
		Hashing
		Signatures
		Watermark
		Guards
	Packing
		How Packers Work
		Is This a Strong Protection?
		Defeating Packing
		PEiD
	Lab: Detecting and Unpacking
		Skills
		Takeaways
	Virtualization
		How Code Virtualization Works
		Layered Virtualization
		Issues with Virtualization
		Is This a Strong Protection?
		Defeating Virtualization
	Cryptors/Decryptors
		Is This a Useful Protection?
		Defeating Cryptors
	Summary
Chapter 14 Detection and Prevention
	CRC
		Is This a Strong Protection?
	Code Signing
		How to Code Sign
		How to Verify a Signed Application
		Is Code Signing Effective?
		Code Signing vs. CRC
		Is This a Strong Protection?
	RASP
		Function Hooking
		Risks of RASP
		Is This a Strong Protection?
	Allowlisting
		How Allowlisting Works
			Breaking Name-Based Allowlists
			Breaking Name and Hash-Based Allowlists
			Example: Metasploit
		Is This a Strong Protection?
	Blocklisting
		Is This a Strong Protection?
	Remote Authentication
		Remote Authentication Example
		Is This a Strong Protection?
	Lab: ProcMon
		Takeaways
	Summary
Chapter 15 Legal
	U.S. Laws Affecting Reverse Engineering
		The Digital Millennium Copyright Act
		Computer Fraud and Abuse Act
		Copyright Act
		Important Court Cases
		Fair Use
		DMCA Research Exception
		Legality
	Summary
Chapter 16 Advanced Techniques
	Timeless Debugging
		Binary Instrumentation
		Intermediate Representations
		Decompiling
		Automatic Structure Recovery
		Visualization
		Deobfuscation
		Theorem Provers
		Symbolic Analysis
	Summary
Chapter 17 Bonus Topics
	Stack Smashing
		Shellcode
		Stack Smashing and Stack Protection
	Connecting C and x86
		Using C Functions in x86 Code
		Using x86 Functions in C Code
		_start vs. main()
		Standard Arguments
		Mixing C and Assembly
	Summary
Conclusion
Index
EULA