Mastering the Lightning Network: A Second Layer Blockchain Protocol for Instant Bitcoin Payments

Copyright
Table of Contents
Preface
	Intended Audience
	Conventions Used in This Book
	Code Examples
	Using Code Examples
	References to Companies and Products
	Addresses and Transactions in This Book
	O’Reilly Online Learning
	How to Contact Us
		Contacting Andreas
		Contacting René
		Contacting Olaoluwa Osuntokun
	Acknowledgments by Andreas
	Acknowledgments by René
	Acknowledgments by Olaoluwa Osuntokun
	Contributions
	Sources
Part I. Understanding the Lightning Network
	Chapter 1. Introduction
		Lightning Network Basic Concepts
		Trust in Decentralized Networks
		Fairness Without Central Authority
			Trusted Protocols Without Intermediaries
			A Fairness Protocol in Action
			Security Primitives as Building Blocks
			Example of the Fairness Protocol
		Motivation for the Lightning Network
			Scaling Blockchains
		The Lightning Network’s Defining Features
		Lightning Network Use Cases, Users, and Their Stories
		Conclusion
	Chapter 2. Getting Started
		Alice’s First Lightning Wallet
		Lightning Nodes
		Lightning Explorers
		Lightning Wallets
			Testnet Bitcoin
		Balancing Complexity and Control
		Downloading and Installing a Lightning Wallet
		Creating a New Wallet
			Responsibility with Key Custody
			Mnemonic Words
			Storing the Mnemonic Safely
		Loading Bitcoin onto the Wallet
			Acquiring Bitcoin
			Receiving Bitcoin
		From Bitcoin to Lightning Network
			Lightning Network Channels
			Opening a Lightning Channel
		Buying a Cup of Coffee Using the Lightning Network
			Bob’s Cafe
			A Lightning Invoice
		Conclusion
	Chapter 3. How the Lightning Network Works
		What Is a Payment Channel?
		Payment Channel Basics
		Routing Payments Across Channels
		Payment Channels
			Multisignature Address
			Funding Transaction
			Commitment Transaction
			Cheating with Prior State
			Announcing the Channel
			Closing the Channel
		Invoices
			Payment Hash and Preimage
			Additional Metadata
		Delivering the Payment
			The Peer-to-Peer Gossip Protocol
			Pathfinding and Routing
		Source-Based Pathfinding
			Onion Routing
			Payment Forwarding Algorithm
		Peer-to-Peer Communication Encryption
		Thoughts About Trust
		Comparison with Bitcoin
			Addresses Versus Invoices, Transactions Versus Payments
			Selecting Outputs Versus Finding a Path
			Change Outputs on Bitcoin Versus No Change on Lightning
			Mining Fees Versus Routing Fees
			Varying Fees Depending on Traffic Versus Announced Fees
			Public Bitcoin Transactions Versus Private Lightning Payments
			Waiting for Confirmations Versus Instant Settlement
			Sending Arbitrary Amounts Versus Capacity Restrictions
			Incentives for Large Value Payment Versus Small Value Payments
			Using the Blockchain as a Ledger Versus as a Court System
			Offline Versus Online, Asynchronous Versus Synchronous
			Satoshis Versus Millisatoshis
		Commonality of Bitcoin and Lightning
			Monetary Unit
			Irreversibility and Finality of Payments
			Trust and Counterparty Risk
			Permissionless Operation
			Open Source and Open System
		Conclusion
	Chapter 4. Lightning Node Software
		Lightning Development Environment
			Using the Command Line
			Downloading the Book Repository
		Docker Containers
		Bitcoin Core and Regtest
			Building the Bitcoin Core Container
		The c-lightning Lightning Node Project
			Building c-lightning as a Docker Container
			Setting Up a Docker Network
			Running the bitcoind and c-lightning Containers
			Installing c-lightning from Source Code
			Installing Prerequisite Libraries and Packages
			Copying the c-lightning Source Code
			Compiling the c-lightning Source Code
		The Lightning Network Daemon Node Project
			The LND Docker Container
			Running the bitcoind and LND Containers
			Installing LND from Source Code
			Copying the LND Source Code
			Compiling the LND Source Code
		The Eclair Lightning Node Project
			The Eclair Docker Container
			Running the bitcoind and Eclair Containers
			Installing Eclair from Source Code
			Copying the Eclair Source Code
			Compiling the Eclair Source Code
		Building a Complete Network of Diverse Lightning Nodes
			Using docker-compose to Orchestrate Docker Containers
			docker-compose Configuration
			Starting the Example Lightning Network
			Opening Channels and Routing a Payment
		Conclusion
	Chapter 5. Operating a Lightning Network Node
		Choosing Your Platform
			Why Is Reliability Important for Running a Lightning Node?
			Types of Hardware Lightning Nodes
			Running in the “Cloud”
			Running a Node at Home
			What Hardware Is Required to Run a Lightning Node?
			Switching Server Configuration in the Cloud
		Using an Installer or Helper
			RaspiBlitz
			Mynode
			Umbrel
			BTCPay Server
			Bitcoin Node or Lightweight Lightning
			Operating System Choice
		Choose Your Lightning Node Implementation
		Installing a Bitcoin or Lightning Node
			Background Services
			Process Isolation
			Node Startup
			Node Configuration
			Network Configuration
		Security of Your Node
			Operating System Security
			Node Access
		Node and Channel Backups
			Hot Wallet Risk
			Sweeping Funds
		Lightning Node Uptime and Availability
			Tolerate Faults and Automate
			Monitoring Node Availability
			Watchtowers
		Channel Management
			Opening Outbound Channels
			Getting Inbound Liquidity
			Closing Channels
			Rebalancing Channels
		Routing Fees
		Node Management
			Ride The Lightning
			lndmon
			ThunderHub
		Conclusion
Part II. The Lightning Network in Detail
	Chapter 6. Lightning Network Architecture
		The Lightning Network Protocol Suite
		Lightning in Detail
	Chapter 7. Payment Channels
		A Different Way of Using the Bitcoin System
		Bitcoin Ownership and Control
			Diversity of (Independent) Ownership and Multisig
			Joint Ownership Without Independent Control
			Preventing “Locked” and Un-Spendable Bitcoin
		Constructing a Payment Channel
			Node Private and Public Keys
			Node Network Address
			Node Identifiers
			Connecting Nodes as Direct Peers
		Constructing the Channel
			Peer Protocol for Channel Management
			Channel Establishment Message Flow
			The Funding Transaction
			Generating a Multisignature Address
			Constructing the Funding Transaction
			Holding Signed Transactions Without Broadcasting
			Refund Before Funding
			Constructing the Presigned Refund Transaction
			Chaining Transactions Without Broadcasting
			Solving Malleability (Segregated Witness)
			Broadcasting the Funding Transaction
		Sending Payments Across the Channel
			Splitting the Balance
			Competing Commitments
			Cheating with Old Commitment Transactions
			Revoking Old Commitment Transactions
			Asymmetric Commitment Transactions
			Delayed (Timelocked) Spending to_self
			Revocation Keys
		The Commitment Transaction
		Advancing the Channel State
			The commitment_signed Message
			The revoke_and_ack Message
			Revoking and Recommitting
			Cheating and Penalty in Practice
			The Channel Reserve: Ensuring Skin in the Game
		Closing the Channel (Cooperative Close)
			The Shutdown Message
			The closing_signed Message
			The Cooperative Close Transaction
		Conclusion
	Chapter 8. Routing on a Network of Payment Channels
		Routing a Payment
		Routing Versus Pathfinding
		Creating a Network of Payment Channels
		A Physical Example of “Routing”
		Fairness Protocol
			Implementing Atomic Trustless Multihop Payments
		Revisiting the Tipping Example
			On-Chain Versus Off-Chain Settlement of HTLCs
		Hash Time-Locked Contracts
			HTLCs in Bitcoin Script
			Payment Preimage and Hash Verification
			Extending HTLCs from Alice to Dina
			Back-Propagating the Secret
			Signature Binding: Preventing Theft of HTLCs
			Hash Optimization
			HTLC Cooperative and Timeout Failure
			Decrementing Timelocks
		Conclusion
	Chapter 9. Channel Operation and Payment Forwarding
		Local (Single Channel) Versus Routed (Multiple Channels)
		Forwarding Payments and Updating Commitments with HTLCs
			HTLC and Commitment Message Flow
		Forwarding Payments with HTLCs
			Adding an HTLC
			The update_add_HTLC Message
			HTLC in Commitment Transactions
			New Commitment with HTLC Output
			Alice Commits
			Bob Acknowledges New Commitment and Revokes Old One
			Bob Commits
		Multiple HTLCs
		HTLC Fulfillment
			HTLC Propagation
			Dina Fulfills the HTLC with Chan
			Bob Settles the HTLC with Alice
		Removing an HTLC Due to Error or Expiry
		Making a Local Payment
		Conclusion
	Chapter 10. Onion Routing
		A Physical Example Illustrating Onion Routing
			Selecting a Path
			Building the Layers
			Peeling the Layers
		Introduction to Onion Routing of HTLCs
			Alice Selects the Path
			Alice Constructs the Payloads
			Key Generation
		Wrapping the Onion Layers
			Fixed-Length Onions
			Wrapping the Onion (Outlined)
			Wrapping Dina’s Hop Payload
			Wrapping Chan’s Hop Payload
			Wrapping Bob’s Hop Payload
			The Final Onion Packet
		Sending the Onion
			The update_add_htlc Message
			Alice Sends the Onion to Bob
			Bob Checks the Onion
			Bob Generates Filler
			Bob De-Obfuscates His Hop Payload
			Bob Extracts the Outer HMAC for the Next Hop
			Bob Removes His Payload and Left-Shifts the Onion
			Bob Constructs the New Onion Packet
			Bob Verifies the HTLC Details
			Bob Sends the update_add_htlc to Chan
			Chan Forwards the Onion
			Dina Receives the Final Payload
		Returning Errors
			Failure Messages
		Keysend Spontaneous Payments
			Custom Onion TLV Records
			Sending and Receiving Keysend Payments
			Keysend and Custom Records in Lightning Applications
		Conclusion
	Chapter 11. Gossip and the Channel Graph
		Peer Discovery
			P2P Bootstrapping
			DNS Bootstrapping
			SRV Query Options
		The Channel Graph
			A Directed Graph
		Gossip Protocol Messages
			The node_announcement Message
			The channel_announcement Message
			The channel_update Message
		Ongoing Channel Graph Maintenance
		Conclusion
	Chapter 12. Pathfinding and Payment Delivery
		Pathfinding in the Lightning Protocol Suite
			Where Is the BOLT?
		Pathfinding: What Problem Are We Solving?
			Selecting the Best Path
			Pathfinding in Math and Computer Science
			Capacity, Balance, Liquidity
			Uncertainty of Balances
			Pathfinding Complexity
			Keeping It Simple
		Pathfinding and Payment Delivery Process
		Channel Graph Construction
			Liquidity Uncertainty and Probability
			Fees and Other Channel Metrics
		Finding Candidate Paths
		Payment Delivery (Trial-and-Error Loop)
			First Attempt (Path #1)
			Second Attempt (Path #4)
		Multipart Payments
			Using MPP
			Trial and Error over Multiple “Rounds”
		Conclusion
	Chapter 13. Wire Protocol: Framing and Extensibility
		Messaging Layer in the Lightning Protocol Suite
		Wire Framing
			High-Level Wire Framing
			Type Encoding
		Type-Length-Value Message Extensions
			The Protocol Buffers Message Format
			Forward and Backward Compatibility
		Type-Length-Value Format
			BigSize Integer Encoding
			TLV Encoding Constraints
			TLV Canonical Encoding
		Feature Bits and Protocol Extensibility
			Feature Bits as an Upgrade Discoverability Mechanism
			TLV for Forward and Backward Compatibility
			A Taxonomy of Upgrade Mechanisms
			Channel Construction-Level Updates
		Conclusion
	Chapter 14. Lightning’s Encrypted Message Transport
		Encrypted Transport in the Lightning Protocol Suite
		Introduction
		The Channel Graph as Decentralized Public Key Infrastructure
		Why Not TLS?
		The Noise Protocol Framework
		Lightning Encrypted Transport in Detail
			Noise_XK: Lightning Network’s Noise Handshake
			Handshake Notation and Protocol Flow
			High-Level Overview
			Handshake in Three Acts
		Conclusion
	Chapter 15. Lightning Payment Requests
		Invoices in the Lightning Protocol Suite
		Introduction
		Lightning Payment Requests Versus Bitcoin Addresses
		BOLT #11: Lightning Payment Request Serialization and Interpretation
			Payment Request Encoding in Practice
			The Human-Readable Prefix
			bech32 and the Data Segment
		Conclusion
	Chapter 16. Security and Privacy of the Lightning Network
		Why Is Privacy Important?
		Definitions of Privacy
		Process to Evaluate Privacy
		Anonymity Set
		Differences Between the Lightning Network and Bitcoin in Terms of Privacy
		Attacks on Lightning
			Observing Payment Amounts
			Linking Senders and Receivers
			Revealing Channel Balances (Probing)
			Denial of Service
			Commitment Jamming
			Channel Liquidity Lockup
		Cross-Layer De-Anonymization
			On-Chain Bitcoin Entity Clustering
			Off-Chain Lightning Node Clustering
			Cross-Layer Linking: Lightning Nodes and Bitcoin Entities
		Lightning Graph
			How Does the Lightning Graph Look in Reality?
		Centralization in the Lightning Network
		Economic Incentives and Graph Structure
		Practical Advice for Users to Protect Their Privacy
		Unannounced Channels
		Routing Considerations
			Accepting Channels
		Conclusion
		References and Further Reading
	Chapter 17. Conclusion
		Decentralized and Asynchronous Innovation
			Bitcoin Protocol and Bitcoin Script Innovation
			Lightning Protocol Innovation
			TLV Extensibility
			Payment Channel Construction
			Opt-In End-to-End Features
		Lightning Applications (LApps)
		Ready, Set, Go!
Appendix A. Bitcoin Fundamentals Review
	Keys and Digital Signatures
		Private and Public Keys
		Hashes
		Digital Signatures
		Signature Types
	Bitcoin Transactions
		Inputs and Outputs
		Transaction Chains
		TxID: Transaction Identifiers
		Outpoints: Output Identifiers
	Bitcoin Script
		Running Bitcoin Script
		Locking and Unlocking Scripts
		Locking to a Public Key (Signature)
		Locking to a Hash (Secret)
		Multisignature Scripts
		Timelock Scripts
		Scripts with Multiple Conditions
		Using Flow Control in Scripts
Appendix B. Docker Basic Installation and Use
	Installing Docker
	Basic Docker Commands
		Building a Container
		Running a Container
		Executing a Command in a Container
		Stopping and Starting a Container
		Deleting a Container by Name
		Listing Running Containers
		Listing Docker Images
	Conclusion
Appendix C. Wire Protocol Messages
	Message Types
	Message Structure
		Connection Establishment Messages
		Error Communication Messages
		Connection Liveness
		Channel Funding
		Channel Closing
		Channel Operation
		Channel Announcement
		Channel Graph Syncing
Appendix D. Sources and License Notices
	Sources
	BTCPay Server
	Lamassu Industries AG
Glossary
Index
About the Authors
Colophon