How Linux Works: What Every Superuser Should Know

Cover......Page 1
Title......Page 5
Copyright......Page 6
About the Technical Reviewers......Page 7
Acknowledgments......Page 22
Who Should Read This Book?......Page 23
A Hands-on Approach......Page 24
What’s New in the Third Edition?......Page 25
A Note on Terminology......Page 26
Chapter 1: The Big Picture......Page 27
 1.1 Levels and Layers of Abstraction in a Linux System......Page 28
 1.3 The Kernel......Page 30
1.3.1 Process Management......Page 31
1.3.3 Device Drivers and Management......Page 32
1.3.4 System Calls and Support......Page 33
 1.4 User Space......Page 34
 1.5 Users......Page 35
 1.6 Looking Forward......Page 36
Chapter 2: Basic Commands and Directory Hierarchy......Page 37
 2.2 Using the Shell......Page 38
2.2.2 cat......Page 39
2.2.3 Standard Input and Standard Output......Page 40
2.3.2 cp......Page 41
 2.4 Navigating Directories......Page 42
2.4.3 rmdir......Page 43
2.4.4 Shell Globbing (“Wildcards”)......Page 44
2.5.1 grep......Page 45
2.5.3 pwd......Page 46
2.5.7 head and tail......Page 47
 2.8 Environment and Shell Variables......Page 48
 2.9 The Command Path......Page 49
 2.10 Special Characters......Page 50
 2.12 Text Editors......Page 51
 2.13 Getting Online Help......Page 52
 2.14 Shell Input and Output......Page 54
 2.15 Understanding Error Messages......Page 55
2.15.2 Common Errors......Page 56
2.16.1 ps Command Options......Page 58
2.16.2 Process Termination......Page 59
2.16.4 Background Processes......Page 60
 2.17 File Modes and Permissions......Page 61
2.17.1 Modifying Permissions......Page 62
2.17.2 Working with Symbolic Links......Page 64
2.18.2 tar......Page 65
2.18.3 Compressed Archives (.tar.gz)......Page 66
2.18.5 Other Compression Utilities......Page 67
 2.19 Linux Directory Hierarchy Essentials......Page 68
2.19.1 Other Root Subdirectories......Page 69
 2.20 Running Commands as the Superuser......Page 70
2.20.2 /etc/sudoers......Page 71
 2.21 Looking Forward......Page 72
Chapter 3: Devices......Page 73
 3.1 Device Files......Page 74
 3.2 The sysfs Device Path......Page 75
 3.3 dd and Devices......Page 76
 3.4 Device Name Summary......Page 77
3.4.1 Hard Disks: /dev/sd*......Page 78
3.4.7 Terminals: /dev/tty*, /dev/pts/*, and /dev/tty......Page 79
3.4.10 Audio Devices: /dev/snd/*, /dev/dsp, /dev/audio, and More......Page 81
 3.5 udev......Page 82
3.5.1 devtmpfs......Page 83
3.5.2 udevd Operation and Configuration......Page 84
3.5.3 udevadm......Page 86
3.5.4 Device Monitoring......Page 87
 3.6 In-Depth: SCSI and the Linux Kernel......Page 88
3.6.2 SCSI and ATA......Page 91
3.6.3 Generic SCSI Devices......Page 92
3.6.4 Multiple Access Methods for a Single Device......Page 93
Chapter 4: Disks and Filesystems......Page 95
4.1.1 Viewing a Partition Table......Page 98
4.1.2 Modifying Partition Tables......Page 101
4.1.3 Creating a Partition Table......Page 102
4.1.4 Navigating Disk and Partition Geometry......Page 104
 4.2 Filesystems......Page 106
4.2.1 Filesystem Types......Page 107
4.2.2 Creating a Filesystem......Page 108
4.2.3 Mounting a Filesystem......Page 109
4.2.4 Filesystem UUID......Page 111
4.2.6 Filesystem Mount Options......Page 112
4.2.7 Remounting a Filesystem......Page 113
4.2.8 The /etc/fstab Filesystem Table......Page 114
4.2.10 Filesystem Capacity......Page 115
4.2.11 Checking and Repairing Filesystems......Page 117
4.2.12 Special-Purpose Filesystems......Page 119
4.3.1 Using a Disk Partition as Swap Space......Page 120
4.3.3 Determining How Much Swap You Need......Page 121
 4.4 The Logical Volume Manager......Page 122
4.4.2 Working with LVM......Page 123
4.4.3 The LVM Implementation......Page 133
 4.6 Inside a Traditional Filesystem......Page 137
4.6.1 Inode Details and the Link Count......Page 139
4.6.2 Block Allocation......Page 140
4.6.3 Working with Filesystems in User Space......Page 141
Chapter 5: How the Linux Kernel Boots......Page 143
 5.1 Startup Messages......Page 144
 5.2 Kernel Initialization and Boot Options......Page 145
 5.3 Kernel Parameters......Page 146
 5.4 Boot Loaders......Page 147
5.4.1 Boot Loader Tasks......Page 148
 5.5 GRUB Introduction......Page 149
5.5.1 Exploring Devices and Partitions with the GRUB Command Line......Page 151
5.5.2 GRUB Configuration......Page 153
5.5.3 GRUB Installation......Page 156
 5.6 UEFI Secure Boot Problems......Page 157
 5.8 Boot Loader Details......Page 158
5.8.2 UEFI Boot......Page 159
5.8.3 How GRUB Works......Page 160
Chapter 6: How User Space Starts......Page 163
 6.1 Introduction to init......Page 164
 6.3 systemd......Page 165
6.3.2 Booting and Unit Dependency Graphs......Page 166
6.3.3 systemd Configuration......Page 167
6.3.4 systemd Operation......Page 170
6.3.5 systemd Process Tracking and Synchronization......Page 173
6.3.6 systemd Dependencies......Page 174
6.3.7 systemd On-Demand and Resource-Parallelized Startup......Page 177
 6.4 System V Runlevels......Page 182
 6.5 System V init......Page 183
6.5.1 System V init: Startup Command Sequence......Page 184
6.5.2 The System V init Link Farm......Page 185
6.5.3 run-parts......Page 186
6.5.5 systemd System V Compatibility......Page 187
 6.6 Shutting Down Your System......Page 188
 6.7 The Initial RAM Filesystem......Page 189
 6.8 Emergency Booting and Single-User Mode......Page 190
 6.9 Looking Forward......Page 191
Chapter 7: System Configuration: Logging, System Time, Batch Jobs, and Users......Page 193
 7.1 System Logging......Page 194
7.1.2 Searching and Monitoring Logs......Page 195
7.1.3 Logfile Rotation......Page 198
7.1.5 A Closer Look at System Logging......Page 199
 7.2 The Structure of /etc......Page 202
7.3.1 The /etc/passwd File......Page 203
7.3.2 Special Users......Page 204
7.3.4 Manipulating Users and Passwords......Page 205
7.3.5 Working with Groups......Page 206
 7.5 Setting the Time......Page 207
7.5.2 Network Time......Page 208
 7.6 Scheduling Recurring Tasks with cron and Timer Units......Page 209
7.6.1 Installing Crontab Files......Page 210
7.6.3 Timer Units......Page 211
 7.7 Scheduling One-Time Tasks with at......Page 213
 7.8 Timer Units Running as Regular Users......Page 214
7.9.2 Process Ownership, Effective UID, Real UID, and Saved UID......Page 215
7.9.3 User Identification, Authentication, and Authorization......Page 217
 7.10 Pluggable Authentication Modules......Page 218
7.10.1 PAM Configuration......Page 219
7.10.2 Tips on PAM Configuration Syntax......Page 222
7.10.3 PAM and Passwords......Page 223
 7.11 Looking Forward......Page 224
Chapter 8: A Closer Look at Processes and Resource Utilization......Page 225
 8.2 Finding Open Files with lsof......Page 226
8.2.1 Reading the lsof Output......Page 227
8.3.1 strace......Page 228
8.4.1 Single-Threaded and Multithreaded Processes......Page 230
8.4.2 Viewing Threads......Page 231
 8.5 Introduction to Resource Monitoring......Page 232
8.5.2 Adjusting Process Priorities......Page 233
8.5.3 Measuring CPU Performance with Load Averages......Page 234
8.5.4 Monitoring Memory Status......Page 236
8.5.5 Monitoring CPU and Memory Performance with vmstat......Page 238
8.5.6 I/O Monitoring......Page 240
 8.6 Control Groups (cgroups)......Page 242
8.6.1 Differentiating Between cgroup Versions......Page 243
8.6.2 Viewing cgroups......Page 245
8.6.3 Manipulating and Creating cgroups......Page 246
 8.7 Further Topics......Page 247
Chapter 9: Understanding Your Network and Its Configuration......Page 249
 9.2 Packets......Page 250
 9.3 Network Layers......Page 251
 9.4 The Internet Layer......Page 252
9.4.2 Subnets......Page 254
9.4.3 Common Subnet Masks and CIDR Notation......Page 255
 9.5 Routes and the Kernel Routing Table......Page 256
 9.7 IPv6 Addresses and Networks......Page 257
9.7.1 Viewing IPv6 Configuration on Your System......Page 258
9.7.2 Configuring Dual-Stack Networks......Page 259
9.8.1 ping......Page 260
 9.9 The Physical Layer and Ethernet......Page 261
 9.10 Understanding Kernel Network Interfaces......Page 262
9.11.1 Manually Configuring Interfaces......Page 263
9.11.2 Manually Adding and Deleting Routes......Page 264
 9.13  Problems with Manual and Boot-Activated Network Configuration......Page 265
 9.14 Network Configuration Managers......Page 266
9.14.2 NetworkManager Interaction......Page 267
9.14.3 NetworkManager Configuration......Page 268
 9.15 Resolving Hostnames......Page 269
9.15.1 /etc/hosts......Page 270
9.15.3 Caching and Zero-Configuration DNS......Page 271
9.15.4 /etc/nsswitch.conf......Page 272
 9.17 The Transport Layer: TCP, UDP, and Services......Page 273
9.17.1 TCP Ports and Connections......Page 274
9.17.2 UDP......Page 276
 9.19 Understanding DHCP......Page 278
 9.20 Automatic IPv6 Network Configuration......Page 279
 9.21 Configuring Linux as a Router......Page 280
 9.23 Network Address Translation (IP Masquerading)......Page 282
 9.24 Routers and Linux......Page 284
9.25.1 Linux Firewall Basics......Page 285
9.25.2 Setting Firewall Rules......Page 287
9.25.3 Firewall Strategies......Page 288
 9.26 Ethernet, IP, ARP, and NDP......Page 290
 9.27 Wireless Ethernet......Page 292
9.27.1 iw......Page 293
 9.28 Summary......Page 294
Chapter 10: Network Applications and Services......Page 295
 10.1 The Basics of Services......Page 296
 10.2 A Closer Look......Page 297
 10.3 Network Servers......Page 298
10.3.1  Secure Shell......Page 299
10.3.2  The sshd Server......Page 300
10.3.3  fail2ban......Page 302
10.3.4  The SSH Client......Page 303
 10.4  Pre-systemd Network Connection Servers: inetd/xinetd......Page 304
 10.5 Diagnostic Tools......Page 305
10.5.1  lsof......Page 306
10.5.2  tcpdump......Page 307
10.5.3  netcat......Page 309
 10.6 Remote Procedure Calls......Page 310
 10.7 Network Security......Page 311
10.7.1  Typical Vulnerabilities......Page 312
 10.8 Looking Forward......Page 313
 10.9 Network Sockets......Page 314
 10.10 Unix Domain Sockets......Page 315
 11.1 Shell Script Basics......Page 317
11.1.1 Limitations of Shell Scripts......Page 318
11.2.1 Literals......Page 319
11.2.2 Single Quotes......Page 320
11.2.4 Literal Single Quotes......Page 321
11.3.1 Individual Arguments: $1, $2, and So On......Page 322
11.3.4 Script Name: $0......Page 323
 11.4 Exit Codes......Page 324
11.5.1 A Workaround for Empty Parameter Lists......Page 325
11.5.4 Logical Constructs......Page 326
11.5.5 Testing Conditions......Page 327
11.5.6 case......Page 330
11.6.2 while Loops......Page 331
 11.7 Command Substitution......Page 332
 11.8 Temporary File Management......Page 333
11.10.1  basename......Page 334
11.10.3  sed......Page 335
11.10.4  xargs......Page 336
 11.11 Subshells......Page 337
 11.14 When (Not) to Use Shell Scripts......Page 338
Chapter 12: Network File Transfer and Sharing......Page 341
 12.1 Quick Copy......Page 342
12.2.1  Getting Started with rsync......Page 343
12.2.2  Making Exact Copies of a Directory Structure......Page 344
12.2.3  Using the Trailing Slash......Page 345
12.2.4  Excluding Files and Directories......Page 346
12.2.6  Compressing Data......Page 347
12.2.9  Further rsync Topics......Page 348
12.3.2  File Sharing Security......Page 349
12.4.1  Server Configuration......Page 350
12.4.2  Server Access Control......Page 351
12.4.3  Passwords......Page 352
12.4.4  Manual Server Startup......Page 353
12.4.7  Home Directories......Page 354
12.4.9  The Samba Client......Page 355
 12.5 SSHFS......Page 357
 12.6 NFS......Page 358
 12.8 The State of Network File Sharing......Page 359
Chapter 13: User Environments......Page 361
 13.2 When to Alter Startup Files......Page 362
13.3.1  The Command Path......Page 363
13.3.3  The Prompt......Page 364
13.3.5  The Permissions Mask......Page 365
13.4.1  The bash Shell......Page 366
13.4.2  The tcsh Shell......Page 368
13.5.1  Shell Defaults......Page 369
 13.6 Startup File Pitfalls......Page 370
 13.7 Further Startup Topics......Page 371
Chapter 14: A Brief Survey of the Linux Desktop and Printing......Page 373
14.1.2  The X Window System......Page 374
14.1.4  Window Managers......Page 375
14.1.7  Applications......Page 376
14.3.1  The Compositing Window Manager......Page 377
14.3.2  libinput......Page 378
14.3.3  X Compatibility in Wayland......Page 379
 14.4 A Closer Look at the X Window System......Page 380
14.4.3 Ways of Exploring X Clients......Page 381
14.4.4  X Events......Page 382
14.4.5  X Input and Preference Settings......Page 383
 14.5 D-Bus......Page 385
 14.6 Printing......Page 386
14.6.2  Format Conversion and Print Filters......Page 387
 14.7 Other Desktop Topics......Page 388
Chapter 15: Development Tools......Page 389
 15.1 The C Compiler......Page 390
15.1.1  Compiling Multiple Source Files......Page 391
15.1.2  Linking with Libraries......Page 392
15.1.3 Working with Shared Libraries......Page 393
15.1.4  Working with Header (Include) Files and Directories......Page 397
 15.2 make......Page 399
15.2.2  Built-in Rules......Page 400
15.2.4  Dependency Updates......Page 401
15.2.5  Command-Line Arguments and Options......Page 402
15.2.6  Standard Macros and Variables......Page 403
15.2.8  Makefile Organization......Page 404
 15.3 Lex and Yacc......Page 405
 15.4 Scripting Languages......Page 406
15.4.3  Other Scripting Languages......Page 407
 15.5 Java......Page 408
 15.6 Looking Forward: Compiling Packages......Page 409
Chapter 16: Introduction to Compiling Software from C Source Code......Page 411
 16.1 Software Build Systems......Page 412
 16.2 Unpacking C Source Packages......Page 413
 16.3 GNU Autoconf......Page 414
16.3.1  An Autoconf Example......Page 415
16.3.3  configure Script Options......Page 416
16.3.4  Environment Variables......Page 417
16.3.6  Autoconf Logfiles......Page 418
16.3.7  pkg-config......Page 419
 16.4 Installation Practice......Page 420
 16.5 Applying a Patch......Page 421
 16.6 Troubleshooting Compiles and Installations......Page 422
16.6.1  Specific Errors......Page 423
 16.7 Looking Forward......Page 425
Chapter 17: Virtualization......Page 427
17.1.1  Hypervisors......Page 428
17.1.2  Hardware in a Virtual Machine......Page 429
17.1.4  Drawbacks of Virtual Machines......Page 430
 17.2 Containers......Page 431
17.2.1  Docker, Podman, and Privileges......Page 432
17.2.2  A Docker Example......Page 433
17.2.3  LXC......Page 440
17.2.5  Pitfalls of Containers......Page 441
 17.3 Runtime-Based Virtualization......Page 443
Bibliography......Page 445
Index......Page 449