Mastering Embedded Linux Programming - Third Edition

Cover
Copyright
Contributors
Table of Contents
Preface
Section 1: Elements of Embedded Linux
Chapter 1: Starting Out
	Choosing Linux
	When not to choose Linux
	Meeting the players
	Moving through the project life cycle
		The four elements of embedded Linux
	Navigating open source
		Licenses
	Selecting hardware for embedded Linux
	Obtaining the hardware for this book
		The Raspberry Pi 4
		The BeagleBone Black
		QEMU
	Provisioning your development environment
	Summary
Chapter 2: Learning about Toolchains
	Technical requirements
	Introducing toolchains
		Types of toolchains
		CPU architectures
		Choosing the C library
	Finding a toolchain
	Building a toolchain using crosstool-NG
		Installing crosstool-NG
		Building a toolchain for BeagleBone Black
		Building a toolchain for QEMU
	Anatomy of a toolchain
		Finding out about your cross compiler
		The sysroot, library, and header files
		Other tools in the toolchain
		Looking at the components of the C library
	Linking with libraries – static and dynamic linking
		Static libraries
		Shared libraries
	The art of cross-compiling
		Simple makefiles
		Autotools
		Package configuration
		Problems with cross-compiling
		CMake
	Summary
	Further reading
Chapter 3: All about Bootloaders
	Technical requirements
	What does a bootloader do?
	The boot sequence
		Phase 1 – ROM code
		Phase 2 – secondary program loader
		Phase 3 – TPL
	Moving from the bootloader to a kernel
	Introducing device trees
		Device tree basics
		The reg property
		Labels and interrupts
		Device tree include files
		Compiling a device tree
	U-Boot
		Building U-Boot
		Installing U-Boot
		Using U-Boot
		Booting Linux
		Porting U-Boot to a new board
		Building and testing
		Falcon mode
	Summary
Chapter 4: Configuring and Building the Kernel
	Technical requirements
	What does the kernel do?
	Choosing a kernel
		Kernel development cycle
		Stable and long-term support releases
	Building the kernel
		Getting the source
		Understanding kernel configuration – Kconfig
		Using LOCALVERSION to identify your kernel
		When to use kernel modules
	Compiling – Kbuild
		Finding out which kernel target to build
		Build artifacts
		Compiling device trees
		Compiling modules
		Cleaning kernel sources
		Building a 64-bit kernel for the Raspberry Pi 4
		Building a kernel for the BeagleBone Black
		Building a kernel for QEMU
	Booting the kernel
		Booting the Raspberry Pi 4
		Booting the BeagleBone Black
		Booting QEMU
		Kernel panic
		Early user space
		Kernel messages
		The kernel command line
	Porting Linux to a new board
		A new device tree
		Setting the board's compatible property
	Summary
	Additional reading
Chapter 5: Building a Root Filesystem
	Technical requirements
	What should be in the root filesystem?
		The directory layout
		The staging directory
		POSIX file access permissions
		File ownership permissions in the staging directory
		Programs for the root filesystem
		Libraries for the root filesystem
		Device nodes
		The proc and sysfs filesystems
		Kernel modules
	Transferring the root filesystem to the target
	Creating a boot initramfs
		Standalone initramfs
		Booting the initramfs
		Booting with QEMU
		Booting the BeagleBone Black
		Building an initramfs into the kernel image
		Building an initramfs using a device table
		The old initrd format
	The init program
		Starting a daemon process
	Configuring user accounts
		Adding user accounts to the root filesystem
	A better way of managing device nodes
		An example using devtmpfs
		An example using mdev
		Are static device nodes so bad after all?
	Configuring the network
		Network components for glibc
	Creating filesystem images with device tables
		Booting the BeagleBone Black
	Mounting the root filesystem using NFS
		Testing with QEMU
		Testing with the BeagleBone Black
		Problems with file permissions
	Using TFTP to load the kernel
	Summary
	Further reading
Chapter 6: Selecting a Build System
	Technical requirements
	Comparing build systems
	Distributing binaries
	Introducing Buildroot
		Background
		Stable releases and long-term support
		Installing
		Configuring
		Running
		Targeting real hardware
		Creating a custom BSP
		Adding your own code
		License compliance
	Introducing the Yocto Project
		Background
		Stable releases and supports
		Installing the Yocto Project
		Configuring
		Building
		Running the QEMU target
		Layers
		Customizing images via local.conf
		Writing an image recipe
		Creating an SDK
		The license audit
	Summary
	Further reading
Chapter 7: Developing with Yocto
	Technical requirements
	Building on top of an existing BSP
		Building an existing BSP
		Controlling Wi-Fi
		Controlling Bluetooth
		Adding a custom layer
	Capturing changes with devtool
		Development workflows
		Creating a new recipe
		Modifying the source built by a recipe
		Upgrading a recipe to a newer version
	Building your own distro
		When and when not to
		Creating a new distro layer
		Configuring your distro
		Adding more recipes to your distro
		Runtime package management
	Provisioning a remote package server
	Summary
	Further reading
Chapter 8: Yocto Under the Hood
	Technical requirements
	Decomposing Yocto's architecture and workflow
		Metadata
		Build tasks
		Image generation
	Separating metadata into layers
	Troubleshooting build failures
		Isolating errors
		Dumping the environment
		Reading the task log
		Adding more logging
		Running commands from devshell
		Graphing dependencies
	Understanding BitBake syntax and semantics
		Tasks
		Dependencies
		Variables
		Functions
		RDEPENDS revisited
	Summary
	Further reading
Section 2: System Architecture and Design Decisions
Chapter 9: Creating a Storage Strategy
	Technical requirements
	Storage options
		NOR flash
		NAND flash
	Accessing flash memory from the bootloader
		U-Boot and NOR flash
		U-Boot and NAND flash
		U-Boot and MMC, SD, and eMMC
	Accessing flash memory from Linux
		Memory technology devices
		The MMC block driver
	Filesystems for flash memory
		Flash translation layers
	Filesystems for NOR and NAND flash memory
		JFFS2
		YAFFS2
		UBI and UBIFS
	Filesystems for managed flash
		Flashbench
		Discard and TRIM
		Ext4
		F2FS
		FAT16/32
	Read-only compressed filesystems
		SquashFS
	Temporary filesystems
	Making the root filesystem read-only
	Filesystem choices
	Summary
	Further reading
Chapter 10: Updating Software in the Field
	Technical requirements
	From where do updates originate?
	What to update
		Bootloader
		Kernel
		Root filesystem
		System applications
		Device-specific data
		Components that need to be updated
	The basics of software updates
		Making updates robust
		Making updates fail-safe
		Making updates secure
	Types of update mechanism
		Symmetric image update
		Asymmetric image update
		Atomic file updates
	OTA updates
	Using Mender for local updates
		Building the Mender client
	Installing an update
	Using Mender for OTA updates
	Using balena for local updates
		Creating an account
		Creating an application
		Adding a device
		Installing the CLI
		Pushing a project
	Summary
Chapter 11: Interfacing with Device Drivers
	Technical requirements
	The role of device drivers
	Character devices
	Block devices
	Network devices
	Finding out about drivers at runtime
		Getting information from sysfs
	Finding the right device driver
	Device drivers in user space
		GPIO
		LEDs
		I2C
		SPI
	Writing a kernel device driver
		Designing a character driver interface
		The anatomy of a device driver
		Compiling kernel modules
		Loading kernel modules
	Discovering the hardware configuration
		Device trees
		The platform data
		Linking hardware with device drivers
	Summary
	Further reading
Chapter 12: Prototyping with Breakout Boards
	Technical requirements
	Mapping schematics to the device tree's source
		Reading schematics and data sheets
		Installing Debian on the BeagleBone Black
		Enabling spidev
		Customizing the device tree
	Prototyping with breakout boards
		Closing the SPI jumper
		Attaching the GNSS antenna
		Attaching the SPI header
		Connecting the SPI jumper wires
	Probing SPI signals with a logic analyzer
	Receiving NMEA messages over SPI
	Summary
	Further reading
Chapter 13: Starting Up – The init Program
	Technical requirements
	After the kernel has booted
	Introducing the init programs
	BusyBox init
		Buildroot init scripts
	System V init
		inittab
		The init.d scripts
		Adding a new daemon
		Starting and stopping services
	systemd
		Building systemd with the Yocto Project and Buildroot
		Introducing targets, services, and units
		How systemd boots the system
		Adding your own service
		Adding a watchdog
		Implications for embedded Linux
	Summary
	Further reading
Chapter 14: Starting with BusyBox runit
	Technical requirements
	Getting BusyBox runit
	Creating service directories and files
		Service directory layout
		Service configuration
	Service supervision
		Controlling services
	Depending on other services
		Start dependencies
		Custom start dependencies
		Putting it all together
	Dedicated service logging
		How does it work?
		Adding dedicated logging to a service
		Log rotation
	Signaling a service
	Summary
	Further reading
Chapter 15: Managing Power
	Technical requirements
	Measuring power usage
	Scaling the clock frequency
		The CPUFreq driver
		Using CPUFreq
	Selecting the best idle state
		The CPUIdle driver
		Tickless operation
	Powering down peripherals
	Putting the system to sleep
		Power states
		Wakeup events
		Timed wakeups from the real-time clock
	Summary
	Further reading
Section 3: Writing Embedded Applications
Chapter 16: Packaging Python
	Technical requirements
		Getting Docker
	Retracing the origins of Python packaging
		distutils
		setuptools
		setup.py
	Installing Python packages with pip
		requirements.txt
	Managing Python virtual environments with venv
	Installing precompiled binaries with conda
		Environment management
		Package management
	Deploying Python applications with Docker
		The anatomy of a Dockerfile
		Building a Docker image
		Running a Docker image
		Fetching a Docker image
		Publishing a Docker image
		Cleaning up
	Summary
	Further reading
Chapter 17: Learning about Processes and Threads
	Technical requirements
	Process or thread?
	Processes
		Creating a new process
		Terminating a process
		Running a different program
		Daemons
		Inter-process communication
	Threads
		Creating a new thread
		Terminating a thread
		Compiling a program with threads
		Inter-thread communication
		Mutual exclusion
		Changing conditions
		Partitioning the problem
	ZeroMQ
		Getting pyzmq
		Messaging between processes
		Messaging within processes
	Scheduling
		Fairness versus determinism
		Time-shared policies
		Real-time policies
		Choosing a policy
		Choosing a real-time priority
	Summary
	Further reading
Chapter 18: Managing Memory
	Technical requirements
	Virtual memory basics
	Kernel space memory layout
		How much memory does the kernel use?
	User space memory layout
	The process memory map
	Swapping
		Swapping to compressed memory (zram)
	Mapping memory with mmap
		Using mmap to allocate private memory
		Using mmap to share memory
		Using mmap to access device memory
	How much memory does my application use?
	Per-process memory usage
		Using top and ps
		Using smem
		Other tools to consider
	Identifying memory leaks
		mtrace
		Valgrind
	Running out of memory
	Summary
	Further reading
Section 4: Debugging and Optimizing Performance
Chapter 19: Debugging with GDB
	Technical requirements
	The GNU debugger
	Preparing to debug
	Debugging applications
		Remote debugging using gdbserver
		Setting up the Yocto Project for remote debugging
		Setting up Buildroot for remote debugging
		Starting to debug
		Native debugging
	Just-in-time debugging
	Debugging forks and threads
	Core files
		Using GDB to look at core files
	GDB user interfaces
		Terminal User Interface
		Data Display Debugger
		Visual Studio Code
	Debugging kernel code
		Debugging kernel code with kgdb
		A sample debug session
		Debugging early code
		Debugging modules
		Debugging kernel code with kdb
		Looking at an Oops
		Preserving the Oops
	Summary
	Further reading
Chapter 20: Profiling and Tracing
	Technical requirements
	The observer effect
		Symbol tables and compile flags
	Beginning to profile
	Profiling with top
	The poor man's profiler
	Introducing perf
		Configuring the kernel for perf
		Building perf with the Yocto Project
		Building perf with Buildroot
		Profiling with perf
		Call graphs
		perf annotate
	Tracing events
	Introducing Ftrace
		Preparing to use Ftrace
		Using Ftrace
		Dynamic Ftrace and trace filters
		Trace events
	Using LTTng
		LTTng and the Yocto Project
		LTTng and Buildroot
		Using LTTng for kernel tracing
	Using BPF
		Configuring the kernel for BPF
		Building a BCC toolkit with Buildroot
		Using BPF tracing tools
	Using Valgrind
		Callgrind
		Helgrind
	Using strace
	Summary
	Further reading
Chapter 21: Real-Time Programming
	Technical requirements
	What is real time?
	Identifying sources of non-determinism
	Understanding scheduling latency
	Kernel preemption
		The real-time Linux kernel (PREEMPT_RT)
		Threaded interrupt handlers
	Preemptible kernel locks
		Getting the PREEMPT_RT patches
		The Yocto Project and PREEMPT_RT
	High-resolution timers
	Avoiding page faults
	Interrupt shielding
	Measuring scheduling latencies
		cyclictest
		Using Ftrace
		Combining cyclictest and Ftrace
	Summary
	Further reading
	Why subscribe?
Other Books You May Enjoy
Index