Python How-To: 63 techniques to improve your Python code

Python How-To
contents
preface
acknowledgments
about this book
	Who should read this book
	How this book is organized: A road map
	About the appendices
	About the code
	liveBook discussion forum
	Other online resources
about the author
about the cover illustration
Chapter 1: Developing a pragmatic learning strategy
	1.1 Aiming at becoming a pragmatic programmer
		1.1.1 Focusing on writing readable Python code
		1.1.2 Considering maintainability even before you write any code
	1.2 What Python can do well or as well as other languages
	1.3 What Python can’t do or can’t do well
	1.4 What you’ll learn in this book
		1.4.1 Focusing on domain-independent knowledge
		1.4.2 Solving problems through synthesis
		1.4.3 Learning skills in context
Part 1: Using built-in data models
	Chapter 2: Processing and formatting strings
		2.1 How do I use f-strings for string interpolation and formatting?
			2.1.1 Formatting strings before f-strings
			2.1.2 Using f-strings to interpolate variables
			2.1.3 Using f-strings to interpolate expressions
			2.1.4 Applying specifiers to format f-strings
			2.1.5 Discussion
			2.1.6 Challenge
		2.2 How do I convert strings to retrieve the represented data?
			2.2.1 Checking whether strings represent alphanumeric values
			2.2.2 Casting strings to numbers
			2.2.3 Evaluating strings to derive their represented data
			2.2.4 Discussion
			2.2.5 Challenge
		2.3 How do I join and split strings?
			2.3.1 Joining strings with whitespaces
			2.3.2 Joining strings with any delimiters
			2.3.3 Splitting strings to create a list of strings
			2.3.4 Discussion
			2.3.5 Challenge
		2.4 What are the essentials of regular expressions?
			2.4.1 Using regular expressions in Python
			2.4.2 Creating the pattern with a raw string
			2.4.3 Understanding the essentials of a search pattern
			2.4.4 Dissecting the matches
			2.4.5 Knowing the common methods
			2.4.6 Discussion
			2.4.7 Challenge
		2.5 How do I use regular expressions to process texts?
			2.5.1 Creating a working pattern to find the matches
			2.5.2 Extracting the needed data from the matches
			2.5.3 Using named groups for text processing
			2.5.4 Discussion
			2.5.5 Challenge
	Chapter 3: Using built-in data containers
		3.1 How do I choose between lists and tuples?
			3.1.1 Using tuples for immutability and using lists for mutability
			3.1.2 Using tuples for heterogeneity and using lists for homogeneity
			3.1.3 Discussion
			3.1.4 Challenge
		3.2 How do I sort lists of complicated data using custom functions?
			3.2.1 Sorting lists using the default order
			3.2.2 Using a built-in function as the sorting key
			3.2.3 Using custom functions for more complicated sorting needs
			3.2.4 Discussion
			3.2.5 Challenge
		3.3 How do I build a lightweight data model using named tuples?
			3.3.1 Understanding alternative data models
			3.3.2 Creating named tuples to hold data
			3.3.3 Discussion
			3.3.4 Challenge
		3.4 How do I access dictionary keys, values, and items?
			3.4.1 Using dynamic view objects (keys, values, and items) directly
			3.4.2 Being cautious with the KeyError exception
			3.4.3 Avoiding KeyError with a hygiene check first: The non-Pythonic way
			3.4.4 Using the get method to access a dictionary item
			3.4.5 Watching for the setdefault method’s side effect
			3.4.6 Discussion
			3.4.7 Challenge
		3.5 When do I use dictionaries and sets instead of lists and tuples?
			3.5.1 Taking advantage of the constant lookup efficiency
			3.5.2 Understanding hashable and hashing
			3.5.3 Discussion
			3.5.4 Challenge
		3.6 How do I use set operations to check the relationships between lists?
			3.6.1 Checking whether a list contains all items of another list
			3.6.2 Checking whether a list contains any element of another list
			3.6.3 Dealing with multiple set objects
			3.6.4 Discussion
			3.6.5 Challenge
	Chapter 4: Dealing with sequence data
		4.1 How do I retrieve and manipulate subsequences with slice objects?
			4.1.1 Taking advantage of the full features of slicing
			4.1.2 Not confusing slices with ranges
			4.1.3 Using named slice objects to process sequence data
			4.1.4 Manipulating list items with slicing operations
			4.1.5 Discussion
			4.1.6 Challenge
		4.2 How do I use positive and negative indexing to retrieve items?
			4.2.1 Positive indexing starts from the beginning of the list
			4.2.2 Negative indexing starts from the end of the list
			4.2.3 Combining positive and negative indices as needed
			4.2.4 Discussion
			4.2.5 Challenge
		4.3 How do I find items in a sequence?
			4.3.1 Checking an item’s presence
			4.3.2 Using the index method to locate the item
			4.3.3 Finding substrings in a string
			4.3.4 Finding an instance of custom classes in a list
			4.3.5 Discussion
			4.3.6 Challenge
		4.4 How do I unpack a sequence? Beyond tuple unpacking
			4.4.1 Unpacking short sequences with one-to-one correspondence
			4.4.2 Retrieving consecutive items using the starred expression
			4.4.3 Denoting unwanted items with underscores to remove distraction
			4.4.4 Discussion
			4.4.5 Challenge
		4.5 When should I consider data models other than lists and tuples?
			4.5.1 Using sets where membership is concerned
			4.5.2 Using deques if you care about first-in-first-out
			4.5.3 Processing multidimensional data with NumPy and Pandas
			4.5.4 Discussion
			4.5.5 Challenge
	Chapter 5: Iterables and iterations
		5.1 How do I create common data containers using iterables?
			5.1.1 Getting to know iterables and iterators
			5.1.2 Inspecting iterability
			5.1.3 Using iterables to create built-in data containers
			5.1.4 Discussion
			5.1.5 Challenge
		5.2 What are list, dictionary, and set comprehensions?
			5.2.1 Creating lists from iterables using list comprehension
			5.2.2 Creating dictionaries from iterables using dictionary comprehension
			5.2.3 Creating sets from iterables using set comprehension
			5.2.4 Applying a filtering condition
			5.2.5 Using embedded for loops
			5.2.6 Discussion
			5.2.7 Challenge
		5.3 How do I improve for-loop iterations with built-in functions?
			5.3.1 Enumerating items with enumerate
			5.3.2 Reversing items with reversed
			5.3.3 Aligning iterables with zip
			5.3.4 Chaining multiple iterables with chain
			5.3.5 Filtering the iterable with filter
			5.3.6 Discussion
			5.3.7 Challenge
		5.4 Using optional statements within for and while loops
			5.4.1 Exiting the loops with the break statement
			5.4.2 Skipping an iteration with the continue statement
			5.4.3 Using else statements in the for and while loops
			5.4.4 Discussion
			5.4.5 Challenge
Part 2: Defining functions
	Chapter 6: Defining user-friendly functions
		6.1 How do I set default arguments to make function calls easier?
			6.1.1 Calling functions with default arguments
			6.1.2 Defining functions with default arguments
			6.1.3 Avoiding the pitfall of setting default arguments for mutable parameters
			6.1.4 Discussion
			6.1.5 Challenge
		6.2 How do I set and use the return value in function calls?
			6.2.1 Returning a value implicitly or explicitly
			6.2.2 Defining functions returning zero, one, or multiple values
			6.2.3 Using multiple values returned from a function call
			6.2.4 Discussion
			6.2.5 Challenge
		6.3 How do I use type hints to write understandable functions?
			6.3.1 Providing type hinting to variables
			6.3.2 Using type hinting in function definitions
			6.3.3 Applying advanced type-hinting skills to function definitions
			6.3.4 Discussion
			6.3.5 Challenge
		6.4 How do I increase function flexibility with *args and **kwargs?
			6.4.1 Knowing positional and keyword arguments
			6.4.2 Accepting a variable number of positional arguments
			6.4.3 Accepting a variable number of keyword arguments
			6.4.4 Discussion
			6.4.5 Challenge
		6.5 How do I write proper docstrings for a function?
			6.5.1 Examining the basic structure of a function's docstring
			6.5.2 Specifying the function's action as the summary
			6.5.3 Documenting the parameters and the return value
			6.5.4 Specifying any exceptions possibly raised
			6.5.5 Discussion
			6.5.6 Challenge
	Chapter 7: Using functions beyond the basics
		7.1 How do I use lambda functions for small jobs?
			7.1.1 Creating a lambda function
			7.1.2 Using lambdas to perform a small one-time job
			7.1.3 Avoiding pitfalls when using lambda functions
			7.1.4 Discussion
			7.1.5 Challenge
		7.2 What are the implications of functions as objects?
			7.2.1 Storing functions in a data container
			7.2.2 Sending functions as arguments to higher-order functions
			7.2.3 Using functions as a return value
			7.2.4 Discussion
			7.2.5 Challenge
		7.3 How do I check functions’ performance with decorators?
			7.3.1 Decorating a function to show its performance
			7.3.2 Dissecting the decorator function
			7.3.3 Wrapping to carry over the decorated function’s metadata
			7.3.4 Discussion
			7.3.5 Challenge
		7.4 How can I use generator functions as a memory- efficient data provider?
			7.4.1 Creating a generator to yield perfect squares
			7.4.2 Using generators for their memory efficiency
			7.4.3 Using generator expressions where applicable
			7.4.4 Discussion
			7.4.5 Challenge
		7.5 How do I create partial functions to make routine function calls easier?
			7.5.1 “Localizing” shared functions to simplify function calls
			7.5.2 Creating a partial function to localize a function
			7.5.3 Discussion
			7.5.4 Challenge
Part 3: Defining classes
	Chapter 8: Defining user-friendly classes
		8.1 How do I define the initialization method for a class?
			8.1.1 Demystifying self: The first parameter in __init__
			8.1.2 Setting proper arguments in __init__
			8.1.3 Specifying all attributes in __init__
			8.1.4 Defining class attributes outside the __init__ method
			8.1.5 Discussion
			8.1.6 Challenge
		8.2 When do I define instance, static, and class methods?
			8.2.1 Defining instance methods for manipulating individual instances
			8.2.2 Defining static methods for utility functionalities
			8.2.3 Defining class methods for accessing class-level attributes
			8.2.4 Discussion
			8.2.5 Challenge
		8.3 How do I apply finer access control to a class?
			8.3.1 Creating protected methods by using an underscore as the prefix
			8.3.2 Creating private methods by using double underscores as the prefix
			8.3.3 Creating read-only attributes with the property decorator
			8.3.4 Verifying data integrity with a property setter
			8.3.5 Discussion
			8.3.6 Challenge
		8.4 How do I customize string representation for a class?
			8.4.1 Overriding __str__ to show meaningful information for an instance
			8.4.2 Overriding __repr__ to provide instantiation information
			8.4.3 Understanding the differences between __str__ and __repr__
			8.4.4 Discussion
			8.4.5 Challenge
		8.5 Why and how do I create a superclass and subclasses?
			8.5.1 Identifying the use scenario of subclasses
			8.5.2 Inheriting the superclass's attributes and methods automatically
			8.5.3 Overriding the superclass's methods to provide customized behaviors
			8.5.4 Creating non-public methods of the superclass
			8.5.5 Discussion
			8.5.6 Challenge
	Chapter 9: Using classes beyond the basics
		9.1 How do I create enumerations?
			9.1.1 Avoiding a regular class for enumerations
			9.1.2 Creating an enumeration class
			9.1.3 Using enumerations
			9.1.4 Defining methods for the enumeration class
			9.1.5 Discussion
			9.1.6 Challenge
		9.2 How do I use data classes to eliminate boilerplate code?
			9.2.1 Creating a data class using the dataclass decorator
			9.2.2 Setting default values for the fields
			9.2.3 Making data classes immutable
			9.2.4 Creating a subclass of an existing data class
			9.2.5 Discussion
			9.2.6 Challenge
		9.3 How do I prepare and process JSON data?
			9.3.1 Understanding JSON’s data structure
			9.3.2 Mapping data types between JSON and Python
			9.3.3 Deserializing JSON strings
			9.3.4 Serializing Python data to JSON format
			9.3.5 Discussion
			9.3.6 Challenge
		9.4 How do I create lazy attributes to improve performance?
			9.4.1 Identifying the use scenario
			9.4.2 Overriding the __getattr_ special method to implement lazy attributes
			9.4.3 Implementing a property as a lazy attribute
			9.4.4 Discussion
			9.4.5 Challenge
		9.5 How do I define classes to have distinct concerns?
			9.5.1 Analyzing a class
			9.5.2 Creating additional classes to isolate the concerns
			9.5.3 Connecting related classes
			9.5.4 Discussion
			9.5.5 Challenge
Part 4: Manipulating objects and files
	Chapter 10: Fundamentals of objects
		10.1 How do I inspect an object’s type to improve code flexibility?
			10.1.1 Checking an object’s type using type
			10.1.2 Checking an object’s type using isinstance
			10.1.3 Checking an object’s type generically
			10.1.4 Discussion
			10.1.5 Challenge
		10.2 What’s the lifecycle of instance objects?
			10.2.1 Instantiating an object
			10.2.2 Being active in applicable namespaces
			10.2.3 Tracking reference counts
			10.2.4 Destructing the object
			10.2.5 Discussion
			10.2.6 Challenge
		10.3 How do I copy an object?
			10.3.1 Creating a (shallow) copy
			10.3.2 Noting the potential problem of a shallow copy
			10.3.3 Creating a deep copy
			10.3.4 Discussion
			10.3.5 Challenge
		10.4 How do I access and change a variable in a different scope?
			10.4.1 Accessing any variable: The LEGB rule for name lookup
			10.4.2 Changing a global variable in a local scope
			10.4.3 Changing an enclosing variable
			10.4.4 Discussion
			10.4.5 Challenge
		10.5 What’s callability, and what does it imply?
			10.5.1 Distinguishing classes from functions
			10.5.2 Revisiting the higher-order function map
			10.5.3 Using callable as the key argument
			10.5.4 Creating decorators as classes
			10.5.5 Discussion
			10.5.6 Challenge
	Chapter 11: Dealing with files
		11.1 How do I read and write files using context management?
			11.1.1 Opening and closing files: Context manager
			11.1.2 Reading data from a file in different ways
			11.1.3 Writing data to a file in different ways
			11.1.4 Discussion
			11.1.5 Challenge
		11.2 How do I deal with tabulated data files?
			11.2.1 Reading a CSV file using csv reader
			11.2.2 Reading a CSV file that has a header
			11.2.3 Writing data to a CSV file
			11.2.4 Discussion
			11.2.5 Challenge
		11.3 How do I preserve data as files using pickling?
			11.3.1 Pickling objects for data preservation
			11.3.2 Restoring data by unpickling
			11.3.3 Weighing the pros and cons of pickling
			11.3.4 Discussion
			11.3.5 Challenge
		11.4 How do I manage files on my computer?
			11.4.1 Creating a directory and files
			11.4.2 Retrieving the list of files of a specific kind
			11.4.3 Moving files to a different folder
			11.4.4 Copying files to a different folder
			11.4.5 Deleting a specific kind of files
			11.4.6 Discussion
			11.4.7 Challenge
		11.5 How do I retrieve file metadata?
			11.5.1 Retrieving the filename-related information
			11.5.2 Retrieving the file's size and time information
			11.5.3 Discussion
			11.5.4 Challenge
Part 5: Safeguarding the codebase
	Chapter 12: Logging and exception handling
		12.1 How do I monitor my program with logging?
			12.1.1 Creating the Logger object to log application events
			12.1.2 Using files to store application events
			12.1.3 Adding multiple handlers to the logger
			12.1.4 Discussion
			12.1.5 Challenge
		12.2 How do I save log records properly?
			12.2.1 Categorizing application events with levels
			12.2.2 Setting a handler’s level
			12.2.3 Setting formats to the handler
			12.2.4 Discussion
			12.2.5 Challenge
		12.3 How do I handle exceptions?
			12.3.1 Handling exceptions with try. . .except. . .
			12.3.2 Specifying the exception in the except clause
			12.3.3 Handling multiple exceptions
			12.3.4 Showing more information about an exception
			12.3.5 Discussion
			12.3.6 Challenge
		12.4 How do I use else and finally clauses in exception handling?
			12.4.1 Using else to continue the logic of the code in the try clause
			12.4.2 Cleaning up the exception handling with the finally clause
			12.4.3 Discussion
			12.4.4 Challenge
		12.5 How do I raise informative exceptions with custom exception classes?
			12.5.1 Raising exceptions with a custom message
			12.5.2 Preferring built-in exception classes
			12.5.3 Defining custom exception classes
			12.5.4 Discussion
			12.5.5 Challenge
	Chapter 13: Debugging and testing
		13.1 How do I spot problems with tracebacks?
			13.1.1 Understanding how a traceback is generated
			13.1.2 Analyzing a traceback when running code in a console
			13.1.3 Analyzing a traceback when running a script
			13.1.4 Focusing on the last call in a traceback
			13.1.5 Discussion
			13.1.6 Challenge
		13.2 How do I debug my program interactively?
			13.2.1 Activating the debugger with a breakpoint
			13.2.2 Running code line by line
			13.2.3 Stepping into another function
			13.2.4 Inspecting pertinent variables
			13.2.5 Discussion
			13.2.6 Challenge
		13.3 How do I test my functions automatically?
			13.3.1 Understanding the basis for testing functions
			13.3.2 Creating a TestCase subclass for testing functions
			13.3.3 Setting up the test
			13.3.4 Discussion
			13.3.5 Challenge
		13.4 How do I test a class automatically?
			13.4.1 Creating a TestCase subclass for testing a class
			13.4.2 Responding to test failures
			13.4.3 Discussion
			13.4.4 Challenge
Part 6: Building a web app
	Chapter 14: Completing a real project
		14.1 How do I use a virtual environment for my project?
			14.1.1 Understanding the rationale for virtual environments
			14.1.2 Creating a virtual environment for each project
			14.1.3 Installing packages in the virtual environment
			14.1.4 Using virtual environments in Visual Studio Code
			14.1.5 Discussion
			14.1.6 Challenge
		14.2 How do I build the data models for my project?
			14.2.1 Identifying the business needs
			14.2.2 Creating helper classes and functions
			14.2.3 Creating the Task class to address these needs
			14.2.4 Discussion
			14.2.5 Challenge
		14.3 How do I use SQLite as my application’s database?
			14.3.1 Creating the database
			14.3.2 Retrieving records from the database
			14.3.3 Saving records to the database
			14.3.4 Updating a record in a database
			14.3.5 Deleting a record from the database
			14.3.6 Discussion
			14.3.7 Challenge
		14.4 How do I build a web app as the frontend?
			14.4.1 Understanding the essential features of streamlit
			14.4.2 Understanding the app’s interface
			14.4.3 Tracking user activities using session state
			14.4.4 Setting up the sidebar
			14.4.5 Showing the tasks
			14.4.6 Showing a task’s details
			14.4.7 Creating a new task
			14.4.8 Organizing your project
			14.4.9 Running the app
			14.4.10 Discussion
			14.4.11 Challenge
solutions to the challenges
	Chapter 1
	Chapter 2
		Section 2.1
		Section 2.2
		Section 2.3
		Section 2.4
		Section 2.5
	Chapter 3
		Section 3.1
		Section 3.2
		Section 3.3
		Section 3.4
		Section 3.5
		Section 3.6
	Chapter 4
		Section 4.1
		Section 4.2
		Section 4.3
		Section 4.4
		Section 4.5
	Chapter 5
		Section 5.1
		Section 5.2
		Section 5.3
		Section 5.4
	Chapter 6
		Section 6.1
		Section 6.2
		Section 6.3
		Section 6.4
		Section 6.5
	Chapter 7
		Section 7.1
		Section 7.2
		Section 7.3
		Section 7.4
		Section 7.5
	Chapter 8
		Section 8.1
		Section 8.2
		Section 8.3
		Section 8.4
		Section 8.5
	Chapter 9
		Section 9.1
		Section 9.2
		Section 9.3
		Section 9.4
		Section 9.5
	Chapter 10
		Section 10.1
		Section 10.2
		Section 10.3
		Section 10.4
		Section 10.5
	Chapter 11
		Section 11.1
		Section 11.2
		Section 11.3
		Section 11.4
		Section 11.5
	Chapter 12
		Section 12.1
		Section 12.2
		Section 12.3
		Section 12.4
		Section 12.5
	Chapter 13
		Section 13.1
		Section 13.2
		Section 13.3
		Section 13.4
	Chapter 14
		Section 14.1
		Section 14.2
		Section 14.3
		Section 14.4
index
	Symbols
	A
	B
	C
	D
	E
	F
	G
	H
	I
	J
	K
	L
	M
	N
	O
	P
	Q
	R
	S
	T
	U
	V
	W
	Y
	Z