Mastering Python

Cover
Copyright
Credits
About the Author
About the Reviewers
www.PacktPub.com
Table of Contents
Preface
Chapter 1: Getting Started – One Environment per Project
	Creating a virtual Python environment using venv
		Creating your first venv
		venv arguments
		Differences between virtualenv and venv
	Bootstrapping pip using ensurepip
		ensurepip usage
		Manual pip install
	Installing C/C++ packages
		Debian and Ubuntu
		Red Hat, CentOS, and Fedora
		OS X
		Windows
	Summary
Chapter 2: Pythonic Syntax, Common Pitfalls, and Style Guide
	Code style – or what is Pythonic code?
		Formatting strings – printf-style or str.format?
		PEP20, the Zen of Python
			Beautiful is better than ugly
			Explicit is better than implicit
			Simple is better than complex
			Flat is better than nested
			Sparse is better than dense
			Readability counts
			Practicality beats purity
			Errors should never pass silently
			In the face of ambiguity, refuse the temptation to guess
			One obvious way to do it
			Now is better than never
			Hard to explain, easy to explain
			Namespaces are one honking great idea
			Conclusion
		Explaining PEP8
			Duck typing
			Differences between value and identity comparisons
			Loops
			Maximum line length
		Verifying code quality, pep8, pyflakes, and more
			flake8
			Pylint
	Common pitfalls
		Scope matters!
			Function arguments
			Class properties
			Modifying variables in the global scope
		Overwriting and/or creating extra built-ins
		Modifying while iterating
		Catching exceptions – differences between Python 2 and 3
		Late binding – be careful with closures
		Circular imports
		Import collisions
	Summary
Chapter 3: Containers and Collections – Storing Data the Right Way
	Time complexity – the big O notation
	Core collections
		list – a mutable list of items
		dict – unsorted but a fast map of items
		set – like a dict without values
		tuple – the immutable list
	Advanced collections
		ChainMap – the list of dictionaries
		counter – keeping track of the most occurring elements
		deque – the double ended queue
		defaultdict – dictionary with a default value
		namedtuple – tuples with field names
		enum – a group of constants
		OrderedDict – a dictionary where the insertion order matters
		heapq – the ordered list
		bisect – the sorted list
	Summary
Chapter 4: Functional Programming – Readability versus Brevity
	Functional programming
	list comprehensions
	dict comprehensions
	set comprehensions
	lambda functions
		The Y combinator
	functools
		partial – no need to repeat all arguments every time
		reduce – combining pairs into a single result
			Implementing a factorial function
			Processing trees
	itertools
		accumulate – reduce with intermediate results
		chain – combining multiple results
		combinations – combinatorics in Python
		permutations – combinations where the order matters
		compress – selecting items using a list of Booleans
		dropwhile/takewhile – selecting items using a function
		count – infinite range with decimal steps
		groupby – grouping your sorted iterable
		islice – slicing any iterable
	Summary
Chapter 5: Decorators – Enabling Code Reuse by Decorating
	Decorating functions
		Why functools.wraps is important
		How are decorators useful?
		Memoization using decorators
		Decorators with (optional) arguments
		Creating decorators using classes
	Decorating class functions
		Skipping the instance – classmethod and staticmethod
		Properties – smart descriptor usage
	Decorating classes
		Singletons – classes with a single instance
		Total ordering – sortable classes the easy way
	Useful decorators
		Single dispatch – polymorphism in Python
		Contextmanager, with statements made easy
		Validation, type checks, and conversions
		Useless warnings – how to ignore them
	Summary
Chapter 6: Generators and Coroutines – Infinity, One Step at a Time
	What are generators?
		Advantages and disadvantages of generators
		Pipelines – an effective use of generators
		tee – using an output multiple times
		Generating from generators
		Context managers
	Coroutines
		A basic example
		Priming
		Closing and throwing exceptions
		Bidirectional pipelines
		Using the state
	Summary
Chapter 7: Async IO – Multithreading without Threads
	Introducing the asyncio library
		The async and await statements
			Python 3.4
			Python 3.5
			Choosing between the 3.4 and 3.5 syntax
		A simple example of single-threaded parallel processing
		Concepts of asyncio
			Futures and tasks
			Event loops
			Processes
		Asynchronous servers and clients
			Basic echo server
	Summary
Chapter 8: Metaclasses – Making Classes (Not Instances) Smarter
	Dynamically creating classes
		A basic metaclass
		Arguments to metaclasses
		Accessing metaclass attributes through classes
	Abstract classes using collections.abc
		Internal workings of the abstract classes
		Custom type checks
		Using abc.ABC before Python 3.4
	Automatically registering a plugin system
		Importing plugins on-demand
		Importing plugins through configuration
		Importing plugins through the file system
	Order of operations when instantiating classes
		Finding the metaclass
		Preparing the namespace
		Executing the class body
		Creating the class object (not instance)
		Executing the class decorators
		Creating the class instance
		Example
	Storing class attributes in definition order
		The classic solution without metaclasses
		Using metaclasses to get a sorted namespace
	Summary
Chapter 9: Documentation – How to Use Sphinx and reStructuredText
	The reStructuredText syntax
		Getting started with reStructuredText
		Inline markup
		Headers
		Lists
			Enumerated list
			Bulleted list
			Option list
			Definition list
			Nested lists
		Links, references, and labels
		Images
		Substitutions
		Blocks, code, math, comments, and quotes
		Conclusion
	The Sphinx documentation generator
		Getting started with Sphinx
			Using sphinx-quickstart
			Using sphinx-apidoc
		Sphinx directives
			The table of contents tree directive (toctree)
			Autodoc, documenting Python modules, classes, and functions
		Sphinx roles
	Documenting code
		Documenting a class with the Sphinx style
		Documenting a class with the Google style
		Documenting a class with the NumPy style
		Which style to choose
	Summary
Chapter 10: Testing and Logging – Preparing for Bugs
	Using examples as tests with doctest
		A simple doctest example
		Writing doctests
		Testing with pure documentation
		The doctest flags
			True and False versus 1 and 0
			Normalizing whitespace
			Ellipsis
		Doctest quirks
			Testing dictionaries
			Testing floating-point numbers
			Times and durations
	Testing with py.test
		The difference between the unittest and py.test output
		The difference between unittest and py.test tests
			Simplifying assertions
			Parameterizing tests
			Automatic arguments using fixtures
			Print statements and logging
			Plugins
	Mock objects
		Using unittest.mock
		Using py.test monkeypatch
	Logging
		Configuration
			Basic logging configuration
			Dictionary configuration
			JSON configuration
			Ini file configuration
			The network configuration
		Logger
			Usage
	Summary
Chapter 11: Debugging – Solving the Bugs
	Non-interactive debugging
		Inspecting your script using trace
		Debugging using logging
		Showing call stack without exceptions
		Debugging asyncio
		Handling crashes using faulthandler
	Interactive debugging
		Console on demand
		Debugging using pdb
			Breakpoints
			Catching exceptions
			Commands
		Debugging using ipdb
		Other debuggers
		Debugging services
	Summary
Chapter 12: Performance – Tracking and Reducing your Memory and CPU Usage
	What is performance?
	Timeit – comparing code snippet performance
	cProfile – finding the slowest components
		First profiling run
		Calibrating your profiler
		Selective profiling using decorators
		Using profile statistics
	Line profiler
	Improving performance
		Using the right algorithm
		Global interpreter lock
		Try versus if
		Lists versus generators
		String concatenation
		Addition versus generators
		Map versus generators and list comprehensions
		Caching
		Lazy imports
		Using optimized libraries
		Just-in-time compiling
		Converting parts of your code to C
	Memory usage
		Tracemalloc
		Memory profiler
		Memory leaks
		Reducing memory usage
			Generators versus lists
			Recreating collections versus removing items
			Using slots
	Performance monitoring
	Summary
Chapter 13: Multiprocessing – When a Single CPU Core Is not Enough
	Multithreading versus multiprocessing
	Hyper-threading versus physical CPU cores
	Creating a pool of workers
	Sharing data between processes
	Remote processes
		Distributed processing using multiprocessing
		Distributed processing using IPyparallel
			ipython_config.py
			ipython_kernel_config.py
			ipcontroller_config.py
			ipengine_config.py
			ipcluster_config.py
	Summary
Chapter 14: Extensions in C/C++, System Calls, and C/C++ Libraries
	Introduction
		Do you need C/C++ modules?
		Windows
		OS X
		Linux/Unix
	Calling C/C++ with ctypes
		Platform-specific libraries
			Windows
			Linux/Unix
			OS X
			Making it easy
		Calling functions and native types
		Complex data structures
		Arrays
		Gotchas with memory management
	CFFI
		Complex data structures
		Arrays
		ABI or API?
		CFFI or ctypes?
	Native C/C++ extensions
		A basic example
		C is not Python – size matters
		The example explained
			static
			PyObject*
			Parsing arguments
		C is not Python – errors are silent or lethal
		Calling Python from C – handling complex types
	Summary
Chapter 15: Packaging – Creating Your Own Libraries or Applications
	Installing packages
	Setup parameters
	Packages
	Entry points
		Creating global commands
		Custom setup.py commands
	Package data
	Testing packages
		Unittest
		py.test
		Nosetests
	C/C++ extensions
		Regular extensions
		Cython extensions
	Wheels – the new eggs
		Distributing to the Python Package Index
	Summary
Index