Software Architecture with Python

Cover......Page 1
Copyright......Page 3
Credits......Page 5
About the Author......Page 6
About the Reviewer......Page 7
www.PacktPub.com......Page 8
Customer Feedback......Page 9
Table of Contents......Page 11
Preface......Page 19
Chapter 1: Principles of Software Architecture......Page 25
 Defining Software Architecture......Page 26
  Software Architecture versus design......Page 27
  An architecture defines a structure......Page 28
  An architecture captures early design decisions......Page 30
  An architecture manages stakeholder requirements......Page 31
  An architecture influences the organizational structure......Page 32
  An architecture is influenced by its environment......Page 33
  An architecture often conforms to a pattern......Page 34
 Importance of Software Architecture......Page 35
 System versus enterprise architecture......Page 37
 Architectural quality attributes......Page 41
  Modifiability......Page 42
  Testability......Page 45
  Scalability......Page 47
  Performance......Page 49
  Availability......Page 50
  Security......Page 51
  Deployability......Page 53
 Summary......Page 54
 What is modifiability?......Page 57
 Understanding readability......Page 58
  Python and readability......Page 59
  Readability - antipatterns......Page 61
  Document your code......Page 63
  Follow coding and style guidelines......Page 71
  Review and refactor code......Page 72
  Commenting the code......Page 73
 Fundamentals of Modifiability – Cohesion & Coupling......Page 74
  Measuring cohesion and coupling......Page 75
  Measuring cohesion and coupling – string and text processing......Page 78
  Providing explicit interfaces......Page 79
  Reducing two-way dependencies......Page 80
  Abstract common services......Page 81
  Using inheritance techniques......Page 82
  Using late binding techniques......Page 86
 Metrics – tools for static analysis......Page 87
  What are code smells?......Page 88
  Cyclomatic complexity – the McCabe metric......Page 89
   Testing for metrics......Page 90
  Running Static Checkers......Page 93
 Refactoring Code......Page 101
  Refactoring code – fixing complexity......Page 102
  Refactoring code - fixing code smells......Page 104
  Refactoring code - fixing styling and coding issues......Page 106
 Summary......Page 107
 Understanding testability......Page 109
  Testability – architectural aspects......Page 110
   Reduce system complexity......Page 113
   Improving predictability......Page 114
   Control and isolate external dependencies......Page 115
 White-box testing principles......Page 119
  Unit testing......Page 120
  Unit testing in action......Page 121
   Extending our unit test case......Page 123
  Nosing around with nose2......Page 125
  Testing with py.test......Page 126
   Measuring coverage using coverage.py......Page 129
   Measuring coverage using nose2......Page 130
   Measuring coverage using py.test......Page 131
  Mocking things up......Page 132
  Tests inline in documentation – doctests......Page 137
  Integration tests......Page 141
   Test automation using Selenium Web Driver......Page 144
 Test-Driven Development......Page 146
 TDD with palindromes......Page 147
 Summary......Page 153
Chapter 4: Good Performance is Rewarding!......Page 155
 Software performance engineering......Page 157
 Performance testing and measurement tools......Page 159
 Performance complexity......Page 160
 Measuring performance......Page 162
  Measuring time using a context manager......Page 163
  Timing code using the timeit module......Page 166
   Measuring the performance of our code using timeit......Page 167
  Finding out time complexity – graphs......Page 169
  Measuring CPU time with timeit......Page 175
  Profiling with cProfile and profile......Page 176
   Prime number iterator class – performance tweaks......Page 180
  Profiling – collecting and reporting statistics......Page 182
   Line profiler......Page 183
   Memory profiler......Page 185
   Substring (subsequence) problem......Page 187
  Objgraph......Page 192
  Pympler......Page 194
 Programming for performance – data structures......Page 196
   Lists......Page 197
   Sets......Page 198
  High performance containers – the collections module......Page 199
   defaultdict......Page 200
   OrderedDict......Page 202
   Counter......Page 204
   ChainMap......Page 205
   namedtuple......Page 206
  Probabilistic data structures – bloom filters......Page 208
 Summary......Page 212
Chapter 5: Writing Applications That Scale......Page 215
 Scalability and performance......Page 217
 Concurrency......Page 220
  Concurrency versus parallelism......Page 221
  Concurrency in Python – multithreading......Page 222
 Thumbnail generator......Page 223
  Thumbnail generator – producer/consumer architecture......Page 225
  Thumbnail generator – resource constraint using locks......Page 230
  Thumbnail generator – resource constraint using semaphores......Page 235
  Resource constraint – semaphore versus lock......Page 238
  Thumbnail generator – URL rate controller using conditions......Page 239
 Multithreading – Python and GIL......Page 247
  A primality checker......Page 248
  Sorting disk files......Page 252
   Sorting disk files – using a counter......Page 253
   Sorting disk files – using multiprocessing......Page 256
 Multithreading versus multiprocessing......Page 259
 Pre-emptive versus cooperative multitasking......Page 260
 The asyncio module in Python......Page 265
 Waiting for a future – async and await......Page 268
 Concurrent futures – high-level concurrent processing......Page 271
  Disk thumbnail generator......Page 273
  Concurrency options – how to choose?......Page 276
  Joblib......Page 277
  PyMP......Page 278
  Fractals – the Mandelbrot set......Page 279
   Fractals – Scaling the Mandelbrot set implementation......Page 283
 Scaling workflows – message queues and task queues......Page 286
  The Mandelbrot set using Celery......Page 288
  Serving with Python on the Web—WSGI......Page 293
  uWSGI – WSGI middleware on steroids......Page 296
  Gunicorn versus uWSGI......Page 298
   Horizontal scalability architectures......Page 299
 Summary......Page 303
Chapter 6: Security – Writing Secure Code......Page 305
 Information Security architecture......Page 306
 Secure coding......Page 308
 Common security vulnerabilities......Page 309
 Is Python secure?......Page 314
  Reading input......Page 315
  Evaluating arbitrary input......Page 318
  Overflow errors......Page 322
  Serializing objects......Page 324
 Security issues with web applications......Page 328
  Server Side Template Injection......Page 329
  Server-Side Template Injection – mitigation......Page 332
  Denial of Service......Page 334
  Mitigation – DoS and XSS......Page 338
 Strategies for security – Python......Page 341
 Secure coding strategies......Page 349
 Summary......Page 350
Chapter 7: Design Patterns in Python......Page 351
 Design patterns - Elements......Page 352
 Categories of design patterns......Page 353
  Pluggable hashing algorithms......Page 355
  Summing up pluggable hashing algorithm......Page 358
  The Singleton pattern......Page 359
   The Singleton – do we need a Singleton?......Page 362
  State sharing – Borg versus Singleton......Page 364
  The Factory pattern......Page 366
  The Prototype pattern......Page 369
   Prototype – deep versus shallow copy......Page 370
   Prototype using metaclasses......Page 371
   Combining patterns using metaclasses......Page 373
   The Prototype factory......Page 374
  The Builder pattern......Page 377
  The Adapter pattern......Page 384
  The Facade pattern......Page 394
   Facades in Python......Page 395
  The Proxy pattern......Page 401
   An instance-counting proxy......Page 402
 Patterns in Python – Behavioral......Page 405
  The Iterator pattern......Page 406
  The Observer pattern......Page 409
  The State pattern......Page 417
 Summary......Page 424
Chapter 8: Python – Architectural Patterns......Page 427
 Introducing MVC......Page 428
  Model Template View (MTV) – Django......Page 430
  Django admin – automated model-centric views......Page 431
  Flexible Microframework – Flask......Page 433
  Chat server and client using I/O multiplexing with the select module......Page 435
  Event-driven programming versus Concurrent programming......Page 442
  Twisted......Page 444
   Twisted – a simple web client......Page 445
   Chat Server using Twisted......Page 446
  Eventlet......Page 452
  Greenlets and Gevent......Page 454
 Microservice architecture......Page 456
  Microservice frameworks in Python......Page 457
  Microservices example – restaurant reservation......Page 459
 Pipe and Filter architectures......Page 462
  Pipe and Filter in Python......Page 463
 Summary......Page 469
Chapter 9: Deploying Python Applications......Page 471
 Deployability......Page 472
  Factors affecting Deployability......Page 473
 Tiers of software deployment architecture......Page 475
  Packaging Python code......Page 476
  Pip......Page 477
  Virtualenv......Page 479
  Virtualenv and pip......Page 481
  Relocatable virtual environments......Page 483
  PyPI......Page 484
  Packaging and submission of an application......Page 486
   The setup.py file......Page 487
   Installing the package......Page 488
   Submitting the package to PyPI......Page 489
  Remote deployments using Fabric......Page 492
  Remote deployments using Ansible......Page 494
  Managing remote daemons using Supervisor......Page 495
 Deployment – patterns and best practices......Page 496
 Summary......Page 498
Chapter 10: Techniques for Debugging......Page 501
 Maximum subarray problem......Page 502
  The power of \"print\"......Page 503
  Analysis and rewrite......Page 505
  Timing and optimizing the code......Page 508
 Simple debugging tricks and techniques......Page 510
  Word searcher program......Page 511
  Word searcher program—debugging step 1......Page 512
  Word searcher program—debugging step 2......Page 513
  Word searcher program—final code......Page 514
  Stopping execution......Page 515
  External dependencies—using wrappers......Page 516
  Replacing functions with their return value/data (Mocking)......Page 518
   Saving to / loading data from files as cache......Page 520
   Saving to / loading data from memory as cache......Page 522
   Returning random/mock data......Page 524
  Simple application logging......Page 529
  Advanced logging—logger objects......Page 531
   Advanced logging—custom formatting and loggers......Page 532
   Advanced logging—writing to syslog......Page 536
 Debugging tools—using debuggers......Page 537
  A debugging session with pdb......Page 538
   iPdb......Page 540
 Advanced debugging—tracing......Page 542
  The trace module......Page 543
  The lptrace program......Page 544
  System call tracing using strace......Page 545
 Summary......Page 546
Index......Page 549