Peanuts: processing technology and product development

Cover......Page 1
Copyright......Page 4
Table of Contents......Page 5
 Why I Wrote a Book About Test-Driven Development......Page 15
 Outline......Page 17
 Conventions Used in This Book......Page 18
 Safari® Books Online......Page 19
 Contacting O’Reilly......Page 20
 Python 3 and Programming......Page 21
 Required Software Installations......Page 22
  Required Python Modules......Page 25
Acknowledgments......Page 27
Part I. The Basics of TDD and Django......Page 29
  Obey the Testing Goat! Do Nothing Until You Have a Test......Page 31
  Getting Django Up and Running......Page 34
  Starting a Git Repository......Page 36
  Using a Functional Test to Scope Out a Minimum Viable App......Page 41
  The Python Standard Library’s unittest Module......Page 44
  Commit......Page 46
 Chapter 3. Testing a Simple Home Page with Unit Tests......Page 49
  Unit Tests, and How They Differ from Functional Tests......Page 50
  Unit Testing in Django......Page 51
  Django’s MVC, URLs, and View Functions......Page 52
  At Last! We Actually Write Some Application Code!......Page 54
  urls.py......Page 55
  Unit Testing a View......Page 58
   The Unit-Test/Code Cycle......Page 59
 Chapter 4. What Are We Doing with All These Tests?......Page 63
  Programming Is like Pulling a Bucket of Water up from a Well......Page 64
  Using Selenium to Test User Interactions......Page 65
   Refactoring to Use a Template......Page 68
  On Refactoring......Page 72
  A Little More of Our Front Page......Page 73
  Recap: The TDD Process......Page 75
  Wiring Up Our Form to Send a POST Request......Page 79
  Processing a POST Request on the Server......Page 82
  Passing Python Variables to Be Rendered in the Template......Page 83
  Three Strikes and Refactor......Page 87
  The Django ORM and Our First Model......Page 88
   Our First Database Migration......Page 90
   The Test Gets Surprisingly Far......Page 91
   A New Field Means a New Migration......Page 92
  Saving the POST to the Database......Page 93
   Better Unit Testing Practice: Each Test Should Test One Thing......Page 96
  Rendering Items in the Template......Page 97
  Creating Our Production Database with migrate......Page 99
  Ensuring Test Isolation in Functional Tests......Page 105
   Running Just the Unit Tests......Page 108
  Small Design When Necessary......Page 109
   REST......Page 110
  Implementing the New Design Using TDD......Page 111
  Iterating Towards the New Design......Page 114
  Testing Views, Templates, and URLs Together with the Django Test Client......Page 115
   A New URL......Page 116
   A New View Function......Page 117
   A Separate Template for Viewing Lists......Page 118
  Another URL and View for Adding List Items......Page 120
   A Test Class for New List Creation......Page 121
   A URL and View for New List Creation......Page 122
   Removing Now-Redundant Code and Tests......Page 123
   Pointing Our Forms at the New URL......Page 124
  Adjusting Our Models......Page 125
   A Foreign Key Relationship......Page 127
   Adjusting the Rest of the World to Our New Models......Page 128
  Each List Should Have Its Own URL......Page 130
   Capturing Parameters from URLs......Page 131
   Adjusting new_list to the New World......Page 132
  One More View to Handle Adding Items to an Existing List......Page 133
   The Last New URL......Page 134
   The Last New View......Page 135
   But How to Use That URL in the Form?......Page 136
  A Final Refactor Using URL includes......Page 138
Part II. Web Development Sine Qua Nons......Page 141
  What to Functionally Test About Layout and Style......Page 143
  Prettification: Using a CSS Framework......Page 146
  Django Template Inheritance......Page 148
  Integrating Bootstrap......Page 149
  Static Files in Django......Page 150
   Switching to StaticLiveServerCase......Page 152
   Large Inputs......Page 153
  Using Our Own CSS......Page 154
  What We Glossed Over: collectstatic and Other Static Directories......Page 155
  A Few Things That Didn’t Make It......Page 158
 Chapter 8. Testing Deployment Using a Staging Site......Page 159
  TDD and the Danger Areas of Deployment......Page 160
  As Always, Start with a Test......Page 161
  Getting a Domain Name......Page 163
   Choosing Where to Host Our Site......Page 164
   User Accounts, SSH, and Privileges......Page 165
   Installing Nginx......Page 166
   Using the FT to Confirm the Domain Works and Nginx Is Running......Page 167
  Deploying Our Code Manually......Page 168
   Adjusting the Database Location......Page 169
   Creating a Virtualenv......Page 170
   Simple Nginx Configuration......Page 172
   Creating the Database with migrate......Page 175
   Switching to Gunicorn......Page 176
   Getting Nginx to Serve Static Files......Page 177
   Switching to Using Unix Sockets......Page 178
   Using Upstart to Make Sure Gunicorn Starts on Boot......Page 179
  Automating......Page 180
   “Saving Your Progress”......Page 184
 Chapter 9. Automating Deployment with Fabric......Page 185
  Breakdown of a Fabric Script for Our Deployment......Page 186
  Trying It Out......Page 190
   Deploying to Live......Page 191
   Nginx and Gunicorn Config Using sed......Page 193
  Further Reading......Page 194
  Validation FT: Preventing Blank Items......Page 197
   Skipping a Test......Page 198
   Splitting Functional Tests out into Many Files......Page 199
   Fleshing Out the FT......Page 202
   Refactoring Unit Tests into Several Files......Page 203
   Unit Testing Model Validation and the self.assertRaises Context Manager......Page 205
  Surfacing Model Validation Errors in the View......Page 206
   Checking Invalid Input Isn’t Saved to the Database......Page 209
  Django Pattern: Processing POST Requests in the Same View as Renders the Form......Page 211
   Refactor: Transferring the new_item Functionality into view_list......Page 212
   Enforcing Model Validation in view_list......Page 214
  Refactor: Removing Hardcoded URLs......Page 215
   Using get_absolute_url for Redirects......Page 216
  Moving Validation Logic into a Form......Page 221
   Exploring the Forms API with a Unit Test......Page 222
   Switching to a Django ModelForm......Page 223
   Testing and Customising Form Validation......Page 224
   Using the Form in a View with a GET Request......Page 226
   A Big Find and Replace......Page 229
   Adapting the Unit Tests for the new_list View......Page 231
   Using the Form in the View......Page 232
  Using the Form in the Other View......Page 233
   A Helper Method for Several Short Tests......Page 234
  Using the Form’s Own Save Method......Page 236
  Another FT for Duplicate Items......Page 239
   Preventing Duplicates at the Model Layer......Page 240
   A Little Digression on Queryset Ordering and String Representations......Page 242
   Rewriting the Old Model Test......Page 244
   Some Integrity Errors Do Show Up on Save......Page 245
  Experimenting with Duplicate Item Validation at the Views Layer......Page 246
  A More Complex Form to Handle Uniqueness Validation......Page 247
  Using the Existing List Item Form in the List View......Page 249
  Starting with an FT......Page 253
  Setting Up a Basic JavaScript Test Runner......Page 254
  Using jQuery and the Fixtures Div......Page 257
  Building a JavaScript Unit Test for Our Desired Functionality......Page 260
  Columbo Says: Onload Boilerplate and Namespacing......Page 262
  A Few Things That Didn’t Make It......Page 263
  Live Deploy......Page 265
  Wrap-Up: git tag the New Release......Page 266
Part III. More Advanced Topics......Page 267
 Chapter 15. User Authentication, Integrating Third-Party Plugins, and Mocking with JavaScript......Page 269
  Exploratory Coding, aka “Spiking”......Page 270
   Frontend and JavaScript Code......Page 271
   The Browser-ID Protocol......Page 272
   The Server Side: Custom Authentication......Page 273
  De-spiking......Page 279
   A Common Selenium Technique: Explicit Waits......Page 281
   Reverting Our Spiked Code......Page 283
   Housekeeping: A Site-Wide Static Files Folder......Page 284
   Mocking: Who, Why, What?......Page 285
   A Simple Mock to Unit Tests Our initialize Function......Page 286
   More Advanced Mocking......Page 292
   Checking Call Arguments......Page 295
   QUnit setup and teardown, Testing Ajax......Page 296
   More Nested Callbacks!  Testing Asynchronous Code......Page 300
  A Look at Our Spiked Login View......Page 305
   Testing Our View by Mocking Out authenticate......Page 306
   Checking the View Actually Logs the User In......Page 309
  De-spiking Our Custom Authentication Backend: Mocking Out an Internet Request......Page 313
   1 if = 1 More Test......Page 314
   Patching at the Class Level......Page 315
   Beware of Mocks in Boolean Comparisons......Page 318
   The get_user Method......Page 319
  A Minimal Custom User Model......Page 321
   A Slight Disappointment......Page 323
   Tests as Documentation......Page 324
   Users Are Authenticated......Page 325
  The Moment of Truth:  Will the FT Pass?......Page 326
  Finishing Off Our FT, Testing Logout......Page 327
  Skipping the Login Process by Pre-creating a Session......Page 331
   Checking It Works......Page 333
  The Proof Is in the Pudding: Using Staging to Catch Final Bugs......Page 334
   Setting Up Logging......Page 335
   Fixing the Persona Bug......Page 337
   A Django Management Command to Create Sessions......Page 339
   Getting the FT to Run the Management Command on the Server......Page 340
   An Additional Hop via subprocess......Page 342
  Baking In Our Logging Code......Page 345
   Using Hierarchical Logging Config......Page 346
  Wrap-Up......Page 348
  Why Prefer “Outside-In”?......Page 351
  The FT for “My Lists”......Page 352
  The Outside Layer: Presentation and Templates......Page 353
  Moving Down One Layer to View Functions (the Controller)......Page 354
   A Quick Restructure of the Template Inheritance Hierarchy......Page 355
   Designing Our API Using the Template......Page 356
   Moving Down to the Next Layer: What the View Passes to the Template......Page 357
  The Next “Requirement” from the Views Layer: New Lists Should Record Owner......Page 358
  Moving Down to the Model Layer......Page 359
   Final Step: Feeding Through the .name API from the Template......Page 361
  Revisiting Our Decision Point: The Views Layer Depends on Unwritten Models Code......Page 365
  A First Attempt at Using Mocks for Isolation......Page 366
   Using Mock side_effects to Check the Sequence of Events......Page 367
  Listen to Your Tests: Ugly Tests Signal a Need to Refactor......Page 369
   Keep the Old Integrated Test Suite Around as a Sanity Check......Page 370
   Thinking in Terms of Collaborators......Page 371
  Moving Down to the Forms Layer......Page 375
   Keep Listening to Your Tests: Removing ORM Code from Our Application......Page 376
  Finally, Moving Down to the Models Layer......Page 379
   Back to Views......Page 381
  The Moment of Truth (and the Risks of Mocking)......Page 382
  Thinking of Interactions Between Layers as “Contracts”......Page 383
   Identifying Implicit Contracts......Page 384
   Fixing the Oversight......Page 385
  One More Test......Page 386
   Removing Redundant Code at the Forms Layer......Page 387
   Removing the Old Implementation of the View......Page 388
   Removing Redundant Code at the Forms Layer......Page 389
  Conclusions: When to Write Isolated Versus Integrated Tests......Page 390
   Onwards!......Page 391
  Installing Jenkins......Page 393
   Configuring Jenkins Security......Page 395
   Adding Required Plugins......Page 396
  Setting Up Our Project......Page 397
  First Build!......Page 399
  Setting Up a Virtual Display so the FTs Can Run Headless......Page 400
  Taking Screenshots......Page 402
  A Common Selenium Problem: Race Conditions......Page 406
  Running Our QUnit JavaScript Tests in Jenkins with PhantomJS......Page 409
   Installing node......Page 410
   Adding the Build Steps to Jenkins......Page 411
  More Things to Do with a CI Server......Page 412
  An FT with Multiple Users, and addCleanup......Page 415
  Implementing the Selenium Interact/Wait Pattern......Page 417
  The Page Pattern......Page 418
  Extend the FT to a Second User, and the “My Lists” Page......Page 421
  An Exercise for the Reader......Page 423
 Chapter 22. Fast Tests, Slow Tests, and Hot Lava......Page 425
   Faster Tests Mean Faster Development......Page 426
   Don’t Take It from Me......Page 427
   Mocky Tests Can Become Closely Tied to Implementation......Page 428
   Clean, Maintainable Code......Page 429
  Architectural Solutions......Page 430
   Functional Core, Imperative Shell......Page 431
  Conclusion......Page 432
Obey the Testing Goat!......Page 435
  Running Firefox Selenium Sessions with Xvfb......Page 437
  Setting Up Django as a PythonAnywhere Web App......Page 438
  The Deployment Chapter......Page 439
  Class-Based Generic Views......Page 441
  The Home Page as a FormView......Page 442
  Using form_valid to Customise a CreateView......Page 443
   The Tests Guide Us, for a While......Page 445
   Until We’re Left with Trial and Error......Page 446
   Back on Track......Page 447
  Best Practices for Unit Testing CBGVs?......Page 448
   Take-Home: Having Multiple, Isolated View Tests with Single Assertions Helps......Page 449
  Installing System Packages and Nginx......Page 451
  Configuring Gunicorn, and Using Handlers to Restart Services......Page 453
   Use Vagrant to Spin Up a Local VM......Page 454
  An Attempted Deploy to Staging......Page 455
   Copying Test Data from the Live Site......Page 456
  Inserting a Data Migration......Page 457
  Testing the New Migrations Together......Page 458
  Conclusions......Page 459
  Switch to Postgres......Page 461
  Investigate a BDD Tool......Page 462
  Async and Websockets......Page 463
  Your Suggestion Here......Page 464
  The Basic TDD Workflow......Page 465
  Moving Beyond dev-only Testing......Page 466
  Selenium/Functional Testing Best Practices......Page 467
  Outside-In, Test Isolation Versus Integrated Tests, and Mocking......Page 468
 Appendix G. Bibliography......Page 469
Index......Page 471
About the Author......Page 478