Pro Python System Administration

Prelim......Page 1
Contents at a Glance......Page 7
Contents......Page 9
About the Author......Page 18
About the Technical Reviewer......Page 19
Acknowledgments......Page 20
Prerequisites for This Book......Page 21
Structure of This Book......Page 22
The Example Source Code......Page 24
Specifying the Requirements......Page 25
Introduction to SNMP......Page 26
The System SNMP Variables Node......Page 28
The Interfaces SNMP Variables Node......Page 29
Querying SNMP from the Command Line......Page 31
Configuring the Application......Page 35
Using the PySNMP Library......Page 37
The SNMP GET Command......Page 38
The SNMP SET Command......Page 39
The SNMP GETNEXT Command......Page 40
Introduction to RRDTool......Page 42
Creating a RoundRobin Database......Page 44
Writing and Reading Data from the RoundRobin Database......Page 46
Plotting Graphs with RRDTool......Page 49
Integrating RRDTool with the Monitoring Solution......Page 52
Creating Web Pages with the Jinja2 Templating System......Page 55
Loading Template Files with Jinja2......Page 56
Accessing Variables......Page 57
Flow Control Statements......Page 58
Generating Web Site Pages......Page 59
Summary......Page 63
What Is the SOAP API?......Page 65
Requesting Services with SOAP......Page 66
Finding Information about Available Services with WSDL......Page 68
SOAP Support in Python......Page 69
Converting WSDL Schema to Python Helper Module......Page 70
Defining Requirements for Our Load Balancer Tool......Page 71
Code Structure......Page 72
Configuration......Page 73
Fixing Issues with Citrix Netscaler WSDL......Page 74
Creating a Connection Object......Page 75
Logging In: Our First SOAP Call......Page 78
Finding What Is Being Returned in the Response from a Web Service......Page 80
How Is the Session Maintained After We Have Logged In?......Page 81
SOAP Methods for Reading Statistical Data and Their Return Values......Page 83
Reading System Health Data......Page 84
Reading Service Status Data......Page 86
Automating Administration Tasks......Page 89
Setting a Service State......Page 90
Logging Levels and Scope......Page 92
Configuring and Using the Logger......Page 94
Handling Exceptions......Page 96
Summary......Page 97
Setting Out the Requirements......Page 99
Making Design Decisions......Page 100
Defining the Database Schema......Page 101
The Search and Display Functions......Page 103
The Basic Concepts of the Django Framework......Page 104
The Model/View/Controller Pattern......Page 105
Installing the Django Framework......Page 106
The Structure of a Django Application......Page 107
Using Django with Apache Web Server......Page 109
Defining the Database Model......Page 112
URL Configuration......Page 115
Enabling the Management Interface......Page 117
Allowing the Administration Plug-in to Manage New Models......Page 119
Viewing Records......Page 120
Using Templates......Page 123
Deleting Records......Page 126
Adding New Records......Page 127
Modifying Existing Records......Page 129
Summary......Page 130
Extending the Design and Requirements......Page 131
Extending the Database Schema......Page 133
Defining Data Models......Page 134
The Add Function......Page 135
Extending DHCP Configuration with Address Pools......Page 137
Displaying DHCP Network Details......Page 138
Reworking the URL Structure......Page 140
Reverse Resolution of URLs......Page 141
Using URL References in the Templates......Page 142
Additions to the Data Model......Page 144
Using Template Inheritance......Page 145
Displaying a List of Objects......Page 146
A Detailed View of the Object......Page 147
Adding and Modifying New Objects......Page 149
Deleting Objects......Page 150
Generating the DHCP Configuration File......Page 151
Resolving IPs to Hostnames......Page 155
Checking Whether the Address Is In Use......Page 156
Summary......Page 159
Specifying the Design and Requirements for the Application......Page 161
Setting Up the Environment......Page 162
Creating a Django Project and Application......Page 163
Configuring the Application......Page 164
Defining the URL Structure......Page 165
The Data Model......Page 166
The Basic Model Structure......Page 167
Customizing the Class Names......Page 171
Adding New Fields to the Object List......Page 172
Reorganizing the Form Fields......Page 174
Adding Custom Object Actions......Page 176
Generating the Configuration File......Page 178
Summary......Page 181
Application Structure and Functionality......Page 183
Plug-in Framework Implementation in Python......Page 184
Interface Model......Page 185
Plug-in Registration and Discovery......Page 186
Discovery and Registration......Page 187
Defining the Plug-in Modules......Page 190
Format of Apache Log Files......Page 191
Log File Reader......Page 193
Calling the Plug-in Methods......Page 197
Plug-in Methods and the Call Mechanism......Page 198
Installing the Required Libraries......Page 201
Using the GeoIP Python Bindings......Page 202
Writing the Plug-in Code......Page 203
Summary......Page 204
Defining the Problem......Page 205
Why We Should Analyze Exceptions......Page 208
What Can We Find in a Typical Log File?......Page 210
The Structure of an Exception Stack Trace Log......Page 211
Handling Multiple Files......Page 213
What Are Generators, And How Do We Use Them?......Page 217
Detecting Potential Candidates......Page 219
Filtering Legitimate Exception Traces......Page 220
Storing Data in Data Structures......Page 221
Generating an Exception Fingerprint for Unknown Exceptions......Page 222
Detecting Known Exceptions......Page 225
The Configuration File......Page 226
Parsing XML Files with Python......Page 227
Storing and Applying Filters......Page 228
Producing Reports......Page 230
Grouping Exceptions......Page 231
Calculating Group Statistics......Page 232
Summary......Page 234
Requirements for the Check System......Page 235
Nagios Plug-In Architecture......Page 236
The Site Navigation Check......Page 237
Retrieving a Web Page......Page 238
Parsing the HTML Pages with Beautiful Soup......Page 240
Adding the New Check to the Nagios System......Page 246
Emulating the User Login Process......Page 248
Summary......Page 253
The Components......Page 255
The Monitoring Server......Page 256
The Data Objects......Page 257
The Data Structures......Page 258
Introduction to Data Normalization......Page 259
Configuration Data......Page 261
Performance Data......Page 264
Scheduling......Page 265
Representing the Information in an ER Diagram......Page 266
Communication Flows......Page 267
Structure......Page 268
Python Support......Page 269
CherryPy......Page 270
Initializing the Database File......Page 271
Accepting Sensor Readings......Page 276
Supplying a New Configuration......Page 277
Providing New Sensor Code......Page 278
Running Multiple Processes......Page 279
Multithreading, Multiprocessing, and GIL......Page 280
Basic Usage Patterns and Examples......Page 281
A Simple Clock Implementation......Page 284
A Cron-Like Scheduler......Page 287
Ticket Dispatcher......Page 288
Summary......Page 291
Actions......Page 293
Submitting Sensor Readings......Page 294
The ConfigParser Library......Page 295
Using the ConfigParser Class Methods......Page 296
The Configuration Class Wrapper......Page 301
The Sensor Design......Page 305
Using the subprocess Library......Page 306
Controlling the Running Processes......Page 310
Using File Objects......Page 313
Using the Pipe Objects......Page 314
Redirecting Standard Error......Page 315
Sending and Receiving Binary Data with XML-RPC......Page 316
Working with Files and Archives (TAR and BZip2)......Page 317
Summary......Page 320
Using the NumPy Library......Page 321
Working with Arrays......Page 322
Basic Mathematical and Statistical Operations......Page 326
Calculating the Mean and Standard Deviation......Page 327
Finding the Trend Line of a Dataset......Page 331
Installing matplotlib......Page 333
Understanding the Library Structure......Page 334
Plotting Graphs......Page 335
Changing the Appearance of the Plot Primitives......Page 336
Drawing Bars and Using Multiple Axes......Page 337
Working with Text Strings......Page 339
Saving Plots to a File......Page 340
Displaying Available Hosts......Page 341
Drawing Timescale Graphs......Page 342
The Index Page......Page 343
Host Details Page......Page 344
Graph Collection Pages......Page 347
Plotting Performance Graphs......Page 349
Summary......Page 352
Requirements Specification and Design......Page 353
System Design......Page 354
Changes to the Host Application......Page 356
Modifying the Plug-in Manager......Page 357
Accessing the MySQL Database from Python Applications......Page 358
Querying the Configuration Variables......Page 363
Querying the Server Status Variables......Page 365
Collecting the Host Configuration Data......Page 366
Writing the Consumer Plug-ins......Page 367
Checking the MySQL Version......Page 368
Checking the Key Buffer Size Setting......Page 370
Checking the Slow Queries Counter......Page 371
Summary......Page 372
The Problem......Page 373
Design Specifications......Page 374
Account Identifier......Page 375
The Simple Storage System Concepts......Page 376
Amazon Machine Images and Instances......Page 378
Elastic Block Store......Page 381
User Interfaces......Page 382
Reusing Existing Images......Page 383
Create and Setup an Elastic Block Store Volume......Page 384
Configure the MySQL Instance......Page 385
Bundling the New AMI......Page 386
Setting Up the Configuration Variables......Page 388
Initializing the EC2 Instance Programmatically......Page 389
Launching the EC2 Instance......Page 390
Mounting the EBS Device......Page 391
Starting the MySQL Instance......Page 392
Unmounting the File System......Page 393
Shutting Down the Instance......Page 394
The Control Sequence......Page 395
Summary......Page 396
¦ ¦ ¦ ¦A......Page 397
B......Page 398
¦D......Page 399
¦E......Page 401
F......Page 402
¦H......Page 403
¦ L......Page 404
¦ M......Page 405
¦O......Page 407
¦P......Page 408
¦R......Page 409
¦S......Page 410
¦U......Page 412
¦W ¦V......Page 413
¦ Z......Page 414