Performance Testing: An ISTQB Certified Tester Foundation Level Specialist Certification Review

Table of Contents
About the Author
About the Technical Reviewer
Acknowledgments
Foreword
Introduction
Chapter 1: The Basic Concepts of Performance Testing
	ISTQB Keywords
	Other Keywords
	1.1 Principles of Performance Testing
		Time Behavior
			Resource Utilization
				CPU Utilization
				Memory Utilization
				Disk Input/Output
				Bandwidth Consumption
				Queueing
			Capacity
	1.2 Types of Performance Testing
		Performance Testing
		Load Testing
			Who
			What
			Where
			When
			How
		Stress Testing
		Scalability Testing
		Spike Testing
		Endurance Testing
		Concurrency Testing
		Capacity Testing
	1.3 Testing Activities in Performance Testing
		Static Testing
			Reviews of Requirements with a Focus on Performance Aspects and Risks
			Capacity
			Time Behavior
			Resource Utilization
		Dynamic Testing
	1.4 The Concept of Load Generation
		Load Generation via the User Interface
		Load Generation Using Crowds
		Load Generation via the Application Programming Interface (API)
		Load Generation Using Captured Communication Protocols
	1.5 Common Performance Efficiency Failure Modes and Their Causes
		Slow Response Under Moderate-to-Heavy Load Levels
			Degraded Response over Time
		Inadequate or Graceless Error Handling Under Heavy or Overlimit Load
		Chapter 1 Questions
Chapter 2: Performance Measurement Fundamentals
	ISTQB Keywords
	Other Keywords
	2.1 Typical Measurements Collected in Performance Testing
		Why Performance Metrics Are Needed
		Collecting Performance Measurements and Metrics
			CPU Utilization
			Memory Utilization
			Disk Input/Output (Physical Disk)
			Bandwidth Consumption
			Queueing
			Technical Environment
			Business Environment
			Operational Environment
		Selecting Performance Metrics
	2.2 Aggregating Results from Performance Testing
	2.3 Key Sources of Performance Metrics
		Performance Test Tools
		Performance Monitoring Tools
		Log Analysis Tools
	2.4 Typical Results of a Performance Test
		Chapter 2 Questions
Chapter 3: Performance Testing in the Software Lifecycle
	ISTQB Keywords
	3.1 Principal Performance Testing Activities
		Test Planning
			Initial Workshop
			Business and Technical Overview
			Definition of Requirements/User Stories
			Volumetric Analysis
			Performance Test Environment Analysis
			Performance Test Tool Analysis/Proof of Concept
			Performance Project Planning
			Performance Test Plan
		Test Monitoring and Control
		Test Analysis
			Test Script Design
			Test Scenario Design
			Monitoring Design
			Performance Test Data Planning
			Scheduling
		Test Design
			Master Data
			User-Defined Data
			Transactional Data
		Test Implementation
			Initial Environment Acceptance Check
			Script Construction
			Scenario Construction
			Test Data Preparation
			Test Environment Preparation
			Test Tool Preparation
			Monitoring Preparation
			Results Capture and Analysis Preparation
			Final Environment Acceptance Test and Readiness Report
		Test Execution
			Initial Test Setup
			Test Execution
			Results Analysis
			Interim Test Report
			Remedial Action (If Needed)
			Test Cycle Report
			Results Review Meeting
			System and Process Improvement
		Test Completion
			Test Completion Report
			Presentation and Recommendations
			Performance Test Pack Creation
			Transition
	3.2 Categories of Performance Risks for Different Architectures
		Single Computer Systems
		Multi-tier Systems
		Distributed Systems
		Virtualized Systems
		Dynamic/Cloud-Based Systems
		Client-Server Systems
		Mobile Applications
		Embedded Real-Time Systems
		Mainframe Applications
			Memory Leaks
			Relational vs. Nonrelational Databases
			Recursive and Iterative Algorithms
	3.3 Performance Risks Across the Software Development Lifecycle
	3.4 Performance Testing Activities
		Sequential Development Models
		Iterative and Incremental Development Models
		Commercial Off-the-Shelf (COTS) and Other Supplier/Acquirer Models
		Chapter 3 Questions
Chapter 4: Performance Testing Tasks
	ISTQB Keywords
	4.1 Planning
		4.1.1 Deriving Performance Test Objectives
		4.1.2 The Performance Test Plan
			Objective
			Test Objectives
			System Overview
			Types of Performance
			Acceptance Criteria
			Test Data
			System Configuration
			Test Environment
			Test Tools
			Profiles
			Relevant Metrics
			Risks
		4.1.3 Communicating About Performance Testing
			Stakeholders with a Business Focus
			Stakeholders with a Technology Focus
	4.2 Analysis, Design, and Implementation
		4.2.1 Typical Communication Protocols
		4.2.2 Transactions
		4.2.3 Identifying Operational Profiles
			Identify Data
				Master Data
				User-Defined Data
				Transactional Data
				Different Types of User Personas and Their Roles
				Different Generic Tasks Performed by Those Users/Roles
				Estimated Numbers of Users for Each Role/Task
			Gather Data
			Construct Operational Profiles
			Off-line Batch Processing Systems
			Systems of Systems
		4.2.4 Creating Load Profiles
		4.2.5 Analyzing Throughput and Concurrency
			System Throughput
			Concurrency
		4.2.6 Basic Structure of a Performance Test Script
			Overall Structure
			Data Collection
			Result Verification and Error Handling
		4.2.7 Implementing Performance Test Scripts
		4.2.8 Preparing for Performance Test Execution
	4.3 Execution
	4.4 Analyzing Results and Reporting
		Analysis Techniques
		Reporting
		Chapter 4 Questions
Chapter 5: Tools
	ISTQB Keywords
	5.1 Tool Support
		Load Generators (2)
		Load Management Console (1)
		Monitoring Tool (5)
	5.2 Tool Suitability
		Compatibility
		Scalability
		Understandability
		Monitoring
		Chapter 5 Questions
Chapter 6: Final Thoughts
	Need an Exam?
Appendix A: References
	Standards
	ISTQB Documents
	Books
	Papers and Articles
Appendix B: Question Answers
	Chapter 1
	Chapter 2
	Chapter 3
	Chapter 4
	Chapter 5
Index