Improving Social Maturity of Cybersecurity Incident Response Teams

FrontCover
Acknowledgements
Project Team
Table of Contents
List of Figures
List of Tables
List of Acronyms
Executive Summary: Quick Reference Guide
Preface
Chapters
Chapter One
Chapter Two
Chapter Three
Chapter Four
Chapter Five
Chapter Six
Chapter Seven
Chapter Eight
Chapter Nine
Chapter Ten
Chapter Eleven
Appendices
Appendix A
Appendix  B
Appendix C
Appendix D
Appendix E
Appendix F
Appendix G
Appendix H
Appendix I
Figure 1.1 Projects Comprising the CSIRT Effectiveness Research Program
Figure 2.1 Elements of CSIRT Social Maturity
Figure 2.2 Example of a Cybersecurity Incident Response Multiteam System
Figure 2.3 Frequency of all 7 Typical Teams Across 28 CSIRTs
Figure 2.4 Challenges in CSIRT MTS Collaboration
Figure 4.1 Cybersecurity Incident Response Decision-Making Model
Figure 5.1 Communication as a Driver of CSIRT Effectiveness
Figure 5.2 Endorsement of Communication Themes by CSIRT Type.
Figure 6.1 Cybersecurity Incident Response Information Sharing Model
Figure 8.1 Focus Group Support for SKUE
Figure 8.2 Example of a Team Knowledge Map Depicting Members of a Team and Their Areas of Expertise (or Specialist Skills)
Figure 9.1 Facets of Trust (Sources:  Bromily & Harris, 2006; Cook & Wall, 1980; Mayer, Davis, & Schoorman, 1995; McAllister, 1995)
Figure 9.2 Focus Group Support for Trust by CSIRT Type.
Table 1.1 Collaboration Chillers in CSIRTs: Quotes from the Field
Table 1.2 Comparison of Top 20 KSAOs
Table 2.1 An Example of an Incident Response MTS Goal Hierarchy (adapted in part from Zaccaro, et al., 2016)
Table 2.2 Component Teams in a Typical Large MTS CSIRT
Table 3.1 Examples of Proactive and Reactive Performance (Quotes From Analyst Interviews)
Table 3.2 Examples of Tasks at Different Levels (Quotes From Analyst Interviews)
Table 3.3 Examples of Organizational Citizenship Behavior and Counterproductive Work Behavior That Managers Should Document
Table 3.4 Psychological Outcome Categories Assessed Using Self-Ratings
Table 4.1 Top Three Decision-Making Knowledge, Skills, Abilities, and Other attributes (KSAOs)
Table 5.1 Principles of Communication in Incident Response
Table 5.2 Incident Response Cycle Communication Examples
Table 5.3 Communication Principles and Strategies to Improve Them
Table 7.1 Collaborative Problem-Solving Process Model
Table 7.2 CSIRT Reactive Problem-Solving Behaviors from our Focus Groups and Interviews
Table 7.3 CSIRT Proactive Problem-Solving Steps and Focus Group Support
Table 7.4 CSIRT Staffing
Table 10.1 Sample Shift Schedule, Including Breaks, for an Analyst
FrontCover
FrontCover
FrontCover
FrontCover
1.0 Introduction to the Handbook
	1.0.1 Team Collaboration Failures
	1.0.2 The Importance of Social Processes to Cybersecurity Incident Response
1.1 Research Foundation for the Handbook
1.2 Major Research Themes and Findings
	1.2.1 Category 1: The Nature and Environment of CSIRT Work
		Theme 1: Cybersecurity incident responders perform individual and collective knowledge work.
		Theme 2: Cybersecurity incident responders often need to work within volatile, uncertain, complex, and ambiguous environments (i.e., “VUCA”; Stiehm, 2002; Scott, 2012).
		Theme 3: Maintaining vigilance (i.e., sustained attention and focus over time) is a substantial problem because of the length of shifts and the nature of CSIRT work.
		Theme 4: Cybersecurity incident response occurs at multiple levels, including individual, team, and multiteam systems.
		Theme 5: Incident response collaboration within and between incident responders and teams is typically discretionary.
		Theme 6: What constitutes good performance among cybersecurity incident responders is not well understood.  Performance should be evaluated directly using appropriate metrics—not indirectly (e.g., not only using existing maturity models).
	1.2.2 Category 2: Individual and Collective Drivers of CSIRT Effectiveness
		Theme 7: Four sets of knowledge, skills, abilities, and other attributes (KSAOs) are necessary for effective cybersecurity incident response work. These include: technical skills, cognitive abilities, social skills, and personal character.
		Theme 8: Team- and MTS-level states and protocols also contribute to CSIRT effectiveness.
	1.2.3 Category 3: Fostering Persistent CSIRT Excellence
		Theme 9: Adaptation and resilience across all levels are vital to effective cybersecurity incident response.
		Theme 10: Effective cybersecurity incident response work requires continuous learning across all levels.
1.3 Summary
FrontCover
2.0 Introduction
2.1 Why are Effective Social Dynamics Important in CSIRTs?
	2.1.1 The Elements of Social Maturity in CSIRTs
2.2 CSIRT Multiteam Systems
	2.2.1 What is a Multiteam System?
	2.2.2 MTS Goal Hierarchies
	2.2.3 Internal Versus External MTSs
	2.2.4 Forming CSIRT MTSs
2.3 Project Findings: Typical CSIRT MTSs
2.4 Challenges in Managing MTSs
2.5 Strategies for CSIRT MTS Managers
	2.6 Summary
FrontCover
3.0 Introduction
3.1 Assessing Performance Measurement
3.2 Background
	3.2.1 Why is a Comprehensive Approach to Performance Metrics Important for CSIRTs?
	3.2.2 Issues with Measurement of Performance
		Errors of Omission (Measurement Deficiency)
			Measuring quality versus quantity.
			Measuring maximum versus typical performance.
			Measuring proactive versus reactive performance.
			Measuring performance at different levels of analysis.
		Errors of Commission (Measurement Contamination)
	3.2.3 Performance-related Outcome Categories
		Performance Outcomes Assessed Using Conventional, Objectively-derived Metrics
		Performance Outcomes Assessed Using Subjectively-derived Ratings [1]
		Psychological Outcomes Assessed Using Subjectively-derived Ratings
3.3 Strategies for Designing a More Complete Performance Measurement Program
	3.3.1 Strategy 1: Balance Measuring Quantity and Quality
	3.3.2 Strategy 2: Measure Maximum Performance in Addition to Typical Performance
	3.3.3 Strategy 3: Measure Both Proactive and Reactive Performance
	3.3.4 Strategy 4: Determine the Appropriate Level of Measurement
	3.3.5 Strategy 5: Create a Balanced Scorecard of Performance Measurement
3.4 Chapter Summary
FrontCover
4.0 Introduction
4.1 Assessing Decision-Making Capacity
4.2 Background
	4.2.1 The Psychological Process of Incident Response Decision-Making
	4.2.2 How Decision-Making Can Go Awry
		Expert Versus Novice Decision-Making
		Decision-Making Problems Affecting Both Experts and Novices
		The Role of Incident Severity
4.3 Strategies for Improved Decision-Making
	4.3.1 Strategy 1: Selecting for Decision-Making Skills
		Problem Sensitivity
		Critical Thinking
		Information Ordering
	4.3.2 Strategy 2: Training Decision-making Skills
		Structured Troubleshooting Training
		Critical Thinking Training
		Expert Modeling
	4.3.3 Strategy 3: Cognitive Prompts to Reduce Overconfidence and Confirmation Bias
		Five-Why Analysis
		Premortem
	4.3.4 Strategy 4: Using Mnemonics to Capture Necessary Information
	4.3.5 Strategy 5: Using Adaptive Case Management
4.4 Chapter Summary
FrontCover
5.0 Introduction
5.1 Assessing Communication Skills
5.2 Background Information and Project Findings
	5.2.1	Principles of Effective Communication
		Communication Errors and Team Failure
		Three Common Challenges to Effective CSIRT Communication
			Time Urgency
			Team Dispersion
			The Impact of National Cultural Differences on Communication
	5.2.2 Project Findings
5.3 Developing Communication Skills in CSIRTs
	5.3.1 Strategy 1: Require teams or MTSs to complete a team charter to plan how, between whom, and when communication will happen
	5.3.2 Strategy 2: Implement checklists and handoff tools to prevent information loss during handoffs
	5.3.3 Strategy 3a: Use scenario-based training approaches to engage members in role-play
	5.3.4 Strategy 3b: Engage teams and MTSs in simulation-based training
	5.3.5 Strategy 4a: Design a virtual display that all team members can use to monitor information
	5.3.6 Strategy 4b: Apply best practices to make wikis more effective
	5.3.7 Strategy 5: Assign a team member to act as the point of contact for between-team communication in a CSIRT MTS
	5.3.8 Strategy 6: Design the work space to increase communication
	5.3.9 Strategy 7: Make team staffing decisions by using situational interviews to assess communication skills
5.4 Chapter Summary
FrontCover
6.0 Introduction
6.1 An Organizational Science Perspective on CSIRT Information Sharing
6.2 Elements of Information Sharing in CSIRTs
	6.2.1. Evidence on Information Sharing from our Study
	6.2.2 Degrees of Interaction between Information Sharing Partners
		Passive Information Sharing Between Partners
		Information Sharing During Handoffs and Escalation
		Active Interaction between Information Sharing Partners
	6.2.3 Recommendations for Effective Passive and Active Information Sharing
	6.2.4 Incident Response Process Requirements
		Mandatory Information Sharing
		Discretionary Information Sharing
	6.2.5 Recommendations for Effective Mandatory and Discretionary Information Sharing
	6.2.6 Information Sharing at Various Levels
		Dyadic Information Sharing
		Within-team Information Sharing
		Multiteam System Information Sharing
		Intra-Organizational Information Sharing
		Inter-Organizational Information Sharing
	6.2.7 Recommendations for Effective Information Sharing at Various Levels
6.3 Summary
FrontCover
7.0 Introduction
7.1 Assessing Collaborative Problem-Solving Capacity
7.2. Background
	7.2.1 Collaborative Problem-Solving Processes
		Shared Situational Awareness
		Collective Information Processing
		Collective Solution Forecasting
	7.2.2. Adaptive Problem-Solving in CSIRTs
7.3 Project Findings
7.4 Improving CSIRT Collaborative Problem-Solving
	7.4.1 Strategy 1: Engage in pre-mission planning (or “pre-briefing”)
	7.4.2 Strategy 2: Use a counterfactual thinking approach to get team members to share unique information
	7.4.3 Strategy 3: Provide team feedback during structured debriefing
	7.4.4 Strategy 4:  Develop adaptive thinking by providing exploratory or active learning experiences with wide problem variety
	7.4.5 Strategy 5: Train leaders to pre-plan strategies for how multiple teams will work together
	7.4.6 Strategy 6: Staff your CSIRT with team members who have a team orientation and teamwork skills
7.5 Chapter Summary
FrontCover
8.0 Introduction
8.1 Assessing Shared Knowledge of Unique Expertise
8.2 Background
8.3 Project Findings
8.4 Developing Shared Knowledge of Unique Expertise
	8.4.1 Strategy 1: Establish knowledge tools (e.g. information board, knowledge map) that display members\' expertise, knowledge, skills, and experiences
	8.4.2 Strategy 2: Train team members in areas other than their specialty
		Lecture/Presentation
		Job Shadowing
		Position Rotation
8.5 Summary
FrontCover
9.0 Introduction
9.1 Assessing Team Trust
9.2 Background
	9.2.1 Swift Trust
	9.2.2 Deep Trust
	9.2.3 Team Climate
		Conflict and Trust
	9.2.4 Trust between Teams, Organizations and External Parties
9.3 Project Findings
9.4 Developing Team Trust
	9.4.1. Strategy 1:  Provide structured opportunities for CSIRT members to learn about the expertise, experiences, and functional backgrounds of other members
	9.4.2. Strategy 2:  Establish clear individual and team goals, roles, and performance standards
	9.4.3 Strategy 3:  Establish norms for communication transparency in teams and MTSs
	9.4.4 Strategy 4: Utilize managerial actions that create a psychologically safe climate in the team and the MTS
	9.4.5 Strategy 5: Create opportunities for building strong social connections among CSIRT members to support conflict management
	9.4.6 Strategy 6: Increase external connections and social networking to facilitate inter-team and inter-organization trust
9.5 Chapter Summary
FrontCover
10.0 Introduction
10.1 Assessing CSIRT Capacity for Sustained Attention
10.2 Background
	10.2.1 The Importance of Sustained Attention During Incident Response
	10.2.2 Sustained Attention in Relevant Professions
10.3 Project Findings
	10.3.1 CSIRT Positions That Require Sustained Attention
	10.3.2 Knowledge, Skills, Abilities, and Other Attributes (KSAOs) Relevant to Sustained Attention
	10.3.3 Cognitive Abilities Relevant to Sustained Attention
10.4 Improving Sustained Attention and Focus over Time
	10.4.1 Strategy 1: Hire Job Applicants Who Display a Capacity For Sustained Attention
		Working Memory Task
		Brief  Vigilance (i.e., Sustained Attention) Tasks
	10.4.2 Strategy 2: Encourage Employees to Incorporate Rest Breaks Into Their Shifts
		Restorative Settings
		Socialization
	10.4.3 Strategy 3: Shift Design – Create a Shift Plan That Reduces Sleep Disturbances and Maximizes Attentiveness
		Work Shift Characteristics
			Shift Length (Eight-Hour Shifts Recommended)
			Shift Rotation Speed (Rapid Shift Rotations Preferred)
			Shift Rotation Direction (Forward Shift Rotation Preferred)
10.5 Chapter Summary
FrontCover
11.0 Introduction
11.1 Assessing Continuous Learning
11.2 Background
	11.2.1 Creativity and Curiosity
	11.2.2 Developmental Networks and Networking Behavior
	11.2.3 Team Learning
		Knowledge Sharing
		Knowledge Storage and Retrieval
11.3 Project Findings
	11.3.1 Findings Relevant to Creativity and Curiosity
	11.3.2 Findings Relevant to Networking
	11.3.3 Findings Relevant to Team Learning
11.4 Strategies and Recommendations
	11.4.1 Strategy 1: Selection of individuals who are creative and curious
	11.4.2 Strategy 2: Leader behaviors to encourage learning
		Recommendation 1. Engage employees’ creativity and curiosity
		Recommendation 2. Facilitate reflection in teams (team reflexivity, or team reflections, and adaptation)
		Recommendation 3. Provide feedback in debriefings (after-action reviews)
		Recommendation 4. Promote trust and respect among team members
	11.4.3 Strategy 3: Design work to enhance learning and development
		Recommendation 5: Improve work design to enhance learning
		Recommendation 6. Put in place mentoring programs
	11.4.4 Strategy 4: Training
		Recommendation 7. Train for networking skills
		Recommendation 8. Train CSIRT professionals on how to establish a professional developmental network
		Recommendation 9. Guided discovery learning
		Recommendation 10. Error management training
11.5 Chapter Summary
FrontCover
FrontCover
A.1 Introduction
A.2 Development of Taxonomic Dimensionality
A.3 Procedure for Taxon Specification
	A.3.1 Generation of Taxa
	A.3.2 Taxon Validation
A.4 How CSIRT Managers Can Use the Performance Taxonomy
	A.4.1 Preparing Position Descriptions
	A.4.2 Developing Performance Evaluation and Management Tools
A.5 Designing Training Programs
A.6 Informing CSIRT Process Models
	References
FrontCover
B.1 Assessment Exercises and Improvement Strategies by Topic Area
FrontCover
C.1 Introduction
C.2 Hiring Validation Considerations
	C.2.1 Test Reliability
	C.2.2 Reliability in CSIRT Applicant Testing
	C.2.3 Test Validity
	C.2.4 Further Reading
C.3 Training Validation Considerations
	C.3.1 Increasing the Performance Impact of Training
		1.  Conduct Training Needs Assessment
		2.  Ensure Employee Training Readiness
		3.  Create a Learning Environment
		4.  Garner Training Support
		5.  Develop an Evaluation Plan
		6.  Select Training Method
		7.  Monitor and Evaluate Training
	C.3.2 Further Reading
FrontCover
FrontCover
FrontCover
F.1 Executive Summary
F.2 Introduction
	F.2.1 Cybersecurity Incident Response Teams (CSIRTs)
F.3 Recommendations to Improve CSIRT Effectiveness
	F.3.1 Military Response Teams
	F.3.2 Emergency Medical Service Teams
	F.3.3 Nuclear Power Plant Operating Teams
F.4 General Discussion
	F.4.1 Top Recommendations Overall
	F.4.2 Top Recommendations for Particular Needs
	F.4.3 Change Management
F.5 Conclusion
FrontCover
G.1 Introduction
G.2 Communication
	G.2.1 Shared Knowledge of Unique Experience (SKUE)
	G.2.2 Trust
G.3 Identifying Important KSAOs for Cybersecurity Incident Response Team Professionals
	G.3.1 Initial Identification of KSAOs
	G.3.2 Reduction of KSAOs
	G.3.3 Expansion of Sources to Organizational Science Literature
	G.3.4 Summary of Identification of Relevant KSAOs
	G.3.5 KSAO Survey
	G.3.6 Comparison of Importance Ratings for the 20 Most Important KSAOs by Category
	G.3.7 Comparison of Importance Ratings for the 26 Least Important KSAOs by Category
G.4 Summary and Conclusions
FrontCover
H.1 Acknowledgements
H.2 Target Audience
H.3 Introduction
H.4 Networking Defined
H.5 Types of Networks
H.6 Development and Maintenance of Networks
	H.6.1 Network Mapping
	H.6.2 Networking in Work Teams
H.7 Conclusion
FrontCover
I.1 Purpose
I.2 Target Audience
I.3 Introduction
I.4 Why Resilience Matters in Cybersecurity
	I.4.1 THE IMPACT OF STRESS
		The Financial Impact of Stress due to Cyber Attacks
	I.4.2 SOCIAL RESILIENCE IN CYBERSECURITY
		INDIVIDUAL RESILIENCE
		PERSONALITY TRAITS
		SOCIAL SUPPORT
		COPING AND PROBLEM-SOLVING
	I.4.3 COLLECTIVE (TEAM) RESILIENCE
		TEAM PROCESSES
		CHARACTERISTICS OF RESILIENT TEAMS
	I.4.4 HOW TO ENHANCE AND DEVELOP SOCIAL RESILIENCE
		Battlemind Training and the Comprehensive Soldier Fitness Program
		Hardiness Training
		Stress Management and Resiliency Training (SMART)
		Additional Ways Leaders Can Enhance the Social Resilience of CSIRTs
I.5 Conclusion