Thermal Guidelines For Data Processing Environments

Table of Contents
Preface
Acknowledgments
Chapter 1 - Introduction
	1.1 Book Flow
	1.2 Primary Users of This Book
	1.3 Adoption
	1.4 Definitions
Chapter 2 - Environmental Guidelines for 
Air-Cooled Equipment
	2.1 Background
		Figure 2.1 Server metrics for determining data center operating environment envelope.
	2.2 New Air-Cooled Equipment Environmental Specifications
		Figure 2.2 2021 recommended and allowable envelopes for Classes A1, A2, A3 and A4. The recommended envelope is for low levels of pollutants verified by coupon measurements as indicated in note 3 of Section 2.2.
		Figure 2.3 2021 recommended and allowable envelopes for Classes A1, A2, A3 and A4. The recommended envelope is for high levels of pollutants verified by coupon measurements as indicated in note 3 of Section 2.2.
		Table 2.1 2021 Thermal Guidelines for Air Cooling— SI Version (I-P Version in Appendix B)
		2.2.1 Environmental Class Definitions for Air-Cooled Equipment
			Figure 2.4 Climatogram of Class A3 illustrating how dew-point limits modify relative humidity specification limits.
			Figure 2.5 World population distribution by altitude. (Courtesy Bill Rankin, www.radicalcartography.net/howhigh.html)
			Figure 2.6 2021 recommended and allowable envelopes for ASHRAE Class H1. The recommended envelope is for low levels of pollutants verified by coupon measurements as indicated in note 3 of Section 2.2.
			Figure 2.7 2021 recommended and allowable envelopes for ASHRAE Class H1. The recommended envelope is for high levels of pollutants verified by coupon measurements as indicated in note 3 of Section 2.2.
		2.2.2 Environmental Class Definition for High-Density Air-Cooled Equipment
			Table 2.2 2021 Thermal Guidelines for High-Density Servers— SI Version (I-P Version in Appendix B)
			Table 2.3 ETSI Class 3.1 and 3.1e Environmental Requirements (ETSI 2014)
		2.2.3 ETSI Environmental Specifications
	2.3 Guide for the Use and Application of the ASHRAE Data Center Classes
		Figure 2.8 Climatogram of the ETSI Class 3.1 and 3.1e environmental conditions (ETSI 2014).
		Table 2.4 Ranges of Options to Consider for Optimizing Energy Savings
	2.4 Server Metrics to CONSIDER IN USING Guidelines
		2.4.1 Server Power Trend versus Ambient Temperature
			Figure 2.9 Server power increase (Class A3 is an estimate) versus ambient temperature for Classes A2 and A3.
			Figure 2.10 Server flow rate increase versus ambient temperature increase.
		2.4.2 Acoustical Noise Levels versus Ambient Temperature
			Table 2.5 Expected Increase in A-Weighted Sound Power Level
		2.4.3 Server Reliability Trend versus Ambient Temperature
			Table 2.6 Relative ITE Failure Rate x-Factor as Function of Constant ITE Air Inlet Temperature
			Figure 2.11 Time-weighted x-factor estimates for air-side economizer use for selected U.S. cities.
		2.4.4 Server Reliability versus Moisture, Contamination, and Other Temperature Effects
		2.4.5 Server Performance Trend versus Ambient Temperature
		2.4.6 Server Cost Trend versus Ambient Temperature
		2.4.7 Summary of Air-Cooled Equipment Environmental Specifications
Chapter 3 - Environmental Guidelines for
Liquid-Cooled Equipment
	3.1 ITE Liquid Cooling
		3.1.1 New Construction
		3.1.2 Expansions
		3.1.3 High-Performance Computing and Other High-Density Workloads
		3.1.4 ITE and Facilities Interface
			Figure 3.1 Liquid-cooled rack or cabinet with external CDU.
			Figure 3.2 Combination air- and liquid-cooled rack or cabinet with internal CDU.
			Figure 3.3 Liquid-cooling systems/loops for a data center.
	3.2 Facility Water Supply Temperature Classes for ITE
		3.2.1 Liquid Cooling Environmental Class Definitions
			Table 3.1 2021 Thermal Guidelines for Liquid Cooling
			Figure 3.4 Liquid-cooling Classes W17 and W27 typical infrastructure.
			Figure 3.5 Liquid-cooling Classes W32, W40, W45, and W+ typical infrastructure.
		3.2.2 Condensation Considerations
Chapter 4 - FacilityTemperature and Humidity Measurement
	4.1 Facility Health and Audit Tests
		4.1.1 Aisle Measurement Locations
			Figure 4.1 Measurement points in aisle.
			Figure 4.2 Measurement points between rows.
			Figure 4.3 Measurement points in a hot-aisle/cold-aisle configuration.
		4.1.2 HVAC Operational Status
		4.1.3 Evaluation
	4.2 Equipment Installation Verification Tests
		Figure 4.4 Monitoring points for configured racks.
		Figure 4.5 Monitoring points for 1U to 3U equipment.
		Figure 4.6 Monitoring points for 4U to 6U equipment.
	4.3 Equipment Troubleshooting Tests
		Figure 4.7 Monitoring points for 7U and larger equipment.
		Figure 4.8 Monitoring points for equipment with localized cooling.
	4.4 COOLING SIMULATION
Chapter 5 - Equipment Placement and Airflow Patterns
	5.1 Equipment Airflow
		5.1.1 Airflow Protocol Syntax
		5.1.2 Airflow Protocol for Equipment
			Figure 5.1 Syntax of face definitions.
			Figure 5.2 Recommended airflow protocol.
		5.1.3 Cabinet Design
	5.2 Equipment Room Airflow
		5.2.1 Placement of Cabinets and Rows of Cabinets
			Figure 5.3 View of a hot-aisle/cold-aisle configuration.
			Figure 5.4 Example of hot and cold aisles for raised-floor environments with underfloor cooling.
			Figure 5.5 Example of hot and cold aisles for non-raised-floor environments with overhead cooling.
			Figure 5.6 Cold-aisle containment.
		5.2.2 Cabinets with Dissimilar Airflow Patterns
			Figure 5.7 Hot-aisle containment.
		5.2.3 Aisle Pitch
			Figure 5.8 Seven-tile aisle pitch, equipment aligned on hot aisle.
			Table 5.1 Aisle Pitch Allocation
			Figure 5.9 Seven-tile aisle pitch, equipment aligned on cold aisle.
Chapter 6 - Equipment Manufacturers’ Heat and Airflow Reporting
	6.1 Providing Heat Release and Airflow Values
	6.2 Equipment Thermal Report
		Table 6.1 Example of Thermal Reporting
	6.3 EPA Energy Starâ Reporting
Appendix A - 2021 ASHRAE Environmental
Guidelines for ITE—
Expanding the Recommended 
Environmental Envelope
	Table A.1 Comparison of 2004, 2008/2011, 2015, and 2021 Versions of Recommended Envelopes
	Figure A.1 Highlighted in red is the 2021 recommended envelope for a low level of pollutants.
	Figure A.2 Highlighted in red is the 2021 recommended envelope for a high level of pollutants.
	A.1 DRY-BULB TEMPERATURE LIMITS
		A.1.1 Low End
		A.1.2 High End
			Figure A.3 Inlet and component temperatures with fixed fan speed.
			Figure A.4 Inlet and component temperatures with variable fan speed.
	A.2 MOISTURE LIMITS
		A.2.1 High End
			Figure A.5 Temperature and humidity distribution of a free-cooled data center.
		A.2.2 Low End
	A.3 ACOUSTICAL NOISE LEVELS
	A.4 Data Center Operation Scenarios for the Recommended Environmental Limits
Appendix B - 2021 Air-Cooled Equipment 
Thermal Guidelines (I-P)
	Table B.1 2021 Thermal Guidelines for Air Cooling— I-P Version (SI Version in Chapter 2)
	Table B.2 2021 Thermal Guidelines for High-Density Servers— I-P Version (SI Version in Chapter 2)
Appendix C - Detailed Flowchart for the Use and Application of the ASHRAE Data Center Classes
	C.1 Notes for Figures
	C.2 Nomenclature for Figures
		Figure C.1 Guidance for applying thermal guidelines.
		Figure C.2 Guidance for applying thermal guidelines to new construction projects.
		Figure C.3 Guidance for applying thermal guidelines to major retrofit projects.
		Figure C.4 Guidance for applying thermal guidelines to existing facilities looking for efficiency gains.
Appendix D - ESD Research and Static Control Measures
	D.1 ESD Background
	D.2 ESD RESEARCH
		Figure D.1 Walking pattern according to ANSI/ESD STM97.2 (ESDA 2016). (Reprinted with permission of EOS/ESD Association, Inc.)
		Figure D.2 Walking voltage test setup according to ANSI/ESD STM97.2 (ESDA 2016). (Reprinted with permission of EOS/ESD Association, Inc.)
		Table D.1 Types of Flooring and Shoes Used in Test Program
		Table D.2 Flooring and Shoes Defined by Electrical Resistance
		Table D.3 Probabilities of Voltages from Walking Tests Greater than Threshold Values
	D.3 Personnel and Operational Issues
	D.4 Flooring Issues
		D.4.1 Measuring Floor Resistance
			Figure D.3 Typical test setup for measuring floor conductivity.
	D.5 Further Reading
Appendix E - Research on the Effect of RH and Gaseous Pollutants on  ITE Reliability
	Figure E.1 2015 thermal environmental conditions of air entering ITE (A1, A2, A3, and A4 represent different environmental envelopes for ITE).
	Figure E.2 Test specimens: a) standard copper and silver coupons and b) printed circuit board (PCB) coupons.
	Figure E.3 Experimental setup for mixed flowing gas testing.
	Figure E.4 Corrosion thicknesses for copper at 50% rh, 70% rh, and 80% rh.
	Figure E.5 Corrosion thicknesses for silver at 50% rh, 70% rh, and 80% rh.
	E.1 Conclusions From the Research
Appendix F - Psychrometric Charts
	Figure F.1 Classes A1–A4 allowable and recommended operating conditions for (a) low level of pollutants and (b) high level of pollutants (SI units).
	Figure F.2 Classes A1–A4 allowable and recommended operating conditions for (a) low level of pollutants and (b) high level of pollutants (I-P units).
	Figure F.3 Class H1 allowable and recommended operating conditions for (a) low level of pollutants and (b) high level of pollutants (SI units).
	Figure F.4 Class H1 allowable and recommended operating conditions for (a) low level of pollutants and (b) high level of pollutants (I-P units).
Appendix G - Altitude Derating Curves
	Figure G.1 Classes A1 to A4 temperature versus altitude.
Appendix H - Practical Example of the Impact of Compressorless Cooling on Hardware Failure Rates
	Figure H.1 Histogram of dry-bulb temperatures for Chicago.
	Figure H.2 Dry-bulb temperatures for Chicago with economization assumptions that include reuse of ITE exhaust heat to maintain a minimum 15°C to 20°C (59°F to 68°F) temperature and a 1.5°C (2.7°F) temperature rise from outdoor air to server inlet.
	Table H.1 Time-Weighted Failure Rate x-Factor Calculations for Air-Side Economization for ITE in Chicago
Appendix I - ITE Reliability Data for Selected Major U.S. and Global Cities
	I.1 Notes on figures and tables
		Figure I.1 Failure rate projections for air-side economizer for selected U.S. cities.
		Figure I.2 Failure rate projections for water-side economizer for selected U.S. cities.
		Table I.1 Time-Weighted Failure Rate x-Factor Calculations for Class A2 for Air-Side Economization for Selected Major U.S. Cities Assuming 1.5°C (2.7°F) Temperature Rise between Outdoor Ambient Temperature and ITE Inlet Air Temperature
		Table I.2 Time-Weighted Failure Rate x-Factor Calculations for Class A2 for Water-Side Economization for Selected Major U.S. Cities Assuming 9°C (16.2°F) Temperature Rise between Outdoor Ambient Temperature and ITE Inlet Air Temperature
		Figure I.3 Failure rate projections for water-side economizer with dry-cooler-type tower for selected U.S. cities.
		Figure I.4 Failure rate projections for air-side economizer for selected global cities.
		Table I.3 Time-Weighted Failure Rate x-Factor Calculations for Class A2 for Water-Side Dry-Cooler-Type Tower Economization for Selected Major U.S. Cities Assuming 12°C (21.6°F) Temperature Rise between Outdoor Ambient Temperature and ITE Inlet Air ...
		Table I.4 Time-Weighted Failure Rate x-Factor Calculations for Class A2 for Air-Side Economization for Selected Major Global Cities Assuming 1.5°C (2.7°F) Temperature Rise between Outdoor Ambient Temperature and ITE Inlet Air Temperature
		Figure I.5 Failure rate projections for water-side economizer for selected global cities.
		Figure I.6 Failure rate projections for water-side economizer with dry-cooler-type tower for selected global cities.
		Table I.5 Time-Weighted Failure Rate x-Factor Calculations for Class A2 for Water-Side Economization for Selected Major U.S. Cities Assuming 9°C (16.2°F) Temperature Rise between Outdoor Ambient Temperature and ITE Inlet Air Temperature
		Table I.6 Time-Weighted Failure Rate x-Factor Calculations for Class A2 for Water-Side Dry-Cooler-Type Tower Economization for Selected Major U.S. Cities Assuming 12°C (21.6°F) Temperature Rise between Outdoor Ambient Temperature and ITE Inlet Air ...
		Figure I.7 Number of hours per year of chiller operation required for air-side economizer for selected U.S. cities.
		Figure I.8 Number of hours per year of chiller operation required for water-side economizer for selected U.S. cities.
		Figure I.9 Number of hours per year of chiller operation required for water-side dry-cooler economizer for selected U.S.cities.
		Figure I.10 Number of hours per year of chiller operation required for air-side economizer for selected global cities.
		Figure I.11 Number of hours per year of chiller operation required for water-side economizer for selected global cities.
		Figure I.12 Number of hours per year of chiller operation required for water-side dry-cooler economizer for selected global cities.
Appendix J - OSHA and Personnel Working in
High Air Temperatures
	Table J.1 Screening Criteria for ACGIH TLVs® and Action Limits for Heat Stress Exposure (ACGIH 2017)
Appendix K - Allowable Server
Inlet Temperature
Rate of Change
	Figure K.1 Examples of tape equipment inlet air temperature versus time that are compliant with the 5°C (9°F) in an hour temperature change requirement for data center rooms with tape. equipment.
	Figure K.2 Examples of tape equipment inlet air temperature versus time that are noncompliant with the 5°C (9°F) in an hour temperature change requirement for data center rooms with tape equipment.
	Figure K.3 Examples of equipment inlet air temperature versus time that are compliant with the 20°C (36°F) in an hour and the 5°C (9°F) in 15 minutes temperature change requirements for data center rooms that contain other types of ITE not includ...
	Figure K.4 Examples of equipment inlet air temperature versus time that: a) are noncompliant with the 20°C (36°F) in an hour requirement, b) are noncompliant with the 5°C (9°F) in 15 minutes requirement, and c) are noncompliant with 5°C (9°F) i...
	Figure K.5 Example of ITE air inlet temperature rate of change (°C/h) calculated over 1 min, 5 min, 15 min, and 60 min time intervals.
	Figure K.6 Example of time delay between inlet air temperature change to storage array and the corresponding temperature change in HDDs of the storage array.
Appendix L - Allowable Server Inlet RH Limits
versus Maximum Inlet
Dry-Bulb Temperature
	Figure L.1 Class A3 climatogram illustrating how dew-point limits modify RH specification limits.
	Figure L.2 Climatogram of recommended ranges for Classes A1 to A4 (see Table 2.1 in Chapter 2 for more details): a) low levels of pollutants and b) high levels of pollutants.
	Figure L.3 Class A1 and A2 operation climatograms.
	Figure L.4 Class A3 and A4 operation climatograms.
	Figure L.5 Class A1 though A4 power off climatogram.
	Figure L.6 Climatogram of recommended ranges for Class H1 (see Table 2.2 in Chapter 2 for more details): a) low levels of pollutants and b) high levels of pollutants.
	Figure L.7 Class H1 operation climatogram.
References and Bibliography