FSSA White Paper - Fire Safety Systems Protecting Data Centers
FSSA White Paper - Fire Safety Systems Protecting Data Centers
Fires in data centers occur and can result in loss of information, property, and life. A recent survey of FSSA members who
install fire extinguishing systems in data centers indicated that hundreds of fires had been successfully extinguished by
gaseous clean agent systems in data centers. In 2018, a fire occurred in a data center at a large university on the East Coast
of the United States. The installed clean agent system extinguished the fire quickly and no data loss occurred. The campus
fire chief related that the information contained on those servers represented years of research for the University and was in
his words “priceless.” Although fires in data centers are not “common,” cases like this where an automatic gaseous agent fire
system extinguishes a fire with no loss of data are the rule when clean agent systems are deployed.
Systems which only sound an alarm require that trained personnel be present to determine the cause of the alarm and then
decide what action should be taken. In some cases, fire department personnel will have to respond to manually fight the fire.
There can be substantial delays with resulting data loss and equipment damage while personnel take action.
There are various types of sprinkler systems which can detect fire and spray water onto the fire. There is typically notable
delay between the onset of fire in a data center and the operation of an automatic sprinkler system. The delay may be
unacceptable in terms of data loss and equipment damage. Many data center operators are concerned about possibility of
consequential water damage to equipment. They choose instead to use a clean, dry fire extinguishing agent for their data
center – a “clean agent.”
Clean agent gases come in two general varieties: liquefied compressed gas agents and inert gas agents. HFC-227ea, HFC-
125 and FK-5-1-12 are examples of liquefied compressed gas agents. Nitrogen, argon, mixtures of nitrogen and argon, and
a mixture of nitrogen and argon with a small percentage of carbon dioxide are examples of inert gas agents.
The discharge of a clean agent is typically initiated by smoke detection. Clean agent systems can be capable of sensing and
extinguishing fires which are extremely small thus greatly limiting fire damage. There have been cases where a system
discharged for “no apparent reason” only to find that the system had reacted to and extinguished a fire in its incipient stage
before the fire had any effect on the IT equipment. Indeed, these systems can greatly limit damage, data loss, and system
downtime caused by fire.
Although the reliability of HDD is generally excellent, HDD are known to fail for numerous reasons. BackBlaze is a cloud
storage company that publishes HDD reliability statistics1 based on their in-house experience. During 2017, the company
had 116,833 drives for which they obtained data. Over 4 drives failed on average each day; a total of 1,508 drives failed
during 2017. Although not an enormous number percentage-wise, the value of the data contained on these drives is
inestimable. Thus, prudence dictates that data stored on HDD be “backed up” to at least one redundant location, and often
to multiple redundant locations. Various schemes have been implemented to provide such backup. We’ll not discuss the
various schemes used to secure data – the point is HDD are known to fail and it is irresponsible to rely on a single HDD to
store valuable information.
Basics of HDD
Typical HDDs consist of round flat metal platters coated with a ferromagnetic film, a motor to spin the platters, read-write
heads which are positioned over the platters, mechanical means to move the read-write heads, circuitry to control the
positioning of the read-write heads, memory buffers, and various other control circuitry. Data is recorded on the magnetic film
on the platters by the write head which changes the
magnetization of individual data cells. The cells are organized
in tracks that encircle the disk. There are about 10,000 parallel
tracks in each millimeter of the platter. The “read” operation is
done as the read head passes over the data cells; the
magnetization of the data cells induces a current in the read
head which is decoded by the HDD circuitry and converted to
binary values for interpretation by the operating system and
software.
When the HDD is stopped, the read-write head controller “parks” the head on the platter near the center in an area where no
data is written. In the photo of the HDD, the head is positioned in the parking zone.
For those interested in more details of HDD design and construction, there are numerous articles available free of charge on
the Internet. For purposes of this paper, however, suffice it to understand that for effective writing and reading of data the
read-write heads must be precisely positioned over extremely tiny data cells and contact between the spinning platters and
the read-write heads can do irreparable damage.
HDD Failures
“Failures” of HDDs can range from momentary failure to write (or read) data to complete failure of the drive’s capability to
retrieve data previously written to the drive.
As with all mechanical devices, failure can be the result of age related wear on moving parts. Such failure can result in
permanent loss of data stored on the HDD. Electronic circuits within the drive can also fail; failures of electronic circuits are
extremely rare, most often occurring early in the life of a HDD. Manufacturing defects are another cause of failures; failure
due to manufacturing defects commonly occur relatively early in the life of a HDD. “Head crashes” occur when the read/write
head contacts the surface of the platter. A head crash can scratch the magnetic coating on the platter resulting in permanent
and typically irreparable damage to the HDD. Head crashes can be caused by vibrating the HDD while the platters are
spinning. The vibration could be caused by moving, striking, or dropping the HDD. The impingement of acoustic waves on
the HDD likewise could cause components of the HDD to vibrate resulting in a head crash.
In line with the findings in the Gregg’s report, interruption of data processing and even permanent damage to HDDs due to
loud sounds in data centers have been reported. The discharge of gaseous fire extinguishing agents3 have been reported to
have caused problems with HDD including loss of data and, in a few cases, permanent damage to HDDs. These events
prompted further investigation into the phenomenon of acoustic interference with HDD operations.
Multiple studies have demonstrated that failures of HDDs due to sound waves are most often linked to vibration of the
mechanical components within the HDD. For example, vibration induced by the impingement of sound waves on the read-
write heads can cause the head assemblies to vibrate and get out of alignment with the data track being read or written –
hence a delay in the read-write operation (as noted by Gregg in his “shouting” experiment). Vibration of the platter may also
be induced and result in a head crash causing permanent and, in some instances, irreparable damage to the HDD.
2
Shouting in the Data Center https://www.youtube.com/watch?v=tDacjrSCeq4
3 Gaseous fire extinguishing agent systems come in two general categories: liquefied agent systems and inert gas systems. Both types of systems
generate noise; further there are audible fire alarms associated with all fire extinguishing systems. Thus far all the reported incidents of interruption
of HDD operation have been during discharge of inert gas systems.
For example, if a sound wave containing the resonant frequency of the read-write head assembly impinges on the HDD, the
read-write head assembly may begin vibrating vigorously; vibrations may be large enough to displace the read-write head
from the required position over the proper data track; hence data throughput will be reduced. Vibration of the platters during
a read-write operation can have a similar effect. If the vibrations of the heads or the platters is severe enough, the heads may
contact the spinning platter causing permanent loss of data and irreparable damage to the HDD. Sound waves containing
the resonant frequencies of the HDD components can induce vibrations at much lower sound pressure levels than waves
which do not include the HDD component’s resonant frequencies.
While much of the focus regarding acoustic interference with HDD performance has been on sound waves in the audible
range, a study by Boltoni et al reported loss of data due to acoustic waves in the ultrasonic range, another potential cause of
HDD failure. Acoustic waves in the ultrasonic range can falsely trigger the shock sensors incorporated in some HDD causing
the read-write heads to park, thus stopping data throughput.
Some Studies
Tyco Fire Protection Products (now Johnson Controls), Siemens, and IBM are among those who investigated the effects of
sound waves on HDD. Studies have also been performed at various colleges and universities including Michigan University,
Zhejiang University, Purdue University, Princeton University, and Michigan Technological University.
In these studies, multiple makes and models of HDD were exposed to sound waves, the frequency and amplitude of which
were controlled. The results conclude that the likelihood that sound waves will induce failures or damage to HDD depends
on three factors:
The 2017 report “Performance of Hard Disk Drives in High Noise Environments” 4 submitted by T. Dutta as part of the
requirements for a Master of Science degree at Michigan Technological University presented data showing the relationship
between Sound Pressure Level and frequency required to cause a 50% reduction in data throughput for 14 different HDD
makes and models. A few representative graphs are shown below. The black diamonds and squares on each graph indicate
the point at which a 50% reduction in throughput was measured. Note that the amplitude of the sound wave needed to be
increased by a relatively small amount from the 50% reduction point to cause complete loss of throughput.
4 https://digitalcommons.mtu.edu/cgi/viewcontent.cgi?article=1491&context=etdr
HDD Graph B indicates that for frequencies between 500 and 630 Hz a
SPL of 125 dB was required to cause a 50% loss of data throughput. For
frequencies between 1 kHz and 3.15 kHz (black circles) no degradation of
throughput was observed with SPL as high as 125 dB. At frequencies of 5
and 8 kHz SPL of 110 dB caused 50% reduction in throughput.
From the above graphs one might conclude that HDD are most sensitive to higher frequency sounds. One more graph for
another of the 14 HDD models tested will dispel that conclusion:
From these graphs and the eleven additional graphs presented in Dutta’s report, it is obvious that the sound pressure level
required to cause serious loss of data throughput depends on the frequency of the sound wave and the design and
construction of the HDD. The response of HDD to sound varies greatly from HDD model to HDD model.
The characteristics of sound waves which will cause notable reduction in data throughput vary from one HDD make and model
to another. For a given HDD, the sound pressure level (SPL) required to substantially reduce throughput also depends on
the frequency of the sound wave.
Tests have shown that sound pressure levels as low as 85 dB in the frequency range of 8 to 10 kHz can result in complete
loss of throughput for at least one make/model of HDD. For another make/model of HDD, tests indicated that throughput
continued at over 90% of “normal” when sound waves of 125 dB impinged on the HDD if the frequency of the sound wave
was in the 800Hz to 2.5 kHz range.
When vibration of the HDD platters or read-write heads causes physical contact between the heads and the spinning platters
(“head crash”) permanent data loss and irreparable damage to the HDD is likely. Such vibration can be induced by anything
that “shakes” the HDD including impingement of sound waves on the HDD. Such serious damage is generally linked to
vibrations at resonant frequency of the HDD components.
Of course, age related mechanical failure of the HDD can also cause loss of data.
Fire Alarms
All of the fire safety systems discussed at the beginning of this document generate sound waves when they are activated.
Smoke detection systems activate fire alarms. Water sprinklers activate fire alarms and water flow alarms. Gaseous agent
systems activate fire alarms and produce sound waves as the fire extinguishing gas discharges into the protected data center.
NFPA Standard 72, the Fire Alarm and Signaling Code, requires the sound pressure level of audible fire alarm devices to be
15 dB above the average ambient sound pressure level5 in the data center. The maximum sound pressure level generated
by alarms and background noise may not exceed 110 dBA.
The average sound pressure levels in an operating data center are on the order of 70 to 85 dBAii. Thus, the minimum SPL in
the data center due to the fire alarms is required to be at least 85 dBA. If the average sound pressure level in the data center
is closer to the upper end of the average reported SPL, the fire alarm must produce an SPL of nearly 100 dBA.
Of course, the SPL which will adversely affect HDD performance is linked to the combination of SPL and frequency – and
required combination of SPL and frequency is known to vary between make/model of HDD. Most standard fire alarm horns
produce a frequency in the range of 2 kHz to 4 kHz. Theoretically, based on the testing of various HDD for noise sensitivity
and the know characteristics of fire alarms, failure of HDD due to sound waves generated by fire alarms appears to be possible.
To date there have been no reported incidents of HDD failure (notable reduction of data throughput or damage to the drive)
due to fire alarms activated by smoke detection systems or water sprinkler systems.
5
For “public mode” NFPA Standard 72 Edition 2016, Chapter 18, requires the SPL from the alarm to be at least 15 dB above the
average ambient sound pressure level or at least 5 dB above the maximum sound level that lasts at least 60 seconds, whichever is
greater.
Theoretically the discharge of any gaseous agent could produce sound waves of sufficient amplitude and in the required
frequency range to cause reduction in data throughput in a HDD, depending on the characteristics of the HDD. To date,
reports of interruption of data throughput due to discharge of a fire extinguishing agent have centered around inert gas agent
systems with no reported incidents linked to discharge of liquefied compressed gaseous agents. In 2012, Rawson and Greeniii
published an excellent article describing the phenomenon and recommending means to avoid loss of data and possible
damage to HDD during a system discharge. The means recommended by Rawson and Green and some additional means
are:
1. IMPROVED DESIGN OF HDD Realizing that sound waves have an impact on the performance of HDD, manufacturers
of HDD could improve designs to reduce the possibility of adverse performance due to sound waves. Manufacturing
equipment having sensitivity over a narrower range of frequency and at higher sound pressure level would reduce the
chance of interruption of operation.
2. SHUTDOWN THE HDD All gaseous agent fire extinguishing agent systems sound an alarm prior to discharge of the
agent. The alarm permits personnel to exit the data center. If HDD are shut down coincident with the sounding of the
alarm the alarm, data loss and HDD damage will be avoided. While an obvious way to avoid data loss due to sound waves
produced by fire safety systems, many modern data centers desire to not shut down HDDs to minimize customer
interruption.
3. REDUCE THE SOUND AT SOURCE Lessening the flow rate from a given nozzle will reduce the amplitude of the sound
waves. Thus, the use of multiple nozzles versus a single nozzle to discharge a given quantity of agent will lessen the
amplitude of the sound waves generated by each nozzle. Some inert gas system manufacturers have developed “silent”
nozzles which, although they are not totally “silent,” do produce sound waves of considerably less amplitude than
“standard” nozzles. Using such nozzles reduces the likelihood of shutting down data throughput and causing permanent
damage to the HDD. Because of range of frequencies and related amplitudes of the sound to which various HDD are
sensitive, use of these “silent” nozzles cannot guarantee that data loss or damage will not occur to all the current and
future HDD which might be found in a given data center.
4. PLACEMENT OF NOZZLES WITH RESPECT TO HDDS Avoid placing HDDs close to nozzles. Increasing distance helps
reduce of the sound pressure level at the drives.
5. USE OF MODELING TOOLS TO FIND WAYS TO REDUCE NOISE REACHING HDD One manufacturer has developed a tool that
can be used to calculate the sound pressure level reaching the HDD. The tool relies on a number of inputs, such as the type of
sound absorbing material used in the construction of the data center (different materials of construction reflect or absorb sound
differently), the number of racks present (as each object in the space will absorb sound), the agent flow rate (as this directly affects
the sound generated), and the number and location of nozzles used in the system). More information on this technology may be
found in a white paperiv by Sandahl, Elder, and Barnard. Use of such modeling tools can help in determining steps to alleviate the
impact of noise on HDDs.
6. REDUCE THE SOUND REACHING THE HDD Many IT equipment manufacturers can provide acoustic sound insulating covers and
doors for IT equipment racks. The use of such acoustic barriers on HDD racks can prevent potentially harmful sound waves from
all external sources from reaching the HDD. Only acoustic barriers approved by the IT equipment manufacturer should be installed.
The installation of “home-made” barriers has potential to block airflow and cause overheating of equipment. In addition to or in lieu
of shielding individual equipment racks from external sound waves, the amount of acoustic energy reaching HDD can be decreased
i “Blue Note: How Intentional Acoustic Interference Damages Availability and Integrity in Hard Disk
Drives and Operating Systems” Connor Bolton, Sara Rampazzi, Chaohao Li, Andrew Kwong, Wenyuan Xu and Kevin Fu, University of Michigan,
Zhejiang University, Reported at the IEEE Symposium on Security and Privacy 2018
ii “Noise within a Data Center” Dubravko Miljković, Hrvatska elektroprivreda, Zagreb, Croatia; [email protected]
iii “Inert Gas Data Center Fire Protection and Hard Disk Drive Damage” Rawson, B. P., & Green, K. C. (2012). Data Center Journal,
http://www.datacenterjournal.com/inert-gas-data-center-fire-protection-and-hard-disk-drive-damage/ .
iv “The impact of sound on computer hard disk drives and risk mitigation measures” Sandal, Elder, & Barnard. https://www.sensecosystems.com/wp-
content/uploads/2017/12/White-Paper-on-Data-Center-Acoustic-Research-1.pdf
v “What the Future Holds for Data Storage” http://www.itpro.co.uk/solid-state-storage-ssd/31387/what-the-future-holds-for-data-storage