A Comprehensive Analysis of the Performance of Gear Fault

Detection Algorithms
Eric Bechhoefer1, Brent Butterworth2
1
GPMS Inc, Cornwall, VT, 05753, USA
[email protected]
2
Garmin International, Midlothian, TX, 76065, USA
[email protected]

ABSTRACT recognized the benefits of monitoring, mandating gearbox

monitoring to protect, in part, against gear fault.
Modern gear fault detection analysis began with algorithms
based on the time synchronous average. Over the course of A vast literature on automate gear fault analysis has been
decades, many gear analyses have been proposed, but with no generated, beginning with the seminal work of R.M.
evidence that the analysis was significantly more powerful in Stewart’s Some Useful Data Analysis Techniques for
terms of fault detection than existing algorithms. This study Gearbox Diagnostics (1977). Stewart gave us the use of the
focuses on a comprehensive comparison of gear fault time synchronous average (Bechhoefer, 2009), the
detection algorithms to evaluate their performance. Using a residual/difference analysis and mesh specific Figures of
large, statistically significant set of data from three nominal Merit. Building on Stewart, P.D. McFadden reported
machines and a damaged machine, the CI responses of 88 Amplitude and Phase Demodulation techniques (1986),
different analysis are compared in terms of their statistical while Ma (1995) introduced Teager’s energy operator to the
significance to detect a cracked tooth. The comparison condition monitoring community. A number of NASA
includes residual, energy operator (and its variants), the researchers (Zakrajsek, Lewicki, 1989, 1993 and 2009)
narrowband analysis (with a comparison of bandwidth investigate the ability of these, and other various algorithms
requirements), the amplitude and frequency modulation to detect gear faults. However, to date, there has not been a
analysis, an analysis of variance of the "factor” analysis: comparison of the power of the algorithms to detect fault.
crest, shape, impulse, and margin, and other standard gear That is to say, which gear analysis algorithms are the best at
fault CIs. Further, the effect of CI selection in the detecting a gear fault?
establishment of gear component health is evaluated, where Further complicating this issue is that the gear themselves
given a set of CIs, a gear health indicator is built, showing have a number of failure modes (Gopinath, 2010), to include:
that CIs with high statistically separability and low Scoring/Frosting, Wear (adhesive, abrasive, corrosive),
correlation have improved fault detection power. This is Pitting (flaking, spalling), Plastic Flow (Ridging/Rippling)
validated on a third, dissimilar gear fault propagation test. and tooth breakage.
1. INTRODUCTION Critical to determining an algorithms fault detection
capability is a metric for measuring the performance of an
Detection of gear faults is challenging. Undetected gear faults analysis. Also, of importance is how this information is
can result in catastrophic gearbox failures, which depending displayed for a user. For this paper, a statistical method is
on the criticality of the application, can be life threatening. proposed to measure separability of a number of standard
For example, as a result of a number of North Sea helicopter analyses. These results will be used to construct in improved
accidents, many large helicopter operators have installed health indicator, which is designed to provide actionable
Health and Usage Monitoring Systems (HUMS). The Civil information to a maintainer. This analysis is based on a large
Aviation Authority (CAA), by issuing ADD 001-05-99, (300+ raw data acquisitions) data set from three nominal
gearboxes, and 300 raw data acquisitions from a damaged
Eric Bechhoefer et al. This is an open-access article distributed under the gear (tooth breakage). The results (e.g. best condition
terms of the Creative Commons Attribution 3.0 United States License,
which permits unrestricted use, distribution, and reproduction in any
indicators) are tested against another, dissimilar gearbox with
medium, provided the original author and source are credited. a known propagating fault. This study is focused on
broken/cracked tooth, as it presents the failure mode that is

1
 ANNUAL CONFERENCE OF THE PROGNOSTICS AND HEALTH MANAGEMENT SOCIETY 2016

associated with catastrophic failure. It is cautioned that other mean separability, one has to ask if it is statically different
failure modes, such as scuffing, may require a different set of from another analysis. For a number of similar analysis
analysis. (energy operator and its related analysis), ANOVA can
determine if there is any statistical different between the CI
2. QUANTIFYING GEAR ALGORITHMS PERFORMANCE populations. This will also be done for the “factor” analysis:
Crest, Shape, Impact and Margin.
Detection: the action or process of identifying the presences
of something concealed. For vibration monitoring, it is
2.2. Building Actionable Information: Gear Health
identifying a feature within the background noise of a
gearbox which is indicative of fault. For critical system, the In order to reduce maintainer workload, and improve fault
risk is a missed detection that allows a mission to proceed detection, the concept of the health indicator was introduced
when it should be aborted. Otherwise, the risk is performing in (Bechhoefer, 2011). This allows the maintainer a common
unnecessary maintenance due to a false alarm. Inherent to paradigm for determining when a maintenance action needs
both Probability of Detection (PD), and Probability of False to be carried out. In general, the Health Indicator (HI) is a
Alarm (PFA) is signal to noise: improving SNR increases the function of distributions of n CIs, where the HI is scaled such
PD and floor a given PFA. A linear measure of signal to noise that:
is signal divided by the standard deviation. Form a statistical
sense, this is related to Hypothesis testing. When nominally • HI ranges from 0 to a positive value, where the
assuming a Gaussian signal, if the normalized signal exceeds probability of exceeding an HI of 0.35 is the PFA
the mean by 3 standard deviation, the PD is 0.9987, while the (probability of false alarm),
PFA is 0.0013. For this test, a similar metric is used: • A warning (yellow) alert is generated when the HI is
statistical separability. greater than or equal to 0.75. Maintenance should be
planned by estimating the RUL until the HI is 1.0.
2.1. Statistical Separability • An alarm (red) alert is generated when the HI is greater
Separability defines the statistical distances between two than or equal to 1.0. Continued operations could cause
populations. The concept inherently implies that for distinct collateral damage.
populations (e.g. a nominal gear vs. a damaged gear) using Note that this nomenclature does not define a probability of
an algorithm that can detect the damage, the separability is failure for the component, or that the component fails when
large. This is quantified using the pooled sampled standard the HI is 1.0. Instead, it suggests a change in operator
deviation. It is noted that this test suggest that the population behavior to a proactive maintenance policy: perform
density is assumed Gaussian, which for some distribution is maintenance before the generations of collateral or cascading
a poor assumption. The test statistic (Medenhall, 1990) used faults. For example, by performing maintenance on a bearing
is the small-sample test for the comparing two population prior the bearing shedding extensive material, costly gearbox
means, where: replacement can be avoided.
𝑌%! − 𝑌%" Hence, from a maintainer perspective, this is a stoplight-
𝑇 = - (1)
𝑆(1⁄𝑛! + 1⁄𝑛" based threshold setting/alerting system: when a component is
yellow, plan maintenance, and when the component turns
where n1 is the number of samples for the nominal gear, n2 is red: do maintenance
the number of samples for the damaged gear, and S, the
pooled sample variance is: 2.3. Importance of Controlling Correlation Between CIs
(𝑛 − 1)𝑆!" + (𝑛" − 1)𝑆""- All CIs have a probability distribution (PDF). Any operation
𝑆 = . ! 𝑛! + 𝑛" − 2 (2)
on the CI to form a health index (HI) is then a function of
The population mean and sample variance are calculated distributions. The HI function is defined as the norm of n CIs
from the gear analysis condition indicators (CIs), where the (energy):
CIs are statistics derived from the gear analysis. 𝐻𝐼 = 0.35<𝑐𝑟𝑖𝑡 √𝒀# 𝒀 (3)
All analysis and comparisons of analysis for this study will
where Y is the whitened, normalized array of CIs, and crit, is
be based on T, the separability of an analysis. This allows a
the critical value.
convent way to measure the performance of one analysis
against another. The function is valid if and only if the distribution (e.g., CIs)
are independent and identical (e.g., IID). As an example, for
2.1.1. Analysis of Variance/Ranking Gaussian distribution, subtracting the mean and dividing by
the standard deviation will give identical Z distributions. The
Further performance metric will be based on Analysis of issue of ensuring independence is much more difficult. In
Variance (ANOVA). For example, while the T represents the general, the correlation between CIs is non-zero. For

2
 ANNUAL CONFERENCE OF THE PROGNOSTICS AND HEALTH MANAGEMENT SOCIETY 2016

instance, Error! Reference source not found. shows the the whitening process, the value for b for each CI will then
correlation coefficients for 6 CIs used for gear fault analysis: be: s2 = 1, such that: b = s2 / (2 - p/2)0.5 = 1.5264.
most are statically significant.
For the HI equation in (4), the normalized energy of the CIs,
Table 1: Correlation Coefficients for the Six CIs Used it can be shown that the function defines a Nakagami PDF
in the Study (Bechhoefer, 2011). The statistics for the Nakagami are h =
n, and w = 1/(2-p/2)*2*n, where n is the number IID CIs used
rij CI 1 CI 2 CI 3 CI 4 CI 5 CI 6
CI 1 1 0.84 0.79 0.66 -0.47 0.74
in the HI calculation.
CI 2 1 0.46 0.27 -0.59 0.36
CI 3 1 0.96 -0.03 0.97 2.5. Advantages of the HI vs. the CI for the Maintainer
CI 4 1 0.11 0.98 The use of a common threshold for damage, as per the HI,
CI 5 1 0.05 simplifies the user experience. It does offer two additional
CI 6 1
benefits. First, the HI allows better control of the probability
of false alarm as it whitens/removes correlation between CIs.
This correlation between CIs implies that for a given function As an example of the importance of correlation, consider a
of distributions to have a threshold that operationally meets simple HI function HI = CI1 + CI2. The CIs will be normally
the design PFA, the CIs must be whitened (e.g., de- distributed with mean 0 and standard deviation of 1. The
correlated). A whitening solution can be found using standard deviation of this HI is:
Cholesky decomposition.
" "
𝜎$% = .𝜎&%! + 𝜎&%" + 2𝜌&%!,&%" 𝜎()! 𝜎()" (5)
The Cholesky decomposition of Hermitian, positive definite
matrix results in A = LL*, where L is a lower triangular, and
L* is its conjugate transpose. By definition, the inverse Where rCI1, CI2 is the correlation between CI1 and CI2. If one
covariance is positive definite Hermitian. It then follows that assumes rCI1,CI2 is 0.0, then 𝜎$% = √2. For a PFA of 10e-6,
if: the threshold is then 6.722. Consider the case in which the
observed correlation is closer to 1 (e.g., rCI1, CI2 is 1.0), then
LL* = S-1, then Y = L x CIT (4) the observed 𝜎$% = 2 . For a threshold of 6.722, the
The vector CI is the correlated CIs used for the HI operational PFA is 4 x 10e-4. This is 390 times greater than
calculation, and Y is 1 to n independent CIs with unit variance the designed PFA. This illustrates the deleterious effect of
(one CI representing the trivial case). The Cholesky correlation on the threshold setting.
decomposition, in effect, creates the square root of the inverse Second, using an HI with multiple CI give a process gain and
covariance. This, in turn, is analogous to dividing the CI by improves sensitivity. Consider a simple example where one
its standard deviation (the trivial case of one CI). It can be sets up a case where there are four CIs that have similar
shown that Y = L x CIT then creates the necessary performance (e.g. for a damaged component, the normalized
independent and identical distributions required to calculate CI value is 1). One can ask the question, at what level would
the critical values for a function of distributions. the CI have to be for the HI to be one? That is:

2.4. Determining the Critical Value for Scaling Gear 1 = (𝐶𝐼!" + 𝐶𝐼"" + 𝐶𝐼*" + 𝐶𝐼+" = (4𝐶𝐼," = 2𝐶𝐼 (6)
Health
Hence, the CI values would be ½: the HI using four CI would
The critical value is taken from the inverse cumulative have a process gain of 3dB. In this study, the HI algorithm
distribution function for the HI. The CIs used have Rayleigh will use six CIs, potentially a gain of 3.9 dB.
like PDFs (e.g., heavily tailed). For magnitude-based CIs, it
can be shown that for the nominal case, the CI probability 3. GEAR FAULT CONDITION INDICATORS
distribution function (PDF) is Rayleigh (Bechhoefer, 2005).
For Gear CIs and Bearing CIs (where magnitudes which are It should be observed that there is analysis (e.g. the Time
Synchronous Average), and then statistics of the analysis,
biased by RMS), a transform is used by make the CI more
which define the condition indicator. Generally speaking,
Rayleigh like. The transform “left shifts” the CI. For
some papers (McFadden, 86, Ma, 1995) describe an analysis,
example, a shift such that the .05 CDF (cumulative
but not specific condition indictors, whereas other papers,
distribution function) is assigned to 0.0.
(Zakrajsek, 1989, and 1993, or Lewicki, 2009) refer the
Consequently, the HI function is based using the Rayleigh condition indicator (CI) which is a statistic of an analysis
distribution. The PDF for the Rayleigh distribution uses a (such as the kurtosis of the Narrowband Analysis). Finally,
single parameter, b, defining the mean µ = b*(p/2)0.5, and Stewart (1977) describe CIs which are derived as gear mesh
variance s2 = (2 - p/2) * b2. The PDF of the Rayleigh is: specific statistics, usually ratio of analysis statistics.
x/b2exp(x/2b2). Note that when applying these operations to For this analysis, the statistics for CIs applied to analysis are:

3
 ANNUAL CONFERENCE OF THE PROGNOSTICS AND HEALTH MANAGEMENT SOCIETY 2016

• RMS: Root Mean Square max(TSA) − 𝑚𝑖𝑛(TSA) (7)

-
• Skewness (1⁄𝑛 ∑-).! 𝑇𝑆𝐴")
• Kurtosis This is the TSA peak to peak divided by the TSA RMS. The
• Peak to peak (max – min value of a waveform) shape factor (SF) is affected by the TSA shape, but is
independent of its dimensions:
• Crest Factor
• Shape Factor (1⁄𝑛 ∑-).! 𝑇𝑆𝐴")-
1⁄𝑛 ∑-).!|TSA) | (8)
• Impulse Factor and
• Margin Factor It is the TSA RMS divided by the mean absolute value of the
TSA. Both the impact factor (IF) and margin factor (MF) =
Whereas the analysis performed were based on the: are similar to the CF, as both are a measure of impact. IF is
• Time Synchronous Average defined as:
• Residual Analysis max(TSA) − 𝑚𝑖𝑛(TSA)
-1⁄𝑛 ∑- |TSA | (9)
).! )
• Energy Operator
• Narrowband Analysis And MF is defined as:
• Amplitude Modulation Analysis and max(TSA) − 𝑚𝑖𝑛(TSA)
- " (10)
• Frequency Modulation Analysis M1⁄𝑛 ∑-).! (|𝑇𝑆𝐴) |N
The flow of the processing is given Figure 1
3.2. Gear Analysis: The Time Synchronous Average
Time Synchronous Average All of the analyses start with the TSA (Figure 1). A solution
FFT
to the TSA starts as a model of vibration from rotating
equipment, as given in (Bechhoefer, 2009):
FM0, ER, GS, G2, SI, SLF 𝑥(𝑡) = ∑0/.! 𝑋/ [1 + 𝑎/ (𝑡)] × 𝑐𝑜𝑠[2𝜋𝑘𝑓1 (𝑡) + 𝜙/ ] + 𝑏(𝑡)
(11)
Remove GM

BP
Fil
where:
te Narrowband Analysis
Residual Analysis r
• Xk is the amplitude of the kth mesh harmonic
• fm(t) is the average mesh frequency
• ak(t) is the amplitude modulation function of the kth
mesh harmonic.
Teager

Hilbert Trans

• fk(t) is the phase modulation function of the kth mesh

Energy Operator AM Analysis harmonic.
• Fk is the initial phase of harmonic k, and
• b(t) is additive background noise.
FM Analysis The mesh frequency is a function of the shaft rotational
speed: fm = Nf, where N is the number of teeth on the gear and
f is the shaft speed. The vibration model in (11) assumes that
that f is constant. This is never the case: there is some wander
is the shaft speed due to changes in load or feedback delay in
Figure 1 Gear Analysis Process Flow the control system. This change in speed will result in
Note that the starting point for all analysis is the time smearing of amplitude energy in the frequency domain.
synchronous average. Observed that the AM and FM analysis Hence, the TSA algorithm (Appendix A), controls this by
start with the Narrowband analysis. interpreting the zero cross time (for ratios that are not 1 and
where the zero cross time is generated from a shaft with more
3.1. The “Factor Analysis” Statistic than one pulse per revolution for the target), and by
The “factor” analysis is a family of related, non-dimensional, resampling the vibration so that, from revolution to
time domain features which have often proven successful in revolution, the x(t) in (11) has the same radian position on the
fault detection (Caesarendra, 2017). The crest factor (CF) is shaft. That is, a one per revolution imbalance represents
a measure of the impact occurring over one revolution of the cosine with period of one revolution. This resampling of the
TSA and is defined as: data mitigates the smearing effect, and reduction of non-

4
 ANNUAL CONFERENCE OF THE PROGNOSTICS AND HEALTH MANAGEMENT SOCIETY 2016

synchronous noise by 1/sqrt(rev), where rev is the number of It should be noted that the performance of the narrowband
revolutions in the TSA. analysis is based on the bandwidth of the bandpass filter used.
As such a separate analysis was conducted to evaluate the
Note that when the Fourier transform (usually implemented
performance of these analysis based on the varying the
at an FFT with radix 2 length), bin 0 is DC, bin 1 is one per
bandwidth.
rev, and the bin for gear mesh is the number of teeth (e.g. 31
tooth gear would be bin 31). This is a very powerful technique, given in Appendix C.

3.3. Gear Analysis: The Residual Analysis 3.5. Gear Analysis: Teagers’ Energy Operator Analysis
The residual, or different signal, is given by Stewart (1977). The energy operator is nonlinear tracker which, given an
A residual signal is defined as a synchronous averaged signal amplitude and phase modulated signal, can measure the
without the gear mesh frequency, its harmonics, driveshaft product of their instantaneous frequency. It has been
frequency and its second harmonics. If the first order observed that in practice the analysis is quite sensitive to
sidebands about the gear mesh frequencies are filtered out, a torque, which is why these CIs often use kurtosis and crest
differential signal is created. Conceptually, the TSA (a time factor (which tends to normalize the change in magnitude due
domain signal) will measure the periodic signal associated a to torque effects).
soft/cracked tooth. This fault feature is not sinusoidal and is
Mayo et al. (2016) reported variants of the energy operator
not associated with 1/Rev or gear mesh, which are removed
(EO) that should have improved performance. Whereas the
in the residual signal, leaving the “impact” as is clearly seen
EO is given as:
in Figure 1.
Ψ23 (𝑇𝑆𝐴- ) = 𝑇𝑆𝐴"- − 𝑇𝑆𝐴-4! × 𝑇𝑆𝐴-5! (13)
This study compared four types of residual signal. The
“classical” residual were the gear mesh and sidebands are The computational enhanced energy operator (CEEO) is
explicitly removed, and the “Threshold Residual”, were a given as:
search algorithm removes frequencies that are statistically
significant (were no gear tooth count is needed). Ψ&223 (𝑇𝑆𝐴- ) = 𝑇𝑆𝐴"- − 𝑇𝑆𝐴-4" × 𝑇𝑆𝐴-5" (14)

The last two residual are based on the envelope of the Mayo then presents a Taylor series expansion of (12) where
residual, square, as given inn Wang (2001). This is an the second order term is kept, giving EO123:
improvement to McFadden’s (1987) paper. In this test, the Ψ23!"* (𝑇𝑆𝐴- ) = 𝑇𝑆𝐴"-4! − 𝑇𝑆𝐴-4" × 𝑇𝑆𝐴- − * [𝑇𝑆𝐴"- −
!
“amRES”, is the envelop of the threshold residual, squared: !
𝑇𝑆𝐴-4! × 𝑇𝑆𝐴-5! ] + !" [𝑇𝑆𝐴"- − 𝑇𝑆𝐴-4" × 𝑇𝑆𝐴-5" ] (15)
𝑎𝑚𝑅𝐸𝑆 = |𝐻𝑖𝑙𝑏𝑒𝑟𝑡(𝑅𝐸𝑆 " )| (12)
The pseudo code is given in Appendix D.
Whereas amRESD is a low pass filtered version of amRES,
(the D is for decimated, as the bandwidth was 0.5, or
3.6. Gear Analysis: Miscellaneous Analysis
decimation by 2).
The Figure of Merit 0 is a well know analysis derived from
An implementation of the residual is given in Appendix B.
Stewart (1977), and is generally calculated as:
3.4. Gear Analysis: The Narrowband/AM/FM Analysis 𝑡𝑠𝑎 𝑝𝑒𝑎𝑘 𝑡𝑜 𝑝𝑒𝑎𝑘
fm0 = -∑* 𝐺𝑀 (16)
).! )
McFadden (1986) understood that in complex gearboxes, it
is was not always clear from the sideband modulations, how Where GM is the gear mesh harmonic taken from the FFT of
sever a gear tooth defect might be. Further, for early faults, the TSA. The peak to peak features is a time domain
such as a fatigue cracks, more effective analysis may be phenomenon (from tooth impact, breathing crack), whereas
needed. McFadden shows that by band pass filtering to the gear mesh in calculated with the FFT with is not sensitive
remove gear mesh harmonics (e.g. the narrow band analysis, to soft toot feature. Hence, as tooth damage increases, so too
where the bandpass is approximately 25% of the gear tooth does FM0.
count), and then enveloping, one can directly measuring the
The energy ratio is the residual RMS divided by the TSA rms.
amplitude and phase (frequency) modulation present in the
The concept is that as the gear fault progresses, the residual
original time synchronous average. McFadden hypothesised
RMS will approach the TSA RMS (e.g. the TSA will be
that as the soft/cracked tooth enters the load zone, the reduced
dominated by gear mesh signatures associated with the gear
stiffness of the damaged tooth transfers load to the
fault). If the residual signal is defined ri, and the TSA is tsai:
surrounding teeth. This both affects the displacement (AM
modulation), and the rotational speed of the gear (FM ∑#
!$%(8! 58̅ )
" ∑# >>>>> "
!$%(;<=! 5;<=)
modulation). 𝑒𝑟 = . -. (17)
- -

5
 ANNUAL CONFERENCE OF THE PROGNOSTICS AND HEALTH MANAGEMENT SOCIETY 2016

The sideband level factor is defined as the sum of the first SKEW 2.39 2.91 2.06
order sideband amplitudes about the gear mesh, divided by KURT 1.76 2.09 1.52
the TSA rms (Kellar, 2003): P2P 0.58 0.68 0.59
CF 1.49 2.27 1.51
𝑇𝑆𝐴?15! + 𝑇𝑆𝐴?14!
𝑆𝐿𝐹 = (18) SF 2.1 3.6 1.81
g ∑#!$%(;<=!5;<=
>>>>>)"
. IF 2.0 2.6 1.82
-
MF 2.0 2.3 1.8
The G2 analysis is simply the ratio of the second gear mesh Note that while for this test, the power of the EO and its
harmonic energy to the first gear mesh harmonic energy. variants may be low compared to other analysis (less than 3).
That said, in other studies looking at scuffing, EO was found
4. RESULTS: BANDWIDTH/AM/FM ANALYSIS to be a good indicator.
The initial study was to evaluate the effect of narrowband 6. RESULTS: FACTOR ANALYSIS
analysis bandwidth selection on the separability of the CIs
generated by the narrowband, AM and FM analysis. Recall The ANOVA analysis indicated that the difference between
that for each analysis, there are eight condition indicators: the factor analysis were statically significant. In table 4, the
RMS, Skewness, Kurtosis, Peak to Peak, Crest, Shape, result for CF, SF, IF and MF are compared with each
Impact and Margin factor. The bandwidth selected was based analysis.
on a percentage of the number of teeth on the damage pinion Table 4 Factor Analysis Separability
(which had 23 teeth). The bandwidth was varied from 3 (e.g. ANAL CF SF IF MF
bandpass from 23 +/-3) to 9 bins, which represents a bandpass
of 13% to 39% of the gear tooth. McFadden (1986) reported THRES 5.9 5.6 6.2 6.2
that 25% was optimal, and in prior studies (Bechhoefer 2015) RES 5.0 3.2 4.4 4.2
found this to be a good value. AMRES 2.3 6.5 3.7 3.8
AMRESD 2.4 7.2 4.8 4.9
As such, the test space for this experiment was 7 x 24 trails. EO 1.5 2.1 2.0 2.0
Representative results are given in Table 2, were “Mean All” CEEO 2.3 3.7 2.6 2.3
is the separability for all CIs (24), EO123 1.5 1.5 1.5 1.5
Table 2 Bandwidth vs. Separability NB 5.7 6.1 5.8 8.2
BW NB NB AM AM FM FM MEAN AM 3.4 11.3 9.8 9.7
RMS CF RMS KRT RMS KRT ALL FM 2.1 2.2 2.6 2.6
13% 1.7 7.3 5.9 -.4 13.2 7.1 5.8 MEAN 3.2 4.9 4.3 4.54
17% 1.92 7.2 6.3 1.6 5.4 6.4 5.7
22% 2 6.5 6.1 2.3 4.6 6.6 5..6
26% 2 6 6.0 2.1 4.3 6.2 5.5 From the results of Table 4. Shape Factor is the more
30% 2.1 4.5 6.2 1.9 3.8 5.2 4.4 powerful analysis.
35% 3 6.6 8.3 -.8 3.7 -.3 4.1
39% 3.2 7.8 8.3 -.3 6.2 -.8 5.6 7. RESULTS: ALL ANALYSIS AND ALGORITHMS
The overall test results are presented in Appendix E. In
It is clear from these results that the separability is a function general, these results can be compressed as the performance
of both the CI algorithm/Analysis and the bandwidth. That of similar analysis do not diverge significantly. Table 5 gives
said, a separability greater than 3 should be considered the overall results for the Residual, Computational Enhanced
significant. As there is some risk of over training a result - Energy Operator, the Narrowband analysis (using a
McFadden’s recommendation of 25% should be considered. bandwidth of 6, or 26%), Amplitude Modulation, Frequency
Modulation and Amplitude Modulated Residual.
5. RESULTS: ENERGY OPERATOR AND ITS VARIANTS Table 5 Mean Separability of Analysis for RMS, Kurtosis
As the EO and EO123 are numerically similar (essentially a and Shape Factor
time shaft), it was found that there is no statistical different RES CEEO NB AM FM AM
between them, as based on ANOVA. However, it was found RES
that the CEEO was statically different and in all case, had SEP 6.4 1.6 4.4 6.7 4.1 3.7
greater separability (Table 3)
Table 3 EO and Variant Separability The reduction from 90 values (Appendix E) to six value in
CI EO CEEO EO123 Table 5 was: that the analysis suggested that skewness and
RMS 0.51 0.75 0.51 peak to peak were in general not powerful statistics.
Similarly, from Table 4, the shape factor was found to be the

6
 ANNUAL CONFERENCE OF THE PROGNOSTICS AND HEALTH MANAGEMENT SOCIETY 2016

highest performance factor analysis. Hence, Table 5 It should be noted that these test results were performed on
represents the mean separability response for RMS, Kurtosis machines running at similar, constant RPM. As acceleration
and Shape Factor. is the second derivative of displacement, the sensor response
is a function of RPM, squared. Hence amplitude-based
From Figure 2, the relationship between the classes of
statistics are more sensitive to changes in RPM, than say,
residual analysis are compared. Clearly, the fault at 0.4
shape-based statistics. For example, if the machine is to
revolution is seen. While the amplitude modulation residual
operate under varying RPM condition, condition indicators
is easily seen as a large change in subplot 3, its performance
based on kurtosis would be less effected by RPM than RMS,
from Appendix E suggest that it is more sensitive to noise,
or Peak to Peak.
hence the lower separability.

Figure 4 Comparison of the narrowband, amplitude and

Figure 2 Comparison of Residual based analysis for good to frequency modulation analysis for good and bad pinion
bad pinion
As noted in Table 3, the performance of the energy operator 7.1. Validation of Condition Indicator Selection
and its variants are all similar. Again, it is clear that the lower
Data was collected on a single stage, spiral bevel gearbox,
separability score that the EO, CEEO and EO123 are
which was run under high load. The instrumentation was
typically a nosier analysis (Figure 3).
installed on a nominal gearbox, with the intent of propagating
a fault. After approximately 24 days, the spiral bevel pinion
showed indications of fatigue cracking at the root of a tooth.
For the test, the health algorithm (eq 3), the gear CI used were
index: Residual Kurtosis, Residual Crest Factor, Energy
Operator Kurtosis, Figure of Merit 0, Amplitude Modulation
RMS and Frequency Modulation RMS (index 2, 3, 5, 7 and
12, Figure 5).
These CIs based on a study (Bechhoefer 2011), with a
consideration of attempting of algorithm diversity. That is, it
was hypothesized that because there is no single CI that
works for every gear fault, it would be wise to use at least one
CI from each type of analysis. It should also be noted that for
(ref 12), the damage was scuffing, not a cracked tooth.
Because the of the result (Appendix E) indicated that the
Figure 3 Comparison of EO and its variants for good and Residual RMS, Narrowband Kurtosis, FM Kurtosis
bad pinion performance (e.g. separability) was improved, and that FM0
did not perform well, the CIs for the health algorithm were
The narrowband, amplitude modulation and frequency changed to Residual RMS, Energy Operator Kurtosis,
modulation analysis are consistently powerful. While it is not Narrowband Kurtosis, Amplitude Modulation and Frequency
as easy to visually identify the gear fault as the residual and Modulation Kurtosis, or index: 1, 5, 8, 10, 12, and 15, Figure
energy operator, the reduced bandwidth (e.g. noise) of the 5. Note that the final HI value changed from approximately
analysis greatly improve the separability score (Figure 4). 0.45 to 1.5, or almost 3x improvement in fault detection. It is

7
 ANNUAL CONFERENCE OF THE PROGNOSTICS AND HEALTH MANAGEMENT SOCIETY 2016

interesting to note that the separability of Residual RMS 8. CONCLUSION

(10.18) is only 41% improved over the Residual CF (5.94).
Few studies have asked the question: how good is a particular
The energy operator, was not a particularly powerful (the
gear analysis algorithm? Generally, the analysis is given on a
separability was 1.55), was left in because it may improve the
small set of data, with no comparison to other analysis. This
HI performance with other gear fault modes.
study, using a statistically significant data set, compared the
performance of gear CI analysis between a good and damage
population. The performance metric used was statistical
2.5

HI Indx: 1,5,8,10,12,15
HI Indx: 2,3,5,7,12,14 separability. Separability is a natural metric as condition
2 monitoring is in effect a decision support tool (e.g. hypothesis
testing), the decision being to bring an asset out of service for
inspection, or not. CI algorithms with a large separability,
1.5
will reduce the probability of a false alarm.
Health

For a decision support tool, this study uses the concept of the
1
health indicator (HI). The HI is a mapping of n CIs into a
common, threshold value which allow the maintainer to make
0.5
decision as to when to perform maintenance. When the
component HI is greater than 1, then it is appropriate to do
maintenance. Using the CIs with the largest separability
0
-16 -14 -12 -10 -8 -6 -4 -2 0
increased the sensitivity of the HI by 3x, or a 4.5 dB
Days improvement. Further, it was shown that using the HI (eq 3)
Figure 5 Comparison of the gear health performance based can give a 3 to 4.5 dB gain over using CIs alone.
on CI selection Caution should be taken in that, while the validation (e.g. HI
One question to ask would be: why not use a greater number improvement) was gained using a different gearbox type than
of the CIs with positive separability? This was tried, but after the original data set used for measuring separability (Table
normalizing, there was no increase in performance. That is, 2), the fault mode was similar: a fatigue crack on a tooth. It
the HI using 15 CI indexes did have a greater HI value. is likely that for scuffing/pitting type of faults, the optimal set
However, its starting (nominal HI) value was 0.275 vs. 0.174. of CIs would be different.
When the new HI was scaled by 0.174/0.275 or 0.64, there
was no observable change in the HI. It is suspected that an REFERENCES
orthogonal transformation, such as principal component Stewart, R.M., (1977). Some Useful Data Analysis
analysis, would indicate that using 15 CI was over Techniques for Gearbox Diagnostics, Machine Health
dimensioned, as using both Residual RMS, Kurtosis and CF Monitoring Group, Institute of Sound and Vibration
(as an example) are highly correlated (Figure 6). Research, University of Southampton, Report
MHM/R/10/77 July.
3
Bechhoefer, E., Kingsley, M., “A Review of Time
HI Using 15 CIs
HI Indx: 1,5,8,10,12,15 Synchronous Average Algorithms”, The Annual
2.5 HI Indx: 2,3,5,7,12,14 Conference of the Prognostics Health Management
Society, 2009.
2 McFadden, P.D., (1986), Detecting Fatigue Cracks in Gears
by Amplitude and Phase Demodulation of the Meshing
Vibration, J. Vib. Acout. Stress Reliab. Design, vol 10,
Health

1.5
no 2, Apr. pp 165-170.
1
Ma, J., (1995). Energy Operator and Other Demodulation
Approaches to Gear Defect Detection. Proceeding of the
MFPT 49, pp. 127-140.
0.5
Zakrajsek, J., “An Investigation of Gear Mesh Failure
Prediction Techniques”, NASA Technical Memorandum
0
-16 -14 -12 -10 -8 -6 -4 -2 0 102340, 1989.
Days
Zakrajsek, J., Decker, H., “An Analysis of Gear Fault
Detection Methods as Applied to Pitting Fatigue Failure
Figure 6 Comparison of HIs using 15 CIs vs. 6 CIs Data”, NASA Technical Memorandum, 105950, 1993.
Lewicki, D., Dempsey, P., Heath, G and Shanthakumaran, P.,
“Gear Fault Detection Effectiveness as Applied to Tooth

8
 ANNUAL CONFERENCE OF THE PROGNOSTICS AND HEALTH MANAGEMENT SOCIETY 2016

Surface Pitting Fatigue Damage” NASA/TM-2009- of 10 years in the USMC and USAF. He held various
215667, 2009. leadership roles within Airbus Helicopters Technical Support
Gopinath, K, Mayuran., M., Machine Design II, Gear Faults, and Engineering group. He is currently a Systems Engineer
Indian Institute of Technology Madras, at Garmin in the G5000H architectures. He received his B.S.
https://nptel.ac.in/courses/112106137/9, 2010. in Professional Aeronautics from Embry Riddle
Mendenhall, W., Wackerly, D., Scheaffer, R., Mathematical Aeronautical. He holds an FAA A&P license, FCC GROL
Statistics with Applications, Boston, PWS-Kent with Radar Endorsement and Amateur Radio License. He is
Publishing, 1990, pp 457 currently pursuing a B.S. in Data Management and Analytics.
Bechhoefer, E., He, D., Dempsey, P., "Gear Health Threshold
Setting Based On a Probability of False Alarm," Annual APPENDIX A
Conference of the Prognostics and Health Management
Society, 2011. Example Time Synchronous Average
Function [tsadata, navgs,rpm] = tsaLinearInterp(
Kellar, J., Grabill, P., “Vibration Monitoring of a HU-60A data, zct, sr, ratio, ppr)
Main Transmission Planetary Carrier Fault”, American %[tsadata,
navgs,rpm]=tsaLinearInterp(data,zct,sr,ratio,ppr,n
Helicopter Society 59th Annual Forum, Phoenix, AZ, avgs)
2003 %Inputs:
Bechhoefer, E., He, D., Dempsey, P., “Gear Health Threshold % data: time domain data in g's
Setting Based On a Probability of False Alarm”, Annual % zct: zero cross time
% sr: sample rate
Conference of the Prognostics and Health Management % ratio: gear ratio/pulse per revolution on
Society, 2011 the tach
Bechhoefer E., Bernhard, A., “Use of Non-Gaussian % ppr: pulse per rev
%Output:
Distribution for Analysis of Shaft Components”, IEEE % tsadata: time synchronous average data
Aerospace Conference, 2005. % navgs: the number of averages in the TSA
Caesarendra, W., Tjahjowidodo T., “A Review of Feature % rpm: mean shaft rpm
Extraction Methods in Vibration-Based Monitoring and %data = data - mean(data);
its Application for Degradation Trend Estimation of
Low-Speed Slew Bearing”, Machine 2017, 5, 21. ndata = length(data);
Mayo, E., Liang, M., Baddour, Na., “Gear Fault Detection dt = ndata/sr; %sample length
rev = 0;
with the Energy Operator and its Variants”, ASME 2016 i = 1+ppr;
International Design Engineering Technical Conference
and Computes and Information in Engineering while zct(i) < dt && i < length(zct)-1
rev = rev + 1;
Conference, Aug 21-24, 2016. i = i + ppr;
Bechhoefer, E., Zhang, X., Improved Fault Detection by end
Appropriate Control of Signal Bandwidth of the TSA,
% Define the number of averages to perform
Annual Conference of the Prognostics and Health navgs = floor(rev * ratio);
Management Society, 2015 trev = zct(navgs*ppr) - zct(1);
Wang, W., “Early Detection of gear Tooth Cracking Using rpm = navgs/trev*60*ratio;
the Resonance Demodulation Technique”, Mechanical
% Determine radix 2 number where # of points in
Systems and Signal Processing (2001), 15(5), 887-903 resampled TSA
McFadden, P.D., “Examination of a Techniques for the Early % is at sample rate just greater than fsample
Detection of Failure in Gears by Signal Processing of the N=(2^(ceil(log2(60/rpm*sr))));
Time Domain Average of the Meshing Vibration”, % now calculate times for each rev (1/ratio teeth
Mechanical Systems and Signal Processing 1, (1987) pass by)
% resample vibe data using zero crossing times to
interpolate the vibe
BIOGRAPHIES yy = zeros(1,N); %data to accumulate the resampled
signal once per rev
Eric Bechhoefer received his B.S. in Biology from the ya = yy; %ya is the resample signal once per rev
University of Michigan, his M.S. in Operations Research iN = 1/N; %resample N points per rev
from the Naval Postgraduate School, and a Ph.D. in General ir = 1/(ratio/ppr); %inverse ratio - how much to
advance zct
Engineering from Kennedy Western University. His is a tidx = 1; %start of zct index
former Naval Aviator who has worked extensively on
condition-based maintenance, rotor track and balance, while (floor(zct(tidx)*sr) == 0)
vibration analysis of rotating machinery and fault detection tidx = tidx + 1;
end
in electronic systems.
zct1 = zct(tidx);%start zct time;
Brent Butterworth has been in Aviation Rotorcraft
Maintenance and Engineering for 25+ years. He is a veteran for k = 1:navgs

9
 ANNUAL CONFERENCE OF THE PROGNOSTICS AND HEALTH MANAGEMENT SOCIETY 2016

tidx = tidx + ir; %get the zct for the % BW is bandwidth, usually 25% of gt.
shaft %Output:
% nb: narrow band signal
stidx = floor(tidx)-1; %start idx for % am: amplitude modulated signal
interpolation % fm: phase modulated signal
dx = tidx - stidx;
yo = zct(stidx); X = rfft(x);
dy = zct(stidx+1)-yo;
zcti = yo + dx*dy; %interpolated ZCT lw = gt-BW;
dtrev = zcti - zct1; %time of 1 rev %calculate the band pass indexes
dtic = dtrev*iN; %time between each sample hi = gt+BW + 2;
zct1c = zct1;
for j = 1:N X(1:lw) = 0;
cidx = floor(zct1c*sr); %idealized filter
yo = data(cidx); y1 = data(cidx+1); X(hi:end) = 0;
x1 = zct1c*sr;
xo = floor(x1); nb = irfft(X);
dx = x1-xo;
dy = y1-yo; n = length(nb);
yaj = yo + dx*dy; %simple linear interp n2 = n/2;
ya(j) = yaj; X = fft(x);
zct1c = zct1 + j*dtic; %increment to the %take the Hilbert Transform
next sample X(1:n2) = X(1:n2) * 2;
end X(n2:end) = 0;
h = ifft(X);
zct1 = zcti; %Analytic Signal

yy = yy + ya; %accumulate the tsa per reve % Amplitude Modulation signal - am

end am = abs(h);

tsadata = yy/navgs; % compute the average % Phase Modulation signal - fm

arg = unwrap(angle(h));
%take the argument
APPENDIX B fm = arg - (arg(end)-arg(1))*linspace(0,1,n);
function [xres] = residualSignal(x, geartooth) %take the derivate
%[xres] = residualSignal(x, geartooth)
%Inputs:
% x :input TSA signal
APPENDIX D
%geartooth :array with number of teeth on a gear function eo = energyOperator(x)
%from Vercer %eo = energyOperator(x);
x = x(:)'; %Inputs:
% x: the TSA
n = length(x); %Outputs:
n2 = n/2; % eo: the energy operator
nHarmonics = 3;
X = rfft(x); %real fft - no n = length(x);
conjugate data = x(:) - ones(n,1)*mean(x);
X(1) = 0; % DC is removed %Generate the x(n-1) and x(n+1) sequences
X(2) = 0; % SO1 is removed dataminus1=[data(end); data(1:end-1,:)];
X(3) = 0; % SO2 is removed dataplus1=[data(2:end); data(1)];
nGears = length(geartooth); %Calculate the E
for j = 1:nGears eo=data.*data-dataminus1.*dataplus1;
crtGear = geartooth(j);
for i = 1:nHarmonics
indx = 1+crtGear*i; APPENDIX E
if indx < n2 %projection
against running over the array Overall Separability Analysis Results
X(indx) = 0; %gear tooth meash
are removed Analysis Sep Analysis Sep
end
end AM P2P 15.564 CEEO IF 2.604
end SBLF 14.938 FM IF 2.592
xres = irfft(X); % residual signal from
AM SF 11.35 FM MF 2.586
the inverses real fft RES RMS 10.185 amRes Krt 2.498
AM IF 9.861 amResd CF 2.432
APPENDIX C AM MF 9.68 EO Skew 2.394
TRES RMS 8.982 CEEO MF 2.336
function [nb,am,fm] = narrowband(x, gt, BW)
%[nb,am,fm] = narrowband(x, gt, BW) TRES P2P 8.487 amRes CF 2.275
% x is the TSA NB MF 8.237 CEEO CF 2.273
% gt is the number of gear teeth and amResd SF 7.207 FM SF 2.199

10
 ANNUAL CONFERENCE OF THE PROGNOSTICS AND HEALTH MANAGEMENT SOCIETY 2016

amRes SF 6.417 CEEO Krt 2.089 SBLF - 2 4.827 EO CF 1.492

TRES MF 6.202 FM CF 2.088 amResd IF 4.804 FM0 1.348
TRES IF 6.188 EO SF 2.08 TRES Krt 4.721 amResdP2P 1.218
NB SF 6.109 EO123 SF 2.062 amResdSkw 4.623 Gear Mesh 1.149
FM Krt 5.966 EO IF 2.025 RES IF 4.401 amRes P2P 1.038
TRES CF 5.939 EO MF 2.008 RES MF 4.193 amResdRMS 0.976
FM P2P 5.926 NB RMS 1.978 amRes Skw 4.188 amRes RMS 0.87
AM RMS 5.914 EO123 IF 1.818 FM RMS 4 CEEO P2P 0.68
NB IF 5.791 EO123 SF 1.815 amRes MF 3.805 GM2/GM1 0.642
NB CF 5.747 EO123 MF 1.803 amRes IF 3.668 EO123 P2P 0.594
TRES SF 5.59 EO Krt 1.759 CEEO SF 3.661 EO P2P 0.583
FM Skw 5.49 RES P2P 1.698 AM CF 3.373 CEEO RMS 0.571
S Index 5.222 Gs RMS 1.643 RES SF 3.179 EO RMS 0.509
NB Krt 5.053 EO123 Krt 1.527 SBLF 3.168 EO123 RMS 0.508
RES CF 4.997 EO123 CF 1.514 NB P2P 3.15 Eng Ops 0.373
amResd MF 4.865 TRES Skw 1.503

11