Comparison of Vehicle-Based Crash Severity Metrics for Predicting
Comparison of Vehicle-Based Crash Severity Metrics for Predicting
2023
Doug Gabauer
Bucknell University, [email protected]
Luke E. Riexinger
Virginia Tech
Hampton C. Gabler
Virginia Tech
Recommended Citation
Dean, Morgan E.; Gabauer, Doug; Riexinger, Luke E.; and Gabler, Hampton C.. "Comparison of Vehicle-
Based Crash Severity Metrics for Predicting Occupant Injury in Real-World Oblique Crashes." (2023) :
505-518.
This Article is brought to you for free and open access by the Faculty Scholarship at Bucknell Digital Commons. It
has been accepted for inclusion in Faculty Journal Articles by an authorized administrator of Bucknell Digital
Commons. For more information, please contact [email protected].
1 Comparison of Vehicle-Based Crash Severity Metrics for Predicting Occupant Injury in Real-World
2 Oblique Crashes
3
4 Morgan Elizabeth Dean
5 Virginia Tech, Center for Injury Biomechanics
6 440 Kelly Hall, 325 Stanger Street (MC 0194), Blacksburg, VA, 24061
7 Email: [email protected]
8
9 Douglas J. Gabauer
10 Bucknell University, Department of Civil and Environmental Engineering
11 Lewisburg, PA, 17837
12 Email: [email protected]
13
14 Luke E. Riexinger
15 Virginia Tech, Center for Injury Biomechanics
16 440 Kelly Hall, 325 Stanger Street (MC 0194), Blacksburg, VA, 24061
17 Email: [email protected]
18
19 Hampton C. Gabler†
20 Virginia Tech, Center for Injury Biomechanics
21 445 Kelly Hall, 325 Stanger Street (MC 0194), Blacksburg, VA, 24061
22
23
24 Word count: 6,363 words text + 5 tables x 250 words each (1,250) + 8 figures = 7,613 words
25
26
27 Funding Acknowledgement: This research was sponsored by the National Academy of Sciences under
28 National Cooperative Highway Research Program (NCHRP) Project 17-90, “Evaluation of Roadside Crash
29 Injury Metrics in MASH.” The authors acknowledge the guidance of Edward Harrigan, program manager
30 for NCHRP 17-90, as well as the feedback provided by the project panel members.
31 Additional acknowledgements are stated at the end of the manuscript.
32
33 † Deceased.
34
35 26 April 2022
Dean, Gabauer, Riexinger, Gabler 2
1 ABSTRACT
2 The flail space model (FSM) is currently used in US roadside hardware crash testing as a means of assessing
3 occupant injury risk using observed vehicle kinematics data. European roadside hardware crash tests use a
4 FSM variant along with the acceleration severity index (ASI). While the FSM and ASI are currently used
5 in roadside hardware testing, other vehicle-based crash severity metrics exist. Previous research has focused
6 on examining the ability of these metrics to predict injury in frontal crashes. Despite MASH prescribing a
7 significant number of oblique crash tests, there has been little research on how well these metrics predict
8 real-world oblique crash injury.
9 This study compared the ability of six different vehicle-based metrics to predict occupant injury in oblique
10 crashes: maximum delta-v (MDV), occupant impact velocity (OIV), ridedown acceleration (RA), ASI,
11 occupant load criterion (OLC), and vehicle pulse index (VPI). The crash severity metrics were calculated
12 from real-world crash pulse data recorded by an event data recorder (EDR). Oblique crashes from the
13 National Automotive Sampling System Crashworthiness Data System (NASS/CDS) were used to train
14 logistic regression models that predict moderate to fatal injuries. The models were then compared on a
15 dataset of oblique crashes from the Crash Investigation Sampling System (CISS).
16 The results of this study confirm that vehicle-based metrics provide a reasonable means of predicting
17 real-world occupant injury risk in oblique crashes and suggest little difference between the investigated
18 metrics. In addition to the vehicle-based metrics, belt use and vehicle damage location were found to
19 influence injury risk.
20
21 Keywords: crash severity metrics, event data recorders, oblique crashes, injury risk curves
Dean, Gabauer, Riexinger, Gabler 3
1 INTRODUCTION
2 Roadside safety hardware, such as guardrails and concrete barriers, are crash tested according to the
3 American Association of State Highway and Transportation Officials’ (AASHTO) Manual for Assessing
4 Safety Hardware (MASH) crash test procedures (1). To assess occupant injury risk, MASH prescribes
5 threshold values based on the flail space model (FSM) (2). The FSM is a vehicle-based metric that uses
6 only measured vehicle kinematics to assess occupant injury risk.
7 Previous research examining the ability of the FSM and other alternative vehicle-based metrics to predict
8 occupant injury in real-world crashes has focused on the frontal crash mode (3–5). Although many MASH
9 tests prescribe an oblique impact, the FSM has yet to be evaluated in this crash mode using real-world crash
10 data. This study uses real-world crashes to examine the ability of the FSM to predict occupant injury in
11 oblique crashes. Alternative vehicle-based metrics are also assessed to determine if any provide a better
12 prediction of occupant injury compared to the FSM. Any metric that outperforms the FSM could potentially
13 be incorporated into a future version of MASH.
14 Crash Severity Metric Background
15 The six vehicle-based metrics analyzed in this study are defined below.
16 Maximum Delta-V (MDV)
17 MDV is the maximum change in the velocity of the vehicle due to the crash (6). Event data recorders (EDR)
18 are required to record a vehicle’s longitudinal crash pulse in the form of delta-v time series data, and the
19 maximum longitudinal delta-v (7). While EDRs are not required to record a vehicle’s lateral crash pulse
20 and lateral maximum delta-v (7), most EDRs began recording these elements in 2006.
21 Flail Space Model: Occupant Impact Velocity (OIV) and Ridedown Acceleration (RA)
22 The FSM, first proposed in 1981 as part of NCHRP Report 230 (8), assumes an occupant is an unrestrained
23 point mass acting as a “free-missile” inside the occupant compartment during a crash. The FSM assumes
24 there is initially 0.6 meters longitudinally and 0.3 meters laterally between the occupant and the vehicle
25 interior, i.e. the FSM boundaries. Injury is assumed a function of two metrics: the occupant impact velocity
26 (OIV) and the occupant ridedown acceleration (RA). OIV is the velocity of the occupant relative to the
27 vehicle velocity at the instant the occupant crosses a FSM boundary (3) and therefore represents the impact
28 between an unbelted occupant and the vehicle interior. RA is the largest 10-ms moving acceleration of the
29 vehicle after the occupant crosses a FSM boundary (1), which accounts for accelerations after the occupant
30 contacts the vehicle interior. Computed longitudinal and lateral OIV/RA values are compared to established
31 threshold values to determine if the level of occupant risk is acceptable.
32 For both lateral and longitudinal directions, the current preferred and maximum OIV thresholds are 9.1 m/s
33 and 12.2 m/s, respectively (1). These OIV thresholds apply to the majority of roadside hardware devices,
34 including all longitudinal barriers, but there are a separate set of more stringent thresholds for support
35 structures and work zone devices. For the longitudinal and lateral directions, the preferred and maximum
36 RA thresholds are 15.0 g and 20.49 g, respectively (1). Additional details on FSM assumptions and the basis
37 for the thresholds can be found in Gabauer and Gabler (4).
38 Occupant Load Criterion (OLC)
39 OLC models a belted occupant in a frontal crash. OLC assumes that the occupant first experiences a period
40 of ‘slack’, i.e. before engagement of the belt, during which the occupant translates at the initial pre-impact
41 velocity. During the slack period (from t=0 to t=t1), OLC assumes that the occupant translates 65 mm with
42 respect to the vehicle. Following the slack period there is a restraint period (from t=t1 to t=t2) where the
43 occupant is modeled as being subjected to restraint forces until the occupant translates 300 mm with respect
44 to the vehicle. OLC is the occupant’s constant rate of acceleration during the restraint period (from t=t1 to
45 t=t2) (6).
Dean, Gabauer, Riexinger, Gabler 4
14
15 Figure 1. Mass-spring-damper system used to calculate the vehicle pulse index.
16 Previous Work
17 Previous studies have used EDR data from real-world crashes to calculate vehicle-based metrics and
18 develop injury risk curves for frontal impacts (3, 5, 6). These studies have used the abbreviated injury scale
19 (AIS) to characterize occupant injury (11). AIS is advantageous over the police-reported injury scales as it
20 categorizes diagnosed injuries into six threat-to-life severity levels, one through six, as agreed upon by
21 medical experts (12). The highest severity injury an occupant suffers is referred to as the maximum AIS
22 (MAIS). Tsoi and Gabler (3) built MAIS3+F injury risk models using data for 334 sampled passenger
23 vehicle drivers in frontal crashes in the National Automotive Sampling System Crashworthiness Data
24 System (NASS/CDS). MAIS3+F indicates any occupant that sustained at least one severe to maximum
25 injury, i.e. AIS of 3 or greater, or was fatally injured. Models were built for MDV, OIV, and VPI.
26 Additionally, Gabauer and Gabler (5) built MAIS2+F and MAIS3+F models for MDV, OIV, and ASI. These
27 models were built using 180 occupants from NASS/CDS in frontal collisions. Both Tsoi’s and Gabauer’s
28 work analyzed belted and unbelted occupants separately. None of these studies, however, consider the
29 ability of vehicle-based metrics to predict oblique crash injury.
30 Objective
31 The purpose of this study was to compare the injury risk prediction capability of six vehicle-based crash
32 severity metrics using real-world oblique crash data.
33 METHODS
34 Injury risk models were developed using each of the six different vehicle-based crash severity metrics for
35 occupants in oblique NASS/CDS crashes, while controlling for potentially confounding factors. The crash
36 severity metrics were calculated using delta-v crash pulse data from EDRs. The developed models were
37 then tested on crashes from the Crash Investigation Sampling System (CISS). The performances of the
Dean, Gabauer, Riexinger, Gabler 5
1
2 Figure 2. To be considered an oblique crash, the occupant must have crossed neither or both FSM boundaries.
3 To cross both, the occupant could have crossed the boundaries simultaneously or crossed one before the other.
4 Calculating the Crash Severity Metrics Using EDR Data
5 To calculate each crash severity metric for a given vehicle, both the longitudinal and lateral delta-v crash
6 pulses were required. NHTSA specifies 100 Hz as the minimum sampling frequency for longitudinal and
7 lateral crash pulses (7). While most EDRs sample velocity change at a frequency of 100 Hz, the crash pulse
8 may be sampled at 250 or 500 Hz depending on the EDR manufacturer and/or module. The coarse sampling
9 rates is primarily a function of the EDR’s limited storage space, but the coarsely sampled velocity change
10 data is based on vehicle acceleration sampled at a much higher rate.
11 To compute resultant MDV, OIV, RA, and ASI, the resultant value was computed at each time point using
12 the longitudinal and lateral crash pulses, and the maximum resultant value was chosen. To compute resultant
13 OLC and VPI, the metric was computed independently in the longitudinal and lateral directions, and then
14 the resultant of the two values was computed. The resultant crash severity metric values were used to build
15 the models primarily for modeling convenience as the resultant value provides a single value representing
16 oblique crash severity.
17 Maximum Delta-V
18 MDV was calculated from the EDR crash pulse by manually finding the maximum velocity change. This
19 method was implemented to avoid using maximum delta-v values from incomplete and/or faulty crash
20 pulses. A crash pulse was considered incomplete if it was cut off while the vehicle was still negatively
21 accelerating. If the recorded delta-v crash pulse did not show a general trend of reduction in vehicle velocity,
22 it was assumed to be a faulty recording. The same method was used for longitudinal and lateral directions;
23 the longitudinal MDV used the vehicle’s longitudinal EDR crash pulse while the lateral MDV used the
24 vehicle’s lateral EDR crash pulse.
25 Flail Space Model: Occupant Impact Velocity and Ridedown Acceleration
26 To calculate OIV, the longitudinal and lateral crash pulses were integrated to obtain vehicle displacement
27 versus time. These data were then interpolated to determine the time values at which the occupant crossed
28 the FSM longitudinal and the FSM lateral boundaries. Of the two time values, the one that occurred first
29 (tfirst) was the time point used to calculate OIV. The longitudinal and lateral crash pulse data were
30 interpolated to calculate the occupant’s relative velocity at tfirst in both directions. The OIV was computed
31 as the resultant of those two velocity values. If neither FSM boundary was crossed, the resultant OIV value
32 was set equal to the resultant MDV value.
33 RA was computed using the longitudinal and lateral crash pulse segments subsequent to time point tfirst. For
34 each 10-ms segment, the longitudinal and lateral RA values were computed and the resultant value was
35 calculated. The maximum resultant value was used as the RA. If neither FSM boundary was crossed, the
36 resultant RA was set to zero. With the exception of using the resultant OIV and RA values, these
37 computations are identical to those prescribed by MASH.
38 Occupant Load Criterion
39 To calculate OLC, the longitudinal and lateral EDR crash pulses were integrated to obtain displacement.
Dean, Gabauer, Riexinger, Gabler 7
1 and interpolated to determine the time points corresponding to a displacement of 65 mm (t1). The velocity
2 values from time t1 and beyond were then integrated to determine the time point at which the occupant was
3 displaced an additional 235 mm. OLC was the calculated slope of the line between t1 and t2 on the crash
4 pulse, i.e. constant acceleration between the two time points. If the occupant did not reach a displacement
5 of 65 mm or 300 mm, OLC was set to zero (6). Therefore, if OLC was zero in only one direction, either the
6 lateral or longitudinal, the resultant value was equal to the OLC in the other direction.
7 Acceleration Severity Index
8 To calculate ASI, only the longitudinal and lateral accelerations were considered (Equation 2). The
9 longitudinal and lateral 50-ms moving average acceleration values (𝑎 , 𝑎 were calculated for the entire
10 crash pulse. For each 50-ms increment, the computed average accelerations were normalized by the
11 corresponding threshold for the appropriate direction (12 g for the longitudinal direction and 9 g for the
12 lateral direction) and an ASI value was computed. The largest of the computed ASI values was selected.
.
𝑎 𝑎
13 𝐴𝑆𝐼 2
12 9
29 The crash must have only one recorded event in both the EDR and NASS/CDS or CISS. In multi-
30 event crashes, it can be difficult to determine which event caused the injuries.
31 Only passenger vehicles were included. Passenger vehicles include cars, light trucks (includes pick-
32 up trucks and SUVs), and vans. Crashes were not limited to roadside hardware or fixed object
33 crashes as the link between vehicle-based criteria and occupant injury should be equally relevant
34 regardless of object struck. In vehicle-vehicle crashes, the body type of the other vehicle was not
35 restricted.
36 Only drivers and right front passengers were considered, since MASH and the FSM focus on front-
37 seated occupants.
38 Occupants were at least 13 years old to restrict the dataset to “adult size” occupants (15).
Dean, Gabauer, Riexinger, Gabler 8
1 The occupant was not ejected, as ejected occupants do not benefit from the enclosed vehicle
2 compartment.
3 The vehicle did not rollover. Occupants in rollovers experience different forces due to the motion
4 of the vehicle and MASH evaluation criteria for passenger vehicles does not permit rollover.
5 The vehicle recorded airbag system deployment or a longitudinal or lateral delta-v of at least 5 mph
6 (16, 17). These are the criterion that cause the EDR to lock in the data, preventing it from being
7 overwritten.
9 Both the lateral and longitudinal EDR crash pulses were available. This data was required to
10 identify oblique crashes, e.g. crashes where an occupant crossed both FSM or neither FSM
11 boundaries.
12 The longitudinal and lateral crash pulse data was deemed complete based on visual inspection. For
13 example, cases were excluded if one of its crash pulses did not appear to capture the entire crash
14 event. Additionally, time points before time zero and trailing zero points at the end of the crash
15 pulse were removed to accurately calculate the crash severity metrics. Cases were also excluded if
16 the crash pulse appeared to not accurately capture the crash event. For example, if the recorded
17 delta-v crash pulse did not show a general trend of reduction in vehicle velocity, it was assumed to
18 be a faulty recording and excluded.
19 Vehicle intrusion did not exceed any MASH intrusion limits (1). NASS/CDS and CISS report
20 intrusion for up to ten documented intrusions per vehicle. Intrusion data is captured via three
21 separate variables for each documented intrusion: (1) intrusion location, (2) intruding component,
22 and (3) intrusion magnitude range. The intruding component and magnitude ranges correspond
23 with the intrusion thresholds outlined in MASH. The location of the intrusion was used to determine
24 whether a driver or right-front passenger should be removed from the dataset. If intrusion exceeded
25 MASH limits only at the right-front passenger location, only the right-front passenger was
26 excluded. If intrusion exceeded MASH limits only at the driver location, only the driver was
27 excluded. More details regarding the MASH intrusion thresholds are discussed in an analysis by
28 Gabauer and Juengpraserstak (18).
29 Case weight was less than 5,000. This restriction was included to avoid a small number of sampled
30 cases dictating the results (19).
31 The predictor variables included in the final models were known for each case.
32 First, driver and right-front passenger NASS/CDS and EDR data were extracted from single-event
33 passenger vehicle crashes. This yielded 1,914 sampled occupants. Following this initial selection, the
34 remaining inclusion criteria were applied (Table 1). A total of 1,723 cases were removed. When the total
35 number of occupants does not change from one row to the next, no cases failed that specific inclusion
36 criterion. CISS selection was done by applying the same inclusion criteria used to form the NASS/CDS
37 dataset (Table 1).
Dean, Gabauer, Riexinger, Gabler 9
1 Table 1. Case selection criteria for the NASS/CDS training and CISS test datasets.
Occupants
Inclusion Criteria NASS/CDS (Training) CISS (Test)
Sampled Weighted Sampled Weighted
EDR Database 14,955 7,280,292 1,863 1,860,430
Drivers and right-front passengers in single-
event crashes with GAD at the front or side of 1,914 1,083,164 1,419 1,086,719
the vehicle
Complete lateral and longitudinal delta-v crash
1,117 537,399 1,288 1,007,999
pulses
Occupant crossed both or neither FSM
479 206,534 580 471,252
boundaries
If the occupant crossed neither FSM boundary,
the specific GAD location was front left, front 218 98,504 253 190,452
right, or at the front side of the vehicle
Vehicle did not roll over 218 98,504 252 190,273
Occupant was not ejected 217 98,418 251 190,183
Occupant age is at least 13 years old 213 98,147 250 189,594
Belt status was recorded by the EDR 202 91,461 250 189,594
Occupant compartment intrusion less than
200 90,752 238 185,675
MASH intrusion limits
Airbag system deployed or vehicle had an MDV
196 86,281 232 182,873
of at least 5 mph (longitudinal or lateral)
Final model predictor variables known 190 86,077 198 164,812
Case weight is less than 5,000 188 69,644 190 110,774
Final dataset 188 69,644 190 110,774
2 Injury Risk Modeling
3 The injury models were developed using logistic regression and the 1998 Abbreviated Injury Scale (AIS)
4 was used to determine injury severity (11). Any occupant with an MAIS2+F rating was placed into the
5 injury category, as the models were built to predict MAIS2+F injury. Occupants with an unknown injury
6 severity were excluded unless their injuries resulted in fatality. Fatally injured occupants were included in
7 the MAIS2+F category, regardless of MAIS.
8 Six oblique crash injury models were built to predict MAIS2+F injury presence. Each model used one of
9 the six crash severity metrics as an independent predictor variable and considered other factors that may
10 influence injury risk, as detailed below:
11 Crash Severity Metrics. All metrics were calculated as the resultant of the metric value in the
12 longitudinal and lateral directions. All metrics were modeled as continuous variables.
13 Belt Status. Belt status was a binary variable, where a value of 1 indicates an occupant using a
14 three-point belt restraint, and 0 indicates the occupant was unbelted. Belt status was determined
15 using the EDR belt status variable.
16 Age. Age was a binary variable, where 1 indicates at least 65 years old and 0 indicates at least 13
17 years old but less than 65 years old.
18 Sex. Sex was a binary variable, where 1 indicates male and 0 indicates female.
Dean, Gabauer, Riexinger, Gabler 10
1 BMI. Body mass index (kg/m2) was a binary variable, where 1 indicates obese (BMI ≥ 30 kg/m2)
2 and 0 indicates not obese (BMI < 30 kg/m2).
3 Occupant Seating Location. The occupant’s seating location was a binary variable, where a value
4 of 1 indicates the occupant is the driver, and 0 indicates the occupant was the right front passenger.
5 General Area of Damage (GAD). GAD describes the area of greatest deformation to the vehicle.
6 It was a binary variable, where a 0 indicates side damage and 1 indicates frontal damage.
7 Vehicle Type. This variable was defined using NHTSA’s Vehicle Body Type Classification (20).
8 All the vehicles in this dataset fall into the category of either ‘Passenger Car’ (PC) or ‘Light Trucks
9 and Vans’ (LTV). Vehicle type was a binary variable, where a value of 1 indicates a PC and 0
10 indicates an LTV.
11 First, logistic regression models were developed using each of the covariates listed above. These initial
12 regression models were examined to determine the statistically significant covariates. A second set of
13 regression models were then developed using only the statistically significant covariates from the initial
14 models. The models were then run on a dataset where only the significant covariates needed to be known,
15 as opposed to the initial dataset where every initial covariate needed to be known for each case. The
16 coefficients, p-values, and odds ratios associated with each of the final model covariates were reported. The
17 logistic regression models were developed using the survey package and SVYDESIGN function in R (21).
18 The R survey package was designed based on the Statistical Analysis System (SAS) and is capable of
19 computing confidence limits which account for the complex NASS/CDS and CISS stratified sampling
20 scheme (22). NASS/CDS and CISS oversample crashes resulting in occupant injuries, so it is necessary to
21 incorporate the case weights when performing analysis.
1 NASS/CDS and CISS weighted values were used to calculate each of these values.
1 β 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 𝑅𝑒𝑐𝑎𝑙𝑙
2 𝐹 6
β ∗ 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 𝑅𝑒𝑐𝑎𝑙𝑙
𝑇𝑃
3 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 7
𝑇𝑃 𝐹𝑃
𝑇𝑃
4 𝑅𝑒𝑐𝑎𝑙𝑙 8
𝑇𝑃 𝐹𝑁
5 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 𝑅𝑒𝑐𝑎𝑙𝑙
5 𝐹 9
4 ∗ 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 𝑅𝑒𝑐𝑎𝑙𝑙
𝑇𝑃 𝑇𝑁
6 𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 10
𝑇𝑃 𝑇𝑁 𝐹𝑃 𝐹𝑁
7 RESULTS
8 Dataset Composition
9 After filtering the data in accordance with the inclusion criteria, 188 sampled occupants (69,644 weighted
10 occupants) comprised the training dataset and 190 sampled occupants (110,774 weighted occupants)
11 comprised the test dataset for the final models (Table 2). The final training and test datasets did not require
12 available data for every initial covariate, as all covariates were not all included in the final models. As a
13 result, some of the data for some of the initial covariates was unknown for a subset of occupants.
Dean, Gabauer, Riexinger, Gabler 12
1 Table 3. Covariate coefficients, p-values, and odds ratios for the final models.
Model Coefficient P-Value Odds Ratio
MDV
Intercept -2.355 -- --
Resultant MDV 0.317 < 0.001** 1.37
Belt Status - Belted -1.813 0.010** 0.16
GAD - Frontal -1.863 0.004** 0.16
OIV
Intercept -2.963 -- --
Resultant OIV 0.396 < 0.001** 1.49
Belt Status - Belted -1.625 0.024** 0.20
GAD - Frontal -1.882 0.004** 0.15
RA
Intercept -0.658 -- --
Resultant RA 0.281 0.045** 1.32
Belt Status - Belted -2.230 0.002** 0.11
GAD - Frontal -1.548 0.039** 0.21
OLC
Intercept -1.244 -- --
Resultant OLC 0.139 0.002** 1.15
Belt Status - Belted -1.544 0.024** 0.21
GAD - Frontal -1.504 0.018** 0.22
ASI
Intercept -4.766 -- --
ASI 2.297 < 0.001** 9.94
VPI
Intercept -2.696 -- --
Resultant VPI 0.009 < 0.001** 1.01
Belt Status - Belted -1.564 0.028** 0.21
GAD - Frontal -1.285 0.045** 0.28
2 Injury Risk Curves
3 For every model besides the ASI model, an injury risk curve was built for both belted and unbelted
4 occupants in vehicles with frontal GAD and side GAD (Figure 3 through Figure 8). For the ASI model
5 (Figure 7), an injury risk curve was constructed for all occupants, since belt status and GAD were not
6 significant. The black vertical lines show the minimum and maximum metric values within the oblique
7 crash training dataset; data outside of the shown range was extrapolated by the corresponding model. The
8 shaded regions represent the 95% confidence intervals.
Dean, Gabauer, Riexinger, Gabler 14
1
2 Figure 3. MDV injury risk curves for front row occupants at least 13 years old in oblique crashes.
3
4 Figure 4. OIV injury risk curves for front row occupants at least 13 years old in oblique crashes.
5
6 Figure 5. RA injury risk curves for front row occupants at least 13 years old in oblique crashes.
Dean, Gabauer, Riexinger, Gabler 15
1
2 Figure 6. OLC injury risk curves for front row occupants at least 13 years old in oblique crashes.
3
4 Figure 7. ASI injury risk curves for front row occupants at least 13 years old in oblique crashes.
5
6 Figure 8. VPI injury risk curves for front row occupants at least 13 years old in oblique crashes.
Dean, Gabauer, Riexinger, Gabler 16
1 Model Comparison
2 The training data was used to determine a decision threshold for each model. These thresholds were used
3 to calculate precision, recall, and the resulting F2 score after the models were applied to the CISS test
4 dataset (Table 4). The accuracy of each model on the test dataset was also calculated.
5 The MDV, OIV, RA, OLC, and VPI final models shared the same set of covariates: the metric, belt status,
6 and GAD. The ASI model, however, had only ASI as a significant covariate. The reason for this difference
7 is not known from the available data but ASI is the only metric of the six investigated metrics that
8 normalizes values as part of the metric computation.
9 The final MDV, OIV, OLC, and VPI models yielded similar decision thresholds. RA and ASI had
10 comparatively high and low decision thresholds, respectively. RA and ASI also yielded the lowest test data
11 F2 scores, indicating these models have less predictive capability than the other models based on the
12 available data. OLC and MDV performed best on the test dataset with F2 scores of 0.61 and 0.57,
13 respectively. Overall, the models perform fairly well and, based on the recall scores, identified at least
14 two-thirds of the injury cases correctly.
15 OLC also yielded the highest accuracy of all the models, however it is important to note why accuracy is
16 not a reliable metric for an unbalanced dataset. Accuracy is a measure of how many predictions were correct,
17 and because the majority of the predictions are true negative injury cases, it is easy for the models to have
18 a high accuracy but poor precision, and therefore a low F2 score.
19 While RA was statistically significant, it is important to note the error associated with RA values computed
20 using EDR data. Based on six full-scale frontal rigid barrier crash tests where vehicle EDR data was also
21 available, Gabauer and Gabler (4) reported the EDR-computed longitudinal RA consistently overestimated
22 the observed RA by 40% on average. An analysis of 23 full-scale frontal rigid barrier crash tests as part of
23 this study found the EDR-based RA produced a root mean squared error of 93.7% with no trend toward
24 overestimating or underestimating the actual RA value. The coarse EDR velocity sampling rate (10 ms)
25 combined with the shorter overall recording time (300 ms maximum) contribute to the error associated with
26 the RA computations. While there is error associated with calculating the other metrics using EDR data, the
27 error is generally low. An analysis by Niehoff et al. (23) found that in full frontal crash tests, there was only
28 a 6 percent error associated with the MDV calculation and Gabauer and Gabler (4) reported an average
29 longitudinal OIV error of 4 percent.
30 Table 4. Precision, recall, and F2 scores for the six final? models on the CISS test dataset.
31 DISCUSSION
32 This is the first study to use vehicle-based crash severity metrics to develop injury risk curves for occupants
33 in oblique crashes. Additionally, this is the first study to analyze all six of these vehicle-based metrics in
34 parallel. The results of the study confirm that vehicle-based metrics provide a reasonable means of
35 predicting real-world occupant injury risk in oblique crashes, the primary crash mode for many roadside
36 hardware crash tests. The F2 scores decrease only minimally when moving from the training dataset results
Dean, Gabauer, Riexinger, Gabler 17
1 to the test dataset results. Additionally, the high recall values indicate the models detected the majority of
2 the injured occupants.
3 Injury Risk Curves
4 The injury risk curves exhibited three consistent patterns. For each model, occupant injury risk increased
5 as the vehicle-based metric value increased. This is consistent with the original FSM assumptions that
6 higher OIV and RA values result in a higher occupant injury risk. At the same metric value, unbelted
7 occupants had a consistently higher injury risk than belted occupants, when holding all other covariates
8 constant. Finally, at the same metric value, injury risk was consistently higher for occupant in vehicles with
9 GAD at the side of the vehicle, as opposed to occupants in vehicles with GAD at the front of the vehicle.
10 Comparing Current Thresholds
11 Since the inception of the FSM in 1981, the OIV and RA thresholds have remained essentially unchanged
12 despite developments in both passive and active safety as well as changes in occupant belt use rates. At the
13 time of the FSM development, belt use rate was less than 20 percent while belt use rates now exceed 80
14 percent (24). Therefore, it is necessary to reevaluate what level of occupant risk these thresholds allow for
15 a modern vehicle fleet.
16 The OIV and ASI models built in this paper were used to compute injury risk values associated with current
17 thresholds in roadside hardware testing (Table 5). The RA model was not included in Table 5 due to the
18 higher error associated with the EDR-computed RA value. This process was completed for the best-case
19 scenario occupant and again for the worst-case scenario occupant, as identified in Table 5. Note that the
20 best- and worst-case scenario occupants were the same for the ASI model. Note that the OIV and ASI values
21 in the models are resultant values, which is different than how MASH specifies the threshold values, i.e.
22 lateral and longitudinal values examined separately. With the resultant value, a given threshold can be
23 reached with a combination of lateral and longitudinal values below each corresponding threshold value.
24 Also note that the injury threshold in this study (MAIS2+F) is lower than the originally intended FSM
25 thresholds (MAIS4+).
26 Table 5. Occupant risk of injury associated with current roadside hardware testing thresholds.
Crash and Occupant Conditions Injury Risk at OIV and ASI Thresholds
Belt Status GAD OIV (m/s) ASI (--)
Scenario
(OIV only) (OIV only) 9.1 12.2 1.0 1.4 1.9
Best Case Belted Frontal 5.3% 16.2%
8.7% 19.2% 42.9%
Worst Case Unbelted Side 65.4% 86.6%
27 As expected, there was an increase in occupant injury risk as the thresholds increase in severity.
28 Additionally, the worst-case scenario occupants had a significantly elevated risk of injury, as compared to
29 the best-case scenario occupants. This suggests that the confounding factors, namely belt use and GAD,
30 substantially influence occupant injury risk. Since these confounding factors are not currently captured in
31 the current MASH procedures, the models developed herein can be used to analyze the injury risk associated
32 with varying occupant populations at varying thresholds. It is also important to note that the OIV and ASI
33 thresholds do not represent a uniform level of MAIS2+F occupant injury risk.
34 The developed injury risk models were also used to compute occupant risk for previously conducted oblique
35 roadside safety hardware crash tests. Longitudinal and lateral OIV data was acquired for 55 roadside barrier
36 MASH crash tests, i.e. MASH test designations 3-10 and 3-11. All of the tests either satisfied all MASH
37 requirements or satisfied all MASH requirements except for the OIV threshold value(s). Each of the 55
38 crash tests either passed the test or failed due to the OIV occupant risk criteria, i.e. a lateral or longitudinal
39 OIV greater than 12.2 m/s. Two tests did not satisfy the MASH OIV limits: one test had a lateral OIV of
40 12.44 m/s while the other had a longitudinal OIV of 12.58 m/s. The provided longitudinal and lateral OIV
Dean, Gabauer, Riexinger, Gabler 18
1 values were used to compute the resultant OIV for each of the tests. Then, the developed oblique crash
2 injury risk model was used to compute the risk of an MAIS2+F injury. To compute the risk, it was assumed
3 the occupant was belted. Additionally, it was assumed GAD at the vehicle front, since this is the case for
4 most MASH tests. The MAIS2+F risk range was 0.5% to 32.9%. The risk associated with the failing tests
5 were 32.9% and 18.8%, respectively. These two cases yielded the two highest risk values. Excluding these
6 cases, the maximum MAIS2+F risk was 17.0%. The MAIS2+F risk average was 5.89%. Excluding the two
7 failing cases, the average risk was 5.13%.
8 LIMITATIONS
9 One limitation of this study was the relatively small number of oblique crash cases available for analysis.
10 EDR data was relatively limited as NASS/CDS only samples approximately 5,000 crashes per year and less
11 than half of these have associated EDR downloads available. Additionally, the inclusion criteria required
12 the availability of both a longitudinal and a lateral crash pulse, which further restricted the size of the dataset.
13 Working with this limited amount of data meant a relatively small number of injury cases available. As a
14 result, the developed models used the MAIS2+F threshold even though the FSM was originally intended to
15 indicate the transition between an AIS 3 and an AIS 4 injury severity (25). Since the models were
16 constructed for a lower AIS threshold, they yielded higher injury risk values relative to risk values that
17 would be yielded by an MAIS4+F prediction model. Despite using a lower AIS threshold, though, the
18 comparisons across the investigated metrics are still helpful for identifying metrics that provide a better
19 injury prediction. In addition to the limited availability of injury cases, there were too few cases available
20 to draw conclusions on correlations between roadside hardware safety crashes and injury risk. Of the 188
21 sampled NASS/CDS cases, 176 of them were collisions with other vehicles. Of the twelve vehicle-object
22 crashes, four were with roadside safety hardware. Of the 190 sampled CISS cases, 166 were collisions with
23 other vehicle, and six of the 24 vehicle-object crashes were with roadside safety hardware.
24 Additionally, each of the metrics analyzed in this study make varying assumptions about the occupant’s
25 belt status. MDV, OIV, and RA assume the occupant was unbelted, while OLC, ASI, and VPI assume the
26 occupant was belted. Despite these assumptions, both belted and unbelted occupants were used to build
27 every model. VPI, in this study, also uses the same vehicle stiffness value and mass to calculate longitudinal
28 and lateral VPI, despite having the ability to be adapted to vehicle variance. An important constant in the
29 model’s governing equation is the natural frequency, which is the square root of the ratio of vehicle stiffness
30 (2,500 N/m) to occupant mass (1 kg). Using the recommended values, the natural frequency is 500 seconds,
31 which is a reasonable assumption for a real-world frontal crash, but may not be representative of a side
32 crash. Finally, VPI and OLC were developed specifically for frontal crashes, and not oblique impacts.
33 There is also error associated with computing vehicle-based metrics for real-world crashes using EDR data.
34 The magnitude of this error, however, has only been investigated extensively for the frontal crash mode (4,
35 23). With the exception of RA, these frontal crash studies generally report that EDR-computed crash
36 severity metrics are reasonably accurate, within 10 percent of the values computed using lab grade
37 instrumentation. A single study (23) reported a lateral delta-V error of 18 percent based on one lateral impact
38 and two offset frontal crash tests. The authors, however, attributed this relatively large error to differences
39 between the EDR and lab grade instrumentation locations and noted the reported error was likely a flawed
40 estimate (23). Although the accuracy of computing vehicle-based metrics from EDR data in oblique crashes
41 is currently not quantified, the good agreement shown for vehicle-based metrics in the frontal crash mode
42 suggest there is likely reasonable accuracy for the majority of vehicle-based metrics in the oblique crash
43 mode. Differences in EDR locations within the vehicle occupant compartment across different vehicles,
44 however, would likely introduce higher errors for oblique crashes compared to either purely frontal or
45 purely side impact crashes.
46 Finally, there are slight differences between the NASS/CDS and CISS databases. For one, CISS codes injury
47 using AIS 2015 codes, whereas CDS codes injury using AIS 1998 and AIS 2008. AIS 1998 was chosen to
Dean, Gabauer, Riexinger, Gabler 19
1 construct these models, as there were case years prior to 2008 included in the dataset. Additionally, while
2 CDS sampled vehicles towed from the scene due to crash damage, CISS samples vehicles that were towed
3 from the crash scene for any reason. As more CISS cases are released, future work could include using
4 CISS cases alone to train and test the model. This would ensure that the models capture the current vehicle
5 fleet as well as current passive safety systems. It would also remove the differences in AIS coding and
6 sampling methods.
7 FINDINGS
8 Based on the development and comparison of the oblique crash models for six different vehicle-based
9 metrics, the findings of the study are summarized below:
10 1. To date, the validation of the vehicle-based metrics using real-world crashes has focused almost
11 exclusively on purely frontal crashes. The results of this study confirm that vehicle-based metrics
12 provide a reasonable means of predicting real-world occupant injury risk in oblique crashes, the
13 primary crash mode for many roadside hardware crash tests.
14 2. There was little difference found in the ability of the investigated vehicle-based metrics to predict
15 occupant injury in oblique crashes. For the training data, there is a 15% or less difference in F2
16 scores among all six metrics. For the test data, the variation is greater than 15%, but still less than
17 25%. While OLC and MDV performed best for the test data, OIV and MDV had the highest scores
18 of the training dataset. This suggests that changing the current vehicle-based MASH occupant risk
19 procedures to use an alternate vehicle-based metric are not warranted.
20 3. Confounding factors, i.e. other than the value of the vehicle-based metric, were found to
21 significantly influence occupant injury risk. For oblique crashes, the most influential confounding
22 factors were occupant belt use and vehicle damage location. While it may not be practical to include
23 these other factors directly in the MASH occupant risk criteria, the developed statistical models
24 could be included as an appendix to MASH to provide users with an ability to compute injury risk
25 for various vehicle-based metric and confounding factor values. Computing injury risk could help
26 agencies make better-informed decisions about roadside safety hardware selection.
27 4. The test dataset suggests the OLC may be better suited for predicting injury in oblique crashes
28 compared to the OIV. While additional injury cases are needed to confirm this finding, an advantage
29 of the OLC metric is that it models a belted occupant, much more representative of occupants
30 present in the current vehicle fleet.
31 ACKNOWLEDGEMENTS
32 This research was sponsored by the National Academy of Sciences under National Cooperative Highway
33 Research Program (NCHRP) Project 17-90, “Evaluation of Roadside Crash Injury Metrics in MASH.” The
34 authors acknowledge the guidance of Edward Harrigan, program manager for NCHRP 17-90, as well as the
35 feedback provided by the project panel members.
36 We would also like to thank the staff of the Midwest Roadside Safety Facility (MwRSF), the Texas
37 Transportation Institute (TTI), and the Federal Highway Administration (FHWA) Federal Outdoor Impact
38 Laboratory (FOIL) for provision of the electronic crash data from previously conducted MASH tests.
39 A special thank you goes to the late Dr. Clay Gabler (deceased: January 11, 2021). His contributions were
40 integral to this work, and his mentorship and friendship are dearly missed.
1 Riexinger; draft manuscript preparation: M.E. Dean, D.J. Gabauer, L.E. Riexinger. All authors reviewed
2 the results and approved the final version of the manuscript with the exception of H.C. Gabler who passed
3 away unexpectedly.
Dean, Gabauer, Riexinger, Gabler 21
1 REFERENCES
2 1. American Association of State Highway and Transportation Officials. Manual for Assessing Safety
3 Hardware Second Edition. 2016.
4 2. National Highway Traffic Safety Administration. Federal Motor Vehicle Safety Standards: Occupant
5 Crash Protection - Supplemental Notice of Proposed Rulemaking. 2004.
6 3. Tsoi, A. H., and H. C. Gabler. Evaluation of Vehicle-Based Crash Severity Metrics. Traffic Injury
7 Prevention, Vol. 16 Suppl 2, 2015, pp. S132-139. https://doi.org/10.1080/15389588.2015.1067693.
8 4. Gabauer, D., and H. C. Gabler. Methodology to Evaluate the Flail Space Model by Using Event Data
9 Recorder Technology. Transportation Research Record, Vol. 1890, No. 1, 2004, pp. 49–57.
10 https://doi.org/10.3141/1890-06.
11 5. Gabauer, D. J., and H. C. Gabler. Comparison of Roadside Crash Injury Metrics Using Event Data
12 Recorders. Accident Analysis & Prevention, Vol. 40, No. 2, 2008, pp. 548–558.
13 https://doi.org/10.1016/j.aap.2007.08.011.
14 6. Wusk, G., and H. Gabler. Evaluation of Vehicle-Based Crash Severity Metrics Using Event Data
15 Recorders. 2017.
16 7. NHTSA, DOT. 49 CFR Part 563: Event Data Recorders, Final Rule. 2006, p. 207.
17 8. Michie, J. D. Recommended Procedures for the Safety Performance Evaluation of Highway
18 Appurtenances. Publication NCHRP Report 230, TRB, National Research Council. Washington, D.C.,
19 1981, p. 48.
20 9. CEN. Road Restraint Systems – Part 2: Performance Classes, Impact Test Acceptance Criteria and
21 Test Methods for Safety Barriers. European Standard EN 1317-2. 2010.
22 10. ISO/TR. Road Vehicles – Traffic Accident Analysis – Part 3: Guidelines for the Interpretation of
23 Recorded Crash Pulse Data to Determine Impact Severity. 2013.
24 11. Association for the Advancement of Automotive Medicine. The Abbreviated Injury Scale 1990
25 Revision Update 98. 1998.
26 12. Burdett, B., Z. Li, A. R. Bill, and D. A. Noyce. Accuracy of Injury Severity Ratings on Police Crash
27 Reports. Transportation Research Record, Vol. 2516, No. 1, 2015, pp. 58–67.
28 https://doi.org/10.3141/2516-09.
29 13. Radja, G. National Automotive Sampling System, Crashworthiness Data System - 2015 Analytical
30 User’s Manual. Publication DOT HS 812 321. 2016, pp. 1–152.
31 14. NHTSA, DOT. Crash Investigation Sampling System: 2018 Analytical User’s Manual. Publication
32 DOT HS 812 958. 2020.
33 15. Bareiss, M., and H. C. Gabler. Estimating near Side Crash Injury Risk in Best Performing Passenger
34 Vehicles in the United States. Accident Analysis & Prevention, Vol. 138, 2020, p. 105434.
35 https://doi.org/10.1016/j.aap.2020.105434.
36 16. Scanlon, J., K. Page, R. Sherony, and H. Gabler. Using Event Data Recorders from Real-World Crashes
37 to Investigate the Earliest Detection Opportunity for an Intersection Advanced Driver Assistance
38 System. Presented at the SAE 2016 World Congress and Exhibition, SAE Technical Paper 2016-01-
39 1457, 2016.
40 17. Gabler, H. C., A. Tsoi, J. Hinch, R. Ruth, D. Bowman, and M. Winterhalter. Light-Vehicle Event Data
41 Recorder Technologies. Publication Report No. DOT HS 812 929. National Highway Traffic Safety
42 Administration, 2020.
Dean, Gabauer, Riexinger, Gabler 22
1 18. Gabauer, D. J., and S. Juengprasertsak. Examination of Current MASH Occupant Compartment
2 Intrusion Limits Using Real-World Crash Data. Transportation Research Record: Under Review.
3 19. Kononen, D. W., C. A. C. Flannagan, and S. C. Wang. Identification and Validation of a Logistic
4 Regression Model for Predicting Serious Injuries Associated with Motor Vehicle Crashes. Accident
5 Analysis & Prevention, Vol. 43, No. 1, 2011, pp. 112–122. https://doi.org/10.1016/j.aap.2010.07.018.
6 20. National Highway Traffic Safety Administration. Fatality Analysis Reporting System (FARS):
7 Analytical User’s Manual, 1975-2017 (2018).
8 21. Lumley, T. Package “Survey”: Analysis of Complex Survey Samples. 2020.
9 22. Statistical Analysis Software, SAS/STAT. https://www.sas.com/en_us/software/stat.html. Accessed Jul.
10 27, 2021.
11 23. Niehoff, P., H. C. Gabler, J. Brophy, C. Chidester, J. Hinch, and C. Ragland. Evaluation of Event Data
12 Recorders in Full Systems Crash Tests. Proceedings of the 19th International Conference on Enhanced
13 Safety of Vehicles. Paper No. 05-0271-O (2005).
14 24. Hedlund, J., S. H. Gilbert, K. Ledingham, and D. Preusser. How States Achieve High Seat Belt Use
15 Rates. Publication DOT HS 810 962. 2008.
16 25. Michie, J. D. Collision Risk Assessment Based on Occupant Flail-Space Model. Transportation
17 Research Board, No. 796, 1981, pp. 1–9.
18