Applied Mathematical Modelling 102 (2022) 137–152

Contents lists available at ScienceDirect

Applied Mathematical Modelling

journal homepage: www.elsevier.com/locate/apm

A new maximum entropy method for estimation of

multimodal probability density function
G. Li, Y.X. Wang, Y. Zeng∗, W.X. He
Department of Engineering Mechanics, State Key Laboratory of Structural Analysis for Industrial Equipment, Dalian University of
Technology, Dalian 116024, China

a r t i c l e i n f o a b s t r a c t

Article history: The high-precision estimation of a multimodal probability density function is a difficult
Received 14 December 2020 problem in many engineering fields. We propose a new method to improve the estimation
Revised 12 September 2021
accuracy based on the fractional moment-based maximum entropy method with a non-
Accepted 23 September 2021
linear transformation and a multi-peak recognition method. For the translation parameters
Available online 2 October 2021
in the nonlinear transformation, three approaches, such as sample-based least square poly-
Keywords: nomial fitting technique, sample-based kernel density estimation and classical maximum
Maximum entropy method entropy method, are presented to determine the parameters. By adjusting the translation
Fractional moment parameter, the valley of the probability density function curve can be translated to the po-
Nonlinear transformation sition with a larger slope, and the distance between adjacent peaks is enlarged to avoid
Multimodal probability density estimation the wrong fitting form of the probability density function curve with multiple peaks. After
the parameters of the transformation are obtained, the fractional moment-based maximum
entropy method is applied to predict the probability density function of the transformed
performance function. Two numerical examples are used to verify the accuracy and sta-
bility of the proposed method. Two engineering examples are introduced to illustrate the
applicability and efficiency of the proposed method in the real-life engineering setting. It is
concluded that the proposed method uses fewer moments with less additional calculation
costs, and has good computational efficiency and applicability for the modeling of multi-
modal probability density functions, compared with the classical fractional moment-based
maximum entropy method.
© 2021 Elsevier Inc. All rights reserved.

1. Introduction

In 2013, Zhang and Pandey [1] proposed the fractional moment-based maximum entropy method (FM-MEM), which
predicts the probability density function (PDF) of the performance function based on statistical moments. The principle of
FM-MEM is to find the PDF with the maximum entropy from many PDFs satisfying the given moment constraints, because
the maximum entropy indicates the prediction of PDF is an unbiased estimation with the highest degree of randomness.
Without introducing prior assumptions, the FM-MEM overcomes the theoretical shortcomings of other classical moment-
based methods. Compared with the classic integer moment maximum entropy method (IM-MEM) [2], FM-MEM is more
flexible in the selection of moment order, which brings an improvement in accuracy and therefore, receives widespread
attention. In recent years, many scholars have conducted in-depth research on various aspects of the FM-MEM.

∗
Corresponding author.
E-mail address: [email protected] (Y. Zeng).

https://doi.org/10.1016/j.apm.2021.09.029
0307-904X/© 2021 Elsevier Inc. All rights reserved.
G. Li, Y.X. Wang, Y. Zeng et al. Applied Mathematical Modelling 102 (2022) 137–152

For the determination of moment constraints, Xu and Kong [3] developed an adaptive scaled unscented transformation
to obtain the fractional moments with only a few of sample evaluations. Based on the generalized dimension-reduction
method, He et al. [4] proposed a hybrid dimension-reduction method, which has high calculation accuracy for cases where
the variables have significant correlation. Xu et al. [5] presented a rotational quasi-symmetric point method (RQ-SPM) to
renew the fractional moment constraints of MEM with high computational efficiency. To improve the efficiency and accuracy,
Xu and Dang [6] proposed the good lattice point method based partially stratified sampling, which weighs the efficiency and
accuracy of high-dimensional reliability problems. Li et al. [7] proposed to combine the Laplace transform with the MEM to
improve the optimization strategy and considerably reduce the calculational cost. Xu et al. [8] determined the initial values
of Lagrange multipliers and moment orders by solving nonlinear equations, and then searched more accurate solutions in
its vicinity to improve the calculation accuracy. The method is also effective for high-dimensional reliability analysis. For
different practical problems, Gzyl et al. [9] used FM-MEM to determine the probability of ultimate ruin as a function of the
initial surplus. Yu et al. [10] combined the extreme moment method and an improved MEM to solve problems with multiple
failure modes and time parameters. Zhang et al. [11] employed the FM-MEM to solve the problem of probabilistic lifetime
modeling of the dynamical and discontinuous stochastic systems.
In fact, fitting the PDF accurately is very challenging when only limited moment information is available, especially for
the multimodal PDF. Although many scholars have conducted extensive and in-depth research on the MEM, the method
is mostly applied to solve the unimodal PDF. When the common MEM is used to model a multimodal PDF, the predicted
failure probability may have significant errors. Actually, modeling multimodal PDFs is needed in many engineering fields.
For example, the PDF of fatigue stress in a steel bridge structure is a bimodal distribution [12], the axle load of a heavy
haul vehicle [13] and the vibrating load of turbine generator blades [14] are random variables with multimodal distribu-
tions. Therefore, some additional processing is required to obtain a more reasonable fitting result for those problems [15].
Jiang [16] proposed a bimodal estimation form of PDFs from normal distributions with unknown parameters when the dis-
tance between two peaks of PDF is appropriate. He et al. [14] proposed an uncertain propagation method combining the
Laplace transform and the first-order second-moment method and solved the reliability analysis problem involving multi-
modal distributions when the first-order reliability method is applicable and accurate. Hu and Du [17] studied the saddle
point approximation of the mean value from the bimodal distributed random function to estimate the failure probability.
Zhang et al. [18] improved the MEM for the bimodal PDF estimation by introducing a new type of the maximum entropy
convergence criterion with the high-order moments. In addition, the FM-MEM [19] has been used to solve some prob-
lems with bimodal PDFs profited from more statistical information of the fractional moment, although increasing the num-
ber of fractional moments leads to more computation efforts for an accurate prediction of a bimodal PDF with two close
peaks.
To increase the efficiency and accuracy of FM-MEM for the problem of multimodal PDF estimation, we propose an im-
proved FM-MEM by introducing a specified nonlinear transformation of the performance function. Through the transfor-
mation, the sampling density in the original space can be rearranged and the multimodal characteristics of the probability
distribution of the samples become more obvious, which facilitates reducing the modeling difficulty and improving mod-
eling accuracy. The proposed method significantly enhances the performance of multimodal recognition, without changing
PDF estimation form or increasing the number of terms of its exponential polynomial; therefore, no additional undetermined
parameters are needed. Moreover, some tricky problems, such as high-order moment variability and function oscillations can
be also avoided. Three types of algorithms, including pre-computation with the MEM, the least squares polynomial fitting,
and the kernel density estimation, are provided for different situations to determine the unknown parameter of the trans-
formation. Several typical examples are tested to verify the validity of the proposed method, compared with the original
FM-MEM and the Monte Carlo simulation.

2. A new FM-MEM for multimodal PDF estimation

In this paper, we propose a new FM-MEM for multimodal PDF estimation by introducing a specified nonlinear transfor-
mation, which leads to a convenient way to capture the property of multiple peaks in the PDF curve. The estimation of the
PDF can be completed with high accuracy by using fewer fractional moments. The algorithm of the new FM-MEM has three
steps to estimate the multimodal PDF: nonlinear transformation of the performance function, determination of translation
parameters, and PDF estimation.
In the first step, the original performance function is transformed into a new form, and then the blurred and clustered
multiple peaks in the PDF of the original performance function can be pulled away to increase the capacity of peak recog-
nition in the PDF of the transformed performance function.
In the second step, three different algorithms are presented to determine the unknown parameters in the nonlinear
transformation function.
In the last step, the standard procedure of the FM-MEM is applied to estimate the PDF of the transformed performance
function. Then, an inverse transformation of the estimated PDF is conducted to obtain the approximate PDF of the original
performance function, matching the given nonlinear transformation in the first step.

138
G. Li, Y.X. Wang, Y. Zeng et al. Applied Mathematical Modelling 102 (2022) 137–152

Fig. 1. Mode comparison of the real PDF and the estimated one from FM-MEM.

2.1. Nonlinear transformation

Although the FM-MEM can improve the accuracy of PDF prediction, it still faces some limitations. Firstly, the upper
and lower bounds of the performance function is difficult to determine accurately; therefore, the value of the performance
function is usually assumed to be contained within the infinite interval. However, a truncation error is inevitable during the
integration, causing a negative effect on the result of FM-MEM. Moreover, when the fractional moments are considered, the
performance function must be non-negative. Adding a large number to the function may shift the performance function to
the non-negative range [20], but it lacks generality and may fail in some cases.
The other limitation of FM-MEM is that the PDF predicted from the FM-MEM often blurs multiple peaks of the real
PDF together when the peaks are close to each other, although the undetermined form of PDF used in the FM-MEM is not
limited to the unimodal problem.
As shown in the Fig. 1, in the calculation process of FM-MEM, K-L (Kullback Leibler) distance needs to be introduced
as Eq. (1) to simplify the calculation [21], the estimated PDF q(x) often blurs multiple peaks of the real PDF p(x) together,
so that the mass of high probability is more uniformly placed on all peaks, instead of accurately fitting the multimodal
situation. This limitation of the FM-MEM has also been introduced on page 74 of the book Deep Learning [22] as follows:
“In this case, we select a q that has high probability where p has high probability. When p has multiple modes, q chooses to blur
the modes together, in order to put high probability mass on all of them.”
 ∞
K [ p( x ) , q ( x ) ] = p(x ) ln [ p(x )/q(x )]dx (1)
−∞

There are two ideas for dealing with multimodal PDF estimation problems by using the FM-MEM at present. One is to
increase the number of the exponential polynomial terms in the pending form of PDF [18], because the real PDF can be
approached infinitely by continuously increasing the number of the terms of the estimation theoretically. However, more
moment constraints are required with the increasing number of terms, which may cause oscillations in the predicted PDF
and expensive calculation cost in the double-loop optimization algorithm of the FM-MEM. Therefore, higher accuracy and
suitable order of the statistical moments are needed. Another idea is to construct a proper multimodal expression of the
estimated PDF in advance [16], but it increases the number of unknown parameters and the computational complexity.
In this paper, a novel way is proposed to deal with the problem of multimodal probability density modeling by
transformation of the performance function based on the work of Li et al. [23]. An improved form of the nonlin-
ear transformation function is proposed, which contains two undetermined parameters, namely, translation and scal-
ing parameters, to change the form of performance functions more suitable for fitting the multimodal PDF. Due to
the nonlinear characteristics, the sparsity of the samples will be redistributed after the transformation, which makes
the location of the valley more obvious, and the distance between adjacent peaks in the multimodal PDF increased,
thereby enhances the recognition capacity of multimodal peaks significantly. The specific form of the nonlinear trans-
formation used in this paper is given below:
N peak −1
 1
1   1   Y − pξ
Z = F (Y ) = arctan Y  ξ + ,Yξ = , N peak ≥ 2 (2)
ξ =1
N peak − 1 π 2 kσ

where pξ is the translation parameter, k is the scaling parameter, and Npeak is the number of peaks.
The translation parameter pξ corresponds to the position of the valley in the multimodal PDF curve. By adjusting pξ , a
part of the valley can be shifted to a position with a larger slope in the PDF curve of the nonlinear transformation, which

139
G. Li, Y.X. Wang, Y. Zeng et al. Applied Mathematical Modelling 102 (2022) 137–152

increases the distance between two adjacent peaks. The translation parameter needs to be determined by some specific
methods which will be provided in section 2.2.
The scaling parameter can be used to adjust the order of magnitude of the performance function. Generally speaking,
based on experience, k=1.5 is a reasonable choice for double peaks. But as the number of peaks increases, the difficulty of
fitting increases significantly, and we need to make the peaks more discrete in order to obtain the desired results. There-
fore, for the three-peak situation, we use k=0.8. When the number of peaks is further increased, k can be further reduced
accordingly.

2.2. Determination of translation parameters

The reasonableness of the translation parameters introduced in Eq. (2) is a crucial issue, whose value needs to be de-
termined accurately and efficiently. The principle of determining its value is based on a rough pre-estimation of the PDF to
judge the position of valley in this pre-estimated PDF. Here, we present three methods to make such rough pre-estimation.
The method can be selected flexibly according to the prior information. When the random distribution of the input
random variable is known and the response is known as bimodal, we recommend using the pre-calculated MEM method
to determine the translation parameter p. This method requires only a few samples, and these samples can be reused in
subsequent calculations without increasing the number of calls of performance functions. In addition, the method has good
robustness and not sensitive to interference information. The obtained p is stable.
When there is no specific distribution information of the input random variable, the other two sample-based methods,
the least squares fitting and the kernel density estimation, can be used to determine the translation parameter p. The least
square fitting requires a pre-judgment on the domain of the performance function. If the performance function is defined
in an infinite interval, the performance function needs to be normalized, or the definition domain should be truncated
appropriately. If the domain of the performance function cannot be pre-judged effectively, we recommend using the kernel
density estimation to determine the translation parameters. The kernel density estimation does not have such restrictions,
but its bandwidth needs to be selected because different bandwidths may have impact on the results. The effects of these
three methods are compared in Example 1.

2.2.1. Pre-computation with MEM

To make a rough estimate of the bimodal PDF, we use the classical MEM (IM-MEM) [2] as the preferred method. Based
on the integer moment constraints, a rough but rapid estimation of the PDF can be obtained using this method, which can
present a more intuitive estimation of the valley position.
As stated in the previous section, there are some limitations of the concept of K-L divergence in dealing with multimodal
problems. For example, when the classical MEM is used to fit the bimodal PDF, only a single peak result can be obtained,
and the peak is usually located in the valley position of the bimodal PDF curve. Therefore, the position of the PDF peak
estimated from the classical MEM pre-calculation process can be assumed as the location of the valley of the true bimodal
PDF. During the pre-calculation process, it is not necessary to find the location of the valley accurately, because the effect of
the nonlinear transformation used here can cover the range with large slope of PDF near the valley.
The procedure of the classical MEM used here is shown in Fig. 2.
The optimization formula of the IM-MEM is the same with Eq. (1). In the proposed method, α i =i, and only the first
i-order integer moments are used as constraints. By using the Lagrangian multipliers, the nonlinear equations about λ can
be constructed from the moment constraints. When λ is solved by the iteration method, a pre-estimation of the PDF can be
obtained. Although the PDF is usually in the form of a single peak, y, the position of the peak can be used as the value of
translation parameter p for subsequent calculations.

2.2.2. Least squares fitting

When the range of the performance function is bounded or can be treated as bounded, we provide a sample-based least
square fitting method. It is used when the specific distribution parameters of each input variable are unknown or there
is only a small amount of sample information. The undetermined pre-estimation of the PDF is set to a polynomial form.
Since the method is based on samples, the approximate multimodal PDF can be directly observed in the histogram, and the
position of the valley can be extracted by manual or automatic selection. The calculation steps are given as follows:
First, select a series of equidistant points yr in the entire range [a, b] of the performance function:
(b − a ) (r − 1 )
yr = a + , r = 1, 2, . . . , R (3)
R−1
where b and a are the upper and lower bounds of the range of performance function, and R is the number of the equidistant
points. If it is difficult to obtain the prior of the bounds, we can also set them as μ+3σ and μ-3σ according to the mean
value μ and standard deviation σ of the available samples of y.
Next, the number of the samples falling into the subdomain [yr -1 , yr ] is counted as ζ r with the probability of Pr =ζ r /ζ ,
in which ζ is the total number of samples. Then, define a polynomial expression of the PDF as:

L
f L (y ) = al yl (4)
l=0

140
G. Li, Y.X. Wang, Y. Zeng et al. Applied Mathematical Modelling 102 (2022) 137–152

Fig. 2. Flowchart of the IM-MEM.

where L is the order of the polynomial expression and al , l = 0, …, L, are undetermined coefficients. To ensure the accuracy
of the result, we take L=10.
Finally, use the least squares fitting method to make a rough estimate of the PDF from the existing data. The polynomial
coefficients can be calculated by the following formula:

R  y  y l 2
 r−1+ yr 2 
R 
L
r−1+ yr
min : L = fL − Pr = al − Pr (5)
2 2
r=2 r=2 l

Then, substitute the coefficients to Eq. (4) to obtain the rough estimate with a multimodal shape, and select the position
of the valley as the translation parameter p.

2.2.3. Kernel density estimation

The kernel density estimation is one of the classical non-parametric methods, which can be used to study the character-
istics of the distribution from samples [24], without using prior knowledge or making assumptions about the distribution.
The classical symmetric kernel function used to achieve the preliminary fitting of the PDF can be given as follows:
  2  2 
K j (y ) = K0 1 − y − v j ,ify ∈ v j − 2h , v j + h
2 (6)
0, otherwise
where K0 is the normalization coefficient, and h is the bandwidth of the kernel function. In order to facilitate the calculation,
normalization of the samples is useful, vj is the j-th normalized sample value. A larger bandwidth leads to a smoother shape
of the PDF estimation.
The PDF estimated by this method is given in the following form:

J
f (y ) = K j (y ) (7)
j=1

in which J is the sample size used.

The normalization coefficient K0 can be obtained from the normalization conditions:
 b
K0 = f (y )dy (8)
a

141
G. Li, Y.X. Wang, Y. Zeng et al. Applied Mathematical Modelling 102 (2022) 137–152

where b and a are the upper and lower bounds of the performance function, and can be set according to practical experi-
ence. If it is impossible to determine the approximate bounds, they can also be set as positive and negative infinity
Through kernel density estimation, an analytical rough estimation of the PDF can be obtained, so that the approximate
location of the valley can be found intuitively in the PDF curve. Generally, a minimum number of about 100 samples is
sufficient to determine the location of the valley accurately enough.
It should be pointed out that there are still adverse effects of small sample size on results of the above two methods
because they are sample-based, which may cause some variability in the results of different samples. Therefore, when the
distributions of the input random variables are given, the MEM is recommended as the first choice to determine the valley
position.

2.3. PDF estimation

When the translation parameter p is obtained through a specific method mentioned above, the PDF of the transformed
performance function Z in Eq. (2) can be estimated by using the FM-MEM. All the samples needed in the FM-MEM for
predicting the PDF of the transformed performance function Z can be obtained from the transformation of samples of the
original performance function Y through Eq. (2). Based on the FM-MEM, the unknown f(z) can be evaluated through the
following optimization formulation:
⎧
⎨find : f (z )
1
maximize : H ( f (z )) = − 0 f (z ) ln f (z )dz (9)
⎩s.t. : 
E [Z αi ] = 0 zαi f (z )dz, i = 1, 2, ..., n
1

where f(z) is the PDF of Z, and E[Zα i ] is the α i -order statistical moment of Z according to the converted samples.
The expression of the PDF f(z) can be derived by the stationary value condition of the Lagrange function as follows:

n
f (z ) = exp(−λ0 − λi zαi ) (10)
i=1

According to the normalization condition of the PDF, we have

 
 1 
n
λ0 = ln exp(− λi zαi )dz (11)
0
i=1

The parameters α i and λi , i=1, 2, …, n, are undetermined, and the optimal estimated result converges to the real PDF, as
n→+∞ [25]. For the computational convenience, we need to construct new optimization problem to get these parameters
in a convenient way by minimizing the K-L divergence between the real PDF f∗ (z) and its approximation f(z),
 ∞
K [ f ∗ ( z ), f ( z )] = f ∗ (y ) ln [ f ∗ (z )/ f (z )]dy
−∞
 ∞
= −H ( f ∗ (z )) − f ∗ (z ) ln [ f (z )]dy (12)
−∞

In statistics, the K-L divergence is usually used to evaluate the difference between two PDFs. The smaller K-L distance
leads to the smaller difference between the real PDF and its approximation, which means the estimation is more accurate.
Combining Eqs. (10) and (12), the new form of the K-L divergence yields:

n
K [ f ∗ (z ), f (z )] = −H ( f ∗ (z )) + λ0 + λi E [zαi ] (13)
i=1

Because the entropy of the real PDF is constant, minimizing the K-L divergence in Eq. (13) is equivalent to the uncon-
strained optimization problem [21]:

find : α, λ 
(14)
min min {(α, λ )}
α λ
where

n
(α, λ ) = λ0 + λi E [zαi ] (15)
i=1

Solving the optimization problem in Eq. (14) and substituting α =[α 1 , α 2 , …, α n ] and λ=[λ0 , λ2 , …, λn ] into Eq. (10),
we can obtain the PDF estimation of Z. This is a two-layer loop optimization process, in which the optimization process of
λ is the inner loop and that of α is the outer loop. This optimization problem can be solved by the simplex method [26].
Note that the result is the PDF estimation of the transformed performance function of Z, and the inverse transformation

142
G. Li, Y.X. Wang, Y. Zeng et al. Applied Mathematical Modelling 102 (2022) 137–152

Fig. 3. Flowchart of the PDF prediction of the transformed performance function.

is needed to convert it back to the PDF of original performance function Y. According to the transformation function F(y)
in Eq. (2) and the probability conservation as Eq. (16), the PDF of the original performance function y can be obtained as
follows:

f (z )dz = f (y )dy (16)

N peak −1
 
dF dy ξ
f (y ) = f (z ) dy
dz
= f (z ) dy ξ dy
ξ =1
N peak −1

= f (z ) dF 1
d y  ξ kσ (17)
ξ =1 
   dF (y ξ ) 
N peak −1
= f F y 1
kσ dy ξ 

ξ =1 y ξ =
y−pξ
kσ

Through this inverse transformation, the PDF of the original performance function can be obtained.
Finally, the flowchart of the PDF estimation is shown in Fig. 3 including the whole procedure of the proposed method.

3. Examples

In this section, four numerical examples and one engineering example are illustrated to verify the feasibility of the
proposed method. The accuracy of PDF estimation is characterized by the cumulative error of the PDF, with the value of [0,
1], expressed by the formula as follows:
 b 
H
ε= | f (y ) − ft (y )| dy ≈ (| f (yh ) − ft (yh )| ) y (18)
a
h=1

where f(y) represents the estimated PDF from classic FM-MEM or the proposed nonlinear transformation-based FM-MEM
(NTBFM-MEM), and ft (y) represents the PDF from Monte Carlo simulation (MCS) with 107 samples. Since ft (y) is obtained
by the MCS, it can be expressed as a series of discrete probability values ft (yh ) ࢞y in an equally spaced interval centered
on yh with the interval length ࢞y. The accuracy of failure probability is characterized by the relative error, which can be

143
G. Li, Y.X. Wang, Y. Zeng et al. Applied Mathematical Modelling 102 (2022) 137–152

Table 1
Example characteristics description.

Example Characteristics

1 Discussion on the three recognition methods of translation parameter p

2 Non-Gaussian distribution variables and three-peak situation
3 Seismic load, engineering example
4 Bimodal vehicle load, engineering example

Table 2
Distribution parameters of X1 .

I 1 2 3

wi 0.4825 0.1524 0.3651

μi 1020.0 1450.0 1520.0
σi 50.000 50.000 60.000

Table 3
Distribution parameters of Xj (j=2, …, 7) [3].

Variable Description Mean Std.

X2 Yield stress of reinforcement 300.0 N/mm2 15.00 N/mm2

X3 Effective depth of reinforcement 770.0 N/mm2 38.50 N/mm2
X4 Stress–strain factor of concrete 0.350 0.035
X5 Compressive strength of concrete 25.00 mm2 5.000 mm2
X6 Width of beam 200.0 mm 40.00 mm
X7 Applied bending moment 100.0 kN•m 30.00 kN•m

expressed as follows:
 
 p fr − p fc 

εp f =   (19)
p fr 
where pfr is the failure probability from MCS with 107 samples and pfc is the failure probability from the other methods.
In the following examples, ε pf , o and ε pf , p are used to characterize the failure probability errors calculated by the classical
FM-MEM and the proposed method respectively. The specific characteristics of each example are shown in Table 1:

3.1. Example 1: Bimodal case with lognormally distributed input variables

In this example, a numerical case of the ultimate bending capacity of a reinforced concrete beam [3], is considered with
the performance function as:
X12 X22 X4
Y = X1 X2 X3 − − X7 (20)
X5 X6
where X1 is the Area of reinforcement. In order to verify the effect of the proposed method, we herein changed the distri-
bution form of X1 to a bimodal Gaussian mixture model, as follows:

3 x − μ 
wi ϕ
1 i
f ( x1 ) = (21)
i=1
σi
According to the mean and standard deviation of X1 and the estimated failure probability in Example 2 of Ref. [3], the
parameters in Eq. (21) are slected reasonably, listed in Table 2.
X2 ∼X5 in Eq. (20) are lognormally distributed random variables with statistical information shown in Table 3. All the
three recognition methods of translation parameter are used in this example, the pre-calculation with MEM, kernel density
estimation and least square fitting. The sample size is 500 in both the least square fitting and the kernel density estimation
with bandwidth of 0.15. The translation parameters and the parameters of estimated PDF from the proposed method are
listed in Table 4.
The cumulative errors of the PDFs obtained from the proposed method with different translation parameters are also
shown in Table 4. Comparing with the cumulative error ε o =0.57669 of the PDF from classical FM-MEM, we can conclude
that all the results from the proposed method with the different translation parameters are more accurate than that from
the classical FM-MEM.
The comparisons of calculation results are given in Fig. 4, where (a), (b) and (c) represent the results of PDF estimated
from the proposed method with MEM pre-calculation, kernel density estimation and least square fitting, respectively. The
results agree well with that of MCS, and the peak value and the position of the bimodal PDF can be accurately captured.

144
G. Li, Y.X. Wang, Y. Zeng et al. Applied Mathematical Modelling 102 (2022) 137–152

Table 4
Parameters of the estimated PDF.

Method p i=1 i=2 i=3 i=4 εp

pre-calculation 2.737 × 10 8
αi 0.64568 2.26032 2.03902 2.13052 0.08641
with MEM λi -1251.47 84461.7 131860 -215294
kernel density 2.779 × 10 8
αi 0.98288 2.00637 2.04594 1.12013 0.08662
estimation λi -21978.4 -96218.4 89469.0 28432.0
least square 2.747 × 108 αi 0.61337 1.07311 2.58150 2.61037 0.08617
fitting λi -2343.00 3237.66 -49496.2 48273.7

Fig. 4. PDF comparison in Example 1. (a) Pre-calculation with MEM. (b) Kernel density estimation. (c) Least square fitting.

It is also confirmed in the figure that the tail of the estimated PDF agree well with that of MCS, which illustrates that the
proposed method can predict the failure probability better. As the cumulative errors are very similar in the estimated PDF
with the three recognition methods of the translation parameter, the pre-computation with MEM is recommended for its
high efficiency and less demand for samples. In this example, the performance function has a higher nonlinearity, which is
more in line with the needs of real engineering problems. In practical problems, it is often necessary to select the method
to determine the translation parameters according to the actual situation. This example verifies that the results obtained by
various methods are satisfactory.
In the comparison of Fig. 4, it can be found that when the classical four-term FM-MEM is used for calculation, the
fitting of multiple peaks fails. In FM-MEM, the accuracy can be improved by increasing the number of items when dealing
with multimodal problems [18]. Thus, the proposed method is compared with the FM-MEM containing different numbers of
fractional moments. The results are shown in Fig. 5. The failure probability errors of different methods are shown in Table
5.
It can be found that when the number of the terms of the fractional moments increases to 6, the rough shape of double
peaks begins to appear. When the number of the items is 10, the results is not improved significantly, but the CPU time

145
G. Li, Y.X. Wang, Y. Zeng et al. Applied Mathematical Modelling 102 (2022) 137–152

Fig. 5. PDF comparison with classic FM-MEM with different items.

Table 5
Comparison of relative error of failure probability.

Method NTBFM-MEM FM-MEM-4 FM-MEM-6 FM-MEM-10 AKDE

ε (%) 4.542 15.449 29.157 31.110 10.105

Fig. 6. PDF comparison with AKDE.

of consumption has increased to 21003.62s. By contrast, the proposed method needs only 6236.55s. Therefore, for some
problems, increasing the items can indeed improve the performance of multimodal recognition, but there are still limitations
in accuracy.
The sample-based method is commonly used for probability density modeling for multimodal problems. For example, the
kernel density estimation method in Section 2.3.3, can realizes the pre-estimation of the translation parameters. In recent
years, kernel density estimation has developed and widely used to deal with various complex problems. The adaptive kernel
density estimation method (AKDE) proposed by Botev et al. [27] is used for comparative calculations in this example.
The calculation result is shown in Fig. 6, in which the AKDE result is based on 10 0 0 samples.
Compared with FM-MEM with increased number of terms, kernel density estimation has better applicability in multi-
modal fitting problems. The positions of the two peaks are accurately fitted, but there is a tiny bump on the right side of
the peak, which brings errors to calculation of the failure probability, as listed in Table 5.

3.2. Example 2: Trimodal case with input variables of Lognormal and gamma distributions

We consider a case of nonlinear performance function involving non-Gaussian distributed input variables, which is con-
sistent with the real engineering situation. The performance function is the plastic bending bearing of a steel beam section
with the following expression [23]:

Y = X1 X2 − 106 X3 (22)

146
G. Li, Y.X. Wang, Y. Zeng et al. Applied Mathematical Modelling 102 (2022) 137–152

Table 6
Distribution parameters of X1 of the trimodal distribution.

I 1 2 3 4

wi 0.2 0.1 0.4 0.3

μi 190 195 265 335
σi 5 5 5 5

Table 7
Distribution parameters of X2 and X3 [23].

Variable Distribution type Mean Std.

3
X2 Lognormal 800000mm 40000mm3
X3 Gamma 125N•mm 17.69N•mm

Table 8
Parameters of the estimated PDF of the trimodal distribution.

I 1 2 3 4 5 6

αi 1.00863 2.01607 3.02517 4.03431 4.87491 6.19710

λi -524.642 3390.78 -10940.5 19609.5 -14885.1 3350.21

Fig. 7. PDF comparison of the trimodal distribution in Example 2.

Similar to Example 1, we construct a more complex PDF fitting problem by adjusting the distribution parameters. The
PDF of X1 is given by the trimodal form as Eq. (23):

4 x − μ 
wi ϕ
1 i
f ( x1 ) = (23)
i=1
σi
In order to make the failure probability roughly the same as the original problem, we took the parameters of X1 accord-
ingly, shown in Table 6. The statistical information of X2 and X3 is given in Table 7.
The problem to be fitted is the PDF with trimodal shape, so we use the kernel density estimation method to determine
the p value in advance, p1 =5.2249 × 107 and p2 =1.2422 × 108 . The first six fractional moments were calculated, with
the results listed in Table 8. The comparison of PDF fitting is shown in Fig. 7. Take y<0 as the failure domain, the
failure probability by MCS is pf=0.02354. The relative errors of the failure probability obtained from the FM-MEM and
the proposed method are εpf,o =18.810% and εpf,p =4.597%, respectively.
The three peaks in this example have obvious characteristics. The peak positions and the cumulative errors of the
PDF obtained by the FM-MEM method and the proposed method are shown in Table 9. It can be found in Fig. 7 and
Table 9 that the classical method of six fractional moments can only fit out the bimodal shape, with unacceptable errors
for high-precision probabilistic modeling. After using the transformation function, we can fit the shape of the trimodal
distribution, and the comparison chart can intuitively show that the fitting has high precision. In Table 9, it can also be
found that the result of the proposed method is more accurate than that of FM-MEM in estimating peak positions.

147
G. Li, Y.X. Wang, Y. Zeng et al. Applied Mathematical Modelling 102 (2022) 137–152

Table 9
Peak position and the error of PDF estimation results.

Method Position of the first peak Position of the second peak Position of the third peak Cumulative error of PDF

MCS 0.318 × 10 8
0.904 × 10 8
1.410 × 10 8
-
FM-MEM 0.452 × 108 1.202 × 108 - 0.13237
NTBFM-MEM 0.286 × 108 0.904 × 108 1.452 × 108 0.05479

Fig. 8. Frame structure diagram.

Furthermore, the attention is often paid to the modeling accuracy of the tail of the PDF in the reliability analysis. As shown
in Fig. 7 the accurate fitting of the trimodal PDF from the proposed method is of great significance to the modeling accuracy
of the tail. If only the single or double peak PDF estimation is obtained, the wrong estimation of the peak position is likely
to be compensated at the tail, resulting in an unacceptably error of the failure probability. However, such problem can be
avoided by the proposed method.

3.3. Example 3: Engineering case of multi-story building

Consider a ten-story reinforced concrete structure damaged in the 1985 Mexico City Earthquake [28]. The structure dia-
gram is given in Fig. 8. The structure consists of four frames spanning 6m in the x-direction, five frames spanning 9m in the
y-direction, and four shear-walls. The story height is 4.5m, and the thickness of the reinforced concrete shear wall is 0.2m.
The concrete material is C40, in which steel bars with diameter of 6mm and spacing of 100mm are distributed. The beams,
with the steel material of Q235, can be divided into three types, including 150 longitudinal beams, 160 transverse beams
and 200 vertical beams, with the cross-sectional areas of 0.25m2 , 0.36m2 and 0.49m2 , respectively. Considering the stiffness
reduction when the structure enters into plasticity, the elastic modulus after yielding is reduced to 10% of the initial value.
Each story has a random concentrated mass with the mean value of 20 0 0 0kg and the COV of 0.1.
The acceleration power spectrum of the seismic load is given as
ωa4 + 4ζa2 ωa2 ω2
S (ω ) =  2 S0 (24)
ω2 − ωa2 + 4ζa2 ωa2 ω2
where ωv =15rad/s, ζ v =0.25, S0 =0.01m2 /s3 .
The structure is excited by the non-stationary seismic load of ground acceleration A(t) with the specific formula as fol-
lows,
A(t ) = g(t )G(t ) (25)
where G(t) is a Gaussian stationary stochastic process with zero mean and power spectral density S(ω) in Eq. (24), and g(t)
is a definite envelope function in the form:

(t/t1 )2 t < t1
g(t ) = 1 t2 ≤ t ≤ t2 (26)
e−υ (t−t2 ) t > t2

148
G. Li, Y.X. Wang, Y. Zeng et al. Applied Mathematical Modelling 102 (2022) 137–152

Fig. 9. PDF comparison in Example 3.

Table 10
Parameters of the estimated PDF.

I 1 2 3 4

αi 1.23758 3.41641 3.47151 10.0990

λi -114.992 -1806.31 1337.83 -113.580

Table 11
Distribution parameters of structural variables.

Variable Mean Std.

A1 -A34 0.07618m2 0.007618m2

A35 -A68 0.03716m2 0.003716m2
A69 -A103 0.03902m2 0.003902m2
A104 -A137 0.01858m2 0.001858m2
E1 -E137 201.097GPa 20.1097GPa

In Eq. (26), we take t1 =8, t2 =40 and υ =0.16.

Since the stiffness of structure changes sharply when the material enters the plastic state, the PDF of transient structural
response may have a bimodal shape. For example, a significant bimodal PDF of the top story displacement response shown
in Fig. 9 appears at 8 seconds during the seismic loading.
It should be noted that, unlike the numerical example, the distance between the two peaks in this example is
very close, the probability mass between the peaks is only 0.3439, and the difference between the peak and valley
probability density is only about 15%, which will cause more difficulty in fitting PDF. To estimate this bimodal PDF, the
translation parameter p= -4.26190 × 10−4 is obtained by the pre-calculation with MEM. The parameters of the estimated
PDF from the proposed method are listed in Table 10. According to Eq. (18), the cumulative errors of the PDF estimated by
the FM-MEM and the proposed method are ε o =0.21067 and ε p =0.06399, respectively. The big error of the FM-MEM comes
from the wrong unimodal form of PDF as presented in Fig. 9. It can be found that the classical FM-MEM cannot simulate
the double peaks of the PDF when they are very close, while the proposed method can accurately do it.
When the safety threshold of the top displacement is set to 0.015m, the failure probability with MCS is 0.0616. The
relative errors of the failure probability obtained from the FM-MEM and the proposed method are ε pf,o =31.022% and
εpf,p =6.5865%, respectively. So the accuracy of the failure probability estimation from the proposed method is also better
than that of the classical FM-MEM when only four terms are reserved in the approximate expression of the PDF.

3.4. Example 4: Engineering case of Burro Creek Bridge

Burro Creek Bridge, built in 1966, is located in Arizona, USA, with a span of 207.264m. The photo and finite element
diagram of the truss structure in the bridge are shown in Fig. 10. The unit system of the dimension in the Fig. 10 is feet.
For the sake of generality, it is converted to meters in this example.
This structure can be divided into vertical, diagonal, upper and lower truss element groups (137 truss elements). There
are 275 random variables in total, including 137 cross-sectional area (A1 -A34 for lower chord, A35 -A68 for upper chord, A69 -
A103 for vertical chord and A104 -A137 for diagonal chord), 137 young’s modulus E1 -E137 of the truss elements and the vertical
load P. The distribution parameters of these structural variables are given in Table 11.

149
G. Li, Y.X. Wang, Y. Zeng et al. Applied Mathematical Modelling 102 (2022) 137–152

Fig. 10. Truss structure diagram. (Quoted from Fig. 12 of the Ref. [29]).

Table 12
Distribution parameters of P of the bimodal distribution.

I 1 2

wi 0.5 0.5
μi 900kN 1400kN
σi 150kN 150kN

Table 13
Parameters of the estimated PDF.

I 1 2 3 4

αi 0.09426 1.76666 0.94607 6.97195

λi -875.856 -226.597 393.044 46.4056

Mei et al. [30] pointed out that the long-term overload of various engineering vehicles may cause great damage to the
bridge. Because of the huge weight difference between engineering vehicles and ordinary vehicles, the distribution of bridge
vehicle load is often difficult to be expressed by unimodal form. In their research, the bimodal Gaussian mixture distribution
is used to describe the vehicle load on the bridge. So we adopt the load P on the bridge of a Gaussian mixture model as
follows,


2 P − μ 
wi ϕ
i
f (P ) = (27)
i=1
σi

whose parameters are shown in Table 12. The failure mode is the maximum displacement in the vertical direction. When
the displacement is greater than 0.35m, the structure fails.
The translation parameter p=0.20594 is also obtained by the pre-calculation with MEM. The parameters of the estimated
PDF from the proposed method are listed in Table 13. According to Eq. (18), the cumulative errors of the PDFs from FM-MEM
and the proposed method are ε o =0.13161 and ε p =0.04452, respectively.
The estimated PDFs of the classical FM-MEM and the proposed method are compared with the MCS result in Fig. 11.
The bimodal PDF is accurately modelled by using the proposed method, which can provide a satisfactory solution for such
practical engineering problem.
The failure probability of from MCS is 0.01014. The relative errors of the failure probability predicted from the classical
FM-MEM and the proposed method are ε pf,o =18.6767% and ε pf,p =6.7929%, respectively, demonstrating the superiority of the
proposed method in accuracy.

150
G. Li, Y.X. Wang, Y. Zeng et al. Applied Mathematical Modelling 102 (2022) 137–152

Fig. 11. PDF comparison in Example 4.

4. Conclusion and remarks

Aiming at the limitation of classical FM-MEM in multimodal probabilistic modeling problem, we propose a new method
for estimating a multimodal PDF, in which the performance function is converted with a specific nonlinear transformation
function to make it more suitable for FM-MEM. The transformation with adjustable parameters can extract the multimodal
feature in PDFs adaptively, thus reduce the difficulty of modeling. Three ways, such as sample-based least square polyno-
mial fitting technique, sample-based kernel density estimation and classical maximum entropy method, are presented
for determination of the translation parameter of the transformation function, and the recommended usage of different
methods are provided.
Through the comparison in the examples, it is concluded that by using the proposed method, the cumulative probability
error and failure probability estimation error can be reduced by 60%∼80% based on four or six fractional moments
according to the examples, which will improve the accuracy of failure probability estimation comparing with the classic
FM-MEM. The proposed method can avoid the difficulties such as the variability of high-order moments and the oscillation
of the PDF curve caused by excessive moment constraints. Two practical engineering cases are illustrated for the applicability
of the proposed method in real-life engineering problems. These practical bimodal PDFs can be well predicted by using the
proposed method although the peaks of the bimodal PDF are very close, as in Example 3.3, the probability mass between
peaks is only 0.3439, which is difficult for the classical FM-MEM.
The proposed method provides a novel analysis idea for such probabilistic modeling problems that are prone to mul-
timodal features. It should be pointed out that the method mainly focuses on the pre-processing part of the performance
function, which does not conflict with other solution methods of multimodal PDF estimation problem, and can be used
flexibly in the calculation.

Acknowledgement

The work is supported by the National Key Research and Development Program (Grant No. 2019YFA0706803), the Na-
tional Natural Science Foundation of China (Grant Nos. 11872142 and 11672052) and Fundamental Research Funds for the
Central Universities of China (Grant No. DUT2019TD37).

References

[1] X. Zhang, M.D. Pandey, Structural reliability analysis based on the concepts of entropy, fractional moment and dimensional reduction method, Struct.
Saf. 43 (2013) 28–40.
[2] D. Ormoneit, H. White, An efficient algorithm to compute maximum entropy densities, Econ. Rev. 18 (1999) 127–140.
[3] J. Xu, F. Kong, Adaptive scaled unscented transformation for highly efficient structural reliability analysis by maximum entropy method, Struct. Saf. 76
(2019) 123–134.
[4] W. He, G. Li, P. Hao, et al., Maximum Entropy Method-Based Reliability Analysis With Correlated Input Variables via Hybrid Dimension-Reduction
Method, J. Mech. Des. 141 (2019) 1.
[5] J. Xu, C. Dang, F. Kong, Efficient reliability analysis of structures with the rotational quasi-symmetric point-and the maximum entropy methods, Mech.
Syst. Sig. Process. 95 (2017) 58–76.
[6] J. Xu, C. Dang, A novel fractional moments-based maximum entropy method for high-dimensional reliability analysis, Appl. Math. Modell. 75 (2019)
749–768.
[7] G. Li, W. He, Y. Zeng, An improved maximum entropy method via fractional moments with Laplace transform for reliability analysis, Struct. Multidis-
cipl. Optim. 59 (2019) 1301–1320.
[8] J. Xu, S. Zhu, An efficient approach for high-dimensional structural reliability analysis, Mech. Syst. Sig. Process. 122 (2019) 152–170.
[9] H. Gzyl, P-L. Novi-Inverardi, A. Tagliani, Determination of the probability of ultimate ruin by maximum entropy applied to fractional moments, Insur-
ance Math. Econ. 53 (2013) 457–463.

151
G. Li, Y.X. Wang, Y. Zeng et al. Applied Mathematical Modelling 102 (2022) 137–152

[10] S. Yu, Z. Wang, D. Meng, Time-variant reliability assessment for multiple failure modes and temporal parameters, Struct. Multidiscipl. Optim. 58 (2018)
1705–1717.
[11] X. Zhang, W. He, Y. Zhang, et al., An effective approach for probabilistic lifetime modelling based on the principle of maximum entropy with fractional
moments, Appl. Math. Modell. 51 (2017) 626–642.
[12] Y.Q. Ni, X.W. Ye, J.M. Ko, Monitoring-based fatigue reliability assessment of steel bridges: analytical model and application, J. Struct. Eng. 136 (2010)
1563–1573.
[13] S.W. Haider, R.S. Harichandran, M.B. Dwaikat, Closed-form solutions for bimodal axle load spectra and relative pavement damage estimation, J. Transp.
Eng. 135 (2009) 974–983.
[14] J. He, X. Guan, R. Jha, Improve the accuracy of asymptotic approximation in reliability problems involving multimodal distributions, IEEE Trans. Reliab.
65 (2016) 1724–1736.
[15] A. Rajan, Y.C. Kuang, M.P.L. Ooi, et al., Moment-constrained maximum entropy method for expanded uncertainty evaluation, IEEE Access 6 (2018)
4072–4082.
[16] R. Jiang, A probability density function with bimodal shape, Transp. Sci. Eng. 013 (1997) 1–8 (in Chinese).
[17] Z. Hu, X. Du, A mean value reliability method for bimodal distributions, in: Proceedings of the ASME 2017 International Design Engineering Technical
Conferences and Computers and Information in Engineering Conference. Volume 2B: 43rd Design Automation Conference, 2017.
[18] Z. Zhang, C. Jiang, X. Han, et al., A high-precision probabilistic uncertainty propagation method for problems involving multimodal distributions, Mech.
Syst. Sig. Process. 126 (2019) 21–41.
[19] G. Li, W. He, Y. Zeng, An improved maximum entropy method via fractional moments with Laplace transform for reliability analysis, Struct. Multidis-
cipl. Optim. 59 (2019) 1301–1320.
[20] Q. Pan, D. Dias, An efficient reliability method combining adaptive support vector machine and Monte Carlo simulation, Struct. Saf. 67 (2017) 85–95.
[21] P.N. Inverardi, G. Pontuale, A. Petri, A. Tagliani, Hausdorff moment problem via fractional moments, Appl. Math. Comput. 144 (2003) 61–74.
[22] I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, The MIT Press, 2016.
[23] G. Li, C. Zhou, Y. Zeng, et al., New maximum entropy-based algorithm for structural design optimization, Appl. Math. Modell. 66 (2019) 26–40.
[24] A.J. Izenman, Recent Developments in Nonparametric Density Estimation, J. Am. Statist. Assoc. 86 (1991) 205–224.
[25] L.R. Mead, N. Papanicolaou, Maximum entropy in the problem of moments, J. Math. Phys. 25 (1984) 2404–2417.
[26] J.C. Lagarias, J.A. Reeds, M.H. Wright, et al., Convergence Properties of the Nelder–Mead Simplex Method in Low Dimensions, Siam J. Optim. A Pub.
Soc. Indust. Appl. Math. 9 (2006) 112–147.
[27] Z.I. Botev, J.F. Grotowski, D.P. Kroese, Kernel density estimation via diffusion, Annals of Stats 38 (2010) 2916–2957.
[28] X. Ye, J. Qian, K. Li, Shaking table test and dynamic response prediction on an earthquake-damaged RC building, Earthq. Eng. Eng. Vib. 3 (2004)
205–214.
[29] B. Keshtegar, S.P. Zhu, Three-term conjugate approach for structural reliability analysis, Appl. Math. Modell. 76 (2019) 428–442.
[30] G. Mei, Q. Qin, D.J. Lin, Bimodal Renewal Processes Models of Highway Vehicle Loads, Reliab. Eng. Syst. Saf. 83 (2004) 333–339.

152