Chemometrics and Intelligent Laboratory Systems 208 (2021) 104196

Contents lists available at ScienceDirect

Chemometrics and Intelligent Laboratory Systems

journal homepage: www.elsevier.com/locate/chemometrics

Improving grasshopper optimization algorithm for hyperparameters

estimation and feature selection in support vector regression
Zakariya Yahya Algamal a, *, Maimoonah Khalid Qasim b, Muhammad Hisyam Lee c,
Haithem Taha Mohammad Ali d
a
Department of Statistics and Informatics, University of Mosul, Mosul, Iraq
b
Department of General Science, University of Mosul, Mosul, Iraq
c
Department of Mathematical Sciences, Faculty of Science, Universiti Teknologi Malaysia, Johor, Malaysia
d
College of Computers and Information Technology, Nawroz University, Kurdistan region, Iraq

A R T I C L E I N F O A B S T R A C T

Keywords: High-dimensionality is one of the major problems which affect the quality of the classification and prediction
Grasshopper optimization algorithm modeling. Support vector regression has been applied in several real problems. However, it is usually needed to
Evolutionary algorithm tune manually the hyperparameters.In addition, SVR cannot perform feature selection. Nature-inspired algorithms
SVR
have been used as a feature selection and as hyperparameters estimation procedure. In this paper, an improving
Feature selection
grasshopper optimization algorithm (GOA) is proposed by adapting a new function of the main controlling
parameter of GOA to enhance the exploration and exploitation capability of GOA. This improving is utilized to
optimize the hyperparameters of the SVR with embedding the feature selection simultaneously. Experimental
results, obtained by running on four datasets, show that our proposed algorithm performs better than cross-
validation method, in terms of prediction, number of selected features, and running time. Besides, the experi-
mental results of the proposed improving confirm the efficiency of the proposed algorithm in improving the
prediction performance and computational time compared to other nature-inspired algorithms, which proves the
ability of GOA in searching for the best hyperparameters values and selecting the most informative features for
prediction tasks.

1. Introduction can calculate 5270 molecular descriptors [18,19]. In high dimensional

QSAR/QSPR modeling, where the number of descriptors, p, exceeds the
In chemometrics, the quantitative structure-activity (property) rela- number of compounds, n, the traditional statistical classification methods
tionship (QSAR/QSPR) is a powerful and a promising model used to are not feasible [7,20]. In addition, the large number of descriptors can
better understand the structural relationship between the chemical ac- degrade the generalizable performance of the used classifier or the pre-
tivity (property) and the chemical compounds by explicitly considering diction performance. Therefore, selecting descriptors that truly affect the
the mathematical, statistical, and informatical methods [1–4]. A common biological activity is an attractive way in QSAR/QSPR modeling [21].
task in these models is the selection of relevant descriptors (variables), The support vector machine technique has appeared which is written
where researchers try to determine the smallest possible set of descriptors briefly (SVM). Support vector machine has attracted much substantial
that can still achieve good predictive performance [4–17]. A typical data attention from many researchers and practitioners and statisticians in
in QSAR/QSPR modeling consist of a small sample size of compounds recent years because of its theoretical and practical advantages that
(molecules) and a very large number of descriptors. Consequently, justify its improved performance in classification and regression [22].
QSAR/QSPR modeling is challenged by the high dimensionality of the Originally, SVM firstly used for solving classification problem. With the
descriptors. introduction of Vapnik’s ε-insensitive loss function, the SVM has been
In chemometrics, today, it is easily come out with thousands of mo- extended to solve the nonlinear regression problem, called the support
lecular descriptors, such as Dragon 7, which is commercial software. It vector regression (SVR) [22–24].

* Corresponding author.
E-mail addresses: [email protected], [email protected] (Z.Y. Algamal), [email protected] (M.K. Qasim), mhl@
utm.my (M.H. Lee), [email protected] (H.T. Mohammad Ali).

https://doi.org/10.1016/j.chemolab.2020.104196
Received 12 July 2020; Received in revised form 4 November 2020; Accepted 6 November 2020
Available online 10 November 2020
0169-7439/© 2020 Elsevier B.V. All rights reserved.
Z.Y. Algamal et al. Chemometrics and Intelligent Laboratory Systems 208 (2021) 104196

Fig. 1. Representation of the proposed solution.

Fig. 2. The flowchart of our proposed framework.

2
Z.Y. Algamal et al. Chemometrics and Intelligent Laboratory Systems 208 (2021) 104196

Table 4
The averaged computational time in seconds.
Datasets CV GOA PGOA

Dataset 1 817.55 * 237.11 * 153.89

Dataset 2 8474.67 * 208.68 * 166.33
Dataset 3 779.14 * 191.22 * 145.13
Dataset 4 889.21 * 202.57 * 165.24

Table 5
Comparative experimental results (on average) for algorithms used based on
testing dataset.
Algorithms Dataset 1 Dataset 2 Dataset 3 Dataset 4

PSO 0.0718 0.1371 0.0331 0.3359

FF 0.0697 0.1335 0.0311 0.3172
Fig. 3. Performance of the controlling parameter by the proposed approach for BA 0.0679 0.1124 0.0291 0.2916
10 iterations. WOA 0.0547 0.1098 0.0254 0.2614
PGOA 0.0522 0.1078 0.0228 0.2532

Table 1
Description of the datasets used. inspiration from nature, have attracted considerable interest and ach-
ieved competitive results when solving optimization problems including
Dataset #Samples #features
hyperparameters tuning problem [28–30] and feature selection. In the
Dataset 1 134 1048 literature, there are numerous studies on tuning the hyperparameters of
Dataset 2 479 2881
SVR using nature-inspired algorithms, such as [30–39]. In recent days,
Dataset 3 65 2540
Dataset 4 212 3107 researchers are developing several new nature-inspired algorithms for
improving and enhancing exploration and exploitation of the existing
algorithms. Among these new algorithms, grasshopper optimization al-
gorithm which has gained popularity due to their high efficiency [40].
Table 2
Experimental results (on average) based on training datasets.
To the best of our knowledge, there are no attempts to perform
feature selection and to tune the hyperparameters of SVR simultaneously.
Datasets CV MSEtrain GOA MSEtrain PGOA MSEtrain
The main objective of this work is to optimize the hyperparameters of the
# # # SVR with embedding the feature selection simultaneously by improving
Features Features Features
the grasshopper optimization algorithm. The efficiency of our proposed
Dataset 104 1.5411 24 1.0377 13 0.0311 algorithm was compared with other previous algorithms.
1 The rest of the paper is organized as follows: Section 2 presents the
Dataset 116 2.6691 31 0.8461 17 0.1357
theoretical part of the support vector regression. The explanation of the
2
Dataset 138 1.0838 35 0.5381 16 0.0180 grasshopper optimization algorithm and the proposed improving are
3 given in Sections 3 and 4. In section 5, the datasets explanation is
Dataset 122 1.8951 27 0.6836 14 0.1829 covered. Section 6, the results are summarized with their discussion.
4
Finally, Section 7 contains a conclusion of this work.

2. Support vector regression

Table 3
Experimental results (on average) based on testing datasets. Support vector machines (SVM) have been successfully employed in
Datasets CV GOA PGOA solving different classification problems. However, with the introduction
of ε-insensitive loss function by Vapnik [41], the SVM has been extended
MSEtest MSEtest MSEtest
to solve the nonlinear regression problems, which is called the support
Dataset 1 1.8472 1.0644 0.0522 vector regression. In SVR, the target variable is a quantitative variable,
Dataset 2 2.4062 0.9724 0.1078
such as chemical activity [42], spectral analysis [25,43], and stock price
Dataset 3 1.2068 0.8517 0.0228
Dataset 4 1.9844 0.8166 0.2532 forecasting.
Given a training dataset of n observations fðxi ; yi Þgni¼1 , where xi ¼
ðxi;1 ; xi;2 ; :::; xi;p Þ 2 Rp represents a vector of the ith feature, yi 2 R for i ¼
SVR has three advantages: (1) guaranteed convergence to optimal
1; :::; n is the target variable, which is a quantitative variable, and
solutions because quadratic programming is used with linear constraints
ε-insensitive loss function, the SVR can be obtained through solving the
for learning the data. (2) Computationally efficient for nonlinear rela-
following optimization problem
tionship modeling by using kernel mapping. And, (3) a superior gener-
( )
alization performance (lower error rates on test set) [25]. X n  
1 T ~
The computational efficient of the SVR is strongly depending on a min w w þ C ζi þ ζi
w;b 2 i¼1
number of hyperparameters and influences that directly or indirectly
8
affect the finding optimal solution. The comprehensive grid search is > ~
> yi  ðw ⋅ ϕðxi Þ þ bÞ  ε þ ζi
> (1)
usually employing to investigate all the combinations of the hyper- <
parameters and cross-validation are used to check predictive ability of S:T: ðw ⋅ ϕðxi Þ þ bÞ  yi  ε þ ζi
>
>
SVR [26]. Despite the excellent characteristics of SVR, there are still >
: ~
ζi ; ζi  0;
several drawbacks, including the selection of features. In other words,
SVR cannot perform feature selection [27]. where C > 0 is a penalized parameter that controls the tradeoff between
Nature-inspired algorithms, which they developed by drawing
the model complexity and training error, ζi and ~ζi are slack variables,

3
Z.Y. Algamal et al. Chemometrics and Intelligent Laboratory Systems 208 (2021) 104196

Fig. 4. The average running time in seconds for the other used algorithms.

r
ϕðxi Þ is a nonlinear mapping which is induced by a kernel function, w is defined as the strength of social forces calculated as sðrÞ ¼ f e I  er ,
the weight vector and bis bias. where f indicates the intensity of attraction, and I is the attractive length
Then, Eq. (1) can be solved by the Lagrangian multipliers after scale. The Gi and Ai are
reformulated it into its dual problem as
Gi ¼  g  be g ; (6)
1 X n   
  Xn   X n  
min α~i  αi α~j  αj K xi ; xj þ ε α~i  αi  yi α~i  αi
α~;α 2 i;j¼1 i¼1 i¼1 Ai ¼ u  be w ; (7)
8 n  
> X
>
< αi  α~i ¼ 0 where gis the gravitational constant, be g shows a unity vector towards the
S:T: i¼1 center of the earth, uis a constant drift and be w is a unit vector in the di-
>
>
: rection of the wind. Nymph grasshoppers have no wings, so their
0  αi ; α~i  C;
movements are highly correlated with wind direction. The mathematical
(2)
model can be extended by replacing the three operators as
where Kðxi ; xj Þstands for kernel mapping, and αi , α
~i are Lagrangian 0 1
multipliers. The regression hyperplane for the underlying regression BX C
B n ub  lb   xj  xi C
problem is then given by xi ¼ ωB
B ω s xj  xi  Cþ T
bd; (8)
@ j¼1 2 dij CA
X    j6¼i
yi ¼ f ðxi Þ ¼ α~i þ αi K xi ; xj þ b ; (3)
xi ¼SV

where ub and lb represent the upper bound and lower bound of the search
where SV is the support vectors set.
b d represents the value of the best solution found
space, respectively; T
thus far. However, in Eq. (8) gravity is not considered, and the wind
3. Grasshopper optimization algorithm
direction is always considered to be toward T b d . The quantity ω is a
The grasshopper optimization algorithm (GOA), which was intro- decreasing coefficient to shrink the comfort zone, repulsion zone, and
duced by Saremi, Mirjalili and Lewis [40], is a new meta-heuristic opti- attraction zone as follows:
mization algorithm inspired from the lives of grasshoppers. Grasshoppers ωmax  ωmin
move in swarms. In GOA, there are two mainly phases of optimization: ω ¼ ωmax  t ; (9)
tmax
exploration and exploitation of the search space. These two phases are
appearing during the food search through these social interactions [44, where ωmax ¼ 1 and ωmin ¼ 0:00001 are, respectively, the maximum and
45]. the minimum value of parameter ω. tmax is the maximum number of it-
In GOA, there are three evolutionary operators in position updating of erations [46,47].
individuals in swarms: the social interaction operator (Si ), the gravity To deal with feature selection, the binary grasshopper optimization
force operator (Gi ), and the wind advection operator (Ai ). The position of algorithm (BGOA) is utilized. Here, each grasshopper is represented by
a grasshopper (xi ) is mathematically modeled as the p-bit binary string, where p is the number of features in the dataset.
To update the position, the transfer function is usually used to force
xi ¼ Si þ Gi þ Ai : (4) grasshopper to fly in a binary space [45,48,49]. In order to build this
Each of these operators is mathematically modeled as follows: binary vector, a transfer function in Eq. (7) can be used, in which the new
solution is constrained to only binary values.
X
n
 
Si ¼ b
d ij s dij b
d ij ; (5) 1 if TðxÞ > rand
xtþ1 ¼ ; (10)
j¼1
j6¼i
i 0 otherwise

where dij is the Euclidean distance between the ith and the jth grass- where rand 2 ½0; 1 is a random number, TðxÞ is the transfer function for
  the position of the current grasshopper. Two familiar transfer functions
hopper that is calculated as dij ¼ xj xi  and b
d ij ¼ ðxj  xi Þ= dij . While sis

4
Z.Y. Algamal et al. Chemometrics and Intelligent Laboratory Systems 208 (2021) 104196

are usually used, namely the S-shaped family and the V-shaped family 5. Datasets
[50].
To test the predicting performance of our proposed algorithm, PGOA,
4. The proposed improving comprehensive comparative experiments with the standard GOA and
cross-validation approach with ten folds (CV) are utilized. Four different
In SVR there are several parameters that are needed to be fixing. sets of chemical datasets were used in this research: antidiabetic activity
These parameters are called hyperparameters, such as penalized of dipeptidyl peptidase-IV inhibitors (Dataset 1) [56], influenza neur-
parameter, C, ε-insensitive loss function, ε, and the kernel parameter. The aminidase a/PR/8/34 (H1N1) inhibitors (Dataset 2), anticancer potency
SVR performance is very sensitive to the selection of these hyper- of imidazo [4,5-b]pyridine derivatives (Dataset 3) [57], diverse series of
parameters and there is no mathematical based procedure for deriving antifungal agents (Dataset 4) [58]. All these datasets include thousands
the exact desired values [51]. As a result, the selection of those hyper- of descriptors as features. Table 1 presents a summary of these datasets.
parameters is a crucial part of the research on SVR. Each dataset was divided into two groups: a training dataset consisting of
In the literature, there several attempts with different procedures to 70% of the total samples, and a test dataset consisting of 30%. This
improve the SVR performance by appropriated choosing of these splitting is repeated 20 times. Two evaluation criteria were used:
hyperparameters [28–30,51–53]. Nature-inspired algorithms are among Mean-squared error of the training dataset (MSEtrain ¼
Pntrain
these different procedures that were employed to select the hyper- i¼1 ðy i;train  b
y i;train Þ2
=ntrain ) and mean-squared error of the testing
parameters of SVR [30–39,54]. However, in all these existence proced- Pntest
b 2
dataset (MSEtest ¼ i¼1 ðyi;test  y i;test Þ =ntest ).
ures regarding the selection of hyperparameters, there is no attempt to
perform feature selection simultaneously.
6. Results and discussion
Our contribution of this work is to optimize the hyperparameters of
the SVR with embedding the feature selection simultaneously by
Table 2 summarizes the number of features selected by each of use
improving the grasshopper optimization algorithm. In our proposed
approach in the training set and the averaged MSEtrain . The number of
procedure, the type of kernel function is Gaussian kernel with parameter
features selected by each approach is an important factor, in which
σ > 0. The illustrative of the solution representation is depicted in Fig. 1.
methods with a small number of selected features are preferred. As can be
The flowchart of our proposed framework is presented in Fig. 2.
seen from Table 2, PGOA possesses a fewer features than the other two
The parameter ω is the main controlling parameter in the GOA, which
methods. For instance, in Dataset 3, PGOA selected 16 features compared
is very similar to the inertial weight particle swarm optimization algo-
to 35 and 138 features for the GOA and CV, respectively. In terms of
rithm (PSO) [44]. The inner ω reduces the movements of grasshoppers
predictive performance, as we can see from Table 2, the PGOA was su-
around the target. In other words, this parameter balances exploration
perior to all the compared methods. Hence, the utilization of the PGOA
and exploitation of the entire swarm around the target. While the outer ω
yielded a lowest MSEtrain . Furthermore, in Dataset 4, it can be seen that
is considered to prevent excessive movement of grasshoppers around the
the reduction in MSEtrain of the PGOA was about 63.92% and 73.24%
target [40,49]. Following Wang, Guo, Duan, Liu and Wang [55], an
lower than that of the GOA and the CV, respectively. Moreover, with
improving of GOA is proposed by replacing Eq. (9) by
respect to the results of Table 2, CV method is ranked last and GOA is
ωmax worse than HHOA in all criteria and is ranked second.
ω
~¼ : (11)
t2 Once again, based on the test set results in Table 3, the proposed
method, PGOA, yields significantly better predictive ability as compared
Equation (11) can decrease quickly within small number of iterations
compared to Eq. (9). In addition, Eq. (11) only needs one value, ωmax , as to GOA and CV. The predictive performance achieved by the PGOA in
Dataset 1, for example, was 0.0522, which was better than 1.0644 and
initial. Fig. 3 shows the performance of Eq. (11).
Each agent has a position contains three quantitative values which are 1.8472 obtained by GOA and CV, respectively. Among the methods used,
it is clearly seen that the CV is the worst predictive method.
representing the C, σ , and ε, and p binary values which are representing
the features. The relevant feature will take the value of 1 and 0 otherwise. To further highlight the computational efficiency, Table 4 shows the
CPU time of the proposed algorithm, PGOA, GOA, and CV. As can be
In other words, each hawk has 3 þ p positions. The steps of our proposed
algorithm are presented as follows. seen, in terms of computational efficiency, the PGOA has less time than
GOA and CV. The p-values (*) from Wilcoxon’s rank sum test
Step 1. The number of agents is ng ¼ 30 and the maximum number of (nonparametric statistical test) with 5% significance level are adopted.
iterations is tmax ¼ 100. The statistical test is needed to indicate that the PGOA provides a sig-
Step 2. The first three positions represents the hypermeters are nificant improvement compared to the other methods. It can be seen that
there is a statistical difference between PGOA and all the others for all
randomly generated from uniform distribution as CeUð0; 5Þ, σ eUð0; 2Þ,
and εeUð0; 1Þ. The rest positions which are representing the feature are datasets. This is not surprising because the CV is computationally
consuming time.
generated as Uð0; 1Þ.
The obtained results in Tables 2–4 strongly prove that high explora-
Step 3. The fitness function is defined as tion of PGOA is able to explore the search area extensively and give
" 2 # promising regions of the search area comparing to GOA. The main reason
ntest 
1 X is that quickly decreasing with small number of iterations strategy is
fitness ¼ min yi;test  by i;test ; (12)
ntest i¼1 adopted.
In order to verify the feasibility and effectiveness of the proposed
where the fitness is calculated for the testing dataset. PGOA in optimizing the SVR hyperparameters and feature selection, the
predictive performance of the PGOA is compared with other widely al-
Step 4. To deal with feature selection, the BGOA is utilized. Here, each
gorithms applied to solve this problem. These algorithms are: Particle
agent is represented by the p-bit binary string. To update the position, the
swarm optimization (PSO), firefly algorithm (FF), bat algorithm (BA),
transfer function is usually used to force agent to be in a binary space as
and whale optimization algorithm (WOA). The parameters of these al-
TðxÞ ¼ ð1 =1 þ expð  xÞÞ, which is the sigmoid transfer function.
gorithms are setting as basic. For the population size and the iteration
Step 5. Steps 3 and 4 are repeated until a tmax is reached. numbers, they are assumed as same as in PGOA. The average MSEtest and
the computational time of the comparative algorithms are depicted in
Table 5 and Fig. 4, respectively.

5
Z.Y. Algamal et al. Chemometrics and Intelligent Laboratory Systems 208 (2021) 104196

From Table 5, it is clear that, PGOA performs better than the other [10] Z.Y. Algamal, M.H. Lee, Regularized logistic regression with adjusted adaptive
elastic net for gene selection in high dimensional cancer classification, Comput.
algorithms in all the datasets which clearly demonstrates the predictive
Biol. Med. 67 (2015) 136–145.
performance strength of the PGOA. GWA is in second place and PSO in [11] Z.Y. Algamal, M.H. Lee, A new adaptive L1-norm for optimal descriptor selection of
the last place. In terms of running time, it indicates from Fig. 4 that PGOA high-dimensional QSAR classification model for anti-hepatitis C virus activity of
has a global search ability and convergence ability which outperformed thiourea derivatives, SAR QSAR Environ. Res. 28 (2017) 75–90.
[12] Z.Y. Algamal, M.H. Lee, A two-stage sparse logistic regression for optimal gene
other algorithms. In general, it is evident from Table 5 and Fig. 4 that selection in high-dimensional microarray data classification, Adv. Data Analy.
PGOA has shown competitive performance in comparison to WOA and it Classification 13 (3) (2018) 753–771.
has shown superior performance in comparison to PSO, FF, and BA. [13] Z.Y. Algamal, M.K. Qasim, H.T.M. Ali, A QSAR classification model for
neuraminidase inhibitors of influenza A viruses (H1N1) based on weighted
penalized support vector machine, SAR QSAR Environ. Res. 28 (2017) 415–426.
7. Conclusion [14] M.K. Qasim, Z.Y. Algamal, H.T.M. Ali, A binary QSAR model for classifying
neuraminidase inhibitors of influenza A viruses (H1N1) using the combined
minimum redundancy maximum relevancy criterion with the sparse support vector
In this paper, a new function of the main controlling parameter was machine, SAR QSAR Environ. Res. 29 (2018) 517–527.
proposed to improve the exploration and exploitation capability of the [15] O.S. Qasim, Z.Y. Algamal, Feature selection using particle swarm optimization-
PGOA in optimizing hyperparameters of SVR and performing feature based logistic regression model, Chemometr. Intell. Lab. Syst. 182 (2018) 41–46.
[16] M. Eklund, U. Norinder, S. Boyer, L. Carlsson, Choosing feature selection and
selection. The experimental results and statistical analysis on four data- learning algorithms in QSAR, J. Chem. Inf. Model. 54 (2014) 837–843.
sets have demonstrated that the performance of our proposed algorithm [17] M. Goodarzi, B. Dejaegher, Y.V. Heyden, Feature selection methods in QSAR
compared with the other methods and algorithms leads to a better per- studies, J. AOAC Int. 95 (2012) 636–651.
[18] R. Todeschini, V. Consonni, A. Mauri, M. Pavan, Dragon 6 (2012).
formance in terms of prediction, number of selected features, and
[19] Dragon 7. www.kode-solutions.net, 2016.
running time. Obviously, the proposed function is a promising way to [20] P. Filzmoser, M. Gschwandtner, V. Todorov, Review of sparse methods in regression
improve the performance of GOA because of its best exploitation ability. and classification with application to chemometrics, J. Chemom. 26 (2012) 42–51.
[21] J. Garcia, P.R. Duchowicz, M.F. Rozas, J.A. Caram, M.V. Mirifico, F.M. Fernandez,
This highly effective predicting framework can be applied to other real
E.A. Castro, A comparative QSAR on 1,2,5-thiadiazolidin-3-one 1,1-dioxide
applications. compounds as selective inhibitors of human serine proteinases, J. Mol. Graph.
Model. 31 (2011) 10–19.
CRediT authorship contribution statement [22] C.-C. Chuang, Z.-J. Lee, Hybrid robust support vector machines for regression with
outliers, Appl. Soft Comput. 11 (2011) 64–72.
[23] Y.-F. Ye, Y.-H. Shao, N.-Y. Deng, C.-N. Li, X.-Y. Hua, Robust Lp -norm least squares
Maimoonah Khalid Qasim: Chemical data preparation, Writing - support vector regression with feature selection, Appl. Math. Comput. 305 (2017)
original draft, preparation. Muhammad Hisyam Lee: Supervision, 32–52.
[24] Y.-P. Zhao, J.-G. Sun, Robust truncated support vector regression, Expert Syst. Appl.
Writing - review & editing. Haithem Taha Mohammad Ali: Coding, 37 (2010) 5126–5133.
writing-reviewing. [25] S. Xu, B. Lu, M. Baldea, T.F. Edgar, M. Nixon, An improved variable selection
method for support vector regression in NIR spectral modeling, J. Process Contr. 67
(2018) 83–93.
Declaration of competing interest [26] H. Kaneko, K. Funatsu, Fast optimization of hyperparameters for support vector
regression models with highly predictive ability, Chemometr. Intell. Lab. Syst. 142
The authors declare that they have no known competing financial (2015) 64–69.
[27] N.A. Al-Thanoon, O.S. Qasim, Z.Y. Algamal, Tuning parameter estimation in SCAD-
interests or personal relationships that could have appeared to influence support vector machine using firefly algorithm with application in gene selection
the work reported in this paper. and cancer classification, Comput. Biol. Med. 103 (2018) 262–268.
[28] J.-S. Chou, A.-D. Pham, Nature-inspired metaheuristic optimization in least squares
support vector regression for obtaining bridge scour information, Inf. Sci. 399
Appendix A. Supplementary data (2017) 64–80.
[29] R. Laref, E. Losson, A. Sava, M. Siadat, On the optimization of the support vector
Supplementary data to this article can be found online at https:// machine regression hyperparameters setting for gas sensors array applications,
Chemometr. Intell. Lab. Syst. 184 (2019) 22–27.
doi.org/10.1016/j.chemolab.2020.104196. [30] S. Li, H. Fang, X. Liu, Parameter optimization of support vector regression based on
sine cosine algorithm, Expert Syst. Appl. 91 (2018) 63–77.
References [31] C.-H. Wu, G.-H. Tzeng, R.-H. Lin, A Novel hybrid genetic algorithm for kernel
function and parameter optimization in support vector regression, Expert Syst.
Appl. 36 (2009) 4725–4735.
[1] S. Gupta, N. Basant, K.P. Singh, Qualitative and quantitative structure-activity
[32] A. Kazem, E. Sharifi, F.K. Hussain, M. Saberi, O.K. Hussain, Support vector
relationship modelling for predicting blood-brain barrier permeability of
regression with chaos-based firefly algorithm for stock market price forecasting,
structurally diverse chemicals, SAR QSAR Environ. Res. 26 (2015) 95–124.
Appl. Soft Comput. 13 (2013) 947–958.
[2] C. Nantasenamat, T. Monnor, A. Worachartcheewan, P. Mandi, C. Isarankura-Na-
[33] M. Nait Amar, N. Zeraibi, Application of Hybrid Support Vector Regression
Ayudhya, V. Prachayasittikul, Predictive QSAR modeling of aldose reductase
Artificial Bee Colony for Prediction of MMP in CO2-EOR Process, Petroleum, 2018.
inhibitors using Monte Carlo feature selection, Eur. J. Med. Chem. 76 (2014)
[34] C.-F. Huang, A hybrid stock selection model using genetic algorithms and support
352–359.
vector regression, Appl. Soft Comput. 12 (2012) 807–818.
[3] E. Pourbasheer, R. Aalizadeh, T.S. Shokouhi, M.R. Ganjali, P. Norouzi,
[35] C.-T. Cheng, W.-C. Wang, D.-M. Xu, K.W. Chau, Optimizing hydropower reservoir
J. Shadmanesh, 2D and 3D quantitative structure–activity relationship study of
operation using hybrid genetic algorithm and chaos, Water Resour. Manag. 22
hepatitis C Virus NS5B polymerase inhibitors by comparative molecular field
(2007) 895–909.
analysis and comparative molecular similarity indices analysis methods, J. Chem.
[36] W.-C. Hong, Y. Dong, L.-Y. Chen, S.-Y. Wei, SVR with hybrid chaotic genetic
Inf. Model. 54 (2014) 2902–2914.
algorithms for tourism demand forecasting, Appl. Soft Comput. 11 (2011)
[4] A.M. Al-Fakih, Z.Y. Algamal, M.H. Lee, H.H. Abdallah, H. Maarof, M. Aziz,
1881–1890.
Quantitative structure-activity relationship model for prediction study of corrosion
[37] J. Cheng, J. Qian, Y.-n. Guo, Adaptive Chaotic Cultural Algorithm for
inhibition efficiency using two-stage sparse multiple linear regression, J. Chemom.
Hyperparameters Selection of Support Vector Regression, International Conference
30 (2016) 361–368.
on Intelligent Computing, Springer, 2009, pp. 286–293.
[5] M. Eklund, U. Norinder, S. Boyer, L. Carlsson, Benchmarking variable selection in
[38] B. Üstün, W. Melssen, M. Oudenhuijzen, L. Buydens, Determination of optimal
QSAR, Mol Inform 31 (2012) 173–179.
support vector regression parameters by genetic algorithms and simplex
[6] A.M. Al-Fakih, Z.Y. Algamal, M.H. Lee, M. Aziz, H.T.M. Ali, QSAR classification
optimization, Anal. Chim. Acta 544 (2005) 292–305.
model for diverse series of antifungal agents based on improved binary differential
[39] J. Zhang, C.-H. Zheng, Y. Xia, B. Wang, P. Chen, Optimization enhanced genetic
search algorithm, SAR QSAR Environ. Res. 30 (2019) 131–143.
algorithm-support vector regression for the prediction of compound retention
[7] A.M. Al-Fakih, M. Aziz, H.H. Abdallah, Z.Y. Algamal, M.H. Lee, H. Maarof, High
indices in gas chromatography, Neurocomputing 240 (2017) 183–190.
dimensional QSAR study of mild steel corrosion inhibition in acidic medium by
[40] S. Saremi, S. Mirjalili, A. Lewis, Grasshopper optimisation algorithm: theory and
furan derivatives, Int. J. Electrochemsc. 10 (2015) 3568–3583.
application, Adv. Eng. Software 105 (2017) 30–47.
[8] Z. Algamal, An efficient gene selection method for high-dimensional microarray
[41] V.N. Vapnik, An overview of statistical learning theory, IEEE Trans. Neural
data based on sparse logistic regression, Electronic Journal of Applied Statistical
Network. 10 (1999) 988–999.
Analysis 10 (2017) 242–256.
[42] G.H. Fu, D.S. Cao, Q.S. Xu, H.D. Li, Y.Z. Liang, Combination of kernel PCA and
[9] Z.Y. Algamal, M.H. Lee, Penalized logistic regression with the adaptive LASSO for
linear support vector machine for modeling a nonlinear relationship between
gene selection in high-dimensional cancer classification, Expert Syst. Appl. 42
bioactivity and molecular descriptors, J. Chemometr. 25 (2011) 92–99.
(2015) 9326–9332.

6
Z.Y. Algamal et al. Chemometrics and Intelligent Laboratory Systems 208 (2021) 104196

[43] B. Malik, K. Chaitanya, M. Benaissa, Support vector regression with digital band [51] P. Tsirikoglou, S. Abraham, F. Contino, C. Lacor, G. Ghorbaniasl, A hyperparameters
pass filtering for the quantitative analysis of near-infrared spectra, J. Chemometr. selection technique for support vector regression models, Appl. Soft Comput. 61
28 (2014) 116–122. (2017) 139–148.
[44] I. Aljarah, A.M. Al-Zoubi, H. Faris, M.A. Hassonah, S. Mirjalili, H. Saadeh, [52] V. Cherkassky, Y. Ma, Practical selection of SVM parameters and noise estimation
Simultaneous feature selection and support vector machine optimization using the for SVM regression, Neural Network. 17 (2004) 113–126.
grasshopper optimization algorithm, Cognitive Comp. 10 (2018) 478–495. [53] K. Ito, R. Nakano, Optimizing support vector regression hyperparameters based on
[45] H. Hichem, M. Elkamel, M. Rafik, M.T. Mesaaoud, C. Ouahiba, A New Binary cross-validation, in: Proceedings of the International Joint Conference on Neural
Grasshopper Optimization Algorithm for Feature Selection Problem, Journal of Networks, IEEE, 2003, pp. 2077–2082, 2003.
King Saud University - Computer and Information Sciences, 2019. [54] G. Cao, L. Wu, Support vector regression with fruit fly optimization algorithm for
[46] A.A. Ewees, M. Abd Elaziz, E.H. Houssein, Improved grasshopper optimization seasonal electricity consumption forecasting, Energy 115 (2016) 734–745.
algorithm using opposition-based learning, Expert Syst. Appl. 112 (2018) 156–172. [55] G. Wang, L. Guo, H. Duan, L. Liu, H. Wang, A modified firefly algorithm for UCAV
[47] H.T. Ibrahim, W.J. Mazher, O.N. Ucan, O. Bayat, A grasshopper optimizer approach path planning, Int. J. Hospit. Inf. Technol. 5 (2012) 123–144.
for feature selection and optimizing SVM parameters utilizing real biomedical data [56] A. Al-Fakih, Z. Algamal, M. Lee, M. Aziz, H. Ali, A QSAR model for predicting
sets, Neural Comput. Appl. 31 (2018) 5965–5974. antidiabetic activity of dipeptidyl peptidase-IV inhibitors by enhanced binary
[48] M. Mafarja, I. Aljarah, H. Faris, A.I. Hammouri, A.M. Al-Zoubi, S. Mirjalili, Binary gravitational search algorithm, SAR QSAR Environ. Res. 30 (2019) 403–416.
grasshopper optimisation algorithm approaches for feature selection problems, [57] Z.Y. Algamal, M.H. Lee, A.M. Al-Fakih, M. Aziz, High-dimensional QSAR prediction
Expert Syst. Appl. 117 (2019) 267–286. of anticancer potency of imidazo [4, 5-b] pyridine derivatives using adjusted
[49] A. Zakeri, A. Hokmabadi, Efficient feature selection method using real-valued adaptive LASSO, J. Chemometr. 29 (2015) 547–556.
grasshopper optimization algorithm, Expert Syst. Appl. 119 (2019) 61–72. [58] A. Al-Fakih, Z. Algamal, M. Lee, M. Aziz, H. Ali, QSAR classification model for
[50] S. Mirjalili, A. Lewis, S-shaped versus V-shaped transfer functions for binary particle diverse series of antifungal agents based on improved binary differential search
swarm optimization, Swarm Evol. Comp. 9 (2013) 1–14. algorithm, SAR QSAR Environ. Res. 30 (2019) 131–143.