A Review of Particle Swarm Optimization Feature Selection Classification and Hybridizations
A Review of Particle Swarm Optimization Feature Selection Classification and Hybridizations
Volume: 3 Issue: 4
ISSN: 2321-8169
1816 1820
_____________________________________________________________________________________________
Amit Saxena
Abstract: Particle swarm optimization (PSO) is a recently grown, popular, evolutionary and conceptually simple but efficient
algorithm which belongs to swarm intelligence category. This paper outlines basic concepts and reviews PSO based techniques
with their applications to classification and feature selection along with some of the hybridized applications of PSO with similar
other techniques.
Keywords: Swarm Intelligence, Classification, Feature selection, Hybridization.
__________________________________________________*****_________________________________________________
I.
INTRODUCTION
The Particle Swarm Optimization (PSO) is a Meta heuristic search technique, biologically inspired from the
natures social behavior, dynamic movements and
communications of insects, birds and fish. It was introduced by
American social psychologist James Kennedy and electrical
Engineer Dr. Russell Eberhart in 1995[1, 2] and is often
considered under the category of Swarm-intelligence (SI) [3].
It is a population based stochastic global optimization
technique evolved to study the social behavior of insects, birds
or fish as why they move in a group searching for food
randomly in some area, knowing only the distance from the
food [4]. A PSO system combines local search methods
(through self experience) with global search methods (through
neighboring experience), attempting to balance exploration and
exploitation [5]. PSO has been successfully applied to various
areas including Prediction analysis [6], Multi-model
Optimization[7], Feature Selection [8], Association rule mining
[9], Text clustering [10], Pattern Recognition [11],
Clustering[12], Classification [13], Data mining[14], Image
Processing [15], Rule extractor[16] etc. Often, to achieve better
performance, a PSO is combined with soft computing
techniques to form a hybridized model discussed later in this
paper.
The rest of the paper is organized as follows. In the Section
II, we present the basic concepts of the PSO and its algorithm.
In section III, Literature Review on development of PSO is
presented. In Section IV, a literature Review on PSO based
Classification and Feature Selection is provided. Section V
presents PSO hybridization and applications followed by the
conclusions in Section VI.
II.
6)
_____________________________________________________________________________________
ISSN: 2321-8169
1816 1820
_____________________________________________________________________________________________
Pseudo code for PSO:
For each particle
Initialize particle
END
Do
For each particle
Calculate fitness value
If the fitness value is better than the best fitness
value (Pbest) in history
Set current value as the new Pbest
End
Choose the particle with the best fitness value of all the
Particles as the gbest
For each particle
Calculate particle velocity according equation (1)
Update particle position according equation (2)
End
while maximum iterations or minimum error criteria are
not attained.
Figure 1. Pseudo code of PSO algorithm[20].
START
Initialize particles
ppiclesparticles
NO
Is current fitness
value
better than pbest?
Assign current
fitness as new pbest
Keep previous
pbest
Yes
III.
S.
N./year
NO
Authors
Purpose*
1/2004
Tiago
Sousa et.
al [13]
2./2007
De Falco
et al.
[34 ]
Yes
Target or maximum
epochs reached?
End
Tools
Description
GA
and
Tree
Induction
Algorithm
and J48a
Java
implementa
tion of C4.5
ANN
Proposed
particle
Swarm Optimizer as a
new tool for Data
Mining algorithm for
classification tasks or
rule discovery.
Proposed
Particle
Swarm Optimization
to face the problem of
classification
of
instances in databases.
1817
IJRITCC |April 2015, Available @ http://www.ijritcc.org
_____________________________________________________________________________________
ISSN: 2321-8169
1816 1820
_____________________________________________________________________________________________
3/2007
Qi Shen
et al.
[ 35]
Support
Vector
machine
(SVM)
Proposed
discrete
particle
swarm
optimization
(PSO)
with support vector
machines (SVM) for
tumor classification.
4/2008
Qi Shen
et al.
[36 ]
Pure Tabu
search (TS)
and hybrid
PSO
Proposed a hybrid
PSO
and
TS
(HPSOTS) approach
for gene selection for
tumor classification.
5/2008
Li-Yeh
Chuang
et al.
[37]
Binary
PSO,
KNN, BPNN
and PNN
Proposed
binary
particle
swarm
optimization with KNN method for gene
expression
data
classification
problems.
6 /2010
Alper
Unler
,Alper
Murat
[38]
logistic
regression
Proposed
modified
discrete particle swarm
optimization
(PSO)
algorithm
for
investigates the feature
subset
selection
problem for the binary
classification problem
using
logistic
regression model.
7/2011
Alper
Unler
et.al.[39]
SVM
Proposed a hybrid
filter wrapper feature
subset
selection
algorithm based on
PSO and SVM.
8/2012
PeiChann
Chang et
al [40]
C, F
Case base
reasoning(
CBR)
Proposed integrating a
case-based reasoning
and particle swarm
optimization
based
hybrid
model
for
medical
data
classification.
9/2013
Abdulha
mit
Subasi
[41]
SVM, PSO
,K-NN
10/2014
Bing
Xue, et.
al
[42 ]
F,C
LDA,
KNN, PSO
11/2014
H.
Hannah
Inbarani
et al.
[43 ]
Subhajit
Kar et al.
[ 44]
Nave
Bayes,
Rough set
and PSO
Proposed A novel
PSO-SVM model that
has been hybridized
the PSO and SVM to
improve the EMG
signal
classification
accuracy.
Proposed a PSO based
feature
selection
approach to selecting a
smaller number of
features for better
classification
performance.
Introduced hybridized
approach
for
the
diseases diagnosis.
PSO, KNN
,SVM
12/2015
13/2015
Yong
Zhang
PSO, 1 NN
Proposed a PSO
adaptive
K-nearest
neighborhood (KNN)
based gene selection
method for proper
classification
of
microarray data.
Proposed binary BPSO
method to find optimal
Dunwei
Gong
et.al.[45]
V.
feature subset.
CONCLUSION
_____________________________________________________________________________________
ISSN: 2321-8169
1816 1820
_____________________________________________________________________________________________
algorithms GAs, it starts with particles (like chromosomes in
GA) flying with an initial random velocities in various
directions (locations). Both locations and velocities are updated
with iterations until a satisfactory solution is reached. The
applications of PSO to feature selection and classification have
been widely published and briefly presented in this paper. PSO
with GA and other techniques when hybridized together can
produce better results compared to technique applied alone.
Such hybridizations on few areas have been provided in this
paper. It can be concluded that slowly but constantly PSO is
also going to become an essential tool for optimization
problems and that will be a challenging task for researchers.
REFERENCES
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
1819
IJRITCC |April 2015, Available @ http://www.ijritcc.org
_____________________________________________________________________________________
ISSN: 2321-8169
1816 1820
_____________________________________________________________________________________________
[41] Abdulhamit Subasi, Classification of EMG signals using PSO optimized
SVM for diagnosis of neuromuscular disorders, Computers in Biology
and Medicine, 43 , 576586, 2013.
[42] Bing Xue, Mengjie Zhang, Will N. Browne, Particle swarm
optimisation for feature selection in classification:Novel initialisation and
updating mechanisms, Applied Soft Computing, 18 , 261276, 2014.
[43] H. Hannah Inbarani , Ahmad Taher Azar, G. Jothi , Supervised hybrid
feature selection based on PSO and rough sets for medical diagnosis,
computer methods and programs in biomedicine, 113 , 175185, 2014.
[44] Subhajit Kar , Kaushik Das Sharma, Madhubanti Maitra, Gene selection
from microarray gene expression data for classification of cancer
subgroups employing PSO and adaptive K-nearest neighborhood
technique, Expert Systems with Applications, 42 , 612627, 2015.
[45] Yong Zhang, Dunwei Gong, Ying Hu,Wanquiu Zhang, Feature selection
algorithm based on bare bones particle swarm optimization,
Neurocomputing, 146 , 150-157, 2015.
[46] Marco Dorigo and Thomas Stutzle, Ant colony optimization, A
Bradford Book, 2004.
[47] S. Kavehand, Talatahari, Particle swarm optimizer, ant colony strategy
and harmony search scheme hybridized for optimization of truss
structures, Computer Structure, 87, 267-283,2009.
[48] K. V. Price, R. M. Storn, and J. A. Lampinen, Differential Evolution:A
Practical Approach to Global Optimization, Springer,
Berlin,
Germany, 2005.
[49] H. Talbi, M. Batouche, Hybrid particle swarm with differential evolution
for multimodal image registration, Proceedings of the IEEE International
Conference on Industrial Technology 3 , 15671573, 2004.
[50] Hajizadeh, A, Norum, L.,Golkar, M.A., A fuzzy-PSO based controller
for hybrid fuel cell power systems during voltage sag, IEEE
International Conference on Clean Electrical Power , pp. 147 153, 2009.
[51] S. Das, A. Abraham, S.K. Sarkar, A Hybrid Rough Set--Particle Swarm
Algorithm for Image Pixel Classification, IEEE Sixth International
Conference on, Hybrid Intelligent Systems 2, HIS '06, pp. 26 DOI:
10.1109/HIS.2006.264909, 2006.
[52] M. Hajihassani, D. Jahed Armaghani, H. Sohaei, E. Tonnizam Mohamad,
A. Marto, Prediction of airblast-overpressure induced by blasting using a
hybrid artificial neural network and particle swarm optimization,
Applied Acoustics, 80 , 5767, 2014.
[53] Yu XM, Xiong XY, Wu YW. A PSO - based approach to optimal
capacitor placement with harmonic distortion consideration, Electr
Power Sys Res 2004, 71(1):2733, 2004.
[54] Shichang Sun, Ajith Abraham, Guiyong Zhang, Hongbo Liu, A Particle
Swarm Optimization Algorithm for Neighbor Selection in Peer-to-Peer
Networks, IEEE, 6th International Conference on Computer Information
Systems and Industrial Management Applications (CISIM'07), 0-76952894-5/07 $20.00 2007, IEEE., 2007.
[55] Peng-Yeng Yin, Jing-Yu Wang, A particle swarm optimization
approach to the nonlinear resource allocation problem, Applied
Mathematics and Computation, 183 , 232242, 2006.
1820
IJRITCC |April 2015, Available @ http://www.ijritcc.org
_____________________________________________________________________________________