Applied Soft Computing 30 (2015) 737747

Contents lists available at ScienceDirect

Applied Soft Computing

journal homepage: www.elsevier.com/locate/asoc

A framework for multi-document abstractive summarization based

on semantic role labelling
Atif Khan a, , Naomie Salim a , Yogan Jaya Kumar b
a
b

Faculty of Computing, Universiti Teknologi Malaysia, 81310 Skudai, Johor, Malaysia

Faculty of Information and Communication Technology, University Teknikal Malaysia Melaka, 76100 Melaka, Malaysia

a r t i c l e

i n f o

Article history:
Received 18 May 2014
Received in revised form 7 September 2014
Accepted 29 January 2015
Available online 20 February 2015
Keywords:
Abstractive summary
Semantic role labeling
Semantic similarity measure
Language generation
Genetic algorithm

a b s t r a c t
We propose a framework for abstractive summarization of multi-documents, which aims to select contents of summary not from the source document sentences but from the semantic representation of
the source documents. In this framework, contents of the source documents are represented by predicate argument structures by employing semantic role labeling. Content selection for summary is made
by ranking the predicate argument structures based on optimized features, and using language generation for generating sentences from predicate argument structures. Our proposed framework differs from
other abstractive summarization approaches in a few aspects. First, it employs semantic role labeling for
semantic representation of text. Secondly, it analyzes the source text semantically by utilizing semantic
similarity measure in order to cluster semantically similar predicate argument structures across the text;
and nally it ranks the predicate argument structures based on features weighted by genetic algorithm
(GA). Experiment of this study is carried out using DUC-2002, a standard corpus for text summarization.
Results indicate that the proposed approach performs better than other summarization systems.
2015 Elsevier B.V. All rights reserved.

1. Introduction
The information on Web is growing at exponential pace. In the
current era of information overload, multi-document summarization is an essential tool that creates a condensed summary while
preserving the important contents of the source documents. The
automatic multi-document summarization of text is a major task in
the eld of natural language processing (NLP) and has gained more
consideration in recent years [1]. One of the problems of information overload is that many documents share similar topics, which
creates both difculties and opportunities for natural language systems. On one hand, the similar information conveyed by several
different documents, causes difculties for the end users, as they
have to read the same information repeatedly. On the other side,
such redundancy can be used to identify accurate and signicant
information for applications such as summarization and question
answering. Thus, summaries that synthesize common information
across many text documents would be useful for users and reduce
their time for nding the key information in the text documents.
Such a summary would signicantly help users interested in single

Corresponding author. Tel.: +60 1126799325.

E-mail addresses: [email protected] (A. Khan), [email protected] (N. Salim),
[email protected] (Y. Jaya Kumar).
http://dx.doi.org/10.1016/j.asoc.2015.01.070
1568-4946/ 2015 Elsevier B.V. All rights reserved.

event described in many news documents [1]. In this paper, we propose a framework that will automatically fuse similar information
across multiple documents and use language generation to produce
a concise abstractive summary.
Two approaches are employed to multi-document summarization: extractive and abstractive. Most of the studies have focused
on extractive summarization, using techniques of sentence extraction [2], statistical analysis [3], discourse structures and various
machine learning techniques [4]. On other hand, abstractive summarization is a challenging area and dream of researchers [5],
because it requires deeper analysis of the text and has the capability to synthesize a compressed version of the original sentence or
may compose a novel sentence not present in the original source.
The goal of abstractive summarization is to improve the focus of
summary, reduce its redundancy and keeps a good compression
rate [6].
Past literature shows that there have been a few research efforts
made toward abstractive summarization. Many researchers have
tried to generate abstractive summaries using various methods.
These abstractive methods can be grouped into two categories: Linguistic (Syntactic) based approach and Semantic based approach.
Linguistic (Syntactic) based approach employs syntactic parser
to analyze and represent the text syntactically. Usually, in this
approach, verbs and nouns identied by syntactic parser are used
for text representation and further processed to generate the

738

A. Khan et al. / Applied Soft Computing 30 (2015) 737747

abstractive summary. On other hand, semantic based approach

aims to produce abstractive summary from semantic representation of document text. Different semantic representations of text
used in the literature are ontology based and template based representation. Titov and Klementiev [7] made a distinction between
syntactic and semantic representation of sentence. They addressed
that syntactic analysis is far away from representing the meaning
of sentences. In particular, syntactic analysis does not dene who
did what to whom (and how, when. . .).
All the linguistic based approaches [1,6,8,9] proposed for the
abstractive summarization rely on the syntactic representation of
source document. These approaches employ syntactic parser to
represent the source text syntactically. The major limitations of
these approaches is the lack of semantic representation of source
text. Since abstractive summarization requires deep analysis of
text, therefore, semantic representation of source text will be a
more suitable representation.
On other hand, a few semantic based approaches have also been
proposed for abstractive summarization and are briey discussed
as follows. A multi-document summarization system, GISTEXTER,
presented in [10] exploits template based method to produce
abstractive summary from multiple newswire/newspaper documents depending on the output of the information extraction (IE)
system. This approach use template for topic representation of document. The major limitation observed in this approach was that
linguistic patterns and extraction rules for template slots were
manually created by humans, which is time consuming. Moreover,
this method could not handle or capture the information about
similarities and differences across multiple documents.
A fuzzy ontology based approach [11] was proposed for Chinese
news summarization to model uncertain information and hence
can better describe the domain knowledge. This approach has several limitations. First, domain ontology and Chinese dictionary has
to be dened by a domain expert which is time consuming. Secondly, this approach is limited to Chinese news, and might not be
applicable to English news.
The methodology proposed in [12] generates short and wellwritten abstractive summaries from clusters of news articles on
same event using abstraction schemes. The abstraction scheme
uses a rule based information extraction module, content selection heuristics and one or more patterns for sentence generation.
The drawback of the methodology was that information extraction
(IE) rules and generation patterns were written by hand, which was
extremely time consuming.
A framework proposed by [13] generates abstractive summary
from a semantic model of a multimodal document. The semantic model is constructed using knowledge representation based on
objects (concepts) organized by ontology. The important concepts
represented by semantic model are rated based on information
density metric and expressed as sentences using the available
phrasings stored for each concept in a semantic model. The limitation of this framework was that it relies on manually built ontology,
which is time consuming. Secondly, it is manually evaluated by
humans i.e. human judges were asked to assess the quality of system summary by comparing it with other summary generated by
traditional extraction methods.
The abstractive approach presented by [14] summarizes a document by creating a Rich Semantic Graph (RSG) for the source
document. Rich semantic graph is an ontology based representation
i.e. graph nodes are the instances of ontology noun and verb classes.
Like other approaches, the limitation of this approach was that it
also relies on manually built ontology, which is time consuming.
The major limitation of all the semantic based approaches for
abstractive summarization is that they are mostly dependent on
human expert to construct domain ontology and rules; which is
a drawback for an automatic summarization system. Our work,

in contrast, aims to treat this limitation by using semantic role

labeling (SRL) technique to build semantic representation from the
document text automatically.
SRL has been widely applied in text content analysis tasks
such as text retrieval [15], information extraction [16], text categorization [17] and sentiment analysis [18]. In the area of text
summarization, [19] introduced a work that combined semantic
role labeling with general statistic method (GSM) to determine
important sentences for single document extractive summary. At
rst, they employed SRL and semantic similarity measure to compute the sentence similarity score. Secondly, the general statistic
method was used to computes the sentence score based on features,
without taking into account their weights. Finally, the sentence
scores obtained from both methods are combined to assign the
overall score to each sentence in the document and the top ranked
sentences are extracted according to 20% compression rate. However, our work is different from [19] in the following manner. We
focus on multi-document abstractive summarization while [19]
focus on single document extractive summarization. We employ
SRL in second phase of the framework, for semantic representation
of text in the document collection. On other hand, [19] employed
SRL and semantic similarity measure to compute sentence semantic similarity score. To the best of our knowledge, semantic role
labeling (SRL) technique, which exploits semantic role parser, has
not been employed for the semantic representation of text in multidocument abstractive summarization.
Therefore, this study proposes a framework that will employ
SRL for semantic representation of text in order to generate a
good abstractive summary. The framework for multi-document
abstractive summarization presented in this study, is different from
previous abstractive summarization approaches in a few aspects.
First, it employs semantic role labeling to extract predicate argument structure (semantic representation) from the contents of
input documents. Secondly, it analyzes the text semantically by utilizing semantic similarity measure in order to cluster semantically
similar predicate argument structures across the text; and nally
it ranks the predicate argument structures based on the features
weighted and optimized by genetic algorithm; since text features
are sensitive to the quality of the generated summary.
The rest of this paper is organized as follows: Section 2 outlines
the proposed framework. The evaluation of the framework is given
in Section 3. Finally we end with conclusion in Section 4.

2. Framework for multi-document abstractive

summarization
2.1. Overview of approach
The framework of our proposed approach is illustrated in Fig. 1.
Given a document collection that need to be summarized, rst of all,
we split the document collection into sentences in such a way that
each sentence is preceded by its corresponding document number
and sentence position number. Next, SENNA semantic role labeler
[20] is employed to extract predicate argument structure from each
sentence in the document collection. In semantic similarity matrix
computation phase (as discussed in Section 2.3), the similarities of
predicate argument structures (PASs) are computed by comparing
them pair wise based on Jiangs semantic similarity measure [21]
and edit distance algorithm. Once the similarity matrix for PASs is
obtained, we perform agglomerative hierarchical clustering (HAC)
algorithm based on average linkage method to cluster semantically
similar predicate argument structures. The number of clusters will
depend on compression rate of summary. Section 2.4 will describe
this phase in detail. The PASs in each cluster are scored based on
features, weighted and optimized by genetic algorithm and the top

A. Khan et al. / Applied Soft Computing 30 (2015) 737747

739

Table 1
Core arguments and adjunctive arguments.
Core Arguments
V
A0
A1
A2
A3
A4
A5

Verb
Subject
Object
Indirect Object
Start point
End point
Direction

Adjunctive arguments
ArgM-DIR Direction
ArgM-MNR Manner
ArgM-LOC Location
ArgM-TMP Temporal marker
ArgM-PRP Purpose
ArgM-NEG Negation
ArgM-REC Reciprocal
AM-DIS Discourse marker

simple predicate argument structure while a sentence containing

more than one complete predicate is represented by a composite
predicate argument as illustrated below.
Example 1. Consider sentence S: Tropical Storm Gilbert formed in
the eastern Caribbean and strengthened into a hurricane Saturday
night.

Fig. 1. Proposed framework.

ranked predicate argument structures are selected from each cluster (as described in Section 2.5). Finally, the SimpleNLG realisation
engine [22] is employed to generate sentences from the selected
predicate argument structures. The generated sentences will form
the nal abstractive summary (as discussed in Section 2.6).
2.2. Semantic role labeling
The aim of semantic role labeling (SRL) is to determine the syntactic constituents/arguments of a sentence with respect to the
sentence predicates, identify the semantic roles of the arguments
such as Agent, Patient and Instrument, and the adjunctive arguments of the predicate such as Locative, Temporal and Manner
[23]. The primary task of SRL is to identify what semantic relation
a predicate holds with its participants/constituents. As abstractive
summarization requires deeper semantic analysis of text, therefore, this study employs semantic role labeling to extract predicate
argument structure from sentences in the document collection. The
framework uses SENNA [20] tool distributed under open source
and non-commercial license, and yields a host of natural language
processing (NLP) predictions: semantic role labeling (SRL), part-ofspeech (POS) tags, named entity recognition (NER) and chunking
(CHK). We employ SENNA tool in our framework for SRL, POS tags
and NER.
At rst, we decompose the document collection into sentences
in such a way that each sentence is preceded by its corresponding
document number and sentence position number. Next, SENNA
semantic role labeler is employed to parse each sentence and
properly labels the semantic word phrases. These phrases are
referred to as semantic arguments. The semantic arguments can
be grouped in two categories: core arguments (Arg) and adjunctive
arguments (ArgM) as shown in Table 1. In this study, we consider
A0 for subject, A1 for object, A2 for indirect object as core arguments, and ArgM-LOC for location, ArgM-TMP for time as adjunctive
arguments for predicate (Verb) V. We consider all the complete
predicates associated with the single sentence structure in order
to avoid loss of important terms contributing to the meaning of
sentence, and the actual predicate of the sentence. We assume
that predicates are complete if they have at least two semantic
arguments. The extracted predicate argument structure is used
as semantic representation for each sentence in the document
collection. A sentence containing one predicate is represented by

After applying semantic role labeling to sentence S, the corresponding two predicate argument structures are obtained as
follows:
P1: [A0: Tropical Storm Gilbert] [V: form] [ArgM-LOC: in the eastern Caribbean]
P2: [A0: Tropical Storm Gilbert] [V: strengthen] [A2: into a hurricane Saturday night]
Both predicate argument structures P1 and P2 are associated
with a single sentence S and hence the sentence S is represented
by a composite (more than one) predicate argument structure. Both
incomplete predicate argument structures (PASs) and the PASs that
are nested in a larger predicate argument structure are ignored.
Example 2. Consider the following two sentences represented by
simple predicate argument structures.
S1: Eventually, a huge cyclone hit the entrance of my house.
S2: Finally, a massive hurricane attack my home
The corresponding simple predicate argument structures P1 and
P2 are obtained after applying semantic role labeling to sentences
S1 and S2:
P1: [AM-TMP: Eventually] [A0: a huge cyclone] [V: hit] [A1: the
entrance of my house]
P2: [AM-DIS: Finally] [A0: a massive hurricane] [V: attack] [A1: my
home]
2.2.1. Processing of predicate argument structures
Once the predicate argument structures (PASs) are obtained,
they are split into meaningful words or tokens, followed by removal
of stop words. The words in PASs are stemmed to their base form
using porter stemming algorithm [24]. Next, SENNA POS tagger [20]
is employed to label each term of semantic arguments (associated
with the predicates), with part of speech (POS) tags or grammatical
roles. The POS tags NN stands for noun, V for verb, JJ for adjective
and RB for adverb, etc. This step is required, as semantic arguments
of the predicates will be compared based on grammatical roles of
the terms. In this study, we compare only terms of the semantic
arguments of the predicates which are labeled as noun (NN) and
the rest are ignored as discussed in Section 2.3. After POS tagging,
the two predicate argument structures P1 and P2 in example 2 are
as follows:

740

A. Khan et al. / Applied Soft Computing 30 (2015) 737747

P1: [A0: a massive (JJ) hurricane NN] [V: attack] [A1: my home
(NN)]
P2: [AM-TMP: Eventually (RB)] [A0: a huge (JJ) cyclone (NN)]
[VBD: hit] [A1: the entrance (NN) of my house (NN)]
We also employ SENNA NER [20] to identify named entities such
as person names (cabral), and organization names (civil defense)
in the semantic arguments of predicate. These named entities are
stored for each predicate argument structure (PAS) and is required
in later phase for scoring the PAS based on proper noun feature.
This study compares predicate argument structures based on
nounnoun, verbverb, locationlocation and timetime arguments. Therefore, the framework extracts only tokens from
predicate argument structure, which are labeled as noun, verb,
location, and time as identied in previous steps. All the PASs associated with the sentence will be included in comparison. Once the
nouns, verbs, and other arguments (time and location) if exist, are
extracted, the predicate argument structures obtained in example
2 after further processing will become
P1: [A0: hurricane NN] [V:attack] [A1: home (NN)]
P2: [AM-TMP: Eventually (RB)] [A0: cyclone (NN)] [VBD: hit] [A1:
entrance (NN), house (NN)]
In next phase, the noun home in the semantic argument A1
of PAS P1 will be compared with both of the nouns entrance and
house in the semantic argument A1 of PAS P2. The temporal (time)
semantic argument Eventually in P2 is skipped from comparison
as there is no corresponding temporal argument in P1.

2.3. Computation of semantic similarity matrix

The objective of this phase is to build matrix of semantic
similarity scores for each pair of predicate argument structure.
In this phase, similarity of the predicate argument structures
(PASs) is computed pair wise based on acceptable comparisons of
nounnoun, verbverb, locationlocation and timetime. Based on
experimental results in the literature [25], Jiang and Conrath measure has the closest correlation with human judgment amongst
all the semantic similarity measures. Therefore, this study exploits
Jiangs semantic similarity measure [21] for computing semantic
similarity between each pair of PASs. Jiangs measure is information content based measure and consider that each concept in the
WordNet [26] hold certain information. According to this measure,
the similarity of two concepts is dependent on the information that
the two concepts share.
Given two sentences Si and Sj , the similarity score between predicate argument structure (PAS) k of sentence Si (v ik ) and PAS l of
sentence Sj (v jl ) is determined using Eq. (5), where simp (vik, vjl )
is the similarity between predicates (verbs) determined using Eq.
(2), simarg (vik, vjl ) is the sum of similarities between the corresponding arguments of the predicates determined using Eq. (1).
Both equations (1) and (2) exploit Jiangs semantic similarity measure for computing similarity between noun terms in the semantic
arguments of the predicate argument structures and the verbs of
predicate argument structures, respectively. Similarity between
corresponding temporal arguments i.e. simtmp (vik, vjl ) is computed
using Eq. (3) and the similarity between corresponding location
arguments i.e. simloc (vik, vjl ) is calculated using Eq. (4). Since Jiangs
measure is based on WordNet, the temporal and location arguments may not be found in the WordNet, therefore we use edit
distance algorithm instead of Jiangs measure in Eqs. (3) and (4) for
computing possible match/similarity between temporal and location arguments of the predicates.

Fig. 2. Computation of semantic similarity score of predicate argument structures.

The similarity score between the two predicate argument structures is computed using Eqs. (1)(5).
simarg (vik , vjl ) = sim(A0i , A0j ) + sim(A1i , A1j ) + sim(A2i , A2j )

(1)

simp (vik, vjl ) = (sim(Pi , Pj ))

(2)

simtmp (vik, vjl ) = (sim(Tmpi , Tmpj ))

(3)

simloc (vik, vjl ) = (sim(Loci , Locj ))

(4)

Eqs. (1)(4) are combined to give Eq. (5) as follows

sim(vik, vjl ) = simp (vik, vjl ) + [simarg (vik, vjl ) + simtmp (vik, vjl )
+ simloc (vik, vjl )]

(5)

The semantic similarity computation of the two predicate argument structures discussed in example 2 is depicted in Fig. 2.
In order to compute the similarity score of the given two
terms/concepts in the semantic argument A0 of predicates P1
and P2; let concept C1 = hurricane and concept C2 = cyclone as
shown in Fig. 2. First, Jiangs measure [21] uses WordNet to compute
the least common subsumer (lso) of two concepts, then determines
IC(C1), IC(C2), and IC(lso (C1, C2)). The information content (IC) of
concept is achieved by determining the probability of occurrence
of a concept in a large text corpus and quantied as follows:
IC(C) = log P(C)

(6)

where P(C) is the probability of occurrence of concept C and is

computed as follows:
P(C) =

Freq (C)
N

(7)

where Freq (C) is the number of occurrences of concept C in the

taxonomy and N is the maximum number of nouns.
Jiangs measure [21] calculates the semantic distance to obtain
semantic similarity between any two concepts as follows:
Jiangdist (C1, C2) = IC(C1) + IC(C2) 2 IC(lso(C1, C2))

(8)

Based on WordNet, Jiang measure determines cylone as a

least common subsumer of the given two concepts hurricane and
cyclone as shown below.
< entity < physical entity < process < phenomenon
< natural phenomenon < physical phenomenon
< atmospheric phenomenon < storm < windstorm < cyclone
< entity < physical entity < process < phenomenon
< natural phenomenon < physical phenomenon
< atmospheric phenomenon < storm < windstorm
< cyclone < hurricane

A. Khan et al. / Applied Soft Computing 30 (2015) 737747

741

In this study, we consider 20% compression rate of summary.

lso(C1, C2) = Cyclone

IC(C1) = 11.0726

2.5. Selection of PAS from each cluster based on optimized

features

IC(C2) = 10.6671
IC(lso(C1, C2)) = 10.6671
According to Eq. (8), the similarity between concepts
C1 = Hurricane and C2 = Cyclone is computed as follows:
Jiangdist = 11.0726 + 10.6671 2 10.6671 = 0.4055
The similarity of other concepts/terms is determined in the same
manner. However, simtmp (vik, vjl ) and simloc (vik, vjl ) are set 0 as there
are no temporal and location arguments for comparison in both
predicate argument structures. According to Eq. (5), the similarity
score of the given two predicate argument structures is computed
as follows:
sim(vik, vjl ) = 0.8571 + [0.4055 + 0.29 + 1 + 0 + 0]
sim(vik, vjl ) = 2.5526
In order to normalize the result in the range of [0,1], we use
a scaling factor eSim(vik, vjl ) [27], Where is constant set to 0.05
(optimal value).
= e0.05X2.5526

sim(vik, vjl ) = 0.8801

Once the semantic similarity score for each pair of predicate
argument structure (PAS) is achieved, then semantic similarity
matrix Mi,j is constructed from the similarity scores of the predicate
argument structure Pi and Pj. Mi,j is dened as follows:
Mi,j =

Msim (Pi , Pj )

if i =
/ j

otherwise

2.5.1. Text features

2.5.1.1. Title feature. This feature is determined by counting the
number of matching contents words in the predicate argument
structure (PAS) and the title of a document [33].
P F1 =

Number of title words in PAS

Number of words in document title

(10)

2.5.1.2. Length of predicate argument structure. We use the normalize length of the PAS, which is the ratio of number of words in the
PAS over the number of words in the longest PAS of the document
[2].

So, the above result will become as

The aim of this phase is to select high ranked predicate argument structures from each cluster based on features weighted and
optimized by genetic algorithm (GA). At rst, all the sentences in
the document collection are represented by corresponding predicate argument structure (PAS) extracted through SENNA SRL. The
following features are extracted for each predicate argument structure (PAS) in the document collection and hence each PAS is
represented by a vector representing the weight of these features
P = {P F1, P F2,. . .P F10}.

(9)

where Msim (Pi, Pj) refers to semantic similarity score of predicate

argument structure Pi and Pj in the Matrix Mi,j .
2.4. Semantic clustering of predicate argument structures
Agglomerative hierarchical clustering is well-known technique
in the hierarchical clustering method, which is quite old but has
been found useful in the range of applications [28]. There are ve
well-known linkage methods of agglomerative hierarchical clustering (HAC) [29] i.e. single linkage, complete linkage, average linkage,
ward and centroid method. Based on different measures (Entropy
and F-Score and Kendall W test), it was found from the literature
studies in [3032] that average linkage is the most suitable method
for document clustering. Therefore, this study exploits HAC algorithm based on average linkage method. This phase takes semantic
similarity matrix as input from previous phase in which the value
at position (i, j) is the semantic similarity between ith and jth predicate argument structures. We consider the value at position (i, j)
in the semantic similarity matrix as semantic similarity between
ith and jth clusters, assuming that the construction of similarity
matrix begins with each predicate argument structure as a single
cluster. The pseudo code for clustering similar predicate argument
structures is given below.
Pseudo code for agglomerative clustering algorithm.
Input: Semantic Similarity Matrix
Output: Clusters of similar predicate argument structures
a. Merge the two clusters that are most similar
b. Update the semantic similarity matrix to represent the pair wise similarity
between the newest cluster and the original cluster based on average linkage
method
c. Repeat step 1 and 2 until the compression rate of summary is reached

P F2 =

Number of words occuring in the PAS

Number of words occuring in the longest PAS

(11)

2.5.1.3. PAS to PAS similarity. For each predicate argument structure P, the semantic similarity between P and other predicate
argument structures in the document collection is computed using
Eq. (5). Once the similarity score for each predicate argument structure (PAS) is achieved, then the score of this feature is obtained by
computing the ratio of sum of similarities of PAS P with all other
PASs over the maximum of summary in the document collection
[34].
P F3 =


sim(Pi , Pj )


Max

(sim(Pi , Pj ))

(12)

2.5.1.4. Position of predicate argument structure. Position of PAS

[33] gives importance of the PAS in the text document and is equivalent to position of sentence from which PAS is extracted. Consider
10 sentences in the document, the score of position feature is 10/10
for the rst sentence, 9/10 for the second sentence and so on. The
score of this feature is computed as follows:
P F4 =

Length of document PAS Position + 1

Length of document

(13)

2.5.1.5. Proper nouns. The predicate argument structure that contains more proper nouns is considered as signicant for inclusion in
summary generation. This feature identies proper nouns as words
beginning with a capital letter. The score of this feature is computed
as the ratio of the number of proper nouns in the PAS over the length
of PAS [33]. Length of PAS is the number of words/terms in the PAS.
P F5 =

Number of proper nouns in the PAS

Length of PAS

(14)

2.5.1.6. Numerical data. The predicate argument structure containing numerical data such as number of people killed, is regarded as
important for inclusion in summary generation. The score for this

742

A. Khan et al. / Applied Soft Computing 30 (2015) 737747

feature is calculated as the number of numerical data in the PAS

over the length of the PAS [33].
P F6 =

Number of numerical data in the PAS

Length of PAS

Initial population of
chromosomes

(15)

Fitness evaluation
2.5.1.7. Number of nouns and verbs. Some sentences may have
more than one predicate argument structure associated with them,
represented by a composite predicate argument structure and considered important for summary. The score of this feature [34] is
computed as follows:
P F7 =

Total number of nouns and verbs in the PAS

Length of PAS

Select generation based on

fitness score

(16)

Perform reproduction using

cross over and mutation

2.5.1.8. Temporal feature. The predicates argument structure containing time and date information for an event is considered as
important for summary generation. The score of this feature is computed as ratio of the number of temporal information (time and
date) in the PAS over the length of PAS [35].
P F8 =

No

Number of temporal information in the PAS

Length of PAS

2.5.1.9. Frequent semantic term. Frequent terms are most probably

related to the topic of document and in this study we consider top
10 as maximum number of frequent semantic terms. Nouns and
verbs are considered as frequent semantic terms in the predicate
argument structure. The score of this feature is calculated as the
ratio of number of frequent semantic terms in the PAS over the
maximum number of frequent semantic terms [23].
P F9 =

Number of frequent semantic terms in the PAS

Max(Number of frequent semanticterms)

(18)

2.5.1.10. Semantic term weight. The score of important term Wi can

be determined by the TF-IDF method [36]. We apply TF-IDF method
to the predicate argument structures in the document collection
and consider the term weights for semantic terms i.e. nouns and
verbs in the predicate argument structure.

Population =

Yes
Terminate?

(17)

Optimized feature
weights

Fig. 3. Genetic algorithm.

genetic learning algorithm (GA) to obtain optimal feature weights

for ranking predicate argument structures for summary generation. GA is chosen since it is a robust and a well-known adaptive
optimization technique used in various elds of research and applications [37]. This particular GA based experiment is evaluated
against multi-documents in DUC 2002 data set [38]. The GA procedure is shown in Fig. 3.
2.5.2.1. Initial population of chromosomes. In the rst step of GA, a
real valued population consisting of 50 chromosomes is randomly
produced. The chromosomes are randomly initialized with values
between 0 and 1. Each individual chromosome represents weights
of features in the form of (w1 , w2 , . . ., w10 ). To produce a random
population of N chromosomes, we have for instance:

0.25 0.08 0.74 0.53 0.26 0.32 0.42 0.73 0.21 0.62

Chromosome 1

0.77 0.67 0.03 0.22 0.31 0.01 0.02 0.73 0.51 0.42

Chromosome 2

Chromosome N

.
..
0.54 0.87 0.52 0.45 0.33 0.71 0.39 0.49 0.83 0.18
The weight of semantic term is calculated as follows:
N
Wi = Tfi x Idfi = Tfi x log
ni

(19)

where Tfi is the term frequency of the semantic term i in the document, N is the total number of documents, and ni is the number of
documents in which the term i occurs. This feature is computed as
the ratio of sum of weights of all semantic terms in the PAS over the
maximum summary of the term weights of PAS in the document
collection [33].

i=1

P F10 =

Wi (P)

i=1

Max (

where each row represents the chromosome and each column

value correspond to the weight of each feature.
2.5.2.2. Fitness evaluation. The tness evaluation function in GA
determines the best chromosome in the population based on its tness value. The chromosome having highest tness value has more
chances to survive and continue in next generation.
We dene tness function F(x) as the average recall obtained
with each chromosome when summarization process is applied on
multi-documents and is given in Eq. (21).

(20)

Wi (P))

where k is the number of semantic terms in the PAS P.

2.5.2. Genetic algorithm for optimal feature weighting
In automatic text summarization, the quality of the summary
is sensitive to the text features i.e. not all features have same
relevance with respect to summary. Therefore, feature weighting
is crucial for summary generation. In this work, we will employ

F(x) =

Countmatch (gramn )

S{Reference Summaries}gramn S

(21)
Count(gramn )

S{Reference Summaries}gramn S

where n is the length of the n-gram, Countmatch is the number of

n-grams shared between system generated summary and reference summaries, and Count is the total number of n-grams in the
reference summaries.

A. Khan et al. / Applied Soft Computing 30 (2015) 737747

2.5.2.3. Selection. This operation determines the chromosome that

will continue to survive in next generation. Stochastic universal
sampling (SUS) method [39] has been used to select parents from
the current population. This method uses M equally spaced steps in
the range [0,Sum], where M is the number of required selections and
Sum is the sum of the scaled tness values over all the individual
chromosomes in the current population.
2.5.2.4. Reproduction operations. These operations describe how
GA creates its next generation. In this study, we use two reproduction operations i.e. cross over and mutation. The cross over
operation produces new chromosomes from the two parent chromosome by exchanging information in the parent chromosomes.
Mutation operation is performed by altering gene of chromosome
by another to generate a new genetic structure.
2.5.2.5. Termination criteria. This process will continue to produce
new generations and evaluate its tness until maximum limit of
generations is reached. In this study, we used 100 maximum generations. When the process nishes, the individual chromosome with
the best tness value is chosen as the optimal feature weights. The
obtained optimal feature weights are given in Section 3.2. Once the
feature weights are obtained, then Eq. (22) is used to determine the
score of predicate argument structures in each cluster.
Score(Pi ) =

10


Wk xP Fk (Pi )

743

Initialization of Basic Constituents

Set Features of Constituents

Combine Constituents into larger

Structures using Simple NLG Realiser
Fig. 4. Process of SimpleNLG engine.

arguments, and ArgM-LOC for location, ArgM-TMP for time as

adjunctive arguments for predicate (Verb) V while the rest of the
arguments are ignored. Thus, the nal summary sentences generated from the predicate argument structures will be compressed
version of the original source sentences in most cases. During summary sentence generation process through SimpleNLG, the simple
heuristic rule implemented in SimpleNLG combine the predicate
argument structures that refer to the same subject (entity). The
following example demonstrates how we generate abstractive
summary from the given source input sentences. For instance, the
following source input sentences:

(22)

k=1

where P Fk (Pi ) is score of feature k for predicate Pi, Wk is the weight

of feature k. The top scored predicate argument structures are
selected from each cluster and are given as input to the summary
generation process in next phase.
2.6. Abstractive summary generation
This phase demonstrates how arguments of the predicates will
be combined, transformed and realize them as summary sentences.
This phase takes top scored predicate argument structures (PASs)
from previous phase, employ SimpleNLG [22] and a simple heuristic
rule implemented in SimpleNLG, to generate summary sentences
from PASs. The heuristic rule states that if the subjects in the predicate argument structures (PASs) refer to the same entity, then
merge the predicate argument structures by removing the subject
in all PASs except the rst one, separating them by a comma (if there
exist more than two PASs) and then combine them using connective
and.
SimpleNLG is an English realisation engine which provides simple interfaces to produce syntactical structures and transform them
into sentences using simple grammar rules. The realisation engine
has excellent coverage of English syntax and morphology, and at the
same time offers API to users, which aim to provide a programming
control over the realisation process.
Moreover, the signicant advantage of this engine is its robustness i.e. the engine will not crash when the input syntactical
structures are incomplete or ill-formed but typically will yield an
inappropriate output. The process of SimpleNLG engine consists of
three main steps as depicted in Fig. 4.
The rst step in SimpleNLG engine requires dening the basic
syntactic constituents that will construct a sentence. The second
step assigns features to the constituents. The different features
that can be set for constituents are: subject, object verb, tense,
etc. In the last step, the SimpleNLG realiser combines the syntactic constituents using simple grammar rules and realize them as
syntactically correct sentence.
As discussed in Section 2.2, we consider specic arguments
i.e. A0 for subject, A1 for object, A2 for indirect object as core

S1: Hurricane Gilbert claimed to be the most intense storm on

record in terms of barometric pressure.
S2: Hurricane Gilbert slammed into Kingston on Monday with
torrential rains and 115 mph winds.
S3: Hurricane Gilbert ripped roofs off homes and buildings.
After applying SENNA SRL, the corresponding three predicate
argument structures P1, P2 and P3 are obtained as follows:
P1: [A0: Hurricane Gilbert] [V: claimed] [A1: to be the most
intense storm on record]
P2: [A0: Hurricane Gilbert] [V: slammed] [A1: into Kingston] [AMTMP: on Monday]
P3: [A0: Hurricane Gilbert] [V: ripped] [A1:roofs off homes and
buildings]
We assume that P1, P2 and P3 are the top scored predicate argument structures selected from different clusters in previous step.
According to the rule stated above, the subject A0 is identied as
repeated in the above example and is eliminated from all predicate
argument structures except the rst one. The SimpleNLG applies
the heuristic rule on the above three predicate argument structure
and form the summary sentence that is compression version of the
original source sentences.
2.6.1. Summary sentence
Hurricane Gilbert claimed to be the most intense storm on
record, slammed into Kingston on Monday and ripped roofs off
homes and buildings.
The next example demonstrates how the SimpleNLG compresses a single sentence from constituents (semantic arguments)
of the sentence predicate.
For instance the source sentence Floods prevented ofcials
from reaching the hotel zone in Cancun and there were no relief
efforts under way by late Wednesday.
After applying SENNA SRL, the corresponding two predicate
argument structures P1 and P2 obtained from the above sentence
are as follows:

744

A. Khan et al. / Applied Soft Computing 30 (2015) 737747

P1: [A0: Floods] [V: prevented] [A1: ofcials] [A2: from reaching
the hotel zone in Cancun]
P2: [A0: ofcials] [V: reaching] [A2: the hotel zone in Cancu]

Step 8: This step denes the constituent from reaching the hotel
zone in Cancun along with the specied indirect object feature.
p.setIndirectObject(from reaching the hotel zone in Cancun);

The second predicate argument structure P2 is ignored as it

is subsumed by the rst predicate argument structure P1. Hence,
the input source sentence is represented by one representative
predicate argument structure P1 as discussed in Section 2.2. The
SimpleNLG will take the semantic arguments (A0, A1, A2) of predicate (V: prevented) in P1 as constituents to form the resultant
sentence as shown in the sample code below.

Step 9: This step uses realiser object created in step3, which

takes different constituents stored in object p obtained in steps 5,6
7 and 8, assemble them using simple grammar rules embodied in
simple NLG, and nally the sentence is realized to make the result
syntactically and morphologically correct:
String output = realiser.realiseSentence(p);

Sample Code Listing

1.
2.
3.
4.
5.
6.
7.
8.

Lexicon lexicon = Lexicon.getDefaultlexicon();

NLG Factory nlg Factory = new NLG Factory(lexicon);
Realiser realiser = new Realiser (lexicon);
SPhraseSpec p=nlgFactory.createClause();
p.setSubject (Floods);
p.setVerb (prevented);
p.setObject (ofcials);
p.setIndirect Object (from reaching the hotel zone in Cancun);
9. String output=realiser.realsieSentence(p);

The stepwise detail of the above code is as follows:

Step 1. Simple NLG comes with a built in lexicon that contain
information (part of speech) about words, and accessed via:
Lexicon lexicon = Lexicon.getDefaultLexicon();
Step 2. Once we get a lexicon, we can create an NLGFactory
object to create SimpleNLG structures.
NLGFactory nlgFactory = new NLGFactory(lexicon);
Step 3. This step creates a realiser object that will transform/combine SimpleNLG structures (constituents of sentence)
into text.
Realiser realiser = new Realiser(lexicon);
Step 4: This step creates a simple NLG construct SPhraseSpec by
calling a method of nlgFactory object, which allows us to dene the
syntactic constituents that will form a sentence.
SPhraseSpec p = nlgFactory.createClause();
Step 5: This step denes the constituent Floods along with the
specication of subject feature.
p.setSubject(Floods);
Step 6: This step denes the constituent prevented along with
the specication of verb feature.
p.setVerb(prevented);
Step 7: This step denes the constituent ofcials along with
the specied object feature.
p.setObject(ofcials);

The output variable will contain the generated sentence i.e.

Floods prevented ofcials from reaching the hotel zone in Cancun.
Once the summary sentences are generated from predicate
argument structures, they are re-arranged based on the position of
predicate argument structures. The position of predicate argument
structure (PAS) is determined from the position of source sentence
in the document collection, from which this PAS is extracted. In
case, the summary sentence is generated from more than one predicate argument structures then its probable position in the summary
is determined based on the position of the rst predicate argument
structure.
3. Results and discussion
3.1. Experimental settings
This section illustrates the evaluation of our proposed framework for abstractive summarization. The proposed framework
is evaluated with DUC 2002 data set [38]. The Document
Understanding Conference (DUC) is a standard corpus used in
text summarization research, which contains documents along
with their human model summaries. DUC 2002 data set contains 59 document sets produced by the National Institute
of Standards and Technology (NIST). Each document set contains documents, single document abstracts, and multi-document
abstracts/extracts with sets dened by different types of criteria
such as event sets and biographical sets. Three tasks were evaluated in DUC-2002: (1) fully automatic summarization of single
newswire/newspaper document, (2) fully automatic summarization of multiple newswire/newspaper documents on single subject
by generating document extracts and (3) fully automatic summarization of multiple newswire/newspaper documents on single
subject by generating document abstracts. The most suitable data
set chosen for our work is DUC 2002 as it refers to task3 (multidocument abstractive summarization on single subject) dened
only for DUC 2002. However, the rest of DUC data sets deal with
other summarization tasks such as question-answering and update
summarization.
At rst, the document collection is segmented into sentences
in such a way that each sentence is preceded by its corresponding
document number and sentence position number. The SENNA
semantic role labeler is employed to extract predicate argument
structure from each sentence in the document collection. In
semantic similarity matrix computation phase (as discussed in
Section 2.3), the similarities of predicate argument structures
(PASs) are computed by comparing them pair wise based on Jiangs
semantic similarity measure and edit distance algorithm. Once the
similarity matrix for PASs is obtained, we perform agglomerative
hierarchical clustering algorithm based on average linkage method
to cluster semantically similar predicate argument structures as

A. Khan et al. / Applied Soft Computing 30 (2015) 737747

discussed in Section 2.4. In order to distinguish among signicant

and less-signicant features, we nd optimize weights for features
by running the genetic algorithm (GA). The training and testing of
GA is performed on 59 multi-documents (obtained from DUC 2002)
using 10-fold cross validation. The initial population consists of 50
chromosomes which were randomly initialized with the real values
between 0 and 1. The tness function is the average recall obtained
with each chromosome when summarization process is applied to
the training corpus. The tness function evaluates the quality of
each chromosome by determining its tness value. Based on tness value, the best chromosomes from the current population are
selected as parents for the next generation. Selected chromosomes
are then reproduced using cross over and mutation operations in
each generation. In order to allow the tness value to converge,
we run 100 maximum generations before terminating the GA
process. The individual chromosome with the best tness value
is chosen as the optimal feature weights. The optimized features
weights are then used to score and select the top ranked predicate
argument structure from each cluster. The selected PASs are fed to
SimpleNLG realisation engine for summary sentence generation.
The two standard evaluation metrics, Recall-OrientedUnderstudy for Gisting Evaluation (ROUGE) [40] and Pyramid
[41] have been widely used in the context of evaluation of text
summary. Previous research studies showed that ROUGE metric
was employed for the evaluation of extractive summaries. ROUGE
score is the n-gram exact matches between system summary and
human model summaries. This metric cannot capture semantically
equivalent sentences. On other hand, Pyramid metric is used for
the evaluation of abstractive summaries. The obvious advantage
of Pyramid metric over the ROUGE is that it can capture different
sentences in the summaries that uses different words but express
similar meanings [41]. Since our work focus on multi-document
abstractive summarization, therefore, we employ Pyramid evaluation metric for the evaluation of our proposed framework. This
metric measures the quality of system generated summary by
comparing it with human model summaries.
Pyramid score (Mean Coverage Score) [41] for peer summary or
candidate summary is computed as follows.
Mean Covarge Score =

Total Peer SCUs Weight

Average SCU in the Model Summary

(23)

where SCUs refers to the summary content units and their weights
correspond to number of model (human) summaries they appeared
in.
The precision for peer summary [41] or candidate summary is
computed as follows.
Precision =

Number of Model SCUs expressed in Peer Summary

Average SCU in the Peer Summary
(24)

745

Fig. 5. Optimal feature weights obtained using genetic algorithm.

Table 2
Comparison of multi-document abstractive summarization results in DUC 2002
based on mean coverage score and average precision.
Method

Mean coverage score

Average precision

Models
AS GA SRL
AS SRL
Best
Avg

0.69
0.50
0.44
0.28
0.17

0.85
0.70
0.60
0.75
0.67

systems participating in multi-document abstractive summarization task.

To compare the performance of our proposed framework
(AS GA SRL), we setup three comparison models (AS SRL, Best and
Avg) besides the average of human model summaries (Models). For
comparative evaluation, Table 2 shows the mean coverage score
and average precision obtained on DUC 2002 dataset for our proposed framework (AS GA SRL), the proposed framework without
the integration of GA (AS SRL), the best automatic summarization
system (Best) in DUC 2002, the average of automatic systems (Avg)
in DUC 2002, and the average of human model summaries (Models). Note that proposed framework without the integration of GA
assumes that all features have equal weights. Fig. 6 visualizes the
summarization results obtained with the proposed framework and
other comparison models.
3.3. Discussion
This section discusses the results presented in previous section.
Table 2 presents the comparative evaluation of proposed framework and other summarization systems based on mean coverage
score and precision measures, in the context of multi-document
abstractive summarization task in DUC 2002. It could be observed

3.2. Results
The results of optimal feature weighting using genetic algorithm are depicted in Fig. 5. The optimal feature weights obtained
are 0.121493, 0.41313752, 0.7118985, 0.351493, 0.24141418,
0.18693995, 0.134772, 0.614672, 0.475472, 0.21308642 correspond to the weights for title, PAS to PAS similarity, position, proper
nouns, numerical data, temporal feature, length, nouns and verbs,
TF IDF, frequent semantic term, respectively.
The proposed framework is evaluated in the context of
multi-document abstractive summarization task, using 59 news
articles/data sets provided by the Document Understanding Evaluations 2002 (DUC, 2002). For each data set, our framework
generates a 100 words summary, the task undertaken by other

Fig. 6. Comparison of summarization results based on mean coverage score and

average precision.

746

A. Khan et al. / Applied Soft Computing 30 (2015) 737747

from the results in Table 2 that on mean converge score, the

proposed framework (AS GA SRL) outperforms the comparison
models and came second to the average of human model summaries (Models). However, on precision, the performance of the
proposed framework (AS GA SRL) drops as compare to the Best system but still better than average of automatic systems (Avg) and
came third to the average of human model summaries (Models).
These results suggest that the summary produced by our proposed
framework (AS GA SRL) is more akin to way humans produce summary as compared to other comparison models (AS SRL, Best and
Avg).
In order to investigate the impact of GA on summarization, we
dropped GA component from the proposed framework for multidocument abstractive summarization, and called it AS SRL. It could
be observed from Table 2, that the performance of the framework
dropped by 12% (from 0.50 to 0.44) on mean coverage score and 14%
(from 0.70 to 0.60) on precision. In order to validate the results, we
also carried out statistical signicance tests (T-Tests) to show the
improvement of our proposed framework (AS GA SRL) with other
comparison model (AS SRL). The Paired-Samples T-Test procedure
is used to compare the means of two results that represent the
same test group and obtained low signicance values of 0.031, and
0.021 for mean coverage and precision, respectively. The low signicance values for the T-test (typically less than 0.05) show that
there is a signicant difference between the results of the proposed
framework (AS GA SRL) and the other comparison model (AS SRL).
This proves that integration of GA into the proposed framework for
abstractive summarization (AS SRL) improves the summarization
results.
4. Conclusion
Although fully abstractive summarization is a big challenge, our
proposed framework shows the feasibility of this new direction
for summarization research. We believe that the proposed work
aims at the real goals of automatic summarization controlling
the content and structure of the summary. Experimental results
show that the proposed framework outperforms the other comparison models and came second to the average of human model
summaries.
Moreover, We exploited GA in our proposed framework to
determine the importance of each feature by assigning them appropriate weights, and then the optimized feature weights were used
to select the top ranked predicate argument structures from each
cluster for summary generation phase. The results indicate that the
integration of GA in the proposed SRL based framework improves
summarization results when compared with the SRL based framework without GA. In future, we plan to integrate graph with SRL
to build a semantic graph for multi-document abstractive summarization.
Acknowledgement
This research is supported by the Universiti Teknologi Malaysia
(UTM).
References
[1] R. Barzilay, K.R. McKeown, Sentence fusion for multidocument news summarization, Comput. Linguist. 31 (2005) 297328.
[2] J. Kupiec, J. Pedersen, F. Chen, A trainable document summarizer, in: Proceedings of the 18th Annual International ACM SIGIR Conference on Research and
Development in Information Retrieval, 1995, pp. 6873.
[3] K. Knight, D. Marcu, Statistics-based summarization-step one: sentence compression, in: Proceedings of the National Conference on Articial Intelligence,
2000, pp. 703710.

[4] B. Larsen, A trainable summarizer with knowledge acquired from robust NLP
techniques, in: Advances in Automatic Text Summarization, MIT Press, 1999,
pp. 71.
[5] H.P. Luhn, The automatic creation of literature abstracts, IBM J. Res. Dev. 2
(1958) 159165.
[6] P.-E. Genest, G. Lapalme, Framework for abstractive summarization using textto-text generation, in: Proceedings of the Workshop on Monolingual Text-ToText Generation, 2011, pp. 6473.
[7] I. Titov, A. Klementiev, A Bayesian approach to unsupervised semantic role
induction, in: Proceedings of the 13th Conference of the European Chapter of
the Association for Computational Linguistics, 2012, pp. 1222.
[8] R. Barzilay, K.R. McKeown, M. Elhadad, Information fusion in the context of
multi-document summarization, in: Proceedings of the 37th Annual Meeting
of the Association for Computational Linguistics on Computational Linguistics,
1999, pp. 550557.
[9] H. Tanaka, A. Kinoshita, T. Kobayakawa, T. Kumano, N. Kato, Syntax-driven
sentence revision for broadcast news summarization, in: Proceedings of
the 2009 Workshop on Language Generation and Summarisation, 2009,
pp. 3947.
[10] S.M. Harabagiu, F. Lacatusu, Generating single and multi-document summaries
with GISTEXTER, in: Document Understanding Conferences, 2002.
[11] C.-S. Lee, Z.-W. Jian, L.-K. Huang, A fuzzy ontology and its application to
news summarization, IEEE Trans. Syst. Man Cybern. B: Cybern. 35 (2005)
859880.
[12] P.-E. Genest, G. Lapalme, Fully abstractive approach to guided summarization,
in: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers vol. 2, 2012, pp. 354358.
[13] C.F. Greenbacker, Towards a framework for abstractive summarization of multimodal documents, in: ACL HLT 2011, 2011, p. 75.
[14] I.F. Moawad, M. Aref, Semantic graph reduction approach for abstractive text
summarization, in: 2012 Seventh International Conference on Computer Engineering & Systems (ICCES), 2012, pp. 132138.
[15] S. Shehata, F. Karray, M.S. Kamel, An efcient concept-based retrieval
model for enhancing text retrieval quality, Knowl. Inf. Syst. 35 (2013)
411434.
[16] L. Del Corro, R. Gemulla, ClausIE: clause-based open information extraction, in:
Proceedings of the 22nd International Conference on World Wide Web, 2013,
pp. 355366.
[17] J. Persson, R. Johansson, P. Nugues, Text Categorization Using
PredicateArgument Structures, vol. 1, 2008, pp. 142149.
[18] N. Jadhav, P. Bhattacharyya, Dive deeper: deep semantics for sentiment analysis, ACL 2014 (2014) 113.
[19] N. Salim, SRLGSM: a hybrid approach based on semantic role labeling and
general statistic method for text summarization, J. Appl. Sci. 10 (2010) 166173.
[20] R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, P. Kuksa, Natural language processing (almost) from scratch, J. Mach. Learn. Res. 12 (2011)
24932537.
[21] J.J. Jiang, D.W. Conrath, Semantic Similarity Based on Corpus Statistics and
Lexical Taxonomy, 1997 arxiv:cmp-lg/9709008.
[22] A. Gatt, E. Reiter, SimpleNLG: a realisation engine for practical applications, in:
Proceedings of the 12th European Workshop on Natural Language Generation,
2009, pp. 9093.
[23] C. Aksoy, A. Bugdayci, T. Gur, I. Uysal, F. Can, Semantic argument
frequency-based multi-document summarization, in: 24th International Symposium on Computer and Information Sciences 2009 (ISCIS 2009), 2009,
pp. 460464.
[24] M.F. Porter, Snowball: A Language for Stemming Algorithms, 2001.
[25] Y. Li, Z.A. Bandar, D. McLean, An approach for measuring semantic similarity
between words using multiple information sources, IEEE Trans. Knowl. Data
Eng. 15 (2003) 871882.
[26] G.A. Miller, WordNet: a lexical database for English, Commun. ACM 38 (1995)
3941.
[27] P. Achananuparp, X. Hu, C.C. Yang, Addressing the variability of natural
language expression in sentence similarity with semantic structure of the sentences, in: Advances in Knowledge Discovery and Data Mining, Springer, 2009,
pp. 548555.
[28] F. Murtagh, V. Contreras, Methods of Hierarchical Clustering, 2011,
arxiv:1105.0121.
[29] S. Takumi, S. Miyamoto, Top-down vs bottom-up methods of linkage for
asymmetric agglomerative hierarchical clustering, in: 2012 IEEE International
Conference on Granular Computing (GrC), 2012, pp. 459464.
[30] M. Steinbach, G. Karypis, V. Kumar, A comparison of document clustering techniques, in: KDD Workshop on Text Mining, 2000, pp. 525526.
[31] Y. Zhao, G. Karypis, U. Fayyad, Hierarchical clustering algorithms for document
datasets, Data Mining Knowl. Discov. 10 (2005) 141168.
[32] A. El-Hamdouchi, P. Willett, Comparison of hierarchic agglomerative clustering
methods for document retrieval, Comput. J. 32 (1989) 220227.
[33] L. Suanmali, N. Salim, M.S. Binwahlan, Fuzzy Logic Based Method for Improving
Text Summarization, 2009, arxiv:0906.4690.
[34] M.A. Fattah, F. Ren, GA, MR, FFNN, PNN and GMM based models for automatic
text summarization, Comput. Speech Lang. 23 (2009) 126144.
[35] J.-M. Lim, I.-S. Kang, J. Bae, J.-H. Lee, Sentence extraction using time features in multi-document summarization, in: Information Retrieval Technology,
Springer, 2005, pp. 8293.
[36] G. Salton, Automatic Text Processing: The Transformation, Analysis, and
Retrieval of Information by Computer, Addison-Wesley, 1989.

A. Khan et al. / Applied Soft Computing 30 (2015) 737747

[37] M. Srinivas, L.M. Patnaik, Genetic algorithms: a survey, Computer 27 (1994)
1726.
[38] DUC, Document Understanding Conference 2002, 2002.
[39] J.E. Baker, Reducing bias and inefciency in the selection algorithm, in: Proceedings of the Second International Conference on Genetic Algorithms, 1987,
pp. 1421.

747

[40] C.-Y. Lin, Rouge: a package for automatic evaluation of summaries, in: Text
Summarization Branches Out: Proceedings of the ACL-04 Workshop, 2004, pp.
7481.
[41] A. Nenkova, R. Passonneau, Evaluating Content Selection in Summarization:
The Pyramid Method, 2004.