Predicting Positive and Negative Links

in Online Social Networks

Jure Leskovec Daniel Huttenlocher Jon Kleinberg

Stanford University Cornell University Cornell University
[email protected] [email protected] [email protected]
arXiv:1003.2429v1 [physics.soc-ph] 11 Mar 2010

ABSTRACT fundamental question is then the following: How does the sign of
We study online social networks in which relationships can be ei- a given link interact with the pattern of link signs in its local vicin-
ther positive (indicating relations such as friendship) or negative ity, or more broadly throughout the network? Moreover, what are
(indicating relations such as opposition or antagonism). Such a mix the plausible configurations of link signs in real social networks?
of positive and negative links arise in a variety of online settings; Answers to these questions can help us reason about how negative
we study datasets from Epinions, Slashdot and Wikipedia. We find relationships are used in online systems, and answers that gener-
that the signs of links in the underlying social networks can be pre- alize across multiple domains can help to illuminate some of the
dicted with high accuracy, using models that generalize across this underlying principles.
diverse range of sites. These models provide insight into some of Effective answers to such questions can also help inform the de-
the fundamental principles that drive the formation of signed links sign of social computing applications in which we attempt to infer
in networks, shedding light on theories of balance and status from the (unobserved) attitude of one user toward another, using the pos-
social psychology; they also suggest social computing applications itive and negative relations that have been observed in the vicinity
by which the attitude of one user toward another can be estimated of this user. Indeed, a common task in online communities is to
from evidence provided by their relationships with other members suggest new relationships to a user, by proposing the formation
of the surrounding social network. of links to other users with whom one shares friends, interests, or
other properties. The challenge here is that users may well have
Categories and Subject Descriptors: H.2.8 [Database Manage- pre-existing attitudes and opinions — both positive and negative
ment]: Database applications—Data mining — towards others with whom they share certain characteristics, and
General Terms: Algorithms; Experimentation. hence before arbitrarily making such suggestions to users, it is im-
Keywords: Signed Networks, Structural Balance, Status Theory, portant to be able to estimate these attitudes from existing evidence
Positive Edges, Negative Edges, Trust, Distrust. in the network. For example, if A is known to dislike people that
B likes, this may well provide evidence about A’s attitude toward
1. INTRODUCTION B.
Social interaction on the Web involves both positive and negative Edge Sign Prediction. With this in mind, we begin by formulat-
relationships — people form links to indicate friendship, support, ing a concrete underlying task — the edge sign prediction problem
or approval; but they also link to signify disapproval of others, or — for which we can directly evaluate and compare different ap-
to express disagreement or distrust of the opinions of others. While proaches. The edge sign prediction problem is defined as follows.
the interplay of positive and negative relations is clearly important Suppose we are given a social network with signs on all its edges,
in many social network settings, the vast majority of online social but the sign on the edge from node u to node v, denoted s(u, v), has
network research has considered only positive relationships [19]. been “hidden.” How reliably can we infer this sign s(u, v) using
Recently a number of papers have begun to investigate negative the information provided by the rest of the network? Note that this
as well as positive relationships in online contexts. For example, problem is both a concrete formulation of our basic questions about
users on Wikipedia can vote for or against the nomination of oth- the typical patterns of link signs, and also a way of approaching
ers to adminship [3]; users on Epinions can express trust or distrust our motivating application of inferring unobserved attitudes among
of others [8, 18]; and participants on Slashdot can declare others users of social computing sites. There is an analogy here to the link
to be either “friends” or “foes” [2, 13, 14]. More generally, arbi- prediction problem for social networks [16]; in the same way that
trary hyperlinks on the Web can be used to indicate agreement or link prediction is used to to infer latent relationships that are present
disagreement with the target of the link, though the lack of explicit but not recorded by explicit links, the sign prediction problem can
labeling in this case makes it more difficult to reliably determine be used to estimate the sentiment of individuals toward each other,
this sentiment [20]. given information about other sentiments in the network.
For a given link in a social network, we will define its sign to be In studying the sign prediction problem, we are following an ex-
positive or negative depending on whether it expresses a positive or perimental framework articulated by Guha et al. in their study of
negative attitude from the generator of the link to the recipient.1 A trust and distrust on Epinions [8]. We extend their approach in a
1
We consider primarily the case of directed links, though our number of directions. First, where their goal was to evaluate prop-
framework can be applied to undirected links as well. agation algorithms based on exponentiating the adjacency matrix,
we approach the problem using a machine-learning framework that
Copyright is held by the International World Wide Web Conference Com- enables us to evaluate which of a range of structural features are
mittee (IW3C2). Distribution of these papers is limited to classroom use,
and personal use by others. most informative for the prediction task. Using this framework, we
WWW 2010, April 26–30, 2010, Raleigh, North Carolina, USA. also obtain significantly improved performance on the task itself.
ACM 978-1-60558-799-8/10/04.
Second, we investigate the problem across a range of datasets, and tion for a task such as this: on all two of the three datasets, we get a
identify principles that generalize across all of them, suggesting boost in improvement over random choice of up to a factor of 1.5.
certain consistencies in patterns of positive and negative relation- This type of result helps to argue that positive and negative links
ships in online domains. in online systems should be viewed as tightly related to each other,
Finally, because of the structure of our learned models, we are rather than as distinct non-interacting features of the system.
able to compare them directly to theories of link signs from so- We also investigate more “global” properties of signed social
cial psychology — specifically, to theories of balance and status. networks, motivated by the local theories of balance and status.
These will be defined precisely in Section 3, but roughly speaking, Specifically, the “friend of my enemy” logic of balance theory sug-
balance is a theory based on the principles that “the enemy of my gests that if balance is a key factor in determining signed link for-
friend is my enemy,” “the friend of my enemy is my enemy,” and mation at a global scale, then we should see the network partition
variations on these [4, 11]. Status is a theory of signed link forma- into large opposed factions. The logic of status theory, on the other
tion based on an implicit ordering of the nodes, in which a positive hand, suggests that we should see an approximate total ordering
(u, v) link indicates that u considers v to have higher status, while of the nodes, with positive links pointing from left to right and
a negative (u, v) link indicates that u considers v to have lower sta- negative links pointing from right to left. Searching for either of
tus. The point is that each of these theories implicitly posits its own these global patterns involves developing approximate optimiza-
model for sign prediction, which can therefore be compared to our tion heuristics for the underlying networks, since the two patterns
learned models. The result is both a novel evaluation of these the- correspond roughly to the well-known maximum cut and maximum
ories on large-scale online data, and an illumination of our learned acyclic subgraph problems. We employ such heuristics, and find
models in terms of where they are consistent or inconsistent with significant evidence for the global total ordering suggested by sta-
these theories. tus theory, but essentially no evidence for the division into factions
suggested by balance theory. This result provides an intriguing con-
Generalization across Datasets. We study the problem of sign
trast with our basic results on sign prediction using local features,
prediction on three datasets from popular online social media sites;
where strong aspects of both theories are present; it suggests that
in all cases, we have network data with explicit link signs. The
the mechanisms by which local organizing principles scale up to
first is the trust network of Epinions, in which the sign of the link
global ones is complex, and an interesting source of further open
(u, v) indicates whether u has expressed trust or distrust of user v
questions.
(and by extension, the reviews of v) [8]. The second is the social
network of the technology blog Slashdot, where u can designate v Further Related Work. Earlier in the introduction, we discussed
as either a “friend” or “foe” to indicate u’s approval or disapproval some of the main lines of research on which we are building; here,
of v’s comments [2, 13, 14]. The third is the voting network of we survey further lines of study that are also related to our work.
Wikipedia; here, the sign of the link (u, v) indicates whether u First, our use of trust networks as a source of data connects to
voted for or against the promotion of v to admin status [3]. a large body of work on trust management in several settings, in-
Despite the fact that link signs have quite different meanings in cluding peer-to-peer networks [12, 25], Semantic Web applications
the three settings, our main results generalize across all three do- [22], and Web spam detection [10]. Related to trust management is
mains in several important ways. First, we find that sign predic- the development of user rating mechanisms on sites such as Slash-
tion performance degrades only slightly when we train our models dot [13, 14] and the development of norms to control deviant be-
on one domain and test them on another. This indicates that our havior [6]. Recent work has also investigated online communities
models are capturing principles that will arguably generalize to a devoted to discussion of controversial topics, where one can expect
range of future contexts in which signed links are employed, rather to find strong positive and negative interactions [2, 24]; and the
than picking up on idiosyncrasies of particular individual domains. analysis of sentiment, subjectivity, and opinion in text has become
Moreover, this generalization holds despite the fact that the qual- an active area in natural language processing [20].
ity of prediction performance is different across the domains: for Our general goal of inferring an individual’s attitudes suggests
example, predicting link signs is more difficult on Wikipedia, yet parallels to a long line of work on recommendation systems [21],
models trained on Wikipedia still perform on other domains with in which the goal is typically to infer how a user would evaluate
very little loss of accuracy compared to models that were explicitly given items based on their evaluation of other items. There are
trained on those domains. crucial differences, however, between an analysis in which a user
Second, we find that the social-psychological theories of balance is evaluating (inert) items, and our case in which a user is evaluating
and status agree with the learned models in certain characteristic other people — in this latter case, the objects being evaluated are
ways, and disagree in other characteristic ways, as we elaborate in themselves capable of forming opinions and expressing attitudes,
Section 3. These similarities and contrasts among the models like- and this provides additional sources of information based on the
wise persist at a general level across the datasets, and thus provide full social network of interactions.
insight into the successes and failures of balance and status as in- As noted above, there is a long history of work in the social sci-
terpretative frameworks for understanding how link signs are being ences on balance theory [4, 11], including more recent work on
used across all these systems. mathematical models that attempt to capture how balance can arise
from dynamic changes to link signs over time [1, 17]. In recent
Additional Tasks. We consider several further issues beyond the
work, we analyzed theories of balance and status in the context
problem of sign prediction. Among these, we ask whether infor-
of social media sites, investigating the extent to which each the-
mation about negative links can be helpful in addressing questions
ory helped explain the linking behavior of users on these sites [15].
that concern purely positive links. Specifically we consider the link
Our work there studied how balance and status effects can act as
prediction problem: given a pair u and v, is there a (hidden) pos-
modifiers on the default behavior of a set of people measured in
itive edge between u and v? We ask how much performance is
aggregate; the problem of making predictions at the level of in-
improved if the negative edges in the network are also visible. In
dividuals was left as an open question. Here, we take some ini-
other words, how useful is it to know where a person’s enemies are,
tial steps toward addressing this question, combining an analysis of
if we want to predict the presence of additional friends? We find
signed networks with machine-learning techniques so as to formu-
that negative links can be a powerful source of additional informa-
late individual-level predictions.
 Epinions Slashdot Wikipedia from x to y. That is, s(x, y) = 1 when the sign of (x, y) is positive,
Nodes 119,217 82,144 7,118 −1 when the sign is negative, and 0 when there is no directed edge
Edges 841,200 549,202 103,747
+ edges 85.0% 77.4% 78.7%
from x to y. Sometimes we will also be interested in the sign of a
− edges 15.0% 22.6% 21.2% directed edge connecting x and y, regardless of its direction; thus,
we write s̄(x, y) = 1 when there is a positive edge in one of the
Table 1: Dataset statistics. two directions (x, y) or (y, x), and either a positive edge or no edge
in the other direction. We write s̄(x, y) = −1 analogously when
2. DATASET DESCRIPTION there is a negative edge in one of these directions, and either a neg-
ative edge or no edge in the other direction. We write s̄(x, y) = 0
We consider three large online social networks where each link
in all other cases (including when there are edges (x, y) and (y, x)
is explicitly labeled as positive or negative: Epinions, Slashdot and
with opposite signs, though this is in fact rare in our datasets). For
Wikipedia2 .
different formulations of our task, we will suppose that for a partic-
Epinions is a product review Web site with a very active user
ular edge (u, v), the sign s(u, v) or s̄(u, v) is hidden and that we
community. Users are connected into a network of trust and dis-
are trying to infer it.
trust, which is then combined with review ratings to determine
which reviews are most authoritative. The data spans from the in- Features. We begin by defining a collection of features for our ini-
ception of the site in 1999 until August 12, 2003. The network tial machine-learning approach to this problem. The features are
contains 119,217 nodes and 841,000 edges, of which 85.0% are divided into two classes. The first class is based on the (signed)
positive. 80,668 users received at least one trust or distrust edge, degrees of the nodes, which essentially record the aggregate local
while there are 49,534 users that created at least one and received relations of a node to the rest of the world. The second class is
at least one signed edge. based on the principle from social psychology that we can under-
Slashdot is a technology-related news website. In 2002 Slashdot stand the relationship between individuals u and v through their
introduced the Slashdot Zoo which allows users to tag each other joint relationships with third parties w: for example, is there some-
as “friends” or “foes.” The semantics of a signed link is similar to one who has a positive relationship toward both u and v, a negative
Epinions, as a friend relation means that a user likes another user’s relationship toward both u and v, or a positive relationship toward
comments, while a foe relationship means that a user finds another one and a negative relationship toward the other? Thus, features of
user’s comments uninteresting. We crawled Slashdot in February this second class are based on two-step paths involving u and v.
2009 to obtain its network of 82,144 users and 549,202 edges of We define the first class of features, based on degree, as follows.
which 77.4% are positive. 70,284 users received at least one signed As we are interested in predicting the sign of the edge from u to
edge, and there are 32,188 users with non-zero in- and out-degree. v, we consider outgoing edges from u and incoming edges to v.
Wikipedia is a collectively authored encyclopedia with an ac- Specifically we use d+ −
in (v) and din (v) to denote the number of in-
tive user community. The network we study corresponds to votes coming positive and negative edges to v, respectively. Similarly we
cast by Wikipedia users in elections for promoting individuals to use d+ −
out (u) and dout (u) to denote the number of outgoing positive
the role of admin. A signed link indicates a positive or negative and negative edges from u, respectively. We use C(u, v) to denote
vote by one user on the promotion of another (+ for a supporting the total number of common neighbors of u and v in an undirected
vote and − for an opposing vote). Using the latest complete dump sense — that is, the number of nodes w such that w is linked by an
of Wikipedia page edit history (from January 2008) we extracted edge in either direction with both u and v. We will also refer to this
all administrator election and vote history data. This gave us 2,794 quantity C(u, v) as the embeddedness of the edge (u, v). Our seven
elections with 103,747 total votes and 7,118 users participating in degree features are the five quantities d+ − + −
in (u), din (v), dout , dout ,
the elections (either casting a vote or being voted on). Out of this to- and C(u, v), together with the total out-degree of u and the total
tal, 1,235 elections resulted in a successful promotion, while 1,559 in-degree of v, which are d+ − + −
out (u) + dout (u) and din (v) + din (v)
elections did not result in the promotion of the candidate. About respectively.
half of the votes in the dataset are by the existing admins, while For the second class of feature we consider each triad involving
the other half comes from ordinary Wikipedia users. The resulting the edge (u, v), consisting of a node w such that w has an edge
network contains 7,118 nodes and 103,747 edges of which 78.7% either to or from u and also an edge either to or from v. There are
are positive. There are 2,794 nodes that receive at least one edge 16 distinct types of triads involving (u, v): the edge between w and
and 1,376 users that both received and created signed edges. u can be in either direction and of either sign, and the edge between
In all networks the background proportion of positive edges is w and v can also be in either direction and of either sign; this leads
about the same, with ≈80% of the edges having a positive sign. to 2 · 2 · 2 · 2 = 16 possibilities. Each of these 16 triad types may
provide different evidence about the sign of the edge from u to v,
3. PREDICTING EDGE SIGN some favoring a negative sign and some favoring a positive sign.
We now consider the problem of predicting the sign of individual We encode this information in a 16-dimensional vector specifying
edges in our dataset. The set-up for this problem follows the frame- the number of triads of each type that (u, v) is involved in.
work of Guha et al. [8]: We are given a full network with all but Learning Methodology and Results. We use a logistic regression
one of the edge signs visible, and we are interested in predicting the classifier to combine the evidence from these individual features
sign of this single edge whose sign has been suppressed. This can into an edge sign prediction. Logistic regression learns a model of
be viewed as leave-one-out cross-validation in the present context, the form
where we learn using the rest of the network and aim to predict the 1
missing sign of a single edge. P (+|x) = Pn
1 + e−(b0 + i bi xi )
3.1 A Machine-Learning Formulation where x is a vector of features (x1 , . . . , xn ) and b0 , . . . , bn are the
Given a directed graph G = (V, E) with a sign (positive or nega- coefficients we estimate based on the training data.
tive) on each edge, we let s(x, y) denote the sign of the edge (x, y) The edges signs in the networks that we study are overwhelm-
ingly positive. Thus we consider and evaluate two different ap-
2
Datasets are available at http://snap.stanford.edu proaches. First, we use the full dataset where about 80% of the
 1 agation model of Guha et al. This suggests that edge signs can be
0.9
(A) meaningfully understood in terms of such local properties, rather
0.8
than requiring a notion of propagation from farther-off parts of the
network.
0.7 Second, consistent with intuition, the triad features perform less
0.6 well than the degree features for edges of low embeddedness, but
0.5 become more effective as the embeddedness increases and a greater
amount of triadic information becomes available.
1
Finally, it is also noteworthy that the accuracy on the Wikipedia
(B)
Predictive accuracy

0.9 network is significantly lower than on the other two networks, even
0.8 for edges with large embeddedness. This discrepancy between
Wikipedia and the other datasets is interesting because the positive
0.7
and negative links on Wikipedia correspond to evaluations that are
0.6 more publicly visible, more consequential, and more information-
0.5 based than for the other two datasets, since they result from public
1 votes on promotion of individuals to adminship, where the candi-
0.9
(C) Em=0
Em=10 dates being voted on have accumulated a long history of activity on
Em=25 Wikipedia. One could conjecture that these aspects of the evalu-
0.8 ations in the Wikipedia dataset make it correspondingly more dif-
0.7 ficult (though still surprisingly feasible) to predict their outcomes
0.6 from simple structural measures.
In all experiments we report the average accuracy and estimated
0.5
logistic regression coefficients over 10-fold cross validation. If not
Random

Degree

16Triads

All23

stated otherwise, we limit our analyses to edges with minimum em-

beddedness 25. We note that our results are robust with respect
to training dataset and evaluation metric. Generally, when using
Figure 1: Accuracy of predicting a sign of edge (u, v) given the full dataset rather than the balanced one, random guessing im-
signs of all other edges in the network. (a) Epinions, (b) Slash- proves accuracy from 50% to approximately 80%. With the full
dot, (c) Wikipedia. dataset the accuracy of our logistic regression method correspond-
ingly jumps to the 90-95% range and maintains roughly a 15% ab-
edges are positive. Second, we follow the methodology of Guha et
solute improvement over random guessing. When evaluating using
al. [8] and create a balanced dataset with equal numbers of positive
AUC rather than accuracy the overall pattern of performance does
and negative edges, so that random guessing yields a 50% correct
not change. The various forms of logistic regression have AUC of
prediction rate. For every negative edge (u, v) we sample a ran-
approximately 90% on the balanced dataset and 95% on the full
dom positive edge, which ensures that the number of positive and
dataset.
negative edges in the data we consider for training and prediction
is balanced. Moreover, we also consider two different evaluation 3.2 Connections to Theories of Balance and
measures: the classification accuracy and the area under the ROC Status
curve (AUC). For ease of exposition we focus on classification ac-
curacy on a balanced dataset. As we discuss later our results are Our goal is to use the machine learning framework not just to
robust to whether we use the full or balanced dataset and whether predict the edge signs themselves, but also for deriving insights
we evaluate using AUC or accuracy. into the usage of these systems based on the observed patterns of
We describe each edge (u, v) in this set using the two classes positive and negative edges.
of features described above. We consider all 23 features together, Specifically, logistic regression provides a coefficient associated
and we also evaluate performance using features of each class sep- with each feature, which suggests how the feature is being used by
arately — that is, representing each edge as a 7-dimensional vector the model to provide weight for or against a positive edge sign. This
of degree features and as a 16-dimensional vector of triad features. provides a natural and appealing connection to classical theories
We also consider performance across different types of edges. In from social psychology, which also offer proposals for how subsets
particular, since the triad features are relevant only when u and v of these features offer evidence for the sign of the edge (u, v).
have neighbors in common, it is natural to expect that they will be We focus here on the second class of features, based on triad
most effective for edges of greater embeddedness. We therefore types, which are motivated by social-psychological theories about
consider the performance restricted to subsets of edges of different local patterns of relationships. Specifically, we say that a theory of
levels of minimum embeddedness. triad types is a function
The classification accuracy is shown in Figure 1, where results f : {types τ } → {+1, −1, 0},
are described for all three datasets, for the two classes of features
separately and together, and for different levels of minimum em- which specifies for each triad type τ whether it constitutes evidence
beddedness (denoted by Em ). Several observations stand out. First, for a positive (u, v) edge (f (τ ) = +1), evidence for a negative
prediction based on the learned models significantly outperform the (u, v) edge (f (τ ) = −1), or whether it offers no evidence (f (τ ) =
results reported in Guha et al. [8] for the Epinions dataset. The low- 0).
est error rate achieved in their paper is 14.7% whereas we obtain Our logistic regression model provides a learned theory of triad
error rates of 11.45% for Degree, 6.64% for 16Triads and 6.58% types for each dataset, in which f (τ ) is equal to the sign of the
for All23. coefficient associated with the feature τ . But several principles
These results are particularly interesting because our features are from social psychology also provide theories of triad types, devel-
based only on local properties in the one-step neighborhood of the oped from plausible assumptions about human behavior rather than
edge (u, v) whose sign is being inferred, in contrast with the prop- through a data-driven approach. In other words, the learned model
and the qualitative models from the literature are expressed in the Feature Balance theory Epinions Slashdot Wikipedia
same language — as mappings from triad types to positive or neg- const 0 0.4321 1.4973 0.0395
pp 1 0.0470 0.0395 0.0553
ative evidence — and we can thus ask questions about how the pm -1 -0.1154 -0.2464 -0.1632
theories align with each other. Through this line of investigation mp -1 -0.2125 -0.3476 -0.1432
we can gain insight into two issues: first, we can evaluate the exist- mm 1 -0.0149 -0.0262 -0.0465
ing theories on our on-line datasets; and second, we can use these
existing theories as an interpretive framework for reasoning about Table 3: Regression coefficients based on Balance attributes
the structure of our learned model. and learned logistic regression.
Balance and Status. We begin by summarizing the two main Feature Status theory Epinions Slashdot Wikipedia
social-psychological theories of triad types that we compare to. const 0 -0.6873 -1.3915 -0.3039
The more well-studied of the two is structural balance theory, u<w<v 1 0.1165 0.0463 0.0258
based on the common principles that “the friend of my friend is u>w>v -1 -0.1002 -0.114 -0.1941
my friend,” “the enemy of my friend is my enemy,” “the friend u<w>v 0 0.0572 0.1558 0.0300
of my enemy is my enemy,” and (perhaps less convincingly) “the u>w<v 0 -0.0064 0.0382 0.0543
enemy of my enemy is my friend.” Concretely, this means that if
w forms a triad with the edge (u, v), then structural balance theory Table 4: The coefficients based on Status Theory and learned
posits that (u, v) should have the sign that causes the triangle on logistic regression.
u, v, w to have an odd number of positive signs, regardless of edge
direction — just as each of the principles above has an odd number ing at the absolute values of the coefficients, we note that certain
of occurrences of the word “friend.” In other words, fbalance (τ ) = features stand out in importance across all three datasets — specif-
s̄(u, w)s̄(v, w), where we recall that the value of s̄ corresponds to ically, there are coefficients of large magnitude for all the FF fea-
the sign regardless of the direction of the edge. tures, as well as the FBmp and BFpm features. In contrast, the BB
An alternate theory, which is implicit in the work of Guha et al. features have coefficients of much smaller magnitude. We also see
[8] and developed further in our recent research [15], is a theory of that balance theory is in notably better alignment with the learned
status. In this theory, a positive edge (x, y) means that x regards model for Epinions and Slashdot than it is for Wikipedia. As dis-
y as having higher status than herself, while a negative edge (x, y) cussed above, Wikipedia differs considerably from the other two
means that x regards y as having lower status than herself. Assum- datasets in that it is a publicly visible voting forum. Given these re-
ing that all participants in the system agree on this status ordering, sults it is interesting to conjecture that in such a setting status may
status theory predicts that when the direction of an edge is flipped, play a stronger role.
its sign should flip as well. It is also interesting to consider the cases in which there are
So to determine fstatus (τ ), we first flip the directions of the relatively stable disagreements among the models across the three
edges between u and w and between v and w, if necessary, so that datasets. In particular, we see that balance theory consistently dis-
they point from u to w and from w to v; we flip the signs accord- agrees with the learned model (and with status theory) when it pre-
ingly as we do this. We then define fstatus (τ ) to be the sign of dicts that a negative (u, w) and negative (w, v) edge should suggest
s(u, w) + s(w, v). This means that status theory makes no pre- a positive (u, v) edge. This is precisely the kind of case that seems
diction when the two signs cancel out, but otherwise, it predicts a somewhat suspect intuitively, namely “the enemy of my enemy is
positive or negative sign based on the imputed status relationship my friend”. Balance theory also consistently disagrees with both
between u and v. the learned model and status theory when it predicts that a positive
Notice that balance and status agree on some types of triads — (v, w) and positive (w, u) edge should result in a positive (u, v)
for example, when u points positively to w and w points positively edge. Here the direction of the two-step path is from v to u rather
to v, then v is both the friend of u’s friend, and also someone of than u to v, and one can conjecture that this opposite direction path
higher status than u, and thus both theories predict a positive sign has a lower predictive power for the (u, v) sign. Indeed these two
for (u, v). But balance and status can also disagree — for example, cases suggest modifications of the models, as we now discuss.
when v points positively to w and w points positively to u, then Comparison of Balance and Status with Reduced Models. To
balance concludes that v is the friend of u’s friend and thus (u, v) fully understand the relationship of the learned model to the the-
is positive, but status posits that v has lower status than u and thus ories of balance and status, it helps to look at “reductions” of the
(u, v) is negative. model that capture just the features essential to these two theories.
Comparison of Balance and Status with the Learned Model. In Let’s begin by considering balance theory. Balance theory has
Table 2, we show the signs of the three theories discussed above generally been applied as a theory of undirected graphs, although
— balance, status, and the learned model — on the three datasets. its extension to directed graphs by ignoring edge direction (as we
For denoting the 16 triad types, in the table and elsewhere, we use a use it here) is standard as well [23]. With this in mind, let’s con-
shorthand in which we record the four binary choices that comprise sider the learning problem using a feature set in which we treat
each type. Thus a type τ will be represented by a string of the form all edges as undirected. In this setting, there are only four differ-
[F |B][F |B][p|m][p|m] to indicate the direction of the edges along ent triad types involving a node w and the edge (u, v), depending
the two-step path u-w-v (Forward or Backward on each step), and on whether the undirected edge {u, w} is positive or negative, and
the signs of these two edges (plus or minus). For example, F Bmp whether the undirected edge {w, v} is positive or negative (since
is the triad type in which u points negatively to w, and v points we can no longer observe the directions of these edges). Thus, we
positively to w (since the first step in the u-w-v path is forward and create a 4-dimensional feature vector for the edge (u, v), by simply
minus, while the second is backward and plus). counting how many undirected triads of each type it is involved in.
At a general level, the results show that both social-psychological We then apply logistic regression to this 4-dimensional problem.
theories agree fairly well with the learned models — with agree- The results are depicted in Table 3. We see that for all triad
ment on more than half the triad types where they make predictions, types other than the “enemy of my enemy” type (mm), and all
and generally on three-quarters or more of the triad types. Look- three datasets, the learned coefficient is the same as the prediction
of balance theory. The disagreement for the mm type is a further
 Feature Bal Stat Epinions Bal Stat Slashdot Bal Stat Wikipedia Bal Stat
const -0.1656 0.018 -0.215
FFpp 1 1 0.4869 0.8504 0.2849
FFpm -1 0 -0.5166 -0.9008 -0.4337
FFmp -1 0 -0.4476 -1.0513 -0.3092
FFmm 1 -1 -0.7331 × -0.5874 × -0.768 ×
FBpp 1 0 0.3416 0.4385 0.0544
FBpm -1 1 -0.0147 × -0.1439 × -0.0131 ×
FBmp -1 -1 -0.8598 -1.1887 -0.1986
FBmm 1 0 0.0436 -0.0719 × -0.0325 ×
BFpp 1 0 0.0814 0.3593 0.116
BFpm -1 -1 -1.3097 -1.0838 -0.3527
BFmp -1 1 -0.1228 × -0.248 × 0.0527 ×
BFmm 1 0 0.0788 -0.024 × -0.0968 ×
BBpp 1 -1 -0.0855 × -0.0873 × -0.0065 ×
BBpm -1 0 -0.0536 -0.2736 -0.0168
BBmp -1 0 -0.0382 -0.2788 0.0507 ×
BBmm 1 1 -0.0242 × × 0.2275 -0.1616 × ×
Total errors 3 3 4 2 7 2
Table 2: Logistic regression coefficients compared to status and structural balance theory. × means there is discrepancy in predic-
tions between the Balance (Status) theory and what is learned from the logistic regression model. Each line represents directions and
signs of the edges on a path (A, B, C) where “BFpm" stands for Backward Forward plus minus and denotes a path A ←+ B → −C.

Feature Epinions Slashdot Wikipedia 1

const -2.2152 -2.8475 -1.4496 0.9
(A)
FF 0.2023 0.2092 0.0773
FB 0.1286 0.1698 0.0286 0.8
BF 0.0077 0.0842 0.0544 0.7
BB -0.0692 -0.0293 -0.0259
0.6
Table 5: Learned logistic regression coefficients for the model 0.5
based on the counts of directed positive paths.
1

indication of the difficulty of the “enemy of my enemy” aspect of (B)

Predictive accuracy

0.9
balance in these domains, and in fact is consistent with an alterna- 0.8
tive formulation of balance theory due to Davis in the 1960s [7],
0.7
which agrees with standard balance theory on the first three triad
types and makes no prediction on mm. We will refer to Davis’s 0.6
variant on balance theory as weak balance. 0.5
We can do a similar reduction for status theory. We begin by pre- 1
processing the graph to flip the direction and sign of each negative
0.9
(C) Em=0
Em=10
edge, thereby creating a positive edge with the same interpretation Em=25
under status theory. The resulting graph has only positive edges, 0.8
and hence there are only four triad types — based on whether the 0.7
(u, w) edge is forward or backward, and whether the (w, v) edge 0.6
is forward or backward. We create a 4-dimensional feature vector
0.5
for the edge (u, v) by counting the frequencies of these four triad
types, and apply logistic regression.
Random

BalanceDet

WeakBalDet

BalanceLrn

StatusDet

StatusLrn

16Triads

The learned coefficients for this problem are shown in Table 4.

Here we see that on both triad types for which status theory makes
a prediction, and across all three datasets, the sign of the learned
coefficient is the same as the sign of the status prediction. Figure 2: Accuracy of predicting a sign of edge (u, v) given
What emerges from the analysis of these reduced-form models is signs of all other edges in the network. (a) Epinions, (b) Slash-
that each of balance theory and status theory are essentially accu- dot, (c) Wikipedia.
rate at the granularity in which they are most naturally formulated
— in the case of balance, on undirected graphs; and in the case
{−1, 0, +1} provided by balance, Davis’s notion of weak balance,
of status, once edge signs and directions have been canonicalized.
and status (denoted BalanceDet, WeakBalDet, and StatusDet in the
This makes it clear that our results for the more detailed 16-type
figure).
model go beyond the scale of resolution at which either balance or
status can provide accurate predictions, and illuminate some more All-positive subgraphs. There is a final reduced model that also
subtle effects that govern social interactions. provides insight into the role of balance theory particularly for these
To further elaborate on this point, we can evaluate the prediction datasets. Suppose that we preprocess the graph by simply deleting
accuracy of each of these reduced models in comparison to the full all negative edges, so that we are left with the subgraph consisting
model on all 16 triad types. The results are shown in Figure 2. of only positive edges. Again, there are now four possible triad
Here, we consider the two kinds of reduced 4-dimensional feature types, and we show the learned coefficients for logistic regression
sets, and evaluate performance using both the coefficients learned on this 4-dimensional problem in Table 5. Balance would predict
via logistic regression (denoted BalanceLrn and StatusLrn in the that all coefficients should be positive, since all relations indicate
figure), as well as the lower performance using coefficients from friendship under the interpretation of balance theory. This agrees
 All23 Epinions Slashdot Wikipedia BalanceLrn Epinions Slashdot Wikipedia
Epinions 0.9342 0.9289 0.7722 Epinions 0.9027 0.9166 0.7319
Slashdot 0.9249 0.9351 0.7717 Slashdot 0.9020 0.9203 0.7439
Wikipedia 0.9272 0.9260 0.8021 Wikipedia 0.8985 0.9145 0.7558
Table 6: Predictive accuracy when training on the “row” BalanceDet 0.7709 0.8414 0.5778
dataset and evaluating the prediction on the “column” dataset. WeakBalance 0.8233 0.8698 0.6081

StatusLrn Epinions Slashdot Wikipedia

with Table 5 except for the last row, where the coefficient of the Epinions 0.8313 0.7514 0.6410
learned model is negative across all three datasets. This corre- Slashdot 0.7682 0.7847 0.6094
sponds to the triad type in which v links positively to w, and w Wikipedia 0.7592 0.6598 0.7163
links positively to u. The data indicates that this in fact serves as StatusDet 0.8083 0.7173 0.6679
evidence of a negative (u, v) link in all datasets, and status theory Table 7: Predictive accuracy when training on the “row”
provides one simple hypothesis for why: if v regards w as having dataset and evaluating the prediction on the “column” dataset.
higher status, and w in turn regards u as having higher status, then
arguably u will view v as having lower status.
3.4 Heuristic Predictors
3.3 Generalization across datasets Having now explored the relationship of balance and status to
We now turn to the question of how well the learned predictors the output of a machine learning model, it is natural to also in-
generalize across the three datasets, in order to investigate the ex- vestigate simple “hand-written” heuristic predictors to determine
tent to which the learned models are based on specific properties baseline performance levels for the sign prediction task.
of each dataset versus more general properties of social network We construct a few such predictors using ideas drawn from sta-
data. That is, in our investigation thus far the learned models have tus and balance theory, as well as considerations of node degree.
been able to take advantage of properties of each specific dataset, We again use the same evaluation framework, in which for every
whereas the base social science models are generic across datasets. negative edge (u, v) we sample a random positive edge, to ensure
Thus it could be the case that the models obtained using logistic that random guessing gets 50% of the predictions correct.
regression perform well only on the individual datasets on which We consider the following heuristic predictors:
they were trained. On the other hand, if the learned models are
• A balance heuristic. For each choice of the sign of (u, v),
able to generalize across datasets it suggests that there are under-
some of the triads it participates in will be consistent with
lying general principles that guide the creation of signed edges in
balance theory, and the rest of the triads will not. We choose
social network graphs and which the regression models are able to
the sign for (u, v) that causes it to participate in a greater
capture.
number of triads that are consistent with balance.
To evaluate the generalization accuracy of the models we per-
• A status heuristic. We define an estimate of a node x’s status
form the following experiment. For each pair of datasets we train
to be σ(x) = d+ − + −
in (x) + dout (x) − dout (x) − din (x). This
the logistic regression model on the first dataset and evaluate it on
gives x status benefits for each positive link it receives and
the second dataset. Table 6 shows the results of these 9 experi-
each negative link it generates, and status detriments for each
ments using the All23 model. The diagonal entries of the table
negative link it receives and each positive link it generates.
show the results already presented above (i.e. leave-one-out cross-
We then predict a positive sign for (u, v) if σ(u) ≤ σ(v),
validation for a single dataset), whereas the off-diagonal entries
and a negative sign otherwise.
show the generalization across datasets. We see the same over- • An out-degree heuristic. We predict the majority sign based
all pattern as before, with prediction accuracy being considerably on the signs given by the edge initiator u. That is, we predict
lower for Wikipedia than for the other two dataset. We also see that + if d+ −
out (u) ≥ dout (u).
the off-diagonal entries are nearly as high as the diagonals, show- • An in-degree heuristic. We predict the majority sign based on
ing that there is very good generalization and thus there indeed are the signs received by the edge target v. That is, we predict +
general cross-dataset properties captured by the model. In both if d+ −
in (v) ≥ din (v).
the first and second columns (testing on the Epinions and Slash-
dot datasets) there is remarkably little decrease in performance re- We show the results of these simple predictors in Figure 3, plot-
gardless of which of the three datasets is used for training. Note ted as a function of embeddedness. First, while these predictors
that in particular, even training on the Wikipedia dataset yields perform considerably better than guessing – and quite well in some
good prediction performance on the other two datasets – thus while cases – they generally perform worse than the models discussed
the Wikipedia dataset is difficult in terms of prediction accuracy it above. Second, in the case of the Epinions data, some of these sim-
seems to provide the same underlying structural information as the ple predictors perform comparably well to the results reported in
other datasets in that it allows training of a more general model. [8] (though not as well as the very best results in that paper). This
Table 7 shows the results of these experiments using the learned underscores the value of having simple baseline measures.
Balance and Status models considered above. For comparison we There are several additional points worth noting. First, the in-
also show the performance of the basic BalanceDet, WeakBalance degree heuristic (InSign) performs relatively poorly across all datasets
and StatusDet models (which are not learned from the data). For (although it beats the other predictors for low embeddeness in the
balance we see that the generalization performance is again very Wikipedia dataset), while out-degree (OutSign) performs well on
good, and also that the prediction accuracy in all cases is higher two of the three; this corresponds to the intuitively natural conclu-
than for the nonlearned balance models. However for status, we sion that the generator of the edge plays a larger role in sign deter-
see a considerably different picture. The model does not generalize mination than the recipient. Second, the triadic features — status
as well, and in fact often performs worse than the baseline status (StatDif) and balance (Balance) — do better with increasing em-
model when trained on one dataset and tested on another. This beddedness, but in most cases performance of these features starts
suggests that the learned balance properties are relatively generic to drop again as embeddedness gets too high. One can understand
across different online settings, whereas the status properties are this in terms of a tradeoff between two types of information spar-
more particular to a given dataset. sity. On the one hand, when an edge participates in relatively few
 0.9 0.9 0.9
Always+ StatDif
Predictive accuracy 0.85 0.85 0.85 OutSign Balance

Predictive accuracy

Predictive accuracy
0.8 0.8 0.8 InSign
0.75 0.75 0.75
0.7 0.7 0.7
0.65 0.65 0.65
0.6 0.6 0.6
0.55 0.55 0.55
0.5 0.5 0.5
0 5 10 15 20 25 0 5 10 15 20 25 0 5 10 15 20 25
Minimum edge embeddedness Ek Minimum edge embeddedness Ek Minimum edge embeddedness Ek

(a) Epinions (b) Slashdot (c) Wikipedia

Figure 3: Accuracy for simple models as a function of minimum edge embeddedness. Refer to main text for model descriptions.

triads (low embedding) then the triadic features provide relatively holds. We can now have the following result, which says that if all
little information, but on the other hand relatively few edges in the three-node sets are status-consistent, then the whole graph obeys a
graph have a high degree of embedding (e.g., significantly above form of status-consistency: there is a total ordering of the nodes in
25). which positive edges all go left-to-right, and negative edges all go
right-to-left.
4. GLOBAL STRUCTURE OF SIGNED NET-
WORKS T HEOREM 2. Let G be a signed, directed tournament, and sup-
pose that all sets of three nodes in G are status-consistent. Then it
When we perform sign prediction for an edge (u, v) using infor-
possible to order the nodes of G as v1 , v2 , . . . , vn in such a way
mation about the two-step paths from u to v, and when we relate
that each positive edge (vi , vj ) satisfies i < j, and each negative
our learned models to the predictions of balance and status, we
edge (vi , vj ) satisfies i > j.
are using local information about the neighborhoods of the nodes.
However, the theories of balance and status also each make global Proof. Following an idea from the previous section, we first re-
predictions about the pattern of the signs in the network as a whole, verse the direction of each negative edge in G and give it a positive
and it is interesting to investigate the extent to which these global sign. Notice that if all sets of three nodes were status-consistent
predictions are borne out by the network structure of our datasets. before this conversion, they remain status-consistent after this con-
Balance and Status: From Local to Global. The global predic- version.
tions of balance and status are best explained in their simplest set- Let G′ be the resulting graph; note that G′ has only positive
tings, which are for networks in which each pair of nodes is con- edges. If any three-node subgraph in G′ were to form a directed cy-
nected by an edge. This complete connectivity clearly does not cle, then the three nodes in this cycle would violate status-consistency.
hold for our datasets, but we will explain how to adapt the predic- Thus, all three-node subgraphs of G′ are acyclic. Applying a well-
tions of the models to our setting nonetheless, and then search for known theorem on tournaments (see e.g. the opening exposition in
evidence of these predictions. [5]) it follows that G′ itself is acyclic. Thus, we can find a topolog-
The global prediction of balance theory is contained in a theorem ical ordering v1 , v2 , . . . , vn of G′ .
of Cartwright and Harary from the 1950s. It asserts that if all tri- Finally, we claim that this ordering satisfies the conditions of the
angles in a completely connected undirected graph obey structural theorem. Indeed, if an edge (vi , vj ) is positive in G, then it is also
balance, then globally the network can be divided into two cliques an edge in G′ , so by the property of the topological ordering we
of mutual friends, such that all edges between the two cliques are have i < j. And if an edge (vi , vj ) is negative in G, then (vj , vi )
negative [4]. is an edge of G′ , whence i > j as required.
T HEOREM 1 (C ARTWRIGHT-H ARARY ). Let G be a signed,
Searching for Evidence of Global Balance and Status. Both
undirected complete graph in which each triangle has an odd num-
Theorem 1 and 2 have more elaborate generalizations, when the
ber of positive edges. Then the nodes of G can be partitioned into
underlying graph is not completely connected, by generalizing the
two sets A and B (where one of A or B may be empty), such that
respective three-node conditions to arbitrary cycles. Under these
all edges within A and B are positive, and all edges with one end
generalized conditions the conclusions remain the same: when bal-
in A and the other in B are negative.
ance holds, we should expect to see a network that is divided into
We can analogously formulate a local-to-global connection for two mutually opposed factions of friends, and when status holds,
status theory, but it leads to a quite different structural prediction. we should expect to see a network whose edges respect a global
Rather than undirected complete graphs, the basic form of this the- ordering. We therefore take these two basic patterns — a division
orem will use completely connected directed graphs, which are re- into two factions, and a global ordering — as potential “signatures”
ferred to as tournaments: directed graphs in which each pair of for the effects of balance and status respectively at a global level.
nodes is connected by a directed edge in one direction or the other. Of course, at best we expect to see balance and status holding in an
First, if we consider the local condition that motivated status the- approximate sense, and so we search for approximate versions of
ory in the previous section, it required that for any edge (u, v), and these two global signatures, rather than exact versions.
any third node w, it should be possible to assign distinct numeri- For balance theory, we attempt to partition the graph into two
cal “status values” to u, v, and w in such a way that the positive sets to maximize the following objective function: the number of
edges among them (if any) go from nodes of lower status to nodes positive edges with both ends in the same set, plus the number of
of higher status, and the negative edges among them (if any) go negative edges with ends in opposite sets. (We will also say that
from nodes of higher status to nodes of lower status. Let us say that those edges that contribute to the objective function are satisfied
the three nodes u, v, and w are status-consistent if this condition by the partition.) We develop the following maximization heuristic
 Balance Epinions Slashdot Wikipedia is very little evidence for the global presence of structural balance
Netwok 0.8344 0.8105 0.7809 in our three network datasets.
Permuted 0.8562 0.7779 0.7866
Rewired 0.8993 0.8310 0.8316
For the global version of status theory, shown in the second part
Status Epinions Slashdot Wikipedia of Table 8, we see quite a different picture. Roughly, we are able
Network 0.7905 0.8221 0.8538
to find orderings for all three datasets that satisfy about 80-85%
Permuted 0.7241 0.7568 0.7767 of all edges. This is much higher than the 50% obtainable from a
Rewired 0.6377 0.6644 0.6321 random ordering; and moreover, it is significantly better than the
performance on either of our two baselines. Thus, we do find evi-
Table 8: Fraction of edges satisfying global balance and status.
dence for an approximate global status ordering in the real datasets,
for this problem. We start by randomly partitioning the nodes into compared to baselines derived from random variations of them.
two sets. Then we repeatedly pick a random node, and change the Overall, then, there is evidence for link formation consistent with
set it belongs to if that would increase the value of the objective a global status ordering on the nodes, in which positive links tend
function. We run this procedure many times from different initial to point left-to-right in this ordering, while negative tend to point
starting sets. On our datasets, we found experimentally that the right-to-left in this ordering. On the other hand, we can find no
variance of the solution is very small, as the heuristic quickly con- significant evidence for the kind of partitioning into factions that
verges to a solution whose objective function value is within a few balance theory suggests at a global level. This forms an intriguing
hundred of the best solution found over all runs. Note there is a contrast with our results at a local level, where there was evidence
trivial solution that would simply declare one of the sets to be the for both balance and status. There is no contradiction here, since
empty set and the other to be the full node set; this would achieve the fidelity of balance and status at a local level is only approxi-
an objective function value equal to about 80% of the edges, since mate, but it does raise interesting questions that suggest the need
in our datasets about 80% of the edges are positive, and the positive for more powerful and general ways to relate the local structure of
edges would be precisely those that are satisfied by this partition. sign patterns to more global forms of structure.
For status theory we employ a different heuristic. First, as in the
proof of Theorem 2, we replace each negative edge with a positive 5. PREDICTING POSITIVE EDGES
edge pointing in the opposite direction, thus obtaining a directed
In the introduction we noted that the sign prediction problem
network with only positive edges. In this transformed graph, the
considered in this paper is closely related to the link prediction
goal is to find an ordering of the nodes that maximizes the number
problem of inferring latent relationships that are present but not
of edges pointing from a node earlier in the ordering to one that is
recorded by explicit links [16]. We now turn to this problem in or-
later in the ordering. (Again, we will refer to such edges as being
der to investigate the role of negative edges here as well. In particu-
satisfied by the ordering.) This is precisely the Maximum Acyclic
lar we consider the question of whether information about negative
Subgraph Problem, which is known to have strong inapproximabil-
links can be helpful in predicting the presence or absence of an un-
ity bounds: a random ordering achieves an objective function value
observed positive link. In other words, how useful is it to know
equal to half the total number of edges in expectation, and it is com-
who a person’s enemies are, if we want to predict the presence of
putationally hard to do asymptotically better than this in the worst
additional friends?
case [9]. Of course, our datasets are not necessarily worst-case in-
Specifically, suppose we are given a social network where the
stances — indeed, status theory suggests they may have additional
task is to predict the presence or absence of a positive edge between
structure — and we employ the following heuristic. We start with
two individuals. This is analogous to the experiments above, only
a random ordering; we then repeatedly pick a random pair of nodes
now it is the presence or absence of an edge in some context rather
and swap their positions in the ordering if that would increase the
than the sign of an edge that is to be predicted. We consider two
value of the objective function. Again, we run this heuristic multi-
cases. In the first case, only information about the positive edges is
ple times and take the best solution found.
used to predict the presence or absence of a hidden positive edge,
Evaluating Global Balance and Status. We now use these two whereas in the second case information about both the positive and
heuristic optimization methods to assess the extent to which each negative edges is used for the prediction.
of the three networks exhibits global balance and status properties. We use the machine learning framework developed in previous
That is, we ask whether the quality of the partition or ordering we sections to build classifiers that predict whether there exists a pos-
find is better than would be expected from simple baselines derived itive edge between a pair of nodes. We train two sets of models
from our datasets. If the quality is significantly above such base- using the same features (16Triads) but in one case we use the full
lines, it provides evidence for these structures. We use two such network with positive and negative edges, while in the other case
baselines. The first a permuted-signs baseline in which we keep we use only the positive edges. We then devise the following ex-
the structure of the network fixed, but we randomly shuffle all the periment. For a positive edge (a, b) we pick a corresponding pair
edge signs. The second is a rewired-edges baseline in which we of nodes (c, d) that are not connected by an edge but have the same
generate a random network where each node maintains the same number of common neighbors (embeddedness) as (a, b). Then we
number of incoming and outgoing positive and negative edges. formulate a binary classification problem where we are given a pair
Table 8 shows the value of the objective function (as a fraction of of nodes and the goal is to determine whether the positive edge is
the total number of edges) for each of balance and status, and across present; that is, we aim to distinguish between pairs of nodes (a, b)
each of our real networks in comparison to the permuted-signs and that are connected by a positive edge and pairs of nodes (c, d) that
rewired-edges baselines. For balance theory, notice that we find are not connected by an edge.
objective function values that are comparable to the total fraction For each pair of nodes we compute two sets of features. For the
of positive edges, which we noted is trivially achievable (and also first set of features we include the information from positive and
trivially achievable in the two baselines, which have the same frac- negative edges by computing the features counting the frequency of
tions of positive edges). Moreover, if we randomize the network each of 16 distinct signed directed triads between a pair of nodes.
structure while preserving the signed in- and out-degrees, we ob- For the second set of features that are based on only positive edges
tain a network that actually achieves a greater objective function we simply compute the frequencies of 4 directed paths composed
value under our heuristics. Taken together, this suggests that there of only positive edges (namely, FFpp, FBpp, BFpp, BBpp).
 Features Epinions Slashdot Wikipedia ported in part by the NSF grant IIS-0705774, IBM Faculty Award,
Positive edges 0.5612 0.5579 0.6983 gift from Microsoft Research and Yahoo! Research Alliance grant.
Positive and negative edges 0.5911 0.5953 0.7114
Table 9: Predicting the presence of a positive edge. 7. REFERENCES
[1] T. Antal, P. Krapivsky, and S. Redner. Social balance on
We then train and evaluate two models: one trained only on fea- networks. Physica D, 224(130), 2006.
tures based on positive edges, and the other trained on the whole
[2] M. J. Brzozowski, T. Hogg, and G. Szabó. Friends and foes:
set of features that also include evidence coming from the nega-
ideological social networking. In Proc. 26th CHI, 2008.
tive edges. Our goal here is to understand how information about
[3] M. Burke and R. Kraut. Mopping up: Modeling wikipedia
negative relationships affects the overall performance of predicting
promotion decisions. In Proc. CSCW, 2008.
existence of positive edges.
Table 9 shows the predictive accuracy for the above task. Since [4] D. Cartwright and F. Harary. Structure balance: A
we pick an equal number of edges and non-edges, random guessing generalization of Heider’s theory. Psych. Rev., 63, 1956.
has an accuracy of 0.50. A logistic regression model using only the [5] M. Chudnovsky, P. Seymour, and B. D. Sullivan. Cycles in
features based on the positive edges improves the performance to dense digraphs. Combinatorica, 28(1):1–18, 2008.
about 0.56 for Epinions and Slashdot, while it gives a much higher [6] D. Cosley, D. Frankowski, S. B. Kiesler, L. G. Terveen, and
boost in Wikipedia, where the classification accuracy almost 0.70. J. Riedl. How oversight improves member-maintained
This is somewhat surprising as we previously saw that for sign pre- communities. In Proc. 23rd CHI, pages 11–20, 2005.
diction Wikipedia was more difficult than the other datasets. [7] J. A. Davis. Structural balance, mechanical solidarity, and
Next we consider a logistic regression model that is trained on interpersonal relations. Am. J. Soc., 68:444–462, 1963.
features based on both positive and negative edges. This model [8] R. V. Guha, R. Kumar, P. Raghavan, and A. Tomkins.
scores 0.59 on Epinions and Slashdot, while it improves the perfor- Propagation of trust and distrust. In Proc. 13th WWW, 2004.
mance on Wikipedia to 0.71. This means that if we use information [9] V. Guruswami, R. Manokaran, and P. Raghavendra. Beating
about negative edges for predicting the presence of positive edges the random ordering is hard: Inapproximability of maximum
in Epinions and Slashdot we get 3 percentage point improvement in acyclic subgraph. In Proc. 49th IEEE FOCS, 2008.
absolute terms, and a 50% increase in the boost relative to random [10] Z. Gyöngyi, H. Garcia-Molina, and J. Pedersen. Combating
guessing, compared to a model based only on positive edges. web spam with trustrank. In VLDB ’04, 2004.
These results clearly demonstrate that in some settings there is [11] F. Heider. Attitudes and cognitive organization. J. Psych.,
a significant improvement to be gained by using information about 21:107–112, 1946.
negative edges, even to predict the presence or absence of positive [12] S. D. Kamvar, M. T. Schlosser, and H. G. Molina. The
edges. Thus it is often important to view positive and negative links eigentrust algorithm for reputation management in p2p
in an on-line system as inter-related, rather than as distinct non- networks. In Proc. 12th WWW, pages 640–651. ACM, 2003.
interacting features of the system.
[13] J. Kunegis, A. Lommatzsch, and C. Bauckhage. The
Slashdot Zoo: Mining a social network with negative edges.
6. CONCLUSION In Proc. 18th WWW, pages 741–750, 2009.
We have investigated some of the underlying mechanisms that [14] C. Lampe, E. Johnston, and P. Resnick. Follow the reader:
determine the signs of links in large social networks where interac- filtering comments on slashdot. In Proc. 25th CHI, 2007.
tions can be both positive and negative. By casting this as a prob- [15] J. Leskovec, D. Huttenlocher, and J. Kleinberg. Signed
lem of sign prediction, we have identified principles that generalize networks in social media. In Proc. 28th CHI, 2010.
across multiple domains, and which connect to social-psychology [16] D. Liben-Nowell and J. Kleinberg. The link-prediction
theories of balance and status. Moreover, our methods for sign pre- problem for social networks. J. Amer. Soc. Inf. Sci. and Tech.,
diction yield performance that significantly improves on previous 58(7):1019–1031, 2007.
approaches. At a global level, we have seen that there is evidence
[17] S. Marvel, S. Strogatz, and J. Kleinberg. Energy landscape of
in all of our datasets for an approximate global status ordering on
social balance. Physical Review Letters, 103, 2009.
nodes, while we find in contrast that there is essentially no evidence
for a global organization of these networks into opposing factions; [18] P. Massa and P. Avesani. Controversial users demand local
this suggests that balance is operating more strongly at a local level trust metrics: an experimental study on epinions.com
than at a global one. Finally, we have seen that employing informa- community. In AAAI ’05, pages 121–126. AAAI Press, 2005.
tion about negative relationships can be useful even for tasks that [19] M. E. J. Newman. The structure and function of complex
involve only the positive relationships in the network, such as the networks. SIAM Review, 45:167–256, 2003.
problem of link prediction for positive edges. [20] B. Pang and L. Lee. Opinion Mining and Sentinment
There are a number of further directions suggested by this work. Analysis. Number 2(1-2) in Foundations and Trends in
A first one is of course to explore methods that might yield still Information Retrieval. Now Publishers, 2008.
better performance for the basic sign prediction problem, and to [21] P. Resnick and H. R. Varian. Recommender systems. Comm.
understand whether the features that are relevant to more accurate ACM, 40(3):56–58, 1997.
methods help in the further development of social theories of signed [22] M. Richardson, R. Agrawal, and P. Domingos. Trust
links. We are also interested in strengthening the connections be- management for the semantic web. In ISWC, 2003.
tween local structure and global structure for signed links. Finally, [23] S. Wasserman and K. Faust. Social Network Analysis:
as noted at the outset, the role of positive and negative relation- Methods and Applications. Cambridge Univ. Pr., 1994.
ships in on-line settings is not limited to domains where they are [24] F. Wu and B. A. Huberman. How public opinion forms. In
explicitly tagged as such. Proc. 4th WINE, pages 334–341, 2008.
Acknowledgements. We thank Michael Macy for valuable discus- [25] L. Xiong and L. Liu. Peertrust: supporting reputation-based
sions, and the anonymous reviewers for a number of comments and trust for peer-to-peer electronic communities. IEEE Trans.
suggestions that helped to improve the paper. Research was sup- Knowl. Data Engr., 16(7):843–857, 2004.