THE ROLE OF CUES AND HEURISTICS IN ONLINE NEWS CREDIBILITY

ASSESSMENTS

by

KATHERINE KEIB

Social Media & Selection

The study of what motivates people to engage with news online has often been centered

on uses and gratifications (U&G) (A. M. Lee, 2013; A. M. Lee & Chyi, 2014; Lin, 1996;

Ruggiero, 2000), on the premise that media selection is driven by an individual’s self-awareness

and expectation that certain media will fulfill those needs, and the prevalence of internet use has

revived its application in communication studies (Ruggiero, 2000). Motivation for internet use

include passing time, interpersonal utility, information seeking, convenience and entertainment

(Papacharissi & Rubin, 2000) as well as problem solving, status seeking, relationship

maintenance, personal insight and persuasion (Flanagin & Metzger, 2001). Regarding news

content specifically, results of a national study of news consumption choices across platforms

revealed four motivations of news consumption: information-motivated, entertainment-

motivated, opinion-motivated, and social-motivated news consumption (A. M. Lee, 2013).

Social media is used prolifically by the public, with 74% of U.S. adults on the platform

(Pew, 2014). People who went beyond just consuming news on social media, but also uploaded

content were found to be motivated by social status and information seeking (C. S. Lee & Ma,

2012). Lee found that when it comes to social media, twitter is used more for entertainment,

opinion and social motivation, Facebook, and twitter are both used least frequently by those with

information motivations (A. M. Lee, 2013). Another study looked at friending journalists and
social media news reception as predictors for sharing news on social media (Weeks & Holbert,

2013). Using a Pew national survey, researchers found that news consumption via social media

and being friends with a journalist did in fact lead to more news sharing, but the behavior was

contingent upon having strong partisan ties. In other words, if the person was consuming the

content regularly, and had strong feelings about it, they were more likely to consume it

positively, and then share it. They also found that people who were news seekers and who got

news via email were more likely to share news socially (Weeks & Holbert, 2013). The sharing

of news via social media is a re-broadcast of the content, therefore it is important for journalists

and researchers to understand how people decide what to share and thus spread further.

One explanation for social sharing is the personal gratification that users gain from

sharing content (C. S. Lee & Ma, 2012b). However, U&G has been criticized because it does

not account for the role of prior experience (C.S. Lee & Ma, 2012b). Prior experience refers to

the past use of social media, in this case, that informs the current and future use of the platform.

In order to overcome this shortcoming, Lee & Ma looked to Social Cognitive Theory to account

for the impact of prior experience on social media use as well as the fact that social media use

may be more passive than active (C. S. Lee & Ma, 2012b). Prior experience, as explained by

SCT, is also used in U&G research to explain the relationship between motivation and self-

efficacy (C. S. Lee & Ma, 2012b). Prior social media experience and socializing have been

found to be the two most important motivators to share content on social media, and led to the

creation of a new U&G model, which combines U&G and SCT (C. S. Lee & Ma, 2012). The

model incorporates prior experience as a predictor between gratifications and intention to share.

The model created by Lee & Ma (2012) was built from previous work that also extended

U&G by applying SCT. One study, for example, addressed behavioral incentives through factor
analysis (LaRose & Eastin, 2004). As people repeatedly use social media, habits are formed,

which can lead to a deficit of self-regulation, and, over time, people may not be as careful in

making decisions (LaRose & Eastin, 2004). The implications of this for shared content here are

that information quickly shared via social platforms may not be carefully processed, and

misinformation can be easily spread, as news sharing intrinsically changes the way users engage

with news (Kümpel, Karnowski, & Keyling, 2015).

Social connections necessarily imply credibility because the source has already been

vetted, whether it is an organization intentionally followed or a friend who is sharing content

they endorse (Metzger, Flanagin, & Medders, 2010b). In addition, users who share news on

social media experience greater involvement with the topic than people who just read the

content, and that involvement increased as people commented on the shared news (Oeldorf-

Hirsch & Sundar, 2015). This greater involvement with the content, coupled with the

credibility automatically given, could result in people mistakenly sharing content that is not

credible. For example, thousands of people shared a false news story about the firing of then

FOX news anchor Megan Kelly in the Fall of 2016 (Ohleiser, 2016). The sharing led to the

story appearing in Facebook’s list of trending stories, which then led to incrementally more

sharing. It was a perfect storm of users acting based on the norms of the platform and the input

from their communities.

Media Credibility

“Journalism is built on credibility” (Cassidy, 2007b)

Conceptualizing the credibility of news information. Credibility, widely considered to

be based on the expertness and trustworthiness of content (Hovland, Janis, & Kelley, 1953), has

been studied in earnest since the 1940s and 1950s. Factors from war time propaganda, to the
growth of television news, and increased advertising, generated research into areas such as who

the public trusted to deliver their news, how citizens decided what was fact and what was hype

and whether people believed advertisers were influencing editorial agendas (Hovland, Janis, &

Kelley, 1953; Metzger, Flanagin, Eyal, Lemus, & McCann, 2003; Roper, 1985). Initially, work

focused on perceptions of the credibility of the message communicator, known as source

credibility (Hovland et al., 1953). The public found information credible if they found the

person delivering the message to be credible, and that assessment could apply beyond the

individual to the organization the person works for or even a medium in general (Hovland et al.,

1953). Over time, the study of credibility became more nuanced, as some critics believed

Hovland’s work was too simplistic (Markham, 1968), and others criticized his lack of theoretical

foundation (Berlo, Lemert, & Mertz, 1969). As scholars undertook source credibility studies,

they began to parse out expertise as a source characteristic and trustworthiness as a characteristic

of the receiver (Metzger, Flanagin, Eyal, Lemus, & McCann, 2003a). Borne of this distinction,

new studies on message and medium credibility began.

Message credibility specifically refers to the media content and includes message

structure, the content itself, language intensity and the delivery of the message (Metzger,

Flanagin, Eyal, Lemus, & McCann, 2003b). Message design characteristics such as font style,

white space, images and color schemes have been shown to influence user’s perceptions of

online credibility (B. Fogg et al., 2003). Assessments of the message influence perceptions of

the source (Kiousis, 2001; Metzger et al., 2003b; Stamm & Dube, 1994). Because of this

relationship, message credibility is often studied along with source or medium credibility. A

new message-specific credibility scale developed by Appelman and Sundar focuses on content
related factors, not design factors as characteristics of message credibility (Appelman & Sundar,

2015).

Another element beyond message credibility or source expertise that has received

attention by researchers is coorientation between the sender and receiver (H. K. Meyer,

Marchionni, & Thorson, 2010; P. Meyer, 1988). Coorientation refers to the relationship between

people and the objects they experience (Avery, Lariscy, & Sweetser, 2010; Newcomb, 1953).

An example would be a journalist, their news story, and the person who reads it. Meyer’s work

revealed that users do feel a connection to the person creating the content, and that this

connection influences judgment about the content (P. Meyer, 1988). The perceived expertise of

the journalist leads the receiver to see the content as more credible precisely because of the

relationship, or coorientation (H. K. Meyer et al., 2010). Considering social media and the

coorientation of journalists and public relations practitioners, researchers found a similar regard

among both groups for the use and importance of social media tools (Avery et al., 2010). As

content has moved online; coorientation suggests there is still a relationship between the source

and the receiver.

In addition, the technological affordances of the internet have refreshed the study of

credibility with inquiry into the influence of message factors such as interactivity and

hypertextuality (Appelman & Sundar, 2015; Chung, Nam, & Stefanone, 2012; Metzger et al.,

2010b; Sundar, Oeldorf-Hirsch, & Xu, 2008). Recent work argues that message credibility is a

separate concept and a new scale was developed and tested for measuring it (Appelman &

Sundar, 2015). In fact, the aspects of online message design are unique to the platform –

hypertextuality is not embedded in messages delivered on television or radio.

the platform that delivers the content. For example, content watched on television carries

specific indicators of credibility applicable only to television, such as cues related to the anchor

speaking (Bucy, 2003), that newspaper would not contain. While primarily focusing on the

communicator, Hovland suggested that medium mattered as well as source (Hovland et al.,

1953). Several years later, Westley and Severin came to the same conclusion, stating that

source is distinct from message and this area needed more research (Westley & Severin, 1964).

These early distinctions between content and medium re-enforce the concept that medium

deserves its own study, and more so, that distinct media should be studied by looking at their

unique attributes.

Online credibility assessments. Study of the credibility of the internet as a medium

began in earnest in the 1990s. Much early work looked at how the public and journalists

themselves evaluated the credibility of the internet vs. other media (Flanagin & Metzger, 2000).

The internet quickly earned credibility ratings equal to or higher than traditional news mediums,

which can be explained in part by the reliance on the medium (Johnson & Kaye, 2010). Recent

work has delved into the complications that social media and unknown sources bring to users

facing an overwhelming number of content items delivered daily via Facebook, Twitter, Google

searches and email (Chung et al., 2012; Sundar et al., 2007). Findings show that new features

specific to the online medium are used by people when assessing credibility and should be

studied along with traditional factors (Chung et al., 2012). Metzger and colleagues (2003)

argued that users must be trained or taught in order to understand the tools available online and

the norms of the medium (Metzger et al., 2003b). Online, people assess the webpage itself as

part of a medium assessment, the content itself as message assessment and the publisher or
individual journalist as a source assessment (Metzger et al., 2003a). Lacking formal training,

time and repeated use has taught us how to assess credibility online and these processes may

become automatic, iterative, and reliant on cues that guide our decision-making process (Sundar

et al., 2008; Wathen & Burkell, 2002).

Recent work summarizing credibility in the 21st century stipulates that scholars should

study various levels of credibility (Hilligoss & Reih, 2008; Metzger, Flanagin, Eyal, Lemus, &

McCann, 2003). The interaction between these levels can cause one to influence the other

(Kiousis, 2001), and it is not always in set path; people do move back and forth between levels in

credibility assessments (Hilligoss & Rieh, 2008).

Why online credibility assessments matter. As was brought to light in the 2016 U.S.

presidential election, the credibility of online content can have significant repercussions in

society (Dougherty, 2016) News that was false was spread, largely via social media, in order to

sway voters. Even before this proliferation of fake news, other researchers have studied online

credibility because of concerns that, for example, a lack of reliable information could lead to

dangerous medical, financial and personal decisions (Markov et al., 2014). Online, common

filters for content, such as government regulators, or journalistic gatekeepers, are not able to

weed out all the incorrect, misleading or manipulative content. Following the 2016 election,

several organizations experimented with fake news filters (Oremus, 2016). While some of these

filters exist online, the volume and diverse nature of online content makes it impossible to filter

all content published. Thus, the public is largely left to decide for itself what is credible.

Although certainly some consumers have become literate in how to navigate online

content, their ability to do so varies widely. The main way that consumers learn to discern what

is credible online is through experience (Flanagin & Metzger, 2000), and that experience informs
their credibility assessments (Fico, Richardson, & Edwards, 2004; Johnson & Kaye, 2016).

Consumers themselves report that experience with a site helps them determine the credibility of

content on the site (B. Fogg et al., 2003). Experience is drawn upon when users notice cues or

markers on websites, and assign meaning to them (B. J. Fogg, 2003), which can lead to habits

and less thoughtful attention to content, i.e., habitual news reading (LaRose & Eastin, 2004;

Schudson, 2003). Habitual news reading has been operationalized as a construct of self-

regulation, which is a part of SCT, and habit is seen as a failure of SCT (LaRose & Eastin,

2004). Specifically, habit means that potentially “…we become inattentive to the reasoning

behind our media behavior, our mind no longer devotes attention resources to evaluating it”

(LaRose & Eastin, 2004 p. 363). This behavior can allow people to employ credibility heuristics

on one website, then apply them to a different site with similar design elements (Hilligoss &

Rieh, 2008). These short cuts in the credibility assessment process can lead to misapplied

credibility on the web.

The websites on which people choose to visit can determine the type of content they are

thus continuously exposed to. Search engines remember our past preferences and deliver custom

search results (McEvoy, 2015). Social media news feeds are populated with posts from friends

and brands we have chosen to follow, and the outlets whose posts we click on more often are

thus served more often in our feed. Selective exposure refers to the phenomenon of content

being limited by self-selected choices in exposure, which, critics warn, can lead to polarization

and lack of open-mindedness (Messing & Westwood, 2012; Zillmann & Bryant, 1985).

Although there has been concern about the effects of selective exposure leading to ideological

polarization, several recent studies have cast doubt that online content or social media is

increasing the problem. Users who are experienced at reading political blogs and partisan
websites are more willing to seek out opposing viewpoints online, good news for those who see

the importance between reading a variety of opinions and a strong democracy (Johnson & Kaye,

2013). In addition, cues such as endorsements in social media increase exposure to a variety of

viewpoints and opinions (Messing & Westwood, 2012). As people are exposed to content from

which the source is not familiar, and the content topic may also be out of their wheelhouse,

assessing the content must rely on elements other than source or topic familiarity.

When online credibility assessments occur. Web users form impressions of the

credibility of a website very quickly, in less than 50 milliseconds, in fact (Lindgaard, Fernandes,

Dudek, & Brown, 2006). They notice design elements, even if that noticing is subconscious &

instinctive (Lindgaard et al., 2006), which then trigger mental shortcuts called heuristics.

Assessments happen quickly, and design can lead to an incorrect assessment. Design elements

that consumers use to assess credibility, such as font style, hyperlinks, and length of copy are

easy to mimic, (Metzger, 2007) and websites wishing to gain credibility can mimic credible

styles, using design to mislead consumers (Metzger, 2007). Additionally, individuals bring

preconceived attitudes into the assessment process (Kiousis, 2001) and researchers often account

for these by assessing relevant differences through their study design. Participants are often

asked about items such as experience with a medium (Fico et al., 2004), familiarity with a brand

(Flanagin & Metzger, 2007) or level of knowledge on a topic (Metzger et al., 2003).

A model of the credibility assessment process developed by Hilligoss & Reih (2008)

integrates common aspects of credibility assessment, regardless of the media, type of information

and environment of use, i.e. context. The authors assert that a credibility assessment is the result

of individual credibility judgments, which may take place in no specific order, and may affect

each other. They developed their model through an observation and analysis of credibility
assessments made over time by 24 subjects. The levels within the assessment are called

construct, heuristics and interaction. Construct is the most abstract level, includes the individual

aspects a person uses to define credibility, and is unique to each person. The heuristics level

encompasses all the shortcuts people use to assess credibility, which may come from the media,

source, aesthetics/design or be endorsement based. The interaction level refers to the specific

source and content cues related to the content at hand. These three sets of characteristics can all

influence situational evaluations of the credibility of a message.

This model shares some characteristics of another model, developed by Wathen &

Burkell (2002). Their model, based on theory, not on actual observations, also suggests a

layered process, but one that happens in linear steps (Wathen & Burkell, 2002). They posit that

users make initial judgments about surface characteristics of a website, then, if the website meets

the standards, they continue to consume the content and simultaneously undertake more

credibility judgments. At each step of their process, users will either continue to consume and

assess the site, or will choose to exit and abandon the content. Given this scenario, the stakes are

high for publishers to get the surface and message characteristics right if they want to attract and

maintain an audience. Conversely, sites wishing to attract an audience just to generate clicks and

revenue, as well as propaganda and advertiser based sites, can mimic credibility cues and present

information that the audience finds difficult to accurately assess. This dissertation will explore

the process that takes place, and, although not explicitly testing these models, will perhaps shed

some insight into them. Specifically, Study I will ascertain the commonalities and differences

that individuals self-report about their credibility assessment process. The eye-tracking portion

in Study II will confirm that people do fixate on aspects of the webpage that they told us they

consider.