1

CONTENT MODERATOR STARTLE RESPONSE

Content Moderator Startle Response: A Qualitative Study

Timir J. Bharucha, Miriah E. Stieger, Priyanka Manchanda

TaskUs

Rainer Mere

Sutherland Global Services

*The pre-print version of this paper is not the copy of record and may not exactly replicate the

final, authoritative version of the article. Please do not copy or cite without the authors’

permission.
 2
CONTENT MODERATOR STARTLE RESPONSE
Abstract

Commercial Content Moderators review user-generated content (UGC) to ensure the

posts meet platform policies, guidelines, community standards, and regional laws. While the

majority of posted UGC is deemed acceptable, a large amount of content remains, which is

classified as non-compliant and may include gore, violence, suicide, child sexual abuse material

(CSAM), and pornography, to name a few. Because of this, content moderators have a greater

prevalence of their nervous system activating a startle response, which can impact emotional,

psychological, and physiological processes.

Prior research on content moderators has failed to explore moderators' initial reactions to

content from the start of employment through tenure as the subjection to material and habituation

increases. This study takes an in-depth look at moderators' experiences from recruiting, through

training, and production to better understand the content moderators' startle response and factors

that enable startle habituation.

The current study sample consisted of a total of 78 respondents - 38 content moderators

located in the Philippines employed by TaskUs Inc. and 40 content moderators located in Estonia

employed by Sutherland Global Services. Employee tenure ranged from 0-6 months. Succeeding

our analysis, transparency, understanding, and preparedness were major themes identified as the

critical factors found within both companies when exploring the activation of the startle response

and facilitation of habituation following content exposure. These themes were prevalent in the

recruiting, training, and production phases of the employment life-cycle.

Keywords: Commercial Content Moderation, Startle Response, Startle Habituation, Wellbeing,

Resilience, Recruitment, Training

 3
CONTENT MODERATOR STARTLE RESPONSE
Introduction

Commercial Content Moderators surveil and examine user-generated content (UGC) on

various social media platforms in an effort to confirm that user posts comply with the set terms,

policies, guidelines, community standards, and applicable regional laws (Roberts, 2016; Steiger

et al., 2021). While the vast majority of posted UGC is deemed acceptable, a heightened volume

of non-compliant media remains in circulation (Steiger et al., 2021). Examples of this can

include gore, violence, murder, suicide, child abuse, sexual violence, and animal abuse (Roberts,

2016; Steiger et al., 2021).

When left unattended, repeated or over-exposure to graphic and disturbing content

increases psychological and emotional distress (Steiger et al., 2021). According to prior research,

the startle response, or reflex, is significantly activated when exposed to unpleasant visual

stimuli (Hartman et al., 2021; Reagh & Knight, 2013). This response hinders an individual's

cognitive and dexterous operations for up to 30 seconds, leading to reductions in decision

making and performance (Deuter et al., 2012; Martin et al., 2012; Vaidyanathan, 2014).

Recurrent subjection to provoking stimuli can lead to startle habituation, the

diminishment of behavioral and psychophysiological reactions (Chen et al., 2014). As the

novelty of the attention-gathering stimuli decreases during autonomic emotion regulation,

emotional responses diminish (i.e., habituation; Chen et al., 2014; Phelps et al., 2012; Valsamis

& Schmid, 2011).

In addition to the graphic nature of the content, moderators have high quota demands.

Moderators are required to review 800-1,000 social media posts per day according to research

(Mukhopadhyay, 2020). Process time averages 20-30 seconds, increasing with complexity

(Mukhopadhyay, 2020). Assignment of material varies, with some queues consisting of specific

and steady, yet egregious content such as pornography or violence. Other queues are more
 4
CONTENT MODERATOR STARTLE RESPONSE
unpredictable with randomization of harmless or benign; and graphic or disturbing content

populating in a single day directed by algorithms (Silbermann, 2020).

Overall, there is a paucity of research on content moderator startle response and

subsequent habituation. Research on commercial content moderation primarily focuses on

automated moderation, artificial intelligence, and algorithms, which overshadows the human

element and the effects on moderators (Carmi, 2019; Chandrasekharan et al., 2019; Dang et al.,

2018). The few studies that do exist on the experience of content moderators primarily explored

prolonged exposure to content and the probability of developing adverse mental health effects

(Arsht & Etcovitch, 2018; Gorwa et al., 2020; Steiger, 2020; Steiger et al., 2021). Content

moderator reactions to content, and subsequent development of habituation, have garnered

minimal attention. This study looks to fill this critical gap in the literature.

We aim to expand the knowledge base of moderator experiences and, specifically, to

better understand their startle response and the development of habituation. The study was

examined through the lens of Schacter and Singer’s (1962) Two-factor Theory of Emotion,

which asserts that emotions are composed of two factors: physiological arousal and cognitive

appraisal. The theory postulates that when an individual experiences an emotion, physiological

arousal takes place (1962). The Two-factor Theory of Emotions is a manner in which to explain

the impact of the stress-strain relationship (Beg, 2015). The fundamentals of Socio-

Physicochemical theory propose that physical and emotional components are stimulated by

situational circumstances, even more so during extreme instances such as violence or terror (Beg,

2015).
 5
CONTENT MODERATOR STARTLE RESPONSE
Methods

Participants

The sample consisted of 38 content moderators located in the Philippines employed by

TaskUs Inc and 40 moderators located in Estonia employed by Sutherland Global Services, both

content-moderation service providers. Employee tenure ranged from 0-6 months. The content

moderators were assigned to categorized queues made up of various media types (images, video,

audio, & text) with a varied level of graphic content ranging from benign to graphic. Categorized

queues require the review of specific and reoccurring content with minimal deviation in assigned

themes.

Procedures & Data Collection

All study procedures were compliant with the code of conduct, legal regulations, and

ethical guidelines of TaskUs Inc and Sutherland Global Services. It is important to note that,

throughout the entire duration of the study, the research teams at both companies remained

independent entities that did not report directly to departments that may oversee content

moderation employees (e.g., Operations). As a result, the researcher teams were not invested in

particular outcomes of the study and could therefore evaluate the data objectively. That is,

special efforts were made to minimize conflict of interest. To further ensure the ethics and

objectivity of this study, an external researcher and licensed counselor from a Carnegie R1

institution independently reviewed the study procedures and data.

To gather candidates, the researchers distributed a flier detailing the study and requested

participants through both companies' employee email distros. Employees were explicitly

informed that (1) participation would be completely voluntary; (2) their continued employment

or performance review would not be contingent upon their participation; (3) their responses

would be stored securely and not shared with anyone beyond the research teams (e.g., not with
 6
CONTENT MODERATOR STARTLE RESPONSE
their supervisors); and (4) their participation would not be associated with monetary

compensation. For employees who indicated interest, the researchers met with all volunteers to

discuss the purpose of the study in greater detail. All interested participants were sent a link that

contained the consent form. The form included the introduction and purpose of the study, risks

and benefits, procedures, confidentiality statement, voluntary nature of the study, participant

rights, and contact information of the researchers, and a 27-item, open-ended questionnaire (see

details below). After the study, participants were thanked, debriefed, and provided with mental

health resources. No adverse outcomes (e.g., mental health deterioration) were observed as a

result of participating in this study.

Questionnaire

This study used a 27-item questionnaire comprised entirely of open-ended questions.

Prior to analysis, questions were placed into the following domains: Training; Production;

Current Cases; and General Experiences. Participants were given a prompt or question and were

able to type in responses.

Training

These questions pertained to participant reactions to training for content moderation.

These included questions about their thoughts (“Tell me your initial thoughts in your first few

weeks of training”), feelings (“What were some of the emotions you experienced during training,

what caused those feelings, and what was occurring at those times?”), and after-effects (“How

did the exposure to graphic content impact the rest of your workday and your day after your shift

ended?”).

Production

During production, content moderators review UGC. This set of questions focuses on the

transition from training to application (“What are some differences you can identify between the
 7
CONTENT MODERATOR STARTLE RESPONSE
jobs/cases you encountered in training versus now in production?”); and changes from training

content to real-world application (“How have your thoughts, feelings, physical sensations, and

reactions about the content changed since moving from training to production?”).

Current Cases

Participants were asked a set of questions about their reactions to the most recent queue

of UGC. These questions focused on how difficult the moderation process is (“What makes a

job/case easy to review? What makes a job/case difficult to review?”); how moderators adapt to

UGC (“What has helped you adapt or become more comfortable with the content?”); and the

effects of content moderation on performance (“What is your average time completing a job in

general, and what is your average time completing a graphic job?”).

General Experiences

Participants were asked several questions about their overall experience with content

moderation. Here, questions related to pre-expectation versus the reality of their job-role

(“In what ways was this job similar and different to what was explained to you during the

recruitment process? What were some things you did not expect to experience?”); reactions to

the job (“Tell me about the time you had to take a break from work due to the material? Please

describe the content and what you were thinking and feeling (emotions and physical

sensations),” “What types of material are you worried about encountering (or used to be)?”);

strategies to mitigate the effects of the content (“What skills, techniques, or resources did you

use so you could move back into your workflow?”); and participants’ sense of adjustment (“I am

curious if you feel you have adjusted to the work? What are your thoughts on adjustment, what

assisted you in adjusting to the work, and how long did it take to adjust?”).
 8
CONTENT MODERATOR STARTLE RESPONSE
Results

Analysis

The study was conducted through the lens of the Two-factor Theory of Emotion

(Schacter & Singer, 1962) and via a bottom-up, deductive, thematic content analysis. The

thematic content analysis approach is appropriate for survey data and when attempting to

understand the phenomenological experience of participants. As we did not enter the study with

a hypothesis, we let the results manifest from the data (i.e., a bottom-up approach).

Thematic Content Analysis

Participant responses were de-identified and uploaded to MaxQDA for analysis. We

followed the traditional steps used in thematic content analysis (Kiger &Varpio, 2020; Maguire

& Delahunt, 2017): familiarization, coding, generating themes, reviewing themes, and defining

themes. Familiarization refers to reviewing, understanding, and becoming comfortable with the

responses. We then examined the data closely and assigned codes to specific responses or

portions of responses. Following that, we grouped the coded data into categories. After carefully

reviewing the coded responses and associated categories, we then began the process of

identifying themes. This involved condensing similar categories into broader, generalizable

themes. These themes were then reviewed by comparing them to the responses to ensure that the

themes were supported by the data. Finally, we appropriately labeled and defined the themes that

emerged from the data.

A number of themes were identified through the content analysis. This section presents a

stratified explanation of each theme as it pertains to training, production, current cases, and

general experiences collectively across both companies. Table 1 provides a list of identified

themes.
 9
CONTENT MODERATOR STARTLE RESPONSE
Training Themes

Concerning the training phase, respondents reported overall enthusiasm, curiosity, and

preparedness for managing their roles. Within this stage, a range of emotions was identified

when addressing first exposure to media; however, generally, respondents remained neutral and

stabilized. Ways of adjustment conveyed included the training department’s capability to teach

relevant fundamental learnings. Specific themes surfaced are outlined below.

Learning Enthusiasm

When asked about their initial thoughts in the first few weeks of training, new hires

stated, "excited to learn new things," "curious about the things we will encounter," and "you're

being taught the things you have never heard before, so it was really neat to me," with a large

majority of participants endorsing the sentiment of excitement regarding the opportunity to learn

new information and work as content moderators. For example, one moderator responded,

"enthusiastic about the world I am about to discover!" In addition, moderators deemed their work

meaningful and purposeful by safeguarding social media; yet, this specific theme emerged at a

greater frequency when working in the production phase versus onboarding with the company.

Readiness

Exploring thoughts, emotions, and physical sensations associated with first exposure to

egregious content, some moderators indicated no impact as they had prior experience viewing

similar media in their personal lives. As such, they were prepared for the sensitive media coming

into training. “I have exposure in my personal life and stuff I seen on social media, so it was an

easy transition for me.” Another went on to add their preferences for horror movies. “I really like

scary movies, and that has graphic contents. Because I watched a lot, it helped me with this

work.” Prior work experience was also found to increase preparedness, “I came from another

similar job so nothing new in that way.”

 10
CONTENT MODERATOR STARTLE RESPONSE
Looking at the process of moving out of training and into production, most participants

indicated no adversity and a smooth transition as they felt ready and eager but anxious. “Because

of training, I could manage my response, emotions, and thoughts of the content I am reviewing.”

Furthermore, candidates emphasize no difference in types and severity of content from training

to production. “Almost all content was discussed in training, and production only had slight

differences, so the change was not hard for me.”

Leaving the training phase, content moderators appreciated the experience and in-depth

instruction provided by the trainers, which included clarifying protocols for actioning the correct

tags on material when reviewing cases and removing ambiguity. They felt training created a

foundation that could be relied upon when facing adversity or uncertainty. “For me the training

was really good. I learned a lot and in the times I’m having confusion I go back to the materials

to help me.”

Production & Current Case Themes

While transitioning from training to production, the first signs of startle response and

habituation were found when moderators were responsible for maintaining specific metrics, and

the content volume increased. Questions related to production and current case themes

overlapped and are therefore combined as one section in relation to emerging themes. Further,

some individuals reported a conflict between what to anticipate and reality, with inconsistency

between training and production relating to content protocols and severity. This resulted in

increased process time with greater cognitive load. However, protective factors materialized in

the form of finding a sense of purpose that allowed moderators to cope with the changes.

Process Ambiguity

Although most respondents indicated a high appreciation for training as it assisted in

solidifying a fundamental understanding of the job, including processes, procedures, and type of
 11
CONTENT MODERATOR STARTLE RESPONSE
content to expect, a select few desired more from the process once in the flow of production. The

moderators expressed concern regarding the largely theoretical approach of training, which

provided insufficient hands-on opportunities within training.

Respondents found training decks outdated and lacking in edgier, diverse examples as a

reference instead of the more assorted level of severity found in production. “There are lots more

variations in content and more intense in production than training.” Likewise, the training

modules were defined as black and white, meaning the review, violations, and decision-making

was more evident than the subjectivity of content in production. “Production has content that is

more diverse in the topic than in training. I believe it should contain more examples.”

Even with these discrepancies, the moderators conceded the impossibility of covering all

potential scenarios and media in training, as material populated in the queues is unpredictable

and ever-changing. Therefore, moderators focused on areas within their control, such as

knowledge of policies, procedures, and wellness interventions instructed. “You cannot learn

everything in training, having experience in production is needed to get used to the content and

job.” Although, participants correspondingly requested discussions surrounding updates to policy

and workflow changes.

Sense of Purpose

The development of a sense of purpose solidified the self-affirmed necessity of the

moderator's role and aided in the adjustment to content. Respondents saw themselves as

protectors and defenders charged with keeping social media a safer place and reducing societal

harm by removing controversial media. Additionally, the moderators believed that they could

bring justice to those afflicted by reviewing the content. "I take pride in what I do no matter how

hard it may be at times, I look back on the purpose why I do this job. And it is fulfilling."

Another moderator responded I always think about the reason I am doing it."
 12
CONTENT MODERATOR STARTLE RESPONSE
Habituation

The participants endorsed habituation and adjustment to egregious content, with the

majority reporting the habituation timeframe ranging from 2 to 4 weeks. In this context,

habituation did not imply a lack of impact for unexpected or severe content but rather a decrease

in emotional, psychological, and physiological responses. For example, one participant stated, “I

get a little startled with the unexpected before, but now it does not affect me as much.”

Although there are still instances when unanticipated content populates, leading to

startle/shock, evidenced by an increased heart rate, sweating, and momentary cognitive

impairment, the moderators report feeling more accustomed to and ready for the exposure when

working in specialized queues. They describe an increased capacity to manage the response and

process the information when working with categorized content with clustered concepts. “By the

time I moved to production, I am used to viewing sensitive content, though sometimes I am still

shocked at the media but doesn’t effect me much. I am able to handle it.” Another content

moderator added, “I could feel myself gradually becoming used to the job and content. Training,

recruiting, and wellness really helped make me ready.”

Hindrance to Habituation

The primary factors that disrupted or slowed adaptation was associated with the

workflow. This included keeping up with the fast-paced environment of changing policies and

trends. Stress, fatigue, and cognitive overload were associated with these frequent updates. In an

effort to manage change, some moderators reported continuing work-related activities during

scheduled breaks. “Daily stress and fatigue can tear down mental fortitude due to random

changing protocols and shifts schedules it deteriorates your ability to focus and process graphic

content.”
 13
CONTENT MODERATOR STARTLE RESPONSE
Concerning content, new, unexpected, intensely egregious content and subjects or topics

moderators were not trained to action altered their ability to adapt. Unfortunately, due to the

ever-changing world and current trends, training and established procedures cannot always

anticipate or account for everything a content moderator may face. When this occurs, moderators

rely on their established forms of support for assistance: wellness staff, managers, trainers, and

teammates.

General Experience Themes

Questions that fell within the “General Experiences” section are listed separately as they

address the recruiting process’s influence on adjustment. Respondents disclosed they were

provided adequate information related to the role requirements and risks involved with

moderation work, allowing them to conduct self-research on moderation and make an informed

decision on whether the role is suitable for themselves. Further, this section identified specific

interventions, techniques, and coping strategies self-reported by moderators to adapt to their

unique work tasks.

Transparency

When in the recruitment process, numerous moderators reported an overall positive

experience. They regarded the recruitment team as clear and accurate with their descriptions of

the job requirements and the potential exposure to sensitive content. These individuals felt as if

they were informed enough to grasp the nature of the job thoroughly. One moderator responded,

"During my recruitment process, they already told me that I'll be doing as a content moderator

handling disturbing content, so before I enter the training, I am prepared for the job."

Conversely, a subgroup of individuals indicated a lack of transparency and the use of

coded dialogue from the recruiter. The respondents stated that the recruiters used words like

"confidential, sensitive media, or graphic" when referencing the content exposure leading to a
 14
CONTENT MODERATOR STARTLE RESPONSE
subjective understanding. Furthermore, there was a failure in fully conveying the time exposed to

and spent reviewing graphic content. The subjectivity, diverse interpretation of terms, and high-

level overview left some feeling unprepared or unaware of the actual reality of work. One

participant wrote, "The interviewer disclosed I will be viewing sensitive medias but not much

more detail." Another responded, "I didn't expect that I had to view so much graphic media."

Concise explanations throughout the recruitment process assisted in the transition to training.

Preparation

After accepting the job offer but before starting training, a few employees researched on

their own to fully understand the day to day and requirements of a content moderator. This

personal quest for knowledge was said to be a key differentiator in increasing readiness.

However, other respondents regarded the content and the potential impact on them as unsettling.

After initial exposure to graphic content, disgust, shock, fear, anger, and sadness were the most

expressed emotions. Physical sensations included the body tensing, palpitations, perspiration,

headaches, and an inability to focus. Moderators reported a disconnect between feelings of

preparedness, perceptions of expectations, and the actual reality of content exposure and its

impact. This disconnect led to a desire for more transparency and preparation early on.

Due to the ever-changing trends in media, even those tenured in the job reported

randomized content populating within their queues that fell far outside of their general

categorization. This resulted in a startle activation and required a break from the material. The

average duration of time necessary for recovery following the startle reflex lasted 30 seconds.

However, when the startle response was unattended, the fight, flight, or freeze response

activated, resulting in a need for “five to ten minutes” to return to baseline functioning. The most

prevalent media reported that led to startle response included violent acts, harm to others, and

child sexual abuse material, as evidenced by replies such as, “...if it’s too bloody or too gory to
 15
CONTENT MODERATOR STARTLE RESPONSE
watch.” “It’s when I saw child abuse, I needed to pause for a few minutes and just listen to music

to divert my attention.”

Programmatic Interventions Employed

Aside from generalized operational practices, participants emphasized the importance of

the psychological health and safety programming facilitated by TaskUs and Sutherland Wellness

& Resiliency Departments in adjusting to the nature of content moderation. The program

provides wellness resources such as psychoeducational training, skill-building groups, individual

sessions, e-learning courses, monthly outreach, and recommendations for tooling and breaking

strategies/interventions based on research findings. These resources are strategically developed

to assist the moderator in reviewing and processing all content levels.

Employees stressed the importance of engaging in wellness training where they could

acquire and develop the mastery to help them in times of crisis. Skills and exercises learned from

the wellness and resiliency departments that facilitated a return to baseline after startle activation

were communicated by the respondents as “physical activity,” “breathing exercises,”

“meditation,” “stretching,” and “communicating with coworkers.” One moderator stated,

“Actually, because of all the wellness and interaction, I’m quite all good.” Access to the

Wellness & Resiliency program was said to be a vital asset assisting in the transition period from

training to production, and content moderators interviewed called for more wellness initiatives to

be placed in the recruiting and training phases.

Discussion

This study addresses a gap in the literature and increases the understanding of

moderators’ initial experiences with content, startle response activation, and startle habituation.

Previous studies on startle response pointed to the emotional, cognitive, and physiological

impairment the phenomenon could have on the individual in response to adversarial or

 16
CONTENT MODERATOR STARTLE RESPONSE
unexpected stimuli (Deuter et al., 2012; Martin et al., 2012; Vaidyanathan, 2014). This

unconscious and protective response were identified in the experiences of content moderators.

Participants in the study were able to identify instances when their startle reflex became

activated. This occurred when they were exposed to unexpected graphic media. Disgust, shock,

fear, anger, and sadness were emotions they experienced. Physical sensations included body

tension, palpitations, perspiration, headaches, and an inability to focus. These responses

impacted both wellbeing and workflow.

Participants identified a reduction in behavioral and psychophysiological reactions over

time and described an increased ability to manage and process graphic media. Key elements that

reduced the prevalence of startle response included open and transparent conversations from

recruiters when describing content exposure and duties of the job to potential moderators. This

was supported by prevalent themes such as preparation and enthusiasm. Those candidates who

fully understood the reality of the job described an easier transition into training and entering

production. Additionally, when in training, content moderators greatly benefitted from in-depth

instruction and examples that were diverse in topic and severity; this was evidenced by the

following themes: Sense of Purpose and Readiness. Conversely, some individuals felt

unprepared as the theme of process ambiguity surfaced in responses.

When in production, most individuals reported impact by content exposure, yet the

majority noted adaptive adjustment to their work as evidenced by the theme of habituation.

Themes of factors that either harm or facilitate habituation were noted. For instance, the theme of

Hindrance to Habituation indicates that specific environmental influences may be a barrier to

adaptive adjustment. Still, programmatic access and a sense of purpose substantially facilitated

habituation. Specifically, the psychological health and safety programs and initiatives assisted

employees in processing and managing content when in production.

 17
CONTENT MODERATOR STARTLE RESPONSE
Implications and Recommendations

This study suggests a need for further research exploring content moderators' responses

to graphic content and the development of interventions or strategies to facilitate habituation.

The results of this study support the necessary creation of strategic initiatives throughout the

employment lifecycle. This includes increased transparency, understanding, expectations, and

requirements of content moderation. Each phase will also greatly benefit from the

implementation of well-being focused interventions.

In the recruitment process, recruiters should be as detailed as possible, describing the

distinct nature of the occupation. This includes verbally detailing examples of content that the

candidate has the potential to be exposed to (i.e., giving detailed examples of content to be

reviewed). Lastly, recruiters should emphasize the availability of wellness support, both

internally and externally available.

When considering elements and structure of training, wellness resources, and coping,

strategy integration is necessary throughout the training processes. Further interventions should

center on acclimation to the rapid pace and change of their work tasks and environment. When

illustrating examples of content, gradually increasing the graphic nature of the material can lead

to greater moderator preparedness, expectations, and ability to process such content. Further,

extending the duration of training allows the content moderator exposure to a variety of media,

topics, procedures, and policies; and decreases the pressure and cognitive overload experienced

while learning.

In production, we recommend rotating teammates' dedicated content queues. This

increases expectations and preparedness for unavoidable content, reducing the possibility of

shock and surprise. Next, moderators should be offered flexible content breaks to be used when

the individual is experiencing a startle. This allows the content moderator time to step away and
 18
CONTENT MODERATOR STARTLE RESPONSE
recompose themselves. Operation leaders should emphasize the importance of mental health as

moderators may worry about not meeting metric standards and protocols.

Limitations and Future Research

No study is without its limitations. While this study’s conclusions are derived from

participant responses, those responses came from survey questions and not live interviews.

Future research on the lived experiences of content moderators should incorporate semi-

structured interviews to furth expound on participant responses. Consistent with all studies using

self-report methods, it is possible that self-report bias may exist. However, several steps were

taken to limit its potential impact on study findings. For instance, participants provided responses

via survey without a live observer. To further encourage honest responses, participants were

informed that their data would be secured and not shared beyond the research teams.

Second, results were acquired from two companies located in different countries. The

current study does not address the cultural differences pertaining to either the country of origin

or the company itself. The differences may have influenced how individuals interpret and adapt

to the content and address the potential risk of moral injury when reviewing material that falls

outside of their accepted cultural norms. We advise future researchers to look into these key

factors.

Future research should also focus on biofeedback and content moderation. The industry

and academics will greatly benefit from biofeedback research in the form of skin conductance

and heart-rate variability. This may provide objective and quantitative data that supplement the

current findings derived from subjective and qualitative data. Following the emerging theme of

Hindrance to Habituation, future research should explore whether programmatic psychological

interventions are a moderating variable to the identified adverse environmental factors.

 19
CONTENT MODERATOR STARTLE RESPONSE
Acknowledgments

The authors express deep gratitude for those content moderators who participated in the

research and so willingly shared their experiences. With this knowledge, we can better support

the well-being of all reviewers. Additionally, the authors would like to thank content moderators

worldwide for the tremendous and much-needed work that they do safeguarding the internet and

social media platforms for the end-user.

Conflict of interest

TaskUs Inc. and Sutherland Global Services employ the authors and content moderators

interviewed for the study.

Funding Statement

The research study was conducted without any external funding.

 20
CONTENT MODERATOR STARTLE RESPONSE
References

Arsht, A., & Etcovitch, D. (2018). The human cost of online content moderation. Harvard Law

Review Online, Harvard University, Cambridge, MA, USA. Retrieved from

https://jolt.law.harvard.edu/digest/the-human-cost-ofonline-content-moderation.

Beg, Mirza. (2015). Socio-Physicochemical interpretation of theory on emotions.

10.13140/RG.2.1.1825.4241.

Carmi, E. (2019). The hidden listeners: regulating the line from telephone operators to content

moderators. International Journal of Communication, 13, 440-458.

Chandrasekharan, E., Gandhi, C., Mustelier, M. W., & Gilbert, E. (2019). Crossmod: A

cross-community learning-based system to assist reddit moderators. Proceedings of the

ACM on human-computer interaction, 3(CSCW), 1-30.

Chen GD, Radziwon KE, Kashanian N, Manohar S, Salvi R (2014) Salicylate-induced auditory

perceptual disorders and plastic changes in nonclassical auditory centers in rats. Neural

plasticity 2014:658741

Dang, B., Riedl, M. J., & Lease, M. (2018). But who protects the moderators? the case of

crowdsourced image moderation. Cornell University.

Deuter, C. E., Kuehl, L. K., Blumenthal, T. D., Schulz, A., Oitzl, M. S., & Schachinger, H.

(2012). Effects of cold pressor stress on the human startle response. PloS one, 7(11).

Gorwa, R., Binns, R., & Katzenbach, C. (2020). Algorithmic content moderation: Technical and

political challenges in the automation of platform governance. Big Data & Society, 7(1),

2053951719897945.

Hartman, M. E., Ladwig, M. A., & Ekkekakis, P. (2021). Contactless differentiation of pleasant

and unpleasant valence: Assessment of the acoustic startle eyeblink response with

infrared reflectance oculography. Behavior Research Methods, 1-13.

 21
CONTENT MODERATOR STARTLE RESPONSE
Kiger, M. E., & Varpio, L. (2020). Thematic analysis of qualitative data: Amee Guide no. 131.

Medical Teacher, 42(8), 846–854. https://doi.org/10.1080/0142159x.2020.1755030

Maguire, M., & Delahunt, B. (2017). Doing a thematic analysis: A practical, step-by-step guide

for learning and teaching scholars. All Ireland Journal of Teaching and Learning in

Higher Education, 9(3), 3351-33514. http://ojs.aishe.org/index.php/aishe-

j/article/view/335

Martin, W. L., Murray, P. S., & Bates, P. R. (2012). The effects of startle on pilots during critical

events: A case study analysis. Griffith University. Aviation Psychology and Applied

Human Factors- working toward sezo accidents Conference 2012 (pp. 387-394). Griffith

University.

Mukhopadhyay, B. (2020, August 26). Warning: The (open) secret lives of content moderators.

Cassandra Voices. https://cassandravoices.com/society-culture/society/warning-the-

open-secret-lives-of-content-moderators/

Phelps, B. J., Doyle-Lunders, L., Harsin-Waite, A., Hofman, N., & Knutson, L. M. (2012).

Demonstrating habituation of a startle response to loud noise. The Behavior Analyst

Today, 13(1), 17-19. http://dx.doi.org/10.1037/h0100714

Reagh, Z. M., & Knight, D. C. (2013). Negative, but not positive emotional images modulate the

startle response independent of conscious awareness. Emotion, 13(4), 782.

Roberts, S. T. (2016). Commercial content moderation: Digital laborers' dirty work. Western

Libraries, 12.

Schachter, S., & Singer, J. (1962). Cognitive, social, and physiological determinants of

emotional state. Psychological Review, 69(5), 379–399. https://doi.org/10.1037/h0046234

 22
CONTENT MODERATOR STARTLE RESPONSE
Silbermann, J (2020, April 8). They’re looking out for the internet. But who’s looking out for

them? Onlabor.

https://onlabor.org/theyre-looking-out-for-the-internet-but-whos-looking-out-for-them/

Steiger, M. (2020). Building a Resilient Workforce: Programming for Commercial Content

Moderation Staff.

Steiger, M., Bharucha, T. J., Venkatagiri, S., Riedl, M. J., & Lease, M. (2021, May). The

Psychological Well-Being of Content Moderators: The Emotional Labor of Commercial

Moderation and Avenues for Improving Support. In Proceedings of the 2021 CHI

Conference on Human Factors in Computing Systems (pp. 1-14).

Vaidyanathan, U., Malone, S. M., Miller, M. B., McGue, M., & Iacono, W. G. (2014).

Heritability and molecular genetic basis of acoustic startle eye blink and affectively

modulated startle response: A genome-wide association study. Psychophysiology, 51(12),

1285-1299.

Valsamis, B., & Schmid, S. (2011). Habituation and prepulse inhibition of acoustic startle in

rodents. Journal of Visualized Experiments, (55), e3446.

 23
CONTENT MODERATOR STARTLE RESPONSE
Table 1

Themes, Definitions, and Exemplars

Themes Definition Exemplar

Transparency Open, honest, and intentional “During my recruitment process,

communication detailing the they already told me that I'll be
true nature, content exposure, doing as a content moderator
and risks associated handling disturbing content, so
pertaining to the role and before I enter the training I am
requirements of a content prepared for the job.”
moderator.

Preparation Developing a thorough “Because of training, I could

understanding and manage my response, emotions,
expectations pertaining to and thoughts of the content I am
workflow, processes, reviewing.”
protocols, content exposure,
and managing exposure
response.

Learning Enthusiasm Curiosity and excitement "Excited to learn new things,"

about expanding worldview "curious about the things we will
and understanding novel roles encounter," and "you're being
and responsibilities of taught the things you have never
moderation work. heard before, so it was really neat
to me."
 24
CONTENT MODERATOR STARTLE RESPONSE

Readiness Moderators’ perception they “Because of training, I could

are capable of manage my response, emotions,
psychologically managing and thoughts of the content I am
their workflows and reviewing.”
assignments.
“Almost all content was discussed
in training and production only had
slight differences so the change was
not hard for me.”

“I have exposure in my personal

life and stuff I seen on social media
so it was an easy transition for me.”

Process Ambiguity The inability to distinguish “Production has content that is

how to action on specific more diverse in the topic than in
material correctly. training. I believe it should contain
more examples.”

Habituation Ability to positively adapt to “I get a little startled with the

a repeated threatening unexpected before, but now it does
stimulus by diminishment in not affect me as much”
physiological or
psychological responses.

Hindrance to Factors that delay or impede “Daily stress and fatigue can tear
Habituation the moderators’ ability to down mental fortitude due to
positively adapt to a repeated random changing protocols and
threatening stimulus shifts schedules it deteriorates your
ability to focus and process graphic
content.”
Sense of Purpose Belief that the work “I take pride in what I do no matter
conducted is essential in how hard it may be at times, I look
protecting the end-user and back on the purpose why I do this
safeguarding the platform, job. And it is fulfilling.”
which results in a positive
outcome.
 25
CONTENT MODERATOR STARTLE RESPONSE

Programmatic The environmental and “Actually, because of all the

Intervention Employed psychological resources wellness and interaction, I’m quite
offered and utilized by all good.”
moderators.