Accepted Manuscript

Motion Analysis System (MAS) for production and ergonomics assessment in

the manufacturing processes

Marco Bortolini, Maurizio Faccio, Mauro Gamberi, Francesco Pilati

PII: S0360-8352(18)30526-6
DOI: https://doi.org/10.1016/j.cie.2018.10.046
Reference: CAIE 5485

To appear in: Computers & Industrial Engineering

Received Date: 1 July 2017

Revised Date: 18 October 2018
Accepted Date: 24 October 2018

Please cite this article as: Bortolini, M., Faccio, M., Gamberi, M., Pilati, F., Motion Analysis System (MAS) for
production and ergonomics assessment in the manufacturing processes, Computers & Industrial Engineering (2018),
doi: https://doi.org/10.1016/j.cie.2018.10.046

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers
we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and
review of the resulting proof before it is published in its final form. Please note that during the production process
errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
 Motion Analysis System (MAS) for production and
ergonomics assessment in the manufacturing processes

Marco Bortolini*, Maurizio Faccio**, Mauro Gamberi* †, Francesco Pilati*

* University of Bologna, Department of Industrial Engineering,

Viale del Risorgimento 2, 40136 Bologna, Italy
** University of Padova, Department of Management and Engineering,
Stradella San Nicola 3, 36100 Vicenza, Italy

† Corresponding Author: [email protected]

 Motion Analysis System (MAS) for production and ergonomics assessment
in the manufacturing processes

Abstract:
Nowadays the Smart Factories operating within the Industry 4.0 revolution, require more and more reliable,
fast and automatic tools for production analysis and improvement. Manufacturing companies, in which the
human labour has a crucial role, need instruments able to manage complex production systems in terms of
resource utilization, product mix, component allocation and material handling optimization.
In this context, this work presents an original hardware/software architecture, Motion Analysis System
(MAS), aimed at the human body digitalization and analysis during the execution of manufacturing/assembly
tasks within the common industrial workstation. MAS is based on the integration of the Motion Capture
(MOCAP) technology with an ad hoc software developed for productive and ergonomic analysis of the
operator during his work. MAS hardware integrates a network of depth cameras initially developed for
gaming (Microsoft Kinect v2™, conceived for markerless MOCAP) and now used for industrial analysis,
while an original software infrastructure is programmed to automatically and quantitatively provide
productive information (human task analysis in terms of time execution and used space within the
workplace, movements of hands and locations visited by the operator) and ergonomic information (full body
analysis implementing all the internationally adopted indexes OWAS, REBA, NIOSH and EAWS). This
double perspective makes MAS a unique and valuable tool for industrial managers oriented to the workplace
analysis and design (in terms of productivity) without neglecting the operator health. This proposed
contribution ends with a real industrial application analysing a water pump assembly station: the system
setup is discussed and the key results obtained adopting MAS are presented and analysed.

Keywords: Motion Capture, Industry 4.0, Manufacturing, Assembly, Ergonomics.

1 Introduction
The industrial environment is currently experiencing its fourth revolution. The use of ubiquitous sensors
connected through communication networks enables the real-time integration of systems, machines, tools,
operators, customers and products defining the so called Smart Factories (Dujin et al., 2014). These features
allow to develop a novel production paradigm, called personalized production. The customers are involved
in the product personalization since the design phase to manufacture and assembly unique products which
fulfil unique needs (Dou et al., 2014). This paradigm dramatically increases the complexity of manufacturing
processes and the variety of assembly operations (Bortolini et al., 2017a). The required production flexibility
is typically ensured by skilled and experienced operators which perform the non-repetitive and added-value
activities (Bortolini et al., 2017b). Thus, the virtual representation of these tasks (e.g. virtual reality) can be a
great help to analyse and improve the manufacturing and assembly processes as well to capitalize the
operator knowledge and expertise (Geiselhart et al., 2016).
In this context, Motion Capture (MOCAP) represents a promising solution both to capitalize the worker skill
and to prevent possible injuries during the execution of manufacturing or assembly tasks. This solution
enables to accurately record the activities of the human body proposing a virtual representation of the
skeleton and its movements. Purpose of all the different MOCAP technologies is to sample many times per
second the postures held by the monitored actor. The recorded data are then mapped into a 3D model of the
human skeleton so that the virtual model performs identical motions compared to the tracked actor.
Considering this current scenario, this paper presents an original Motion Analysis System (MAS) for human
body digitalization and analysis for manufacturing and assembly processes. This research develops a
hardware system adopting commercial MOCAP devices (conceived for gaming) extending their applicability
to the industrial sector and integrating them with an original analysis software programmed for the dynamic
assessment of the work content. MAS acquires the operator activities during his manufacturing or assembly
tasks and it evaluates them from a double perspective: productive and ergonomic viewpoint. The productive
viewpoint deals with the time and the space resulting from the analysis of human tasks, movements of
focused body parts, occupied locations over time and travelled distances (body, hands, feet, etc). The
ergonomic viewpoint is estimated with a full body analysis measuring the human skeleton movements during
the operator working activities investigating the evolution of the joint angles and the bone postures.
According to this purpose, the remainder of this paper is organized as in the following. Section 2 analyses
the different technologies commercially available for MOCAP, the most relevant contributions to MOCAP
usage in the industrial environment and the methods and approaches proposed by the literature to assess the
ergonomics of working conditions. Section 3 presents the hardware and software architecture developed for
the automatic and quantitative evaluation of the technical and ergonomic performances of an operator during
manufacturing or assembly activities. Section 4 describes the MAS application to a case study of a manual
assembly process of a gearbox in an industrial assembly station, whereas Section 5 presents and discusses the
case study key results and main outcomes. Finally, Section 6 concludes the paper and suggests future
research opportunities.

2 Literature review
This Section presents the most relevant contributions proposed by literature which investigate the adoption
of MOCAP technologies in the industrial environment (Section 2.1) and the different methods and
approaches available to assess the ergonomics of operator working conditions (Section 2.2).

2.1 MOCAP technologies in the industrial environment

The adoption of MOCAP technologies in the industrial environment is a relatively recent field of research.
Until few years ago, these solutions were extremely costly, required meticulous and time expensive set-up
procedures and did not offer completely reliable results from the precision perspective (Oyekan et al., 2017).
Du and Duffy (2007) proposed the first contribution to digitalise the human body of an operator during
assembly activities through a MOCAP technology. Three different technologies have been developed so far
to ease the tracking of human movements.
Marker-based optical MOCAP exploits active or passive markers properly displaced in specific part of
human body. A bunch of cameras detects the position of each marker in its own two-dimensional (2D) field
of view, whereas the relative position and orientation of cameras enable to triangulate the location of markers
in the 3D space of action (Tian and Duffy, 2011). The markers can be either active or passive. Active
markers are LEDs which typically emits their own light one at a time at high frequency. On the contrary,
passive markers are small plastic spheres coated with a retroreflective material to reflect the light that is
generated near to the camera lens by an infrared emitter (Ceseracciu et al., 2014).
Inertial MOCAP technology is based on miniaturized inertial sensors which are properly displaced on the
body parts to monitor. Each inertial measurement unit (IMU) is equipped with a gyroscope, a magnetometer
and an accelerometer to record their relative measures on each of the three geometrical axis (Bourke et al.,
2008). However, compared to optical MOCAP, the inertial approach is affected by a lower accuracy of the
absolute location of the limbs due to positional drift which can compound over recording time. Thus, this
technology presents several limitations and drawbacks for its application in manufacturing or assembly shop
floors, and, in general, in the industrial environment.
Marker-less optical MOCAP represents a recent advance in the technology to avoid the awkward suits which
have to be worn by the operator in case of marker-based optical or inertial MOCAP. Indeed, both these
technologies typically require active and passive markers as well as IMUs to be mounted on cumbersome
suits. On the contrary, marker-less optical MOCAP free the operator to perform his activities in his regular
outfit (Geiselhart et al., 2016). Nguyen et al. (2013) first adopt this solution to monitor the postures and
movements of a human operator during manual manufacturing processes with promising results both in term
of measurement accuracy and system set-up easiness.
The adoption of MOCAP technologies in the industrial environment achieved a relevant importance with the
Factory of the Future (FoF) concept (Jardim-Goncalves et al., 2017, May et al., 2016). In these factories the
operators perform complex and non-repetitive tasks to comply to the personalized production paradigm
(Faccio et al., 2015). The operator expertise is one of the most valuable competitive advantages owned by
the FoFs (Fantini et al., 2014). MOCAP technologies are of major help for the tracking of the unique manual
assembly or manufacturing processes to later capitalize the operator knowledge (Alnahhal and Noche, 2015,
Romero and Vernadat, 2016). Furthermore, the recent socio-economic trends suggest or even forces the FoFs
to evaluate and improve the working conditions of their operator and the related ergonomics (Honglun,
2007). Finally, production and ergonomic performances are strongly interrelated. The actions designed and
implemented to improve one performance category have an impact on the other, not necessarily positive (Xu
et al., 2012, Accorsi et al., 2017). Thus, any approach or solution aimed at the simultaneous optimization of
the manufacturing and assembly processes from both the production and ergonomic perspectives is strongly
encouraged (Gragg et al., 2013).
From the production perspective, marker-less optical MOCAP is adopted by Agethen et al. (2016a and
2016b) to analyse the discrepancies between the planned and the real manual production process, which
typically remain unknown without appropriate monitoring tools. The authors exploit this tracking system to
reconstruct the operator motion within an assembly line. Similarly, Geiselhart et al. (2016) adopted a marker-
less optical MOCAP system to measure the production performances in real working conditions compared to
the one forecasted by simulation models and methods. From the ergonomic perspective, the pioneering
contribution of Jayaram et al. (2006) exploits inertial MOCAP to automatically assess the ergonomics of
different postures assumed by the operators during their working activities. From the ergonomic analysis
perspective, two contributions aim at improving the ergonomic evaluation. Vignais et al. (2013) assess the
risk of possible disorders for each body part adopting an inertial MOCAP technology along with a specific
ergonomic index. Kim and Nussbaum (2013) further detail the ergonomic evaluation assessing the evolution
over time of the angle of most of the skeleton relevant joints. Furthermore, some authors recently exploited
the emerging marker-less optical MOCAP technology for the ergonomic assessment in real industrial
environment. Both Geiselhart et al. (2016) and Plantard et al. (2016) integrate multiple depth cameras to
increase the accuracy and the covered area of the monitored human motions with promising results.

2.2 Methods and approaches for the ergonomic assessment of working conditions
During the last decades, several methods and approaches are proposed by literature contributions to assess
the ergonomics of operator working conditions (Li and Buckle, 2017). In the last years, these indices are
adopted by the authors which developed MOCAP systems to quantitatively and univocally assess the
ergonomic performance of the operators during manual assembly or manufacturing activities (Shikdar et al.,
2002). The ergonomic indices are classified in the following with respect to the targeted manual handling
activity.
Lifting and carrying tasks are traditionally assessed through the NIOSH equation (Waters et al., 1993). This
method determines the recommended load weight limit for human lifting operations. Pushing and pulling
activities along with force limit considerations are assessed by Snook and Ciriello (1991). The manual
handling of low loads at high frequency is carefully analysed by Occhipinti with the proposed OCRA index
(1998). Similarly to OCRA, the Strain Index (Moore and Garg, 1995) estimates the risk of distal upper
extremity disorders analysing different features of a performed task. Furthermore, operator posture and
movements are carefully assessed by three indices. Indeed, both OWAS, RULA and REBA analyse the
working posture of an operator evaluating the position of the different body parts and the angle of several
skeleton joints. However, the OWAS index qualitatively estimate the body posture (Karhu et al., 1977).
RULA carefully assess the upper limbs but it poorly estimates the posture of lower limbs, the legs in
particular (McAtamney and Corlett, 1993). REBA index is distinguished by the advantages of RULA one
along with a proper and thorough evaluation of lower limbs posture (Hignett and McAtamney, 2000).
Finally, Schaub et al. (2013) aim to integrate the different features of most of the aforepresented ergonomic
indices in a unique indicator for manual assembly or manufacturing activities. The proposed EAWS index
consists of four sections for the evaluation of, respectively, working postures and movements with low
additional physical efforts (similarly to REBA index), action forces of the whole body or hand-finger system
(Snook and Ciriello, 1991), manual materials handling (NIOSH index) and repetitive loads of the upper
limbs (OCRA index). The following Table 1 summarizes the features of the presented indices for the
ergonomic assessment of manual assembly or manufacturing activities.
Snook & Strain
Feature NIOSH OCRA OWAS RULA REBA EAWS
Ciriello Index
Posture x x x x x x
Upper limbs x x x x x
Lower limbs x x x x
Spine x x x x x
Quantitative x x x x x x x
Load/Force x x x x x x
Frequency x x x x x
Duration x x x x
Recovery x x
Table 1. Features of the ergonomic indices for the manual assembly or manufacturing activities.

Considering the revised literature and as far as these Author knowledge, this paper presents one of the first
research contribution of MOCAP technologies for industrial applications which proposes the simultaneous
assessment of the production and ergonomic performances of a human operator during assembly or
manufacturing activities. Despite the previous contributions, the developed MAS, based on marker-less
optical MOCAP technology, avoids any interference with the worker activity, which is typical of the
MOCAP solutions which adopt cumbersome suits. Furthermore, the selected marker-less cameras ensure a
remarkable measurement accuracy of both the operator posture themselves and his movements in relation to
the machines and components displaced in the shop floor. Finally, the proposed MAS automatically and
quantitatively assesses a bunch of ergonomic indices to estimate the operator musculoskeletal performance
and it measures a set of productive key performance indicators (KPIs). The latest research trends have been
investigated to identify the most relevant productive KPIs to be assessed through the developed MAS.
Agethen et al. (2016b) remark the importance of accurately evaluate the paths travelled by the operators
within an assembly or manufacturing station, whereas Bin Che Anu et al. (2014) suggest to analyse in detail
the production processes to measure the added value portion of the working time. Concerning manual
picking activities, a proper analysis has necessarily to monitor the instant, frequency and duration of retrieval
tasks from each possible storage location of the shop floor (Thomas and Meller, 2015).

3 Motion Analysis System description

MAS is an original hardware/software architecture conceived for the analysis of human manufacturing and
assembly systems. It is developed for adapting itself to the typical workplace configurations and its aim is to
analyse the human work providing the production management with a very detailed report from both the
productive (time and space) and ergonomic point of view (see Figure 1). The aim is achieved by a human
markerless MOCAP hardware system developed for the digitalization of the operator body during his work
and an ad hoc software programmed to perform dynamic analysis.

INPUTS
 Product infos (bill of materials, dimension)  Workplace layout
 Component/tool infos (weight, dimensions)  Control volume infos (dimensions, location)
 Operator physical features  Component /Tool– Control volume matching

HARDWARE
Markerless mocap

SOFTWARE HUMAN BODY DIGITALIZATION

Productive and Ergonomic analysis .TRC output file

OUTPUTS
Productive perspective: Ergonomic perspective:

 Operator spaghetti chart on layout Trend over time and analysis of:
 Hand movement analysis  Operator articulation angles
 Human body time and space analysis  Ergonomic indices
 Working time classification
 Picking activity analysis

Figure 1. MAS conceptual framework.

3.1 Hardware architecture

The hardware structure of MAS is constituted by a Wi-Fi network with up to four depth cameras connected
each one via USB port to dedicated PCs. The adopted PCs have to be equipped with high-performance
graphic cards which allow to process the huge data flow of images acquired from each camera during
MOCAP operations. Between the PCs, one acts as master while the others are the slaves. This system
configuration, depicted in Figure 2, allows to synchronize the four image flows thanks to the Wi-Fi
communication between the master and slave PCs.
 MASTER

Industral SLAVE
Workplace

SLAVE SLAVE
SYMBOLS
Camera Control PC ND optical Filter
IR Depth Camera Router
Figure 2. MAS hardware architecture

The used depth cameras are the Microsoft Kinect v.2™ (https://developer.microsoft.com/en-
us/windows/kinect/hardware) derived from the gaming sector but thanks to the Microsoft™ Software
Development Kit (SDK) they are now configured and used for human tracking in industrial applications. The
cameras have two parallel sensors for a best depth evaluation: a color RGB sensor and a IR depth sensor.
Their features and performance are summarized in the following data and in Figure 3:
 RGB sensor, resolution 1920x1080 @ 30fps;
 IR sensor, resolution 512x424 @ 30fps
 FOV (field of view), horizontal 70°, vertical 60°;
 Min/Max depth distance, ~1.4m / ~5m;
 Skeleton detectable joints, 26;
 Contemporarily acquirable operators: 2.

~ 5m
~ 5m
full length visibility
full length visibility

~ 1.4m

~ 1.4m

~ 70°
~ 1.8m

~ 60°

Top VIEW
TOP view Side
SIDEviewVIEW

Figure 3. Depth camera field of view.

The depicted spatial field of view of the cameras is the result of an experimental campaign aimed at the
investigation of the operating limits of the adopted hardware. A real industrial workplace is settled within the
Laboratory of the Department DIN of the University of Bologna for this purpose.
 ~ 5m ~3
.5m

~ 1.4m

~ 3m
~10cm

case a) case b) case c)

config. a) config. b) config. c)

Figure 4. Ideal camera configurations for MAS with two (case a), three (case b) and four (case c) cameras.

As result of the on field analysis, the position of the cameras must be carefully chosen to maximise the
acquisition precision and the industrial area covered. According to the camera performance three ideal
configurations can be used and the Figure 4 depicts them. These configurations are defined ideal because in
these conditions the best measurement precision for a skeleton acquisition can be achieved: the positional
error between the digital and the real skeleton position is measured in about ~3-4cm.
Despite of these ideal configurations, the system can excellently work adapting the position and the number
of the cameras to the specific case study, according to the real layout and constraints of the
manufacturing/assembly workplace. Several real configurations are experimented with two, three and four
cameras and large obstacles (shelves, tables, tool trolleys etc.) within the camera field of view. The next
Figure 5 shows the trial workplaces distinguished by low and high obstruction levels (for sake of brevity, all
the intermediate configurations investigated but not presented, are omitted).

Camera position Camera position

L 5-8m
Camera position

Camera position

Camera position

Camera position

Table
Shelves
L 5-8m

Table
Trolley

Trolley

Camera position Camera position

a) Low obstructed workplace b) High obstructed workplace

 Figure 5. Trial workstation for experimental campaign.

Seven people, both female and male, with different physical morphologies participated to the experimental
campaign. The people heights vary from 157 to 193 cm accordingly to the following values: 157, 168, 175,
178, 182, 186 and 193 cm. Each participant performs 9 times a specific set of predetermined identical
activities for a duration of 8.4 min. To evaluate the 3D spatial precision of MAS, each task requires the
operator to touch with his hand or foot several predetermined markers on the workplace (e.g. tools, markers
on tables, on shelves or on the floor). The experimental campaign aims to evaluate for each monitored
operation the error between the real and the tracked 3D position of the considered joint. The average tracking
error and its standard deviation are calculated accordingly and they result in a remarkable tracking precision
distinguished by an average accuracy of 5.2 cm with a standard deviation of 0.8 cm.
Another important detail necessary for the application of such depth cameras to a real industrial
environment, is the light level management. Indeed, depth cameras are very sensitive to high light level and
they cannot properly operate if the environment is highly illuminated. To overcome this limit, the authors
provided each camera with a tailored Neutral Density (ND) filter mounted on the RGB and IR sensors. This
ND filter is necessary to enlarge the light contrast range of the Kinect v.2™ device ad it must be chosen from
ND=0.3 to ND=3.8 in relation to the intensity of the bright surfaces (windows, lights, reflective metal
surfaces, etc.) within the area to analyse. Without the ND filter, the system performance dramatically
decreases.

3.2 Software architecture

The software providing the dynamic analysis of the working operator is developed in Matlab™ environment
and it is conceived to elaborate the human body digitalization coming from the depth camera network
previously discussed. Up to two operators can be contemporarily monitored within the covered area
presented in Figure 4. The digitalization of their bodies consists in the process of recording the movement of
their skeletons. The resulting output file stores the positions of all body joints over time. The chosen standard
for the proposed architecture is the .TRC file format (introduced by Motion Analysis inc.
https://research.cs.wisc.edu/graphics/Courses/cs-838-1999/Jeff/TRC.html) which supplies the position of all
the joints of the analysed human body. Such joints are named with the MOCAP standardized terminology
widely adopted in the cinematography industry and by MOCAP commercial software (see Figure 6).
 Figure 6. Skeleton joints of the acquired human body

The set of joints together with their position vector are the necessary information to analyse the
manufacturing/assembly process together with additional information regarding the area in which the
operator works and the product he manufactures or assembles.
The required input information is of different type:
 Physical features of the operator, height in particular: necessary for the MAS to acquire and evaluate
the skeleton dimensions (length of each limb);
 3D workplace layout including position and geometrical dimensions of machines, racks, shelves,
workbenches, etc.: the 3D environment in which the operator is immersed and works has to be
detailed in terms of object geometrical dimensions and zones;
 Information of the product to be assembled or manufactured: the product components, dimension
and weight (in synthesis the product BOM, e.g. bill of materials);
 Information of the tools necessary for the manual operations: position of tools, their dimension and
weight;
 Relation between components and tools used for the final product manufacturing.

The aforementioned information can be easily obtained by the analyst extracting them from the company
Manufacturing Execution System and provided to the MAS as .CSV data in a massive way.
For each MOCAP activity all the previous data must be collected at least once. However, it is not necessary
to provide them for every replication of the same scene because the information is invariant from the
replication.
The developed software has a double perspective providing information on both the productive and
ergonomic and point of view. Indeed, MAS is focused on the whole manufacturing analysis integrating the
working condition improvement (ergonomic viewpoint) with the production enhancement (productive
viewpoint).
These viewpoints are not related to the production mix. The MAS is not affected or influenced by the
product type assembled by the operator. Indeed, the proposed architecture is able to track the operator
motion whatever the assembled or manufactured products are. The performed experimental campaign
validates the adoption of MAS for production systems distinguished by small to medium size products of
whatsoever shape, volume and weight. As previously stated in the Section 3.1, the MAS is able to
contemporarily track 2 operators which perform their activities in the same shared workstation even in case
of interaction or collaboration.
Aim of the MAS is to evaluate the performance of manual operations during an assessment trial quantifying
the productivity and the ergonomics of a workstation. The productive viewpoint is assessed through a
dynamic analysis of the operator movements in relation to the workplace layout in which the tasks are
executed (manufacturing activities, task execution time, component locations, workspace usage, racks or
workbenches utilization, hands position, etc.). The ergonomic viewpoint consists in the evaluation of several
ergonomic indexes internationally defined and approved (e.g. OWAS, REBA, NIOSH and EAWS).
The aforementioned results are obtained evaluating the .TRC file containing the position vector of all the
operator joints over time. The structure of this file format is presented in Table 2. The position vectors (X, Y,
Z) of each joint is listed and stored frame by frame providing a dynamic representation of all the movements
executed by the operator.

Frame# Time Reference Joint1: Hips Joint2: RightUpLeg Joint3:….

X1 Y1 Z1 X2 Y2 Z2 X3 Y3 Z3 …
1 0 0 0 0 0 1000.71 28.2 -133.42 985.79 -226.8 …
2 0.033 0 0 0 23.01 1001.14 -16.14 -167.27 985.20 -177.4 …
3 0.067 0 0 0 22.81 1000.95 -16.00 -173.33 984.54 -179.4 …
4 0.1 0 0 0 22.44 1001.10 -15.87 -174.61 983.99 -184.76 …
5 0.133 0 0 0 22.13 1005.53 -15.91 -180.52 988.521 -194.92 …
6 0.167 … … … … … … … … … …

Table 2. TRC file structure example.

All this information enables to determine the angle of every human body articulation for each monitored
frame, thus all displacements and postures of the operator are considered and quantitatively measured.
 Knee angle Trunk rotation Shoulder lateral elevation
Figure 7. Examples of angles calculated by MAS.

All the angles are automatically determined in the 3D space (see Figure 7) and, along with the joint position
vectors, they enable to evaluate the productive and ergonomic KPIs illustrated in detail in the next
paragraphs.
3.2.1 Productive viewpoint
The information about the operator working performance provided by the MAS is:
 Time and space analysis of the workplace areas (task execution times, travelled paths by the
operator, time spent in adjacency of the workstation objects (machines, racks, shelves, workbenches,
etc.));
 Hand displacement over time and velocity trend;
 Cumulative vertical movements for lifting and lowering;
 Control volume analysis1 for the distinction between added-value and no added-value activities;

3.2.2 Ergonomic viewpoint

The information about the operator ergonomic performance provided by the MAS deals with the evaluation
over time of several indexes measuring the body postures and movements:
 Articulation angle analysis applying the ISO 11226 standard to classify as acceptable or not the
worker postures;
 OWAS, Ovako Working posture Assessment (Karhu et al., 1977), for entire body analysis;

1
In the MAS environment, the analyst can define 3D control volumes within the workplace (defining their dimensions
and 3D position) to achieve an in-depth statistic about the locations most visited by the worker hands over time.
Creating and placing control volumes on the workbenches, in the picking positions, within specific shelfs or racks or in
whatsoever location, allows to distinguish between the operator picking or travelling time (no added-value activity) and
the manufacturing time (added-value activity) in relation to the number of visits to control volumes and their average
duration for both the operator hands.
  REBA, Rapid Entire Body Assessment (Hignett and McAtamney, 2000), for entire body analysis;
 NIOSH, National Institute for Occupational Safety and Health index (Waters et al., 1993), for
weight lifting activities;
 EAWS European Assembly Worksheet (Schaub et al., 2013), for entire body analysis and specific of
the automotive sector.

4 Real case application setup

To apply and to validate the developed MAS architecture, the case study of a real assembly station designed
to assembly industrial water pumps is presented. This real application follows the tests and validations
performed in the Mechanical Laboratory of Department DIN of the University of Bologna previously
mentioned in Section 3.1. This Section presents a real application to highlight the feasibility of the proposed
architecture showing the main results and findings.
The final aim of MAS it to automatically and quantitatively assess the operator productive and ergonomic
performance. In particular, the most relevant KPIs concern the operator walking path within the station
layout, his hand distribution on workbench, the added value portion of the cycle time, the body posture
assessment and the REBA ergonomic index evaluation.
The analysed product is assembled over a moving workbench and an operator has to manage several
components placed in racks and pallets around him. The pump assembly must be executed within the cycle
time of 5.5 min/pcs (the cycle time coincides with the MOCAP time horizon). The product (see Figure 8) and
its components are well known as well as the detailed bill of materials, the component dimensions and
weights.
For the considered case study, one male operator 178cm high is involved in the experiments capturing his
motions for the duration of one cycle time (5.5 min) without any interruptions while he is performing the
required 18 assembly tasks on the only product considered, e.g. the commercial water pump pictured in
Figure 8.

Figure 8. Water pump assembly (left) and main components (right).

The analysed workplace, depicted in Figure 9, is organized with an assembly station connected with the
other production phases through conveyor rollers. The area is provided with front and rear racks to
respectively store the small- and medium-sized product components, an europallet for large components and
a mobile trolley containing the tools and bits required by the operator. All their dimensions and 3D positions
are known and acquired. The gross area is approximatively 20sqm and a quasi-rectangle displacement of the
depth camera is adopted to capture the movements of the operator which can dress any type of quite slim
clothes (to reduce the noise in the joint position acquisition) without restriction of colours.
From the privacy perspective, the utilisation of MAS architecture within an industrial environment is
compliant with the recent EU General Data Protection Regulation (GDPR) (2016). In particular, three major
GDPR principles have been fully considered and implemented to ensure an adequate protection of the
personal data of the involved operators. First, the “privacy by design principle” is implemented during the
MAS development to ensure the personal data protection at any stage of the assessment activity. Second, the
“right to be forgotten” principle is ensured through the possibility for any worker to request at any time and
immediately obtain the removal and deletion of all the data dealing with his person. Third, the “principle of
transparency” of the data usage is guaranteed through a formal agreement between the involved operator and
the employer. The aforementioned agreement should ensure the operator privacy and personal data
protection through regulating the relationship between the parties in terms of:
 Purpose of MAS adoption. The employer has to inform the employee for which purpose the collected
data are used, to evaluate which aspect of the performed tasks.
 Privacy protection. The collected data are acquired completely anonymously without any possibility
to match the information obtained with a specific employee.
 Data protection. The collected data are stored only for the time and to the extent necessary to the
workstation analysis. The aforementioned data can be processed and analysed only by the authorized
analysts which have be clearly designated in this agreement indicating their name, role and division
within the company.
 Duration of tracking period. The employer has to explicitly inform the employee every single time
in which the MAS is used for tracking purpose. In any case, the frequency and duration of the
recording phase has necessarily to represent a limited portion of the worker shift duration with no
intent or purpose of continuous monitoring.
 Recording technology specifications. The employer has to indicate the model of the depth camera
adopted for MOCAP purpose along with the camera number and location within the workstation
layout. Further specifications to be provided are the camera field of view, the tracking distance and
the areas of the layout monitored by the cameras.
Furthermore, a specific demonstrative session has to be organised by the employer which adopts the MAS
architecture to actively involve the operator in the assessment process. During this session, the architecture
has to be tested by the workers providing them all the specifications and details about its functioning
 principles. The employer has to clearly list all the data which the MAS is able to obtain, how they are
processed and for which purpose they are used. The session ends with the collection of the worker feedbacks
and the possibility for them to make whatsoever question and obtain detailed answers.
Finally, MAS is conceived to be adopted to analyse an industrial workstation to assess if there are
improvable configurations of work both from the productivity and from the ergonomics point of view. From
the privacy perspective, the MAS architecture does not differ from the traditional processes adopted in the
industrial workshop to assess the operator productive and ergonomic performances (e.g. Time and Methods
or Ergo-analyses) which consists in recording a video of worker usual tasks for a limited time period (e.g. a
cycle time). Nowadays this is a common practice considering the increasing penetration of ICT devices in
the industrial production management.

1m 1m

Rear racks
Europallet

Trolley

Assembly
station
Material
flow

Front racks

Figure 9. Assembly station: scheme (left) and front-view picture (right) masked for confidentiality reasons.

The four cameras are connected to one Intel i7-processor PC (master) and three Intel i3-processor PC
(slaves) with adequate GPU performance necessary to guarantee an acquisition frame rate equal or higher
than 30fps. The used operating system is Windows 10™. The average storing capacity required is 30Mbyte
per second of acquisition and camera. Thus, the four camera motion analysis and the cycle time of 5.5
min/pcs require a total hard disk space of 39.6 Gbyte.

5 Results and discussion

In this context, MAS is used to analyse the work execution and the consequent output results both from the
productivity and from the ergonomics perspectives. Considering the several and different KPIs which the
MAS is able to evaluate, the following table of contents presents a rationale and offers an overview of this
Section 5 articulation:
 Productive perspective:
 analysis of the operator and upper limbs movements,
  operator walking path within the station layout,
 hands distribution on workbench,
 cycle time partition between the different working activities;
 Ergonomic perspective:
 articulation angle analysis and posture assessment through ISO 11226,
 dynamic evolution of REBA ergonomic index,
 average REBA score for each body part.

From the productive perspective, MAS automatically and quantitatively evaluates several KPIs. For sake of
brevity, the most significant results are presented in this Section.
The next Table 3 proposes the movement analysis of the operator body and both his hands. During the
monitored cycle time, the operator walks for a total distance of 34.7 m at an average speed of 6.32 m/min.
He experiences the alarming value of 6.0 m as the cumulative vertical movements over the monitored period
to lift components from storage locations at ground level. Concerning the upper limbs, the operator performs
the assembly tasks with a balanced proportion between the right and left hand usage, e.g. right vs left hand
travelled distances 83.4 m vs 81.2 m.
Traveled distance Vertical drop Average speed
Body part
[m] [m] [m/min]
Right Hand 83.4 34.9 13.6
Left Hand 81.2 33.7 13.3
Operator 34.7 6.0 6.32
Table 3. Analysis of the operator and upper limbs movements.

The movements of the operator and his hands within the assembly station are further in-depth analysed.
Figure 10 presents the spaghetti chart of the operator walking path within the station layout detailing the
locations visited by the operator during the 34.7 meters listed in Table 3. The irregular pattern is determined
by the different picking activities which the operator has to perform between the industrial equipment of the
workstation (front and rear racks, europallet and trolley).
The hands distribution on the workbench is presented by Figure 11 (left and right hands separately). The left
hand activity is quite confined to the left side of the workbench whereas the right hand performs assembly
operations in the central/right portion. Furthermore, the left hand movements are concentrated in a narrow
area of the workbench (e.g. 34.2% of the cycle time spent in a 100 cm2 area, with a peak of 14.9% spent in
25 cm2). On the contrary, the right hand movements span across the entire workbench. Compared to the left
hand, the right one spends a similar amount of the cycle time (37.9%) in an area of double size (200 cm2).
 Figure 10. Operator walking path within the station layout.

Figure 11. Hands distribution on workbench.

Concerning the different activities performed by the operator during the cycle time, MAS evaluates for each
control volume the number of visits and their duration for both the operator hands. This analysis is adopted
to assess and to distinguish between the time spent by the operator to execute added-value activities
(assembly tasks) and picking/travelling activities. As presented in Figure 12, most of the cycle time (61%) is
spent for value added activities (e.g. assembly), whereas a non-negligible portion of it (18%) is wasted to
walk inside the station area. Regarding the picking activity (21% of the cycle time), front and rear racks are
similarly exploited for component storage purpose (10% and 8% of cycle time, respectively), thus a different
disposition of components on storage locations should be addressed in future improvements of the station
layout fostering the front locations instead of the rear or lateral ones.
 Walking
18%
front racks
10%

Picking
21% rear racks
Assembly
61% 8%
pallet
3%

Figure 12. Cycle time partition between the different working activities.

From the ergonomic perspective, MAS proposes a set of quantitative indicators to monitor the operator
health during the assembly process discussed in the case study. Several full body ergonomic indexes can be
evaluated. For sake of brevity, in this Section the REBA index is the only one presented (see Figure 13)
during the cycle time of 5.5min/pcs (330 seconds/pcs). The global target of the REBA index suggests that, in
this case study, the operator spends most of the cycle time holding postures with null or low risk for his
health. However, this index highlights some criticalities at ~110sec and ~290sec in which the operator health
is seriously threatened by to medium/high risky postures.

Figure 13. Dynamic evolution of REBA ergonomic index over the cycle time and classification of the risk level for the
operator health.

Thus, a further extensive evaluation is required. Figure 14 presents an in-depth analysis of the articulation
angle according to the ISO 11226. In particular, the trunk frontal bending together with its rotation are here
measured and assessed. The critical tasks overcoming the admissible angles can be highlighted and the
related postures are extracted from the 3D skeleton capture.
 Angle [degree]

ISO11226 limit

0 60 120 180 240 300 seconds

Angle [degree]

ISO11226 limit

0 60 120 180 240 300 seconds

~ 110sec ~ 290sec

Assembly activity Picking activity

Figure 14. Example of MAS posture assessment applying the articulation angle analysis of ISO 11226.

This analysis allows the production managers to understand what/where/when the manufacturing process
should be modified and corrected to achieve a great advantage for the operator health.
Finally, Figure 15 proposes the average score of the REBA index split for each limb: left and right upper and
lower limbs are independently assessed to analyse the differences between the two body sides. No
differences are experienced between the left and right limbs from the ergonomic perspective, expect for the
upper arms: the movements performed by the right one are riskier than the activities carried out by the left
one. In conclusion, these results suggest that both the neck and the legs do not represent a risk for the
operator health, whereas the other body parts are affected by a low risk level (maximum score of 2.5 for left
lower arm).

2.5

RE 2
BA Risk level

ind Null
1.5
ex Low
Medium
1
 Figure 15. Average score of the REBA ergonomic index for each body part.

6 Conclusions
This paper proposes an innovative hardware/software architecture, called by the Authors Motion Analysis
System (MAS), developed for an in-depth evaluation of the human labour content within the
manufacturing/assembly workstations.
The MAS exploits commercial Motion Capture (MOCAP) devices (conceived for gaming) extending their
applicability to the industrial sector and integrating them with an original analysis software programmed for
the dynamic assessment of the human labour within an industrial workplace. MAS acquires the 3D
representation of the operator during his manufacturing or assembly tasks by means of depth cameras
(Microsoft Kinect v2™) using a markerless technology for the human body digitalization. The digital 3D
skeleton of the operator is acquired at 30fps storing all the dynamic information of the human movements.
This information is exploited by the software section of MAS in which the manufacturing process is assessed
from a double perspective, namely productive and ergonomic viewpoints. The productive viewpoint deals
with the task execution time and the workspace utilization (travelled distances of the operator and his hands,
spaghetti chart, etc.), moreover MAS can distinguish between time and space spent for added-value or non
added-value activities thanks to the proposed control volume analysis. The ergonomic viewpoint concerns a
full body assessment measuring the human skeleton movements during the activity execution and
implementing the evaluation of several international ergonomic indexes (OWAS, REBA, NIOSH, EAWS).
The applicability and usefulness of MAS is discussed in the case study application of a real assembly
workstation. A system configuration with four depth cameras is adopted and a single operator is analysed
providing both productive and the ergonomic information about the assembly process. The case study results
suggest how MAS is a valuable hardware/software architecture to assess a manual manufacturing/assembly
workstation highlighting the productive and ergonomic aspects of possible improvements (workstation
layout, location of tools or components, musculoskeletal workload etc.).
Concerning further research activities, this paper represents the starting point of a wider project aimed at the
manufacturing workplace optimization. The meaningful information provided by MAS concerning different
aspects of manual activities has to be integrated with a manufacturing optimization tool able to rearrange the
location of equipment and components within the workstation to improve both the productive and the
ergonomic performances of the operator. In this context, the MAS is going to be exploited to automatically
and quantitatively assess the operator tasks within the industrial workplace before and after the implemented
optimization to measure the achieved improvements.

7 References
Accorsi, R., Bortolini, M., Gamberi, M., Manzini, R., & Pilati, F. (2017). Multi-objective warehouse
building design to optimize the cycle time, total cost, and carbon footprint. The International Journal of
Advanced Manufacturing Technology, 92 (1-4), 839-854.
Agethen, P., Otto, M., Gaisbauer, F., & Rukzio, E. (2016a). Presenting a Novel Motion Capture-based
Approach for Walk Path Segmentation and Drift Analysis in Manual Assembly. Procedia CIRP, 52, 286-
291.
Agethen, P., Otto, M., Mengel, S., & Rukzio, E. (2016b). Using Marker-less Motion Capture Systems for
Walk Path Analysis in Paced Assembly Flow Lines. Procedia CIRP, 54, 152-157.
Alnahhal, M., & Noche, B. (2015). Dynamic material flow control in mixed model assembly lines.
Computers & Industrial Engineering, 85, 110-119.
Bin Che Ani, M. N., Hamid, A., & Binti, S. A. (2014). Analysis and Reduction of the Waste in the Work
Process Using Time Study Analysis: A Case Study. In Applied Mechanics and Materials (Vol. 660, pp. 971-
975). Trans Tech Publications.
Bortolini, M., Ferrari, E., Gamberi, M., Pilati, F., & Faccio, M. (2017a). Assembly system design in the
Industry 4.0 era: a general framework. IFAC-PapersOnLine, 50(1), 5700-5705.
Bortolini, M., Faccio, M., Gamberi, M., & Pilati, F. (2017b). Multi-objective assembly line balancing
considering component picking and ergonomic risk. Computers & Industrial Engineering, 112, 348-367.
Bourke, A. K., Donovan, K. J. O., & Olaighin, G. (2008). The identification of vertical velocity profiles
using an inertial sensor to investigate pre-impact detection of falls, Medical Engineering & Physics 30 (7),
937–946.
Ceseracciu, E., Sawacha, Z., & Cobelli, C. (2014). Comparison of Markerless and Marker-Based Motion
Capture Technologies through Simultaneous Data Collection during Gait: Proof of Concept, PloS one, 9(3),
1–7.
Dou, R., Zhang, Y., & Nan, G. (2016). Customer-oriented product collaborative customization based on
design iteration for tablet personal computer configuration. Computers & Industrial Engineering, 99, 474-
486.
Du J. C. & Duffy, V. G. (2007). A methodology for assessing industrial workstations using optical motion
capture integrated with digital human models. Occupational Ergonomics, 7(1), 11-25.
Dujin, A., Geissler, C., & Horstkötter, D. (2014). Industry 4.0 The new industrial revolution How Europe
will succeed. Roland Berger Strategy Consultants, Munich.
European Commission (2016). General Data Protection Regulation, regulation 2016/679.
Faccio, M., Gamberi, M., Pilati, F., & Bortolini, M. (2015). Packaging strategy definition for sales kits
within an assembly system. International Journal of Production Research, 53(11), 3288-3305.
Fantini, P., Palasciano, C., Taisch, M., Berlin, C., Adams, C., & Stahre, J. (2014). Socially sustainable
manufacturing: exploring the European landscape. In IFIP International Conference on Advances in
Production Management Systems, 474-481. Springer Berlin Heidelberg.
Geiselhart, F., Otto, M., & Rukzio, E. (2016). On the use of Multi-Depth-Camera based Motion Tracking
Systems in Production Planning Environments. Procedia CIRP, 41, 759-764.
Gragg, J., Cloutier, A., & Yang, J. (2013). Optimization-based posture reconstruction for digital human
models. Computers & Industrial Engineering, 66(1), 125-132.
Hignett, S., & Mcatamney, L. (2000). Rapid Entire Body Assessment (REBA), Applied ergonomics, 31(2),
201–205.
Honglun, H., Shouqian, S., & Yunhe, P. (2007). Research on virtual human in ergonomic simulation.
Computers & Industrial Engineering, 53(2), 350-356.
ISO 11226 (2000) Ergonomics — Evaluation of static working postures
Jardim-Goncalves, R., Romero, D., & Grilo, A. (2017). Factories of the future: challenges and leading
innovations in intelligent manufacturing. International Journal of Computer Integrated Manufacturing. 30
(1), 4-14.
Jayaram, U., Jayaram, S., Shaikh, I., Kim, Y., & Palmer, C. (2006). Introducing quantitative analysis
methods into virtual environments for real-time and continuous ergonomic evaluations, Computers in
industry, 57(3), 283–296.
Karhu, O., Kansi, P., & Kuorinka, I. (1977). Correcting working postures in industry: A practical method for
analysis. Applied ergonomics, 8(4), 199–201.
Kim, S., & Nussbaum, M. A. (2013). Performance evaluation of a wearable inertial motion capture system
for capturing physical exposures during manual material handling tasks. Ergonomics, 56(2), 314–326.
Li, G., & Buckle, P. (2017). Current techniques for assessing physical exposure to work-related
musculoskeletal risks , with emphasis on posture-based methods, . Ergonomics, 42(5), 674-695.
May, G., Stahl, B., & Taisch, M. (2016). Energy management in manufacturing: Toward eco-factories of the
future–A focus group study. Applied Energy, 164, 628-638.
McAtamney, L., & Corlett, E. N. (1993). RULA: a survey method for the investigation of work-related upper
limb disorders. Applied ergonomics, 24(2), 91-99
Moore, J., & Garg, A. (1995). The strain index: a proposed method to analyze jobs for risk of distal upper
extremity disorders. American Industrial Hygiene Association, 56(5), 443-458.
Nguyen, T. D., Kleinsorge, M., Postawa, A., Wolf, K., Scheumann, R., Krüger, J., & Seliger, G. (2013).
Human centric automation: using marker-less motion capturing for ergonomics analysis and work assistance
in manufacturing processes. In Proceedings of the 11th Global Conference on Sustainable Manufacturing
(GCSM)—Innovative Solutions, Berlin, 586-592).
Occhipinti, E. (1998). OCRA: a concise index for the assessment of exposure to repetitive movements of the
upper limbs, Ergonomics, 41(9), 1290-1311
Oyekan, J., Prabhu, V., Tiwari, A., Baskaran, V., Burgess, M., & Mcnally, R. (2017). Remote real-time
collaboration through synchronous exchange of digitised human–workpiece interactions. Future Generation
Computer Systems, 67, 83-93.
Plantard, P., Shum, H. P. H., Pierres, A. Le, & Multon, F. (2016). Validation of an ergonomic assessment
method using Kinect data in real workplace conditions. Applied Ergonomics, IN PRESS.
Romero, D., & Vernadat, F. B. (2016). Future perspectives on next generation enterprise information
systems. Computers in Industry, 79, 1-2.
Schaub, K., Caragnano, G., Britzke, B., & Bruder, R. (2013). The European assembly worksheet. Theoretical
Issues in Ergonomics Science, 14(6), 616-639.
Shikdar, A., Al-Araimi, S., & Omurtag, B. (2002). Development of a software package for ergonomic
assessment of manufacturing industry. Computers & industrial engineering, 43(3), 485-493.
Snook, S. H., & Ciriello, V. M. (1991). The design of manual handling tasks: revised tables of maximum
acceptable weights and forces. Ergonomics, 34(9), 1197-1213.
Thomas, L. M., & Meller, R. D. (2015). Developing design guidelines for a case-picking warehouse.
International Journal of Production Economics, 170, 741-762.
Tian, R., & Duffy, V. G. (2011). Computers & Industrial Engineering Computerized task risk assessment
using digital human modeling based Job Risk Classification Model. Computers & Industrial Engineering,
61(4), 1044–1052
Vignais, N., Miezal, M., Bleser, G., Mura, K., Gorecky, D., & Marin, F. (2013). Innovative system for real-
time ergonomic feedback in industrial manufacturing. Applied Ergonomics, 44(4), 566–574.
Waters, T. R., Putz-Anderson, V., Garg, A., & Fine, L. J. (1993). Revised NIOSH equation for the design
and evaluation of manual lifting tasks. Ergonomics, 36(7), 749-776.
Xu, Z., Ko, J., Cochran, D. J., & Jung, M. C. (2012). Design of assembly lines with the concurrent
consideration of productivity and upper extremity musculoskeletal disorders using linear models. Computers
& Industrial Engineering, 62(2), 431-441.
HIGHLIGHTS

 Body digitalization and analysis during the execution of manufacturing/assembly

 Microsoft Kinect v2™ markerless motion capture

 Focus on both ergonomic improvement and production enhancement

 Control volume analysis for operator performance evaluation