Available online

Available atonline
www.sciencedirect.com
online
Available at www.sciencedirect.com
at www.sciencedirect.com
Available online at www.sciencedirect.com

ScienceDirect
ScienceDirect
Procedia CIRP 00 (2017)
Procedia CIRP 000–000
105 000–000
(2022) 1–6
Procedia CIRP 00 (2022) www.elsevier.com/locate/procedia
Procedia CIRP 00 (2022) 000–000 www.elsevier.com/locate/procedia
www.elsevier.com/locate/procedia

29th CIRP Life Cycle Engineering Conference

29th CIRP Life Cycle Engineering Conference
Vision-Based Screw Head Detection for Automated Disassembly for
Vision-Based 28th
Screw
CIRPHead
Design Detection
Conference, May for2018,
Automated
Nantes, FranceDisassembly for
Remanufacturing
Remanufacturing
A new methodology
Simon to analyze
Mangold
Simon Mangold
∗
∗
, Christianthe
, Christian functional
Steiner,
Steiner,
Marco Friedmann,
Marco Friedmann, andJürgen
physical
Jürgen
Fleischer architecture of
Fleischer
existing products for an assembly oriented product
Karlsruhe Institute of Technology, 76131 Karlsruhe, Germany
Karlsruhe Institute of Technology, 76131 Karlsruhe, Germany family identification
* Corresponding author. Tel.: +49-1523-950-2634. E-mail address: [email protected]
Paul Stief *, Jean-Yves Dantan, Alain Etienne, Ali Siadat
* Corresponding author. Tel.: +49-1523-950-2634. E-mail address: [email protected]

École Nationale Supérieure d’Arts et Métiers, Arts et Métiers ParisTech, LCFC EA 4495, 4 Rue Augustin Fresnel, Metz 57078, France
Abstract
Abstract
*Remanufacturing
Corresponding author. is commonly
Tel.: +33 3 87perceived
37 54 30;asE-mail
a promising
address:field for future challenges such as resource efficient production. For an economic oper-
[email protected]
Remanufacturing is commonly
ation of remanufacturing perceived
facilities, as a promising
an automation field for future
of the currently manual challenges such as resource
labor is mandatory. Thus, theefficient production.
automation playsFor an economic
a vital oper-
role in order to
ation of remanufacturing facilities, an automation of the currently manual labor is mandatory. Thus, the automation
realize high rates of re-utilization and therefore a significant reduction of waste. Screw connections allow for non-destructive dismantling and are plays a vital role in order to
realize
commonly highused
ratesconnection
of re-utilization
elements.and Especially
therefore athe significant
automation reduction of waste. Screw
of the disassembly stepconnections allowasfor
is a key element non-destructive
products dismantling
from the field and are
are of unknown
commonly
Abstract
specification used
uponconnection
feeding toelements. Especially the
the remanufacturing automation
line of the disassembly
due to alterations during theirstep lifeiscycles.
a key element
State of astheproducts from for
art solutions theautomated
field are ofdisassem-
unknown
specification upon feeding
bly lack flexibility to adaptto tothe remanufacturing
different products and lineproduct
due to conditions.
alterations during their life cycles.
This contribution State
presents of the art
a highly solutions
flexible for automated
approach disassem-
for the localization
Inbly
and lack
today’s flexibility
businessofto
classification adapt to
environment,
screws in different
the trend
electric products
motors.towards
Theand product
more
presented conditions.
product
systemvariety This
anda contribution
utilizes equippedpresents
customization
tool a highly
is unbroken.
industrial robotDueflexible
with approach
toanthis for
development,
integrated the the
localization
eye-in-hand need of
vision
and
agile classification
system
andand of screws
an industrial
reconfigurable in electric
computer.
production motors.
The system
systems The is presented
emerged abletotocope system
locate
with utilizes
andvarious
classifyaproducts
tool equipped
six different industrial
types
and product robotheads
offamilies.
screw with an
of integrated
To design varying eye-in-hand
sizes
and optimize using vision
machine
production
system
learningas
systems and an industrial
approaches
well as to in computer.
order
choose to adapt
the The
optimalthesystem
robot’sisend-effector.
product able to locate
matches, andanalysis
Because
product classify six
of themethods different
presented types ofconcept
arehardware
needed. screw heads
Indeed, the
most of
of varying
system thedepends
knownsizes
uponusing
methods machine
a minimum
aim to
learning
analyze approaches
of constraints
a product in
concerning order
or one product to adapt
the presentation the
family on of robot’s
theobjects.end-effector.
physical This paper
level. Because
compares
Different of the presented
different
product network
families, hardware concept
architectures
however, may differ the system
and largely
peripheral depends
settings
in terms upon a minimum
andnumber
of the presentsand
the
of constraints
most
nature suitable concerning
solution
of components. Thisthefact
to the presentation
use impedes of objects.
case. A dataset
an This
consisting
efficient paper
of six
comparison compares
classes different
of
and choice different network
screw architectures
of appropriate heads was family
product and peripheral
created settings
to train neural
combinations and
the presents
networks
for the
to detect
production
most
screws suitable
in an solution to the
experimental use consisting
set-up case. A datasetof consisting
metal blocks of six classes
holding of different
different screws ofscrew
diverseheads
typeswas andcreated to train
conditions. neuralare
Results networks
validatedtoon
detect
two
system. A new methodology is proposed to analyze existing products in view of their functional and physical architecture. The aim is to cluster
screws
differentinelectric
an experimental
motors set-up
from the consisting
automotive ofsector
metalbeing
blocksprocessed
holding different screws ofdisassembly
on an automated diverse types and conditions. Results are validated on two
line.
these products in new assembly oriented product families for the optimization of existing assembly lines and the creation of future reconfigurable
different electric motors from the automotive sector being processed on an automated disassembly line.
assembly systems. Based on Datum Flow Chain, the physical structure of the products is analyzed. Functional subassemblies are identified, and
a©© 2022
2022 The
functional Authors.
Authors.isPublished
Theanalysis Published
performed. by Elsevier
Elsevier B.V.
by Moreover, B.V.
a hybrid functional and physical architecture graph (HyFPAG) is the output which depicts the
© 2022
This is anThe
openAuthors. Published by Elsevier B.V. (https://creativecommons.org/licenses/by-nc-nd/4.0)
similarity between product families by providing designlicense
access article under the CC BY-NC-ND support(http://creativecommons.org/licenses/by-nc-nd/4.0/)
to both, production system planners and product designers. An illustrative
This is an open
Peer-review access article under thescientific
CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
example of a under
Peer-review under responsibility
nail-clipper is usedof
responsibility ofthe
to the scientific
explain committee
committee
the proposed of of
thethe
29th
methodology. CIRP
29th CIRP
An Life Cycle
Life
industrial Engineering
Cycle
case study onConference.
Engineering twoConference.
product families of steering columns of
Peer-review under responsibility of the scientific committee of the 29th CIRP Life Cycle Engineering Conference.
thyssenkrupp Presta France is then carried out to give
Keywords: Automated disassembly; Remanufacturing; Screw Head Detection a first industrial evaluation of the proposed approach.
©Keywords:
2017 The Authors.
AutomatedPublished
disassembly; by Remanufacturing;
Elsevier B.V. Screw Head Detection
Peer-review under responsibility of the scientific committee of the 28th CIRP Design Conference 2018.
1. Introduction lines with similar parts and connectors with small alterations
Keywords: Assembly; Design method; Family identification
1. Introduction lines with similar parts and lines connectors
and variations, disassembly addresswith
partssmall alterations
and connectors
In order to keep planet earth vigorous and for the mainte- and variations, disassembly lines address parts
with large variations. In the case of remanufacturing, the parts and connectors
In order
nance of ourto societies,
keep planet earth vigorous
humankind needs and for the sustain-
to achieve mainte- with
have large
passedvariations.
at least one In prior
the caselife of remanufacturing,
cycle the parts
where many inaccessi-
nance
1.ability of
Introductionour societies, humankind needs to achieve
and climate neutrality. Therefore private lifestyles and ofblethe sustain- have passed at
product have
alterations least one
rangechanged prior life
and characteristicscycle where many
the conditionsmanufactured inaccessi-
of the parts and and/or
con-
ability
industrialandoperations
climate neutrality. Therefore
need to change. Oneprivate
measure lifestyles
on this and ble alterations
nectors.
be- assembled An have
in automated
this changed
system. the conditions
dismantling
In this system
context, theof theneeds
thus
main parts and con-
to adapt
challenge in
industrial
Due
half operations
to the technology
in production need to change.
fast development One measure
in the
is the so-called on this
domain of
remanufacturing, be- nectors.
modelling An
to the individual automated
conditions
and analysis dismantling
is nowof the system
processed
not thus
only to parts. needs
cope Among to adapt
with singleoth-
half
where in used
production
communication and and technology
partly anworn-out is the
ongoing so-called
parts
trend areofbeingremanufacturing,
reworked and
digitization to
ers,the
and products, individual
this limitedconditions
isa considered product of
as onerangeofthe processed
theormain parts.
challenges
existing Among
product oth-
forfamilies,
produc-
where used and partly worn-out parts are being reworked and ers, this
tionalso is
systems considered as one of the main challenges for produc-
brought to "asmanufacturing
digitalization, new" conditions [1]. Within
enterprises arethis process
facing chain, but
important to be [3].
ableThe disassembly
to analyze and tosystem
compare needs to adapt
products on the
to define
brought to in
"as today’s
new" conditions [1]. Within this process chain, new tion systems
physical layer [3].
as The disassembly
in special system needs to adapt on[4].
the
the dismantling
challenges of products is mandatory.
market environments:Nowadays, especially
a continuing product families. It canthe be selection
observed andthat usage of tools
classical existing
the dismantling
disassembly of products
for remanufacturingis mandatory.
is being Nowadays,
conductedtimes especially
mainly physical layer
Within families
in product as in
this contribution,special the selection and
an image-processing usage of
system tools [4].
for the
tendency towards reduction of product development and are regrouped in function of clients or features.
disassembly
manual labor for
[2].remanufacturing
However, In is being
foraddition,
future conducted mainly in
large-scale Within
detection thisof contribution,
screw heads isanpresented.
image-processing
The system system for the
is applied to
shortened product lifecycles. there is anapplications
increasing However, assembly oriented product families are hardly to find.
manual labor [2]. However, for future large-scale applications detection
theOnadaption of screw
of robot heads
tools.is presented.
To maintain The system is applied to
with high
demand of numbers of remanufactured
customization, being at theparts, same ittimeis mandatory
in a global to the product family level, productsthe industrial
differ mainlyapplica-
in two
with high numbers of remanufactured parts, it is mandatory to the adaption of
bility,characteristics:robot
the system is (i) tools.
embedded To maintain
in an the industrial applica-
automate this
competition task.competitors
with At the samealltime, overthetheautomation
world. This of the dis- main
trend, the number of industrially
componentsrelevant
and (ii) theuse
automate
assembly this task. At the sameimplies
for remanufacturing time, the automation of the pro-
dis- bility,
case.ofThethe addressed
system is coresembedded are two in different
an industrially relevant mo-
supplementary use
which is inducing the development many from challenges
macro to for micro type components (e.g. mechanical, electrical, electronical).
assembly for
duction systems. remanufacturing implies many challenges for pro- case.
tors The addressed
from themethodologies cores are
automotive discipline. two different supplementary mo-
markets, results inIndiminished
comparisonlot to sizes
manufacturing production
due to augmenting Classical considering mainly single products
duction systems. In comparison to manufacturing production tors from the automotive discipline.
product varieties (high-volume to low-volume production) [1]. or solitary, already existing product families analyze the
To cope with this augmenting variety as well as to be able to product structure on a physical level (components level) which
2212-8271 © 2022 The Authors. Published by Elsevier B.V.
identify
2212-8271
possible
© 2022
optimization
Thearticle
Authors. Published
potentials in
by Elsevier B.V.
the existing causes difficulties regarding an efficient definition and
This is an open access under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
production
This is an open
Peer-review system,
access
under it is important
article under
responsibility to have
CC BY-NC-ND
of the scientific a precise
license
committee knowledge comparison ofConference.
different product families. Addressing this
(http://creativecommons.org/licenses/by-nc-nd/4.0/)
of the 29th CIRP Life Cycle Engineering
Peer-review under responsibility of the scientific committee of the 29th CIRP Life Cycle Engineering Conference.
2212-8271 © 2022 The Authors. Published by Elsevier B.V.
This is an©open
2212-8271 2017access article Published
The Authors. under theby CC BY-NC-ND
Elsevier B.V. license (https://creativecommons.org/licenses/by-nc-nd/4.0)
Peer-reviewunder
Peer-review under responsibility
responsibility of the
of the scientific
scientific committee
committee of theof28th
the CIRP
29th CIRP
DesignLife Cycle Engineering
Conference 2018. Conference.
10.1016/j.procir.2022.02.001
2 Simon Mangold et al. / Procedia CIRP 105 (2022) 1–6
Author name / Procedia CIRP 00 (2022) 000–000 2

2. Related Work was achieved by utilizing a combination of pre-processing,

object detection and visual reasoning. The authors suggest the
Since bolted connections are commonly used connectors, vi- extension of the described system to other screw types.
sion based screw detection has found interest from research in Another notable mention goes to [3]. The developed system
prior works. In a contribution by Vongbunyoung et al., the au- is able to perform all actions to completely disassemble
tomatic detection of connective components is described as a electromechanical devices such as hard disk drives. Screw
highly desirable task for a disassembly system [5]. Bolted con- detection in this system also relies on the OpenCV Hough
nections are described as rather easy to detect in comparison Transformation in order to detect possible locations of screws.
to other commonly used connective components such as rivets, Afterwards a binary classification algorithm is used to distin-
tape or glue. This is as screw heads are highly standardized and guish screws from or any other circular object. This classifier
generally visible components. Nonetheless, there are two major achieved an evaluation accuracy of 93,6 %. The evaluation of
challenges highlighted and shown by a case study of Liquid- the disassembly process concluded that 10 out of 180 actions
crystal display (LCD) panels: failed with 9 of them being unscrew actions.
• Small size of connectors poses high demands to the uti-
lized camera system. Recent developments in the wide field of machine learning
• High variety because of different materials, corrosion or create a variety of possibilities for applications in the industrial
lighting conditions pose high demands on the flexibility environment. As summarized in [10] there are two major
of a visual detection system. categories of object detectors. One-Stage or Two-Stage Object
The presented screw detection system focuses on the distinction Detectors are distinguished by the way they generate the class
between product components and connector elements but does probabilities and location coordinates of an object. In general
not include the classification task to different screw heads. An- one-stage object detectors are considered to have a lower
other related image processing system is proposed in [6]. The detection accuracy than two-stage object detectors. The main
authors derive a visual intelligence scheme for the disassembly advantage of a one-stage object detection is the high detection
of hard drives. The underlying detection algorithm was first speed.
introduced in [7]. In the later publication, the screw detection One of the representatives of one-stage object detectors is
is accomplished in a two step approach: First, candidates were YOLO (You Only Look Once). YOLOv3 is the latest version
generated under the assumption that all screw heads have a developed by the original developer Joseph Redmon [11].
circular shape. Therefore an OpenCV Hough Transformation YOLOv4 was then released in 2020 and improved YOLOv3’s
is utilized to detect circles. Found candidates are then further AP and frames per second (FPS) by 10% and 12% respectively
examined since not all found circular shapes are screw heads. [12].
The contribution examines several classifiers to the task and YOLOv5 is the newest version of the YOLO architecture1 de-
derives a model which achieves an accuracy of 99%. However, veloped by Glenn Jocher2 using Python’s PyTorch framework.
both described systems are only capable of distinguishing
between the two classes "screws" and "artifacts". There is no Summarizing state of the art solutions, one can state that the
further subdivision into different kinds of screw heads. required variety in the detection of screw heads for the applica-
In the work of [8] the authors describe an architecture to tion in automated disassembly for remanufacturing has not been
distinguish between four different types of screw heads in solved. The existing algorithms and approaches are not contain-
different sizes. The final classification achieves an accuracy ing the desired amount of different screw heads for the applica-
of 97%. The pipeline’s overall accuracy is limited by the tion. On the other hand, machine learning based approaches in-
candidate generation done by searching for circular structures herently provide mandatory flexibility. The YOLO architecture
in the image using the OpenCV Hough Transformation. It was in the newest version provides the required framework and is
shown that this step succeeds in approximately 75 % of all utilized in the following.
attempts. This resulted in an average precision (AP) of 0.757
for the complete pipeline. The authors suggest the exploration
3. Methodology
of alternative methods for the detection of region of interests
(ROI). It is also noted that most misclassifications can be
3.1. Overview
directly linked to how Hough cuts regions.
A possible alternative approach to detect crosshead screws
The scope of the presented system is the adaption of a robot
is described by [9]. The proposed system relies on a digital
tool towards a presented object containing screw heads. There-
single-lens reflex camera (DSLR) camera connected to a com-
fore a camera system is aligned to capture the device. To ad-
puter that performs the screw detection using several image
dress the autonomous adaption of the robot tools, a two step ap-
processing techniques and the Faster-RCNN with Inception
proach is used. At first, the screw head classification algorithm
Version 2 deep learning architecture. It was shown that the
image processing steps had an impact on the precision and
recall of the deep learning model. The highest precision and 1 There has not yet been a publication on YOLOv5. However sourcecode and
recall rates of the system amount to 91.8 % and 83.6%. This documentation can be accessed by: https://github.com/ultralytics/yolov5
2 https://github.com/glenn-jocher

2
 Simon Mangold et al. / Procedia CIRP 105 (2022) 1–6 3
Author name / Procedia CIRP 00 (2022) 000–000 3

is applied. Based on this result, robot movements are executed

to configure an appropriate tool for the disassembly task in the
second step. This contribution focuses on the development and
optimization of the screw detection part which is addressed by
the following steps. Generated screw poses can subsequently be
used to reach the screw head and conduct the dismantling steps.
• Localization of screws within the image
• Classification of screws into discrete classes

create list
start
of screws

initialize
model
choose tool

acquire
image insert
tool tip
into screw

screw
detection unscrew
screw

screws yes
stop
no >0

Fig. 1: Screw detection and disassembly workflow.

Fig. 2: Hardware components implemented as an eye-in-hand effector system
during on-site validation.
3.2. Hardware

The focus of the contribution is on industrial applicability. detection architecture. For the training, a dataset containing 550
Therefore, industrially relevant components are utilized. images is created. Three different kinds of objects are included
The system consists of a Universal Robots UR5 e-series robot in the dataset to create a versatile system:
with an eye-in-hand camera system. Fig. 2 shows the combi- • Metal bars with equidistant bores holding screws of vary-
nation of the camera and screw driver module attached to the ing sizes, as seen in Figure 3(a) and (b).
robot flange. • Electric window lifter motors with different conditions of
A monochromatic camera with 1280x1024 pixels3 equipped connectors and parts (Fig. 3(c) and (d)).
with a 6 mm lens4 is used. • Starter motors for automotive combustion engines (Fig.
The object detection is deployed to a Siemens SIMATIC 3(e) and (f)).
IPC227E5 . Since computational power on this device is lim- Within the data set the following six different classes of
ited, it is necessary to achieve an acceptable runtime in order screw heads are included. All classes include the two sizes M3
to allow for real world applications in a throughput driven and M6 respectively:
environment.
• Pozidriv
• Hex socket
3.3. Screw Detector • Slot drive
• Phillips
The localization and classification of screws in the perceived • Hex
images is primarily implemented using the YOLOv56 object • Torx
For the training dataset, no specified position and orientation
3 MER-134-93U3M USB3 camera by Daheng Imaging: https://www.get- of objects in the image were used. During the image acquisition
cameras.com/USB-Camera-OnSemi-Python1300-MER-134-93U3M for the training dataset, the camera is moved around the object.
4 LCM-5MP-06MM-F2.0-1.8-ND1: https://www.get-cameras.com/LENS-C-
This leads to a large number of points of view and therefore in
mount-5MP-6MM-F2.0-1/1.8-LCM-5MP-06MM-F2.0-1.8-ND1 a desirable variety of images of a limited number of objects.
5 SIMATIC IPC227E; Intel Celeron N2930; 8 GB RAM; 240 GB SSD; In-

dustrial Edge
Furthermore, different backgrounds, lighting conditions and
6 Repository: https://github.com/ultralytics/yolov5 apertures are used.
Official Documentation: https://docs.ultralytics.com/ Both YOLOv5 screw detectors were trained for 500 epochs
3
4 Simon Mangold et al. / Procedia CIRP 105 (2022) 1–6
Author name / Procedia CIRP 00 (2022) 000–000 4

mance of the different detectors was performed using the Inter-

section over Union (IoU) method. Table 2 shows the achieved
precision of different screw detectors.

Detector [email protected] [email protected]:0.95

YOLOv5s 0.984 0.834
YOLOv5m 0.980 0.826

Table 2: Experimental evaluation of different screw detectors.

(a) (b)
It can be seen that there is no significant difference in overall
precision. Even though the YOLOv5s version is a smaller and
faster model it outperforms the YOLOv5m version on the used
dataset. To evaluate the performance of the screw detector the
confusion matrices (see Figures 4 and 5) are generated in order
to get a further understanding on possible sources of errors.

(c) (d)

(e) (f)

Fig. 3: Exemplary images contained in the dataset.

Fig. 4: Confusion matrix acquired for YOLOv5s on the screw detection dataset.

with a batch size of 32. The size of incoming images was

reduced from the original size to 640 × 640 pixels. Table 1
shows the number of layers and number of parameters, such
as the number of floating-point operations (GFLOPs). In order
to allow reproducibility and an efficient training process,
transfer learning using the yolov5s.pt and yolov5m.pt
weights provided by ultralytics is conducted. The developed
software package is integrated into the Siemens Industrial
EDGE environment.

Detector Layers Parameters GFLOPs

YOLOv5s 224 7,067,395 16.4
YOLOv5m 308 21,057,843 50.4

Table 1: Comparison of YOLOv5 detector architectures.

Fig. 5: Confusion matrix acquired for YOLOv5m on the screw detection

4. Experimental Results dataset.

4.1. Overall Precision 4.2. Runtime Evaluation

In order to evaluate the different detectors with respect to the In addition to the precision metric, the runtime is regarded
present use case, different metrics were examined. The perfor- in special. It is defined as the average time of the pre-loaded de-
4
 Simon Mangold et al. / Procedia CIRP 105 (2022) 1–6 5
Author name / Procedia CIRP 00 (2022) 000–000 5

tectors to acquire an image and achieve a result. Table 3 shows 5. Result Assessment
the results of the experimental runtime evaluation. Experiments
were performed with 100 random images on the aforemen- The confusion matrix and the classification results prove
tioned Siemens SIMATIC IPC227E. the general functionality of the approach and the screw detec-
tors. Especially the distinction between the Philips and Pozidriv
Detector av. runtime [s] rel. av. runtime works precisely even though the screw heads appear rather sim-
YOLOv5s 7.53 1.0 ilar to the human eye. Hexagon headed screws and hexagon
YOLOv5m 18.09 2.4 socket screws are major sources for misclassifications. How-
ever, the biggest source of errors are False Positive detections.
Table 3: Experimental runtime evaluation of two YOLOv5 screw detectors on
Siemens SIMATIC IPC227E. During the on-site validation it showed that these False Posi-
tive errors usually show a lower value of confidence provided
4.3. On-Site Validation by the screw detector. In a production environment this could
be handled by applying a confidence threshold and ignoring de-
The entire software package containing the YOLOv5s model tections that show a low confidence value.
is deployed to the site of usage. The validation is conducted us- As denoted in section 4.1, there is no significant advantage in
ing unseen objects for the system, similar to the ones contained terms of precision between the two applied classifiers.
in the training dataset. A small motor (window lifter motor as in Regarding the runtime evaluation, it can be seen that the larger
Figure 3 (c) and (d)) is repeatedly presented to the system with screw classifier takes around twice as much time to evaluate a
an alteration of the background, an alteration of the screw head single image. In order to allow a productive usage of the de-
type and a adaption of camera parameters. Likewise, a large veloped package we advise the use of the YOLOv5s model
motor (Starter motors as in Figure 3 (e) and (f)) is presented. at this stage of the development as it has significant perfor-
The contained screw heads were partly damaged, dirty and or mance advantages while maintaining a comparable accuracy as
corroded. Table 4 contains the results of the test series. the YOLOv5m model.
The system is applied to scenes without any fixtures. In real en-
Test Object Present screws Correct detections vironments, fixtures for the clamping of parts are used. Those
20x Small motor 45 39 may contain screws which could potentially lead to misclassifi-
7x Large motor 17 15 cations. In this work, this effect is not considered. One approach
could be the usage of image pre-processing steps such as filters
Table 4: Experimental on-site validation on unseen objects for the repeating sections of the input images. In real production
environments it is not guaranteed that screw heads face the cam-
During the application of the system to the small motor, era plane as assumed in the contribution. Even though images
some errors occur. If the Torx head screws is changed to a Slot within the presented training data set are taken from different
drive, the system falsely detects a Torx head. One may conclude angles, multiple views may be required for the detection of a
that the system has learned alternative factors than the screw screw in real production environments. Within this contribution
head form. Additionally there are False Positive results occur- only the ideal case is regarded and not-aligned perspectives are
ring. Some screw heads are being detected doubled. Despite neglected. In real world applications one may conduct a clas-
these misdetections, the system appears rather robust against sification into component levels and in a second step apply the
an alteration of the background and camera settings such as the presented screw detection in this contribution. The objects in
focal length and the aperture setting. At the application of the the training dataset are screws of mainly good physical condi-
large motor, there are fewer misdetections occurring. Yet there tions. Yet in case of disassembly applications, the production
are False Positive errors occurring if the motor is only party system needs to work on damaged or worn out screws. It is not
visible in the image. All in all, the object detector is proven regarded profoundly in this contribution how the detector works
for robust functionality and has an acceptable overall runtime. on those objects. Further research should include this.
However, it has to be mentioned that several influences on pro-
cess quality could be observed.
Since it is not possible to automatically re-focus the camera op- 6. Conclusion
tics, the positioning of the camera in relation to the object and
therefore the image quality influence the outcome of the screw We present a system for the classification of screw heads into
detection. The detection accuracy can be increased by using the discrete classes corresponding to their physical shapes using
robot to adjust the distance between camera and object in order state of the art object detectors. Together with the classification
to acquire a clear image. Screw detection on the electric motors results, the location in the camera frame is derived. The system
works sufficiently reliable. improves state of the art solutions as it is capable of classifying
Another influence on the detection quality is the background of into multiple different classes. With the utilized hardware, we
the images respectively the board under the cores. The detector prove the application in an industrial environment. Especially
is easily confused by the holes in the base plate (Figure 2) espe- the Industrial EDGE Environment of company Siemens can be
cially if the image has a poor quality. There are False Positive denoted here. In future research, the correlation of the image
detections of hexagon socket screws observed. quality to the screw detection results need to be assessed. In-
5
 6 Simon Mangold et al. / Procedia CIRP 105 (2022) 1–6
Author name / Procedia CIRP 00 (2022) 000–000 6

creased computational power would allow for more complex

and presumably more accurate detectors. A larger dataset would
augment the detection results and allow a better generalization
and therefore a wider variety of possible applications of the
system. This should be focused in further examinations. Ex-
periments on the complete dismantling pipeline are not con-
ducted since the contribution focuses on the image processing
task solely. This task remains to future work. Furthermore, the
influence of the training data on the outcome of the process, in
special the reduction in variability will likely lead to an increase
in overall precision.

Acknowledgements

The project AgiProbot is funded by the Carl Zeiss Founda-

tion.

References

[1] Jawahir, I. S., Bradley, R., 2016. Technological Elements of Circular Econ-
omy and the Principles of 6R-Based Closed-loop Material Flow in Sustain-
able Manufacturing. Procedia CIRP 40, 103–108.
[2] Javier Ramírez, F., Aledo, J., Gamez, J., Pham, D., 2020. Economic mod-
elling of robotic disassembly in end-of-life product recovery for remanu-
facturing. Computers & Industrial Engineering 142.1
[3] Klas, C., Hundhausen, F., Gao, J., Dreher, C., Reither, S., Zhou, Y., Asfour,
T., 2021. "The KIT Gripper: A Multi-Functional Gripper for Disassem-
bly Tasks" IEEE International Conference on Robotics and Automation
(ICRA) 2021. Xi’an, China
[4] Fleischer, J., Gerlitz, E., Rieß, S., 2021. Concepts and Requirements for
Flexible Disassembly Systems for Drive Train Components of Electric Ve-
hicles. Procedia CIRP 98, 577-582.
[5] Vongbunyong, S., Chen, W. H., 2015. Disassembly Automation. Auto-
mated Systems with Cognitive Abilities. Springer International Publishing
[6] Yildiz, E., Brinker, T., Renaudo, E., Hollenstein, J., Haller-Seeber, S., Pi-
ater, J., Wörgötter, F., 2020. A Visual Intelligence Scheme for Hard Drive
Disassembly in Automated Recycling Routines. Proceedings of the Inter-
national Conference on Robotics, Computer Vision and Intelligent Systems
(ROBOVIS 2020)
[7] Yildiz, E., Wörgötter, F., 2019. "DCNN-Based Screw Detection for Au-
tomated Disassembly Processes" 15th International Conference on Signal-
Image Technology & Internet-Based Systems (SITIS) 2019. Sorrento, Italy
[8] Yildiz, E., Wörgötter, F., 2020. DCNN-based Screw Classification in Auto-
mated Disassembly Processes. Proceedings of the International Conference
on Robotics, Computer Vision and Intelligent Systems (ROBOVIS 2020),
61–68.
[9] Foo, G., Kara, S., Pagnucco, M., 2021. Screw detection for disassembly of
electronic waste using reasoning and re-training of a deep learning model.
Procedia CIRP 98, 666–671.
[10] Xiao, Y., Tian, Z., Yu, J., Zhang, Y., Liu, S., Du, S., Lan, X., 2020. A
review of object detection based on deep learning. Multimedia Tools and
Applications 79.11, 23729–23791
[11] Redmon, J., Farhadi, A., 2018. "YOLOv3: An incremental improvement".
arXiv:1804.02767
[12] Bochkovskiy, A., Wang, C., Liao, H., 2020. "YOLOv4: Optimal Speed and
Accuracy of Object Detection". arXiv:2004.10934