1997 IEEE International Conference on Intelligent I’rocessing Systems Octobcr 2X - 3 I . Uci,jing.
Chinil
Machine Vision System for Total Quality Control of SMT Electronic Cards
Perfecto Mariiio Miguel Angel Dominguez
Electronic Technology Department and Electronic Technology Department and
Applied Electronics Institute Applied Electronics Institute
. UniversiQ of Vigo University of Vigo
E.T.S. Ingenieros Industriales. Apartado) Oficial E.T.S. Ingenieros Industriales, Apartado Oficial
36200 Vigo. Spain 36200 Vigo, Spain.
e-mail: pmarino(@uvigo.es e-mail:
[email protected] Fay: +14-86-469547 Fax: +34-86-469547
Tel: +34-86-812162 Tel: +34-86-812143
Abstract - The authors have been involved in the spectrum (50 + 850 MHz). This ATE LAN node is supported
implementation o f a total quality control project in a in its performance for a RF analyzer connected to it through
communications equipment company, whose products are an IEEE 488 bus [j].
mounted and tested in a manufacturing plant controlled by an The last implementation is an automated inspection LAN
IBM Token Ring LAN. The most significant products are RF
node to detect the missing of SMT and non SMT components
and microwave amplifier boards for satellite TV receivers, with
electronic components based on SMT technology. mounted in the electronic cards of above mentioned
The first implementation o f total quality control project was amplifiers. This LAN node for automated inspection is based
an ATE (Automatic Test Equipment) LAN node that runs a on artificial vision [6] [7] and will be called MVI (Machine-
program of verification for single-channel and multichannel Vision Inspection).
electronic amplifiers, which bandwith is in TV spectrum (50 + The first implementation is a case study of on-
850 MHz). This ATE LAN node is supported in its performance line/postprocess inspection because the ATE LAN node
for a RF analyzer connected to it through an IEEE 488 bus. immelately follows the production process. whereas the last
The last implementation is an automated inspection LAN one based on artificial vision (MVI) is the type of on-Imehn-
node to detect the missing of SMT and non SMT components
process inspection [ 11, and therefore can be used in different
mounted in the electronic cards of above mentioned amplifiers.
This LAN node for automated inspection is based on artificial stages along the mounting shop floor of the Communications
vision and will be called MVI (Machine-Vision Inspection). enterprise [SI.
1. INTRODUCTION 11. FLEXIBLE MANUFACTURING SYSTEM
Total quality control is perfonned at shop floor level along The total quality control project of communications
whole operations cycle performed over the product, from raw enterprise implemented over a flexible manufacturing system
materials, through parts and semifinished shapes until its (FMS) [91. covers manufacturing activities over electronic
final shipping. amplifier boards provided with SMT components and
Automated inspection improves the data collection about devices. such as insertion of non SMT Components. welding
the features and charactenstics of products and processes, in and screwing of them. The essential physical elements of this
order to assure production conformance to given I N S are depicted in figure 1:
- Scara type robots from IBM [SI capable of performing
specifications and to detect machine malfunctions or
unaccuracies of logistic processes [ 11. insertion. welding and screwing tasks, provided with
The authors have been involved in the implementation of a adequate end effectors [IO], over the electronic amplifier
total quality control project in a communications equipment boards
company [2], whose products are mounted and tested in a - An automated material handling system to move the
manufacturing plant controUed by an IBM Token Ring LAN amplifier boards between robots and inspection stations,
131. The most significant products are RF and microwave implemented through two conveyor belts. The main loop
amplifier boards for satellite TV receivers. with electronic conveyor (MLC) provides boards with SMT electronic
components based on SMT technology [4]. components and devices, to the robotic stations and the
The first implementation of total quality control project machine-vision inspection station (MVI). The secondary
was an ATE (Automatic Test Equipment) LAN node that loop conveyor (SLC) supplies completely mounted
runs a program of verification for singlechannel and boards, with SMT and non SMT components to the
multichannel electronic amplifiers. which bandwith is in TV automated test equipment station (ATE).
~
01997 IEEE
0-7803-4f53-4/97/$10.00
- 1427 -
t
REFLECTOMETRIC
- - - ELEMEWS
-----
3-
BRIDGE -TESTING
I HANDLING ELEMENTS
m
I
I !
:.MECHANICAL _ _ . .:_ . _ _ . _ _
. . _ . . _COUPLING
Fig. 1 FhtS layout of communications enterprise for automated mounting
and inspection of electronic amplifier boards Fig. 2 Connections of board testing and handling elements of ATE
- Automated storage and retrieval systems (AS/RSs) such IV. MACHINE VISION LAN NODE
as: product identification system. part-washing devices
for automatic removal of a fault board from conveyor The machine-vision inspection (Mvr) node is devoted to
belts, and a general feeding and cleaning pallet-based detect the missing of SMT components and screws for
system of the two conveyor belts. clamping coaxial cables, when RF amplifier boards are
- Control elements connected through a Token Ring Local mounted in the FMS (Fig. 1). Boards are checked by means
Area Network (LAN) that perform a shop floor of pictures which are tested if all board’s defined components
distributed system. and therefore are LAN nodes with match a pattern of a previous correct image. If the matching
specific local tasks. The two conveyor belts are controlled is verified the board will pass to following FMS stages,
by a Programmable Logic Controller (PLC). This PLC otherwise a signal is activated to introduce the failed board in
and the above mentioned production identification system the part-washing device (Fig. 1). For thishNI node was
are connected to LAN through a communications server developed a vision program that runs in a PC to which an
191. The robot movings are controlled by a PC directly Image Digitizing Card (IDC) is connected. The vision
connected to it through an I/O card. The PC is under program controls th~sIDC, which performs the capture,
..DOS and runs an only program which is a programming processing, storage, and display of images taken by a
language interpreter called AML/2 1111. A videocamera (Fig. 5). This vision program defines the
communication server can link until sixTeen robots and patterns and analyzes the images of boards under test to
thirty PLC’s to the LAN [9]. decide if all pattern’s components are present or not.
111. AUTOMATIC TEST EQUIPMENT
The specific local task managed for the ATE LAN node is
the verification of amplifier boards mounted in the FMS of
communications enterprise’s shop floor. Amplifiers must be
FIXTURING HEAD
checked for each channel in next parameters: gain, I I w I DEVICE
maximum variations of gain, return loss and rejection of
adjacent frequency.
The ATE station in the FMS besides its main task of
testing electronic amplifier boards, is involved in the
qb3 ED3
SECONDARY LOPP
C M Y W BELT
handling of these boards on the secondary loop. For that
reason its hardware configuration is divided in two parts:
Fig. 3 Confguration of electro-pneumaticplacement system and fixturing head
boards testing and handling elements (Figs. 2 and 3). The of ATE on the secondwy loop conveyor, including the part-washing device
verification program is shown in figure 4.
- 1428 -
a) Main program b) Handling and testing subprogram
WAIT FOR
PLUGGING
&
OFRFW
c
I L I E A S i S
U
I I
I
AND DATA
YES
2%
Fig. 5 Hardwart: configuration of MVI station for quality control of mounted
RF amplifier components
CORRECT
MEASUREMENT involved besides the kind of element to be detected, the most
changing characteristic of it from the situation of presence in
the PCB to the missing one. In this quality control
W M T FOR
UNPLUGGING application the main elements to be detected are screws (non
HANDLING Sh4T) and Sh4T components.
1..9
AND
TESTING In &e PCBs under test there is a set of metallic brackets
- OPFRATION
- ESCAPE?
and cable terminals. and it is very important to know if
insertion robots of FMS (Fig. 1) have put them or not, Given
that each of these parts has a screw that hold down it against
the PCB, the problem to detect the missing of any non SMT
component can be reduced to check the presence or absence
of its respective screw. When a screw is missing in the PCB
Fig. 4 Flowchart of verification program for ATE node
remains its twisted hole, which is translated to videocamera
(Fig. 5 ) as a low lighting area, and the result in the IDC is a
set of pixels with low grey level. On the contrary, when the
A. Main features for detection screw is present, its metallic condition gives a deep
brightness under the lights making a very clear area with
Printed circuit boards (PCBs) of RF amplifiers under high grey level. This is a very useful feature of PCB screws.
inspection, have a big number of SMT and non SMT SMT components of PCB show a particular difficult for
components and strips, that make dificult the task of their detection given its small size. Their shape is rectangular and
recognition and visual detection 161. This multiplicity of are placed in the PCB crossing an insulating area that
elements and colour scheme changes make a high variation uncouples the two conductive strips, to which outermost
in grey levels of PCB image, and therefore binaryzations or parts of a SMT component are welded. This insulating area
filtering procedures over the image that would simplify has an uniform dark green colour. When a SMT component
notably the tasks of recognition and detection, are not is missing in a given area, the image taken by the IDC shows
suitable 171. Visual control is even more complex given the in that area a set of pixels with the same tonality. On the
shadows projected from metallic parts or in screw heads contrary, when the S M T component is present, in that area
because its slots, taking a great significance the lighting the IDC image has pixels with the same tonality related to
applied over the PCB under inspection. Another problem is the isolating area near SMT boundaries, and other pixels
the existence of PCB areas where SMT components are very with more clear intensity level (white Sh4Ts) or more dark
close, which makes very difficult their detection because the one (black SIMTs) related to S M T surface points. Therefore
borders of a SMT component can be confused with the SMT’s detection over PCB will be based in this feature
neighbouring ones. Hence in PCB visual control must be using gradient techniques.
- 1429 -
B. Detection algoritliiiis grcy lcvcls is uscd Wlicti a SCICW IS prcsciil ;i liigli grcy lcvcl
IS dctcctcd 111 that wca, but its iiiissiiig crcates a low grcy
The visioii prograiii dcfiiies a pattcrii of PCB uudcr lest lcvcl a ~ c apiovidcd by tlic twistcd Iiolc. For selcctiiig tlic
sliowiiig tlic arcas wlicrc ;i coiiipoiiciit must csist. Tlic SMT scicws a littlc sqiiarc iiic'i is uscd with a SILC that allows to
coiiiponciits arc sclcctcd with a rcctaiigular arcs iitovcd ovcr iitcludc i t I I I S I ~ Cthc wliolc daik arca wlieii tlic scrcw is
the wliole image. This area is placed over a SMT coiiipoiiciit . iiiissiiig Oiily arc uscd tlic pixels from vertical lmc that cross
iii such a way that the iiuddlc liiic wliicli pass through thc tlic sqiiarc by its half Tlic dccisioii threshold IS a grcy lcvcl
inorc iiarrow a m cross to S M T froiii sidc lo sidc by its half, of 70 If tlicic is 110 poiiits froiu vcrtical IIIIC with ii value
just aloug tlic insulatiilig slrip uiidcr tlie S M T conipoticiit. liiglicr tliiiii 70, iio S C I C ~IS i i i tlic arca bccausc all is dark
Points wider tllis middle line are used for iiiipleiiieiitiiig the Or1 the cotitiary ifaiip poirit from v e r b a l liiic has a value
oricdiiiieiisioiial gradieiit of pixels difference. In each poiiit liiglicr tliaii 70, tlic screw IS preseiit given 11s metallic
tltc' gradient is calculated as the value of lumiiioits iiitciisity briglitliess Tlic explained grey level tecliruque and 11s
iii tlie next poiiit iiuiius tlie S a m value in the previous poiiit rcspccttvc detcctioii algoritliiri IS stated in figure 7 Botli
[grad(x) = I(x+l) - I(x-I)]. A positive gradicnt shows tlic tecliiiiques arc uscd 111 tlic dcvclopcd visioii prograiii of MVI
cliaiige froin a dark area to other clearest, aiid a Iicgativc (Fig 8)
gradicnt will show the opposilc. For takitig a dccisioii aborit
if a gradiciit csist iii a given poiiit, a levcl of 35 is sclcctcd. V. RESULTS
Wlieu a SMT coiiipoiiciit is iiiissiiig, tlic obtaiiicd gradicuts
over the ii~iddlcliiic points arc less tliaii 35 bccausc Ilicsc Tliis tolal qmlity coiitrol piojccl of FMS 111 llic shop floor
points bcloiig to the isolatiiig strip a i d linvc siiiiilar toiiality, of COllllllUlllCalIOllSC t l l C l pi ISC 1s glvlllg tl1c IlCXt 1csulls
tlicrcforc 110 gradient is got. 011 tlie con('l.ary, if tlic SMT - 11it1oductioii of ccililicd IS0 9000 quality for F M S based
coiiipoiieiit exists, great levcl c1i;iiigcs arc dctcctcd in the two 011ATE aiid MVI pcrfoniiaiiccs, givcii tlic bcst arid quick
sides of it (liiglicr tliari 35) aud tlie obtaiiicd gradiciits dctcct itifoi iliatioil p i ovidcd about thc maudacturing pioccss
tlie preseiicc of SMT coiiipoiicnt. Tlic csplaincd gradiciit - Oii-liiic/poslproccss tcstiiig 01 I1iial products ( A I E). w~tli
tccltiuquc aiid its rcspcctivc dctcctioii algoritluii is dcpictcd ;I IOW I;ltC or m c d OW, ~ C C ; I W C or Il1lpicl1lcmi 011-
in figurc 6. Iiiic/iii-proccss iiispcctioii (MVI) Also failcd products
For fillding tlic prcsciicc of scrcws a tccli~iic~uc bascd oii Iiavc bccii cflicictitly Ictricvcd providiiig fccdback
Line ofgicy
lcvcls eo Sclcctioti iircu
Linc of gradients Linc or yradiciits
,ScIwtioii area c " 1 1 t s i o i i a m l'wistiiig are:
wliiro*w . :.........,.....
....).. /""
.......................... ol' screw
liisulatiiig strip 255( Lcvcl orgrey 2551 Lcvcl ol'grcy
Grad(x) = I(x+l)-l(x-I)
U
Gradient
I
Grad(x) = l(x+l)-l(x-l)
Gtadiciil
7o
01
Ltlccisioii IcvcI
- -
;
1)ixcls
7o
U
wdecisioii levcl
I'lhCk
Grey l e i e l of decbioii line pixels G I cy level of decisioii h i e pixels
nlieii tlie S C ~ C I Vis iiiissiiig wlieii the screw is preseiit
Fig. 1 Gradient tecluiique and detection flowchart of ShlT couiponenls
Fig 7 Grey level tecluiique and detection flowcliarl of non SMT components
- 1430 -
mounting lines to previous inanufacturing stages. based VI. REFERENCES
on the exact location of defaults detected by MVI station.
Just-In-Time (JIT) manufacturing methods are best [ 11 N. Singh. Computer-integrated design and
supported by the feedback information provided by on- manufacturing, WILEY; 1996.
line inspection stations such as MVI and ATE.
Low cost implementation of MVI and ATE stations 121 P. Mariiio and F. Poza, “Distributed system for
doesn’t affect to economy of “critical point” e.xpenses assembly of SMT electronic equipment”, in
derived of capital investments for FMS resources, and Proceedings of the 1995 5th NATIONAL
therefore a reduction of manufacturing costs is achieved, CONFERENCE ON COMPUTER AIDED PROJECT,
based in the best quality furnished to process and PLANNING AND PRODUCTION, University of
products in real-time. Minho, Guimaraes (Portugal), pp. 33-38.
[31 P. Mariiio, Enterprise-wide communications:
standards, network and services, Ed. RA-MA; 1995.
141 Ph. P. Marcoux, Fine pitch surface mount technology,
Van Nostrand; 1992.
P. Mariiio, E. Mandado and A. Lago, Electronic
instrumentation, Ed. MARCOMBO; 1995.
H. Hang and J. Woods (editors), Handbook of visual
communications, ACADEMIC PRESS; 1995.
D. Pearson (editor), Image processing, McGRAW-
HILL Book Company (UK) Limited; 1991.
P. Mariiio, F. Poza and J. Nogueira, “Integrated
information system based on industrial standard
protocols’’, From Balanced Automation Systems 11, Ed.
CHAPMAN & HALL, London, pp. 201-208, 1996.
P. Mariiio, F. Poza and J. Nogueira, “Flexible
manufacturing communications network in distributed
V p l r s and in- robotics”, in Proceedings of the 1996 MHS’96, E E E
‘bypm3ofpdl
96TH 8213, pp. 265-272.
S. Soloman, Sensors and control systems in
47
Put code
Fig. 8 Flowchart of vision program based on image digitizing card
manufacturing, McGRAW-HILL; 1994.
IBM, AML/2 manufacturing control system, user’s
guide. PN67X1370; 1986.
- 1431 -
