HIGH ACCURACY DIMENSIONAL MEASUREMENT USING NON ..

TARGETED OBJECT FEATURES

Annin Gruen and Dirk Stallmann

Institute of Geodesy and Photogrammetry

ETH-Hoenggerberg, CH-8093 Zurich, Switzerland
Tel.: +41-1-3773038, Fax: +41-1-3720438, e-mail: [email protected]

Commission V

ABSTRACT
In industrial measurement applicatiom: a great variety of tasks require the measurement of non-targeted
("natural") objects. This paper describes a novel approach to the Multi-Photo Geometrically Constrained
(MPGC) matching algorithm which allows for high accuracy dimensional positioning of these features.
Depending on the quality of the feature definition a relative accuracy of 1:25000 is attainable. The major
algorithmic aspects will be addressed. An example [rom the measurement of a test object will demonstrate the
performance and potential of this new approach.
KEY WORDS: Industrial inspection, image matching, edge measurement, high accuracy

1.INTRODUCTION 4& The simultaneous estimation of object space

coordinates with measures for quality control of the
It has been shown that under ideal conditions image algorithm.
edges can be measured with very high accuracy (e.g.
41) The application of the algorithm for an image/object
0.006 pixels in Raynor, Seitz, 1990). In many practical
tracking procedure.
applications, in particular in dimensional industrial
inspection tasks, the highly accurate measurement of A typical part to be measured is an aeroplane engine
. natural, non-targeted object features (e.g. edges) is nozzle as shown in Figure 1 with 25 cm diameter and 5
required. With the current off-the-shelf sensor-, camera-, cm height. This type of object has a complex surface
signal transfer- and A;D conversion technology we shape with pulse and step edges, corners, surface paint
anticipate a relative accuracy of 1:25000 in object space patches, different surface orientations and mixed surface
to be attainable. This paper describes a new automatic
measurement algorithm which can potentially deli ver
such accuracies. The aIgorithm is applied in a 3-D-vision
system for precise measurements 01' industrial parts. The
algorithm is a modi1'ication and extension 01' the MPGC
algorithm. It finds the edge, matches respective patches
in multiple images (theoretically unlimited in number)
and determines 3-D object coordinates simultaneously.
The paper gives a short description of the basic
algorithm, its implementation and the results of a
practical accuracy test. A compiete algorithmic treatment
can be found in Gruen, Stallmann, 1991.

The main features of this algorithm are:

.. The free selection of image edge templates (synthetic

or real, varying edge spread functions, different edge
types, straight and curved edges).
41) Use of collinearity constraints for the imaging rays, to
reach a high precision and reliability and substantial
reduction of the solution space.
41) Additional image space constraints to force
translations of the edge template to be perpendicular to Figure 1 Aeroplane cngine nozzle. Image acquired
the edge in the image. with a Videk Megaplus camera (1024 x
1024 pixels), image brightness and contrast
41) An unlimited number of sensor frames to be included enhanced with a Wallis filter.
in the matching procedure.

694
geometry. The development of an algorithm for high equations with usually high weights and integrated into
accurate measurement of non-targeted but well-defined the adjustment system.
image edges was to be achieved. For inspection the
The joint system is solved in a least squares adjustment.
checking of the diameter d between the outer flanges is of
Beeause of the non-linearity of the funetional model the
interest. With a diameter of 25 cm an accuracy in
final solution is obtained iteratively, whereby
diameter measurement of 10 J.lm was requested. Because
approximate values for the parameters are required: the
the method gives 3-D coordinates and the measurement is
geometrie transformation parameters for each patch and
done in a plane the standard deviation of a single
the objeet coordinates of the objeet point. The iterations
coordinate translates to 7.1 J.lm. A partial report on this
are stopped if each element of the solution vector falls
project can be found in Ei-Hakim, 1990 and Gruen,
below a threshold.
Stallmann, 1991.
2.2 Modifications for edge matching
2. EDGE MATCHING ALGORITHM
In order to convert the 11PGC algorithm into an optimal
The used edge matching method is a modifieation and and non-biased proeedure for the measurement of edges,
extension of the 11PGC matching algorithm. This image the following modifications and extensions had to be
matching algorithm is based on least squares matching introdueed:
(LSM) and belongs to the class of area based mate hing .. Introduetion of a synthetie edge template.
methods. In an iterative least squares adjustment proeess
• Reduction of the image shaping parameters to those
the algorithm matches multiple images, finds the edge
whieh are safely determinable by the given image
and determines 3-D objeet coordinates simultaneously.
edge strueture.
More detailed deseriptions of the mathematieal eoneepts
ofLSM are published in Gruen 1985, 11PGC matching in • Additional image space constraints for the shifts to
Gruen 1985 and Gruen, Baltsavias 1988 and edge prevent a movement of the template along the edge.
matching in Gruen, SIal/mann 1991. Operational details • Creation and pre-rotation of individual templates for
on 11PGC can be found in Baltsavias, 1992. each image.

2.1 Mathematical concept of MPGC matching The used templates are synthetic straight ramp edges
from dark (grey value 30) to light (grey value 226) wirh
The LSM is a method to find similar structures in two varying linear ramp steepness (Figure 2).
corresponding image windows, the reference patch
(template) and a seareh image (patch). The patch is
transformed upon the template such that the squared sum
of grey value differenees is minimised. The Iocation of
the pateh against the template is described by a shift
vector. The template ean be an artifieial pattern for point
loeation, or areal window of an image to find
homologous image windows for parallax measurement.
In our case of edge matching it is a synthetic ramp edge. Figure 2 Synthetie straight ramp edges with
varying ramp widths (1,2 and 3 pixels)
The systematie differenees between the template and the
patch. eaused by perspective and sensor effeets, can be
modelled by geometric and radiometric transformations. The geometrical transformation of the patch has to be
Since the radiometric parameters are largely orthogonal restricted, because the character of the image edges does
to the other system parameters, the radiometrie correetion not allow the determination of a11 six parameters. They
can be applied prior to the adjustment. Due to the small allow a rotation of the patehes without ascale change. A
patches the bundles of rays are very narrow and for the scale perpendieular to edge is implicitly included by the
geometric transformation the strict perspective projeetion linear radiometrie fit.
can be approximated by an affine transformation and
modelIed by six linear parameters. The radiometrie fit is Sinee the image edges to be measured are essentially uni-
done by two parameters to form a linear function. This directional, a linear template edge would slide
leads to equal brightness and contrast of the patehes and continuously along the cdgc during matching. This efrect
the template. is eompensated by rcstrieting the shift veetor of one patch
approximately perpendieular to the Ioeal edge direetion.
The LSM model uses only the grey value information.
The restrietions of the number of the reshaping
But very often additional information is available, wh ich
parameters and the shift veetor eondition are also
can be used to support the model. If the sensor geometry
formulated as observation equations and addcd with a
for each object point is based on perspeetive projeetion
high weight to the least squares equation system (for a
the collinearity conditions ean be formulated. These
eompiete algorithmie representation see Gruen,
conditions allow us to replaee stereo LSM with MPGC
Stallmann, 1991).
matching, using a theoretieally unlimited number of
patches simultaneously for matching. Additionally, the The orientation of the corresponding edge elements in the
object coordinates can be determined simultaneously. trames differ depcnding on the camera orientation.
The geometrieal conditions are formulated as observation Therefore the template must be pre-rotated into the

695
approximate edge direction of the patch (Figure 3). The number of iterations to determine the unknown
only difference between the pre-rotated template and the parameters can be decreased. The tracking is basically
"original" is the rotation angle. The approximate edge done in image space. Through the simultaneous
direction is derived from the maximum edge gradient computation of consistent object space coordinates this
direction. which in turn is computed from the Sobel generates implicitly an object tracking procedure.
gradients in a 3 x 3-pixel window.
3. IMPLEMENTATION OF THE EDGE
Essentially LSM and MPGC are area-based matching
techniques. For high accuracy edge matching the method MATCHING ALGORITHM
is transformed into a combination of an area-based and The previously described algorithm is implemented as a
feature-based technique. This is achieved by introducing module in DEDIP (Development Environment for
as reference template a synthetic (or real) edge pattern, DIgital Photogrammetry, Gruen, Beyer 1991). The
which is to be matched with the actual image edges. following functional features and options are realised in
Compared to the conventional feature-based matching the program:
techniques our method does not require the extraction of
image edges, but matching is done directly by using the • Interactive measurement of image coordinates in
original grey value edges. digital images.
• Visualisation of the edge matching on the display
2.3 Edge tracking (Figure 4).
• Interface to bundle block adjustment program in
The edge measurement procedure has been extended to DEDIP.
an edge tracking technique, wh ich automatically tracks
and measures the fuH edge. The new approximate match • Single-point edge matching and edge tracking with an
point for the next patch of the first image is determined unlimited number of images.
by using the previously matched position, its local edge • Edge matching with and without collinearity
direction and a user-defined in cremen tal distance (a constraints.
certain number of pixels). In the other images the
A stand-alone version of the program is also available
previously determined position is used for the new start
whieh allows a higher execution sp~ed, since the time-
position. The edge tracking stops either after the
consuming visualisation is not used.
measurement of a user-specified number of edge points
or if matching fails, e.g. because of the end of an edge is Figure 4 shows the visualisation of the edge matching. In
reached. Other termination conditions can be formulated. the top row are the corresponding image regions around
The result of tracking is a polyline which approximates the matching point with the start and final positions and
the fuU edge. An additional advantage is the automatie the epipolar Une from the first image. The ffames and
determination of good initial values through which the crosses show the size and the position of the template and

template patch 1 patch 2 patch 3 patch 4 other patches

collinearity constraints
edge
constraint
I I
grey value matehing equations

pre-rotation

template 1 template 2 template 3 template 4

Figure 3 Edge matching as implemented

696
 epipolar line

final position initial position

Figure 4 Visualisation of the edge matching procedure using four images

the patch in the images. In the bottom row are the template defined by DIN 874 (German Industrial Standard). All
and the geometrically and radiometrically transformed points determined in object space should thus He on a
patches.
straight line defined by the knife edge.
The matching algorithm is only usable for fine
measurement, because the LSM requires good 4.1 Set-up
approximate values for the iteration process of the least
squares adjustment. The initial values consist of image To test of the algorithm a single camera/multiple frames
coordinate pairs of corresponding image points in a11
set-up was used (see Figure 5). A CCD camera was
images. The initial values must be delivered, e.g. by an
mounted on a optical bench construction and pointed
operator or automatically from a given CAD model.
Currently only the manual mode is implemented. The towards the object. The object was supported by a ro~1ting
images are displayed and the approximate image table, and fixed in a 3-D calibration'field which was used
coordinates of a corresponding point are measured with to determine the camera orientation. By rotating the table
the cursor. For measuring of more edge points the edge an arbitrary number of CCD frames could be produced.
tracking is used. The average puIl-in range is half a patch The calibration field consists of a black coloured plate (56
size. cm x 56 cm) and towers (10 and 30 cm height) with retro-
reflective targets. The ~'lrgets are illuminated with a
In a possible automatic mode the initial edge points in
specially designed fibre-optic lighting system around the
image space can be computed by resections of the model
edge point using the known camera orientation. In order to camera lens. The images are taken with a SONY XC77-
exclude those images in which the edge points are CE CCD camera with aSchneider XENOPLAN 1:1.7/17
invisible a hidden-surface algorithm must be applied. lens and a Datacube MAX-SCAN framegrabber. The
framegrabber is controlIed by a Sun-3E workstation.
From there the data is transferred via Ethernet to a
4. ACCURACY TEST
network of Sun workstations for further processing.
The primary aim of this test was to verify the accuracy of
the edge matching algorithm. For this it must be weIl The framegrabber uses the composite video signal for
understood that the accuracy of edge matching depends - digitisation. The effective image size is 592 (H) x 574 (V)
among other parameters (orientation, etc.) - on the pixels with a pixel spacing of 13.5 J.1m (H) and 11.0 J.1m
definition of the edge in object space, the contrast of the (V). At each camera position 4 images are acquired and
edge and the edge strength in image space and the their average is used for further processing.
amount of noise in the image data. Therefore it is of
utmost importance to have an almost ideal object edge for For the test 12 images are acquired: 8 images with a 4SO-
accuracy testing available and to control illumination and interval table rotation and 4 images with a 90' rotation of
imaging in an optimum way. the camera about its axis (K rotati'on). Figure 6 shows one
The test object is a short knife edge (125 mm) with an of the images used for orientation and edge matching. The
exactly polished cd ge for controlling the planarity of height-base-ratio is h/b = 8.81 and the average image scaIe
surfaces. The technical specification for the planarity is 1:44, i.e. 10 J.1m in object space correspond to 0.23 J.1m
tolcrance of the edges is 2.5 J.1m (probability P=95%), in image space or 0.017 pixel with 13.5 J.1m pixel spacing.

697
 CCD camera

diffuse diffuse
illumination fibre optic illumination

rotation axis
...
/,'"
...
./camera axis
calibration field on
......
rotation table
.....
"

Figure 5 Image acquisition arrangement for testing

can only be reached by a good illumination. Bad lighting

can fade features into the background, make edges fuzzy
and produce pseudo edges by shadows. The lighting
depends on the surface shape, surface texture, surface
reflectance and surface colour of the object. furthermore
on the physical constraints imposed by the manufacturing
environment, the light level and the light character Ce.g.
polarised light). The problemsof surface texture,
reflectance and colour can be avoided by a special
surface treatment. The knife edge has a dazzle-free edge
and the nozzle a special surface treatment.

In the accuracy test there are two objects to be measured:

the object and the targets of the calibration Held. The
illumination must be chosen such that both are weIl
illuminated. To reach this aim a different illumination for
each of the two objects has been used. The retro-
reflective targets need only a weak illumination. They are
illuminated with a specially designed fibre-optic lighting
system arranged around the camera lcns. The fibre-optie
system is adjustable and thus an optimal target
illumination can be selccted.

The use of diffuse lighting allows a uniform illumination

8 x zoom of edge of the object and produces little shadows and reflections.
The diffuse lighting is achieved by a white curtain around
the whole set-up and an illumination with high frequency
Figure 6 CCD frame wilh knife edge fluorescent tubes.

4.3 TEST RESULTS

4.2 Illumination
All tasks of image acquisition, target location, bundle
The chosen lighting conditions have a decisive influence block adjustment with self calibration and the edge
on the image quality. An optimal brightness and contrast matching werc performed with the program DEDIP.

698
The measurement of image target Iocation for orientation
was done with least squares template matching. The
camera orientation information was determined by a
bundle block adjustment, using 10 additional parameters
for compensating systematic image eITors. All 12 frames and also listed in Table 1.
were used. Minimum datum information was applied. The As evidenced by the RMS values of Table 1 the
adjustment gave the estimated image coordinate errors as deviations of the estimated knife edge points from an
Jlx = 0.50 J.lm (0.037 pixel) and J.ly = 0.611lm (0.055 pixel) adjusted straight line stay with 6 J.lm weIl within the
with a system redundancy of r = 430. 1: 25000 relative accuracy measure as far as the
horizontal distance (Y) is concerned. The respective Z-
The camera orientation parameters were introduced in the (depth-) values exceed this specification, but they are not
edge matching as deterministic. The cd ge matching was of interest in the original application example. The
done with 4, 6 and 8 frames respectively. For the edge estimated standard deviations of the related object
coordinates, with values of about 14.5 Ilm in Y and 28.0
tracking function a 3 pixel wide template ramp and a
Ilm in Z, are less precise, because they also include eITors
patch size of 5 x 5 pixels were used. 173 edge points were in the orientation of the CCD-frames and are dependent
measured. on the datum choice. Since in many inspection tasks the
accuracy of Iocation and orientation of measured object
A straight line was fitted through the object points features is only of minor interest, these absolute precision
projected onto the X-Y -plane and the X-Z-plane. The estimates are mostly of little value here.
differences between the observed coordinates and the
mean line (DXY, DXZ) are plotted in Figure 7 and their This accuracy test and other tests with different set-ups,
minimum and maximum values and root mean square camera configurations, templates etc. allow formulation
of some more general observations:
errors (RMS) derived (Table 1). Additionally, the
standard deviations of the determined object coordinates • If big differences between the template and the patches
are plotted and their average values derived as occur, e.g. at corners, junctions and occlusions, the

0.040

0.024
,.....,
E
E
'--'
N
U1 0.008
>-"
(/)

x
(/)

N -0.008 l,
I
x \ I I
I I
\ I I
0 ,\\ 1
\'111 I
x>-"
0
1,111'1
1'11
I'
I 1
f
f
\/'
IJ
,I
t J \, \1
-0.024 , \~
1\1
\I
DXY -
- - -- DXZ -
0 SX
0 SY 306 332 358 384 410
!::. SZ X [mm]

Figure 7 Edge matching results using the knife edge and six image frames. Differences DXY, DXZ
between estimated and straight line adjusted edge point positions. Standard deviations SX, SY,
SZ of estimated point locations.

699
 Table 1 Edge matching results (values in /.lm given in object space)

number RMS minimum maximum RMS minimum maximum

SY SZ
of frames DXY DXY DXY DXZ DXZ DXZ
4 5.8 -12.0 +14.8 15.4 13.1 ·40.1 +27.4 28.1
6 5.9 ·15.0 +11.4 13.6 13.5 -35.0 +25.3 28.3
8 6.6 -17.7 +14.9 14.3 12.5 -36.1 +39.5 27.8

matching faHs, because the adjustment solution does 6. ACKNOWLEDGEMENTS

not converge.
The software for this vision system was partially
• The use of more than 4 images does not result in a developed under a contract with Standard Aero Limited,
significant improvement of the matching results if the and in cooperation with NCR, Canadian Institute of
camera configuration is good. Industrial Technology, both Winnipeg, Canada. We are
• The number of iterations per matched point using the grateful for the support and cooperation.
edge tracking procedure depends on the curvature of
the edge and the tracking step width. For the examples
7. REFERENCES
with 4 images the average number of iterations was 3.
• The accuracy of LSM also depends on the design of Baltsavias, E. P., 1992: Multiphoto Geometrically Con-
the template. If a very steep edge template (grey level strained Matching. Ph. D. Dissertation, Mitteilungen Nr.
ramp, one pixel wide) is used, the radiometric 49, Institute of Geodesy and Photogrammetry, ETH Zu-
corrections will also cause the patch patterns to be rich.
compressed into very steep gradients. This leads to Gruen, A., 1985: Adaptive least squares correlation - a
inaccurate LSM results. powerful image matching technique. South African Jour-
• The use of a larger patch size (7 x 7 or 9 x 9 pixel) nal of Photogrammetry, Remote Sensing and Cartogra-
phy, 14 (3), pp. 175-187.
smooths the difference curve, because for the edge
matching more grey values are used along the edge. Gmen, A.W., Baltsavias. E.P., 1988: Geometrically Con-
However, the improvement in the results is not strained Multiphoto Matching. Photogrammetric Engi-
significant. neering and Remote Sensing, Vol. 54, No. 5. May 1988,
pp. 633-641.
The computing performance is 6.7 points/sec, measured Gruen, A., Stallmann. D., 1991: High accuracy edge
on a Sun 4/490 workstation and relates to an average matching with an extension of the MPGC-matching algo-
number of 4.3 iterations per matched point. The rithm. SPIE, Vol. 1526, Industrial Vision Metrology,
performance was measured with the pro gram version Winnipeg, 1991, pp. 42-55.
without screen display using 4 images. 652 object points Omen, A., Beyer, H., 1991: DIPS II - Turning a Standard
were thus matched in a total time of 97.3 sec. Computer Workstation into a Digital Photogrammetric
Station. ZPF - Zeitschrift für Photogrammetrie und
5. CONCLUSIONS Fernerkundung, 1, pp. 2-10.
EI-Hakim, S.F., 1990: Some solutions to vision dimension-
As evidenced by the controlled practical test presented in al metrology problems. SPIE, Vol. 1395, Close-Range
this paper our algorithm has the capability to measure Photogrammetry Meets Machine Vision, Zurich, pp. 480-
natural object edges with a relative accuracy of 1:25000. 487.
This translates into 0.023 pixel in image space and was
Raynor, J.M., Seitz, P., 1990: The technology and practical
obtained with standard image acquisition hardware and
problems of pixel-synchronous CCD data acquisition for
under non-optimized camera orientation conditions. A
optical metrology applications. SPIE, Vol. 1395. Close-
further improvement can be expected through the use of
Range Photogrammetry Meets Machine Vision, Zurich,
higher resolution CCD-imager chips, better signal
pp. 96-103.
transfer and digitisation hardware and more accurate
camera orientations.
However, in order to obtain such high accuracies a
number ofprerequisites must be met, like the existence of
a sufficiently well-defined object edge, an appropriate
illumination, and the usage of more than two CCD
frames. At this level, a elose-range vision system can
weIl compete with a CMM (Coordinate Measurement
Machine), having the additional advantages of high er
processing speed and non-contact measurement
characteristics.

700