MODULE 5

Robotic vision system - Image

acquisition: Vidicon, charge
cou 1p led device (CCD) and charge
injection device (CID) cameras
Image processing techniques:
histogram processing: sliding,
stretching,
equalization and thresholding.
 Robotic vision system
• Robot vision (machine vision) may be defined a.'> the process of
extracting.characterising and interpreting informat ion from images
of a three dimens ional wortd.
• In human vis ion system, eyes senses the images and brain analyses
the infonnation and takes action on the basis of analysis.
• Similarly in robotic vision system. visual sensing of infonnation is
done after fonning an image of the object and infonnati on is
analyzed to make a useful decision about its content.
• Robotic vision (machine vision )system thus needs both visual
sensing and interpretive capabilities.
• Image sensing device may consist-. of vidicon camera or a charge
coupled device(C CD) camera or charge injection device (CID)
camera.
 • Image sensing device receives light through a lens system and
converts this light into electrical signals.
• Micro-computers are used to refine and analyze these electrical
signals to provide an interpretation of the object.
• Machine vision system is ideally suited for in-process gauging and
provides timely corrective actions to control the process.
• Machine vision systems are used for functions like eaueme or
dimensions, identification of shapes, measurement or distances,
determining orientation of part~ quantifying motion, detecting
surface shading etc.
• A big advantage of machine vision systems is that they can be
designed to operate for wavelength in ultraviolet and infrared
ranges to which human eye is insensitive.
• It is a noncontact measuring technique and thus suited for fragile
work pieces and which would distort under measuring force and
would get contaminated by touch.

Steps of machine vision process

• The machine vision system involves follow ing four basic steps :
(i) Image formation
(ii)Processing of image in a form suitable for analysis by computer
(iil)Defining and analyzing the characteristics of image
(iv)lnterpretatlon of Image and decision making.

I. l!UiC teqllkillOtl 2. lfflllt procelliq l . llllefpl'elation

lllld ■llal)iM
.. .ad dipkalloe

O.itiuuAc.

Up• o
tOutce I
Dedllo•
end
I aaloon,
\ I
\ I
,..,,.\
\ : ,'
\ I/
I

-t>b O~O Q -
App,c,Mion
tLY ,
{i) Image Acquisition (l,mage Formation}

• Image acquisition is the creation of digital images,

typically from a physical scene.
• The first link in the vision chain is the camera. It
plays the role of robotic eye or the sensor.
•The visual information is converted in to electric
signals in the camera and when sampled spatially
and quantized ,these signals give a digital image in
real time by a process is called digitizing.

• ligh1ing for camera: Dacie lighting is suiled when a image is required 10 obiain
maximum image contrast. Front lighting is used when
certain key fcalUrcs on the surface of the object arc to be
inspocllc..~ .

l l • t -ces
Back lighting Front liahtina

• An imase sensor like ,•idicon camera. CCO or CID camera is used 10 generate the
electronic signal representing the image.
• The image sensor collects light from the scene through a lens and using a photosensitive
target. convens it into electronic signal.
• Most image sensors generate signals representing two-dimensional arrays (scans of the
entire image).
 Vidicon camera

Vldlcon Clffllfl tube

~
' oonduc:IDrr
pholOrttisht
~ ·~
beam
gllHttaltd
vaaunW..

-----c1l1 ------
OU~
vldeolignal -- oc po.er -,.,ply
~ 2002 £ncyclo,-di1 S-rit1nnlt1, ku,,
----------------

6..\. \ 'IDICON CMUCIIA (,\.'(ALOG CAMIUIA)

I Advwnaaes DIAdvanQp5
• Is more reliable. • Doesn't break the image in Pixel as CCD camera.
• Low cost. • Analog output
• More flexible. • Imposes constraints on System designer
I• Grid and pixel shape can be Varied.

"·"''
• Used in closed circuit television systems, can be used for machine
vision systems.
• In it, an image is formed by focusing the incoming light through a
series of lenses on a transparent conductor coated with a photo
resistive material.
• Photo resistive material is composed of a large number of tiny
photo resistive elements.
• The resistance of elements decreases with increasing illumination.
• Once the image forms on the photo conductor, a charge is
accumulated which is the function of intensity of the impinging light
over a specified time.
• The charge built up is read by scanning the photo sensitive layer by
focused electron beam by electron gun, creating a video signal that
represents the varying amounts of light in the image.
• It provides a great deal of information of a scene at very fast
speeds. However they tend to distort their image due to their
construction and are subject to image burn-in on the photo
conductive surface.
• These are also susceptible to damage
... ' by shock and vibration
~

Solid state camera(Digital camera)

• These arc commonly used in machine vision systems.
• Th~ employ charge coupled dcviGC(CCD) or charge injected device (CID)image sc:nsors.
• They contain matrix or linear amy of small, acc.ura1ely spaced photo J.cnsi1ive element!\
fabricated on silicon chips using integrated circuit technology.
• Each detector converts the l~gh1 falling on it through the camera lens, in to analog elcc1ric
signal corresponding to light intensity.
• The entire image is thus broken down in to an array of individual pit-1ure dement,;(pixels).
• Solid state cameras are smaller, rugged and their sensors do not wear out with use.
• They exhibit less image distonion because of accurate placcmen11 of the photo detectors.
• CCD and CID differ primarily in how the "·oltagcs arc cxrractcd from the sensors

CCD IMAGE CID IMAGE

 6.4..1. CCD

* The other approach tor o bcaining a digitlz.ed i.q,age is by lhe use of

the Cha rae Couplet.I I.Jcvice (CCD) .

In this tocbnology the image is projected by a vldco c amera onto the

* CCD whic h dete~ta. jlt.o rea and read& out the accumulated c harge
gcnc,J'Bted by lhc Ught on
. each portion of me imlllF•

* . .a.~--
Ught deeecb o n oocun uu"':'6"•
the __ ,,....,ion of lqJhl on • pbolO
- r
conductive s ubsttate (e .g . $lhcon).

* Charges accumuJ--
_.... under nndtivc control electrode& in •solaCCd wellt
r - --
due to vo ltases applied to the central e lc:c:trodea.

* cl
Each isolated well rcpre!COO one p1'L w
hid, can he u-anslerrcd to

output ,noraae regifler by vuyin• tbe voltage OD

the rnetal cc,nllOI

electrode. which is shown in die ......,. 6.2(a). (I»).

• A CCD is a mollaic of tiny light !ICTlsitive detectors culled pixels or 'photosites'.

• The pixels att ammgcd as a flat n:ctungular 51.1rfacc onto which an image is projected using a
~mera or telescope lens.
• Each pixel accumulates an electrical charge depending on the amount of light fall ing uron it.
• When an image is captured, the e lectrical charge from each pixel is mca~1.1rcd
and converted to a num~-r (digitiw d) by the clc.-ctronic circuiti. within the camma.

ADVANTAGES DISADVANTAGES
• More sensitive at low light. • Since Scilicon processing is still new
• Prod1Kes better image even in extreme variation in science, so there seem5 to be variation
brightness In light sen5itivity from pixel to pixel.
• Small size
• Longer life
• les.s power consumption
 6A.l. Cbarp lnJection Devka (CID)

* CID c amera i, .a type or image c apturing device whic h u1Ulus 1111

im.asc scnscw 10 re,aliue.r viJ1iblc liah1 as an electro nic signal. These
C ID camu.u do no« we pho10-chcrnlc■I Olm lO capcure s11lls o r vldeu,
ln11cid lhe elc:ct.ronic ai,nal Is tt:«>rdcd 10 either ■ remotely connected
dcvk.-e or an internal memory.

* Operadon: When the camer:a'a record func tion is initiated, the

c-.amer.a's lens focus.rs lish• thnlua), the camc:r. apcnture, light filters
~ 0 •n~ the dcctrook bwigc sensor. TIN, tfna8e ;1cn,-or is a;~
•n • arid pattern. with each individual 9 q ~ calJed u
1
tnase ~ c.annoc determine the colour or liaht recorded..

,
•V

90 . -
\G,@(!) I
0

'1;"t: ), _J_1.i_J.i_ ,t J-5 J,

-
0 • V O O O

! \ _ _ ,,__
Ieea
0

r~
ltM•c~- • - -
£ De«--••
----
l<I
1,dHl ••---....t - - clt"'iid

1 <1• •- --~ nu,..,,_, ,..

u-.,.m..,.._,,.,~ . .. .,~le'd---

-
, t.... wtt._,, c,o1 _..·1w,#
"-d to flo¥1

I•

CID camera

• In CID , array of photosensitive capacitor elements are arranged.

• Each pixel of the CID image sensor is arranged over two intersecting
capacitors.
• After the intensity of the light is read, it is stored on the sensor.
• Unlike Charge Coupled Device (CCD) cameras which transfer collected
cba~e out of the pixel during readout (aod hence erase the ~~e stored
on tlie sensor), charge does not transfer from site to site in the CID array.
• Instead, a displacement current proportional to the stored si~al charge is
read when charge '"packets" are shifted between capacitors within
individually selected pixels.
• S_incc the image stays on the sensor, a displacement current equal to the
signal on the sensor 1s created.
• The displaccmc..'11t currc.."lll is n."ad when the charge is shifted lx.-twccn
capacitors, and is then amplified and converted to a voltage.
• This voltage is the c.amera output which is created as a video signal. Since
the image 1s still on the CID image sensor, this image is non-destructive.
• Allows accurate exposure even in low light scenarios

I
(ll) Image Processing:
• An image is an array or a matrix of square pixels(picture elements)
arranged in columns and rows.
• Images are obtained using various image acquisition devices like
cameras.
• The series of voltage levels available on detectors representing light
intensities over the area of the image.
• An analog to digital converter is used to convert analog voltage of
each detector into digital value.
• After acquiring the images they arc stored in computer memory as
digitized image.

• Acquired images ,viii be having errors during their acquisition .

• If we remove these errors we will get clear images.
• The removal of errors for making the image more bright and clear
is done by image processing.
• A camera may typically fonn an image 30 times per sec.
• At each time interval the entire image has to be captured and frozen
for processing by an image processor.
• Image processing is a method to convert an image in to digital
form and perfo·rm some operations on it,inorder to get an
enhanced image or to extract some useful information from it.

,,.,.,
• If voltage level for each pixel is given either O or 1 value depending
on some threshold value, it is called Binary System.
• On the other hand Gray scale system assigns up to 256 different
values depending, on intensity to each pixel. Thus in addition to black
and white (binary system), many different shades of gray can be
distinguished. This thus permits comparison of objects on the basis
of surface characteristics like texture, color, orientation, etc., all of
which produce subtle variations in light intensity distributions.
• Gray scale systems are used in applications requiring higher degree
of image refinement.
• For simple inspection tasks, binary system may serve the purpose.
• It may be appreciated that gray-scale system requires huge storage
processing capability It is, therefore, essential that some means be
used to reduce the amount of data to be processed.

I'

Binary image Gray scale Image

Histogram Processing:
• Histogram is a graphical representation showing a visual
impression of the distribution of data. It is nothing but a graph that
shows frequency of occurrence of data.
• The histogram of a digital image with gray levels in the range (0, L-
I] is a discrete function h(rk) = n1t, where rk is the k th gray level and
nk is the number of pixels in the image having gray level r1c.
L=maximum value of gray level (eg.256).
Important types of histogram processing are:
• Histogram equalization
• Contrast Stretching or Histogram stretching
• Histogram sliding
• Histogram Thresholding

i)Histogram equalization:
Histogram equalization is n technique for adjusting i1nage
intensities to enhance contrast by equalizing all the pixd values
of an i1nage.it results a high contrast image.

ILli
 Histogram Equalization
Example

6 - --
2 3 3 2 5 - --
4 2 4 3 4 - --

3 2 3 5 3 - --
2 _,_
2 4 2 4

◄x4 1mage
1 ---
I
I , , ~~Y leve
I
' ' ' ' '
0 1 2 3 4 5 6 7 8 9
Gray scale = (0, 9]
histogram

...... ••

Example

Gray
0 1 2 3 4 5 6 7 8 9
LevelU)
No. of 0 0 6 5 4 1 0 0 0 0
pixels
'
L", 0 0 6 11 15 16 16 16 16 16
,~1
6 11 15 16 16 16 16 16
.r-r-
' n
,
/ JJ •
0 0 I I I I I I I I
16 16 16 16 16 16 16 16
3.3 6.1 8.4
sx9 0 0 9 9 9 9 9
~3 ~6 ~s
ii) Contrast Stretching or Histogram stretching
• Contrast stretching aims to increase (expand) the dynamic
range of an image. It transfonns the gray levels in the range
(0, 1... L-1) by a piecewise linear function.

'

<
c-
(q _-
. ,~
_ c,oar,.....,.
_

iv)Histogram sliding:
• In histogram sliding, we just simply shift a complete
histogram rightwards or leftwards. Due to shifting
or sliding of histogram towards right or left, a clear
change can be seen in the image.
• We can Increase brightness using histogram sliding
• Histogram of this image has been shown below.
• In order to bright it, we will slide its histogram
towards right, or towards whiter portion.
• By applying desired sliding transformation in order
to change the bri~htness, the histogram of the image
shifted towards nght or left
• The imllgc and its histogram is shown below.
• Lets compare these two images and their histograms to sec that what change have
to got

A fter h istogr am 5Ddlng

f1pn6.l:H....,._Slidiaa

iii)Histogram Thresholding:
• It is a type of Image Segmentation
• Segmentation divides an image into its constituent regions or
objects.
• The goal of segmentation is to simplify and/or change the
representation of an image into something that is more
meaningful and easier to analyze.
• Image thresholding is a simple, yet effective, way of
partitioning an image into a foreground and background
• Thresholding is a limited case of contrast stretching, it
produces a two-level (binary) image.
• Thresholding is a non-linear operation that converts a gray-
scale image into a binary image where the two levels are
assigned to pixels that are below or above the specified
threshold value.
• Thresholding creates binary Images from grey-level ones by turning all pixels

below some threshold to zero and all pixels above that threshold to one.

• The drawback of this algorithm is that it will always segment the image in to

two categories.

o Edge detect ion is an Image processing technique for finding the boundaries of

objects within Images.

• It works by detecting discontinuities in brightness.

• Edge detection is used in areas such as image processing, computer

vision, and machine vision

❖ Once we have computed a measure of edge

strength (typically the gradient magnitude), the

next stage is to apply a threshold, to decide

-------------
whether edges are present or not at an image t-t.....,

point.

❖ The lower the thre.shold, the more edges will be

detected, and the result will be increasingly - - - - - - - - - - - - -

susceptible to noise and detecting edges of

Irrelevant features in the Image.

❖ Conversely a high threshold may miss subtle

edges, or result in fragmented edges. ''

(iii) Image Analysis:
• Digital image of the object fonned is analyzed in the central
processing unit of the system to draw conclusions and make
decisions.
• Analysis is done by describing and n1easurin~ the properties of
several image features which may belong to either regions of the
image or the image as a whole.
• Analysis is carried for describing the position of the object, its
geometric configuration, distribution of light intensity over its
visible surface, etc.
• Three important tasks perfonned by machine vision systems are :
measuring the distance of an object from a vision system camera,
detennining object orientation, and defining object position.
• The distance of an object from a vision system ca1nera can be
detennined by stadimetry or by triangulation technique.
• The object orientation can be detennined bY. the methods of
conncctmg of three points ,light intensity distribution , structured
light method. .~ ,

(iv) Image Interpretation:

• Image interpretation involves identification of an object based on

recognition of its image.
• Various conclusions are drawn by comparing the results of the
analysis with a pre stored set of standard criteria.
• In a binary system, the image is segmented or windowed on the
basis of clustering of white and black pixels. Then all groups of
black pixels within each segment (called blocks) and groups of
white pixels (called holes) are counted and total quantity is
compared with expected numbers to determine how closely the real
image matches the standard image.
• The complex images can be interpreted by use of gray-scale
interpretation technique and by the use of various algorithms.

lL"f'