03 01 FindPatterns Logic
03 01 FindPatterns Logic
Section 3 | Slide 2
The third section of the In-Sight Spreadsheets Standard training will focus on the Pattern and Logic At the end of this section Participants will be able to:
Tools.
- Apply the Property Sheet parameters and auto-inserted information for FindPatterns to a
sample image
- Identify uses for the PatMax technology
- Configure the PatMax Pattern tools
- Create basic mathematical formulas involving If, And, InRange and Not functions
1. Analyze Problem
1
2. Create Prototype Job 2
3
3. Design Operator Interface
e
3. Design Operator Interface 3. Block must have correct length (within tolerance)
- Decide how the results will be given to the user (visual, Ethernet, discrete, etc.) - Tool used: FindSegment
4. Complete & Deploy 4. Block must have correctly sized holes (within tolerance)
- Final details for longevity and ease of use - Tool used: ExtractBlobs
- Maintenance functions like back-up and restore
FindPatterns
In this section we will find a model using the FindPattern Tool. The goal of a vision application is to determine whether a part is good or bad, based on image
analysis of the part. You need to define as precisely as possible what distinguishes a good part from
a bad part in order to design an inspection system. Try listing all the criteria that must be met for the
part to pass, keeping in mind all the variations of “good” and “bad” parts.
Considerations:
- Which vision tools might you use for analysis? (Sometimes only one tool will work,
sometimes there will be a choice.)
- Do you require real world units of measure rather than pixels?
- How many cameras and which model(s) will you need to capture all of the required detail?
- Variations in color can affect how the part appears to the vision system.
- If the parts move, will you require strobed lighting?
- For parts whose position varies, how will you locate and fixture the part?
- How will you trigger your system to acquire an image?
- What outputs will your system need to send results to external hardware, for example, to
reject a part?
- How will the operator interact with the system?
Find Region
Model Region
(x, y)
Angle
When the part moves in the Field of View (FOV) the tools that inspect a certain area on the part must Pattern Match - Train a model part, followed by the model part searching at run time FindPatterns &
also move, this is done by “Fixturing” with a Locating tool to: PatMax®
- Use FindPatterns to Train and Search for a specific model or pattern in the image
- Determine (x, y) part location
- FindPatterns performs pattern-matching
- Determine rotation (angle)
- FindPatterns is available on all In-Sight vision systems at no additional cost
- Determine scale variations within ±10%
The x, y, and angle results will be used to locate the part for other tools.
To use a FindPatterns Function: To use a vision function, drag it from the Tool Palette and drop it into the cell in the spreadsheet.
- Leave blank rows at the top for building custom operator interface later
- Always insert descriptive comments using an apostrophe
Model Region
Graphics
Find Region
Graphics
A Property Sheet is used to configure In-Sight Explorer functions whenever a function returns a Model Region and Find Region graphics (red lines) can: move, resize, and rotate a region. The
structure or the function has many input parameters. mode depends on where you place the curser.
Every Property Sheet has a table that lists the default parameters, values and expressions of a The Regions define the area of the image where the Location or Inspection Tool will perform its
function. Each row of a Property Sheet consists of a parameter name and value, or a group of operation, and are also referred to as the Region of Interest (ROI).
expanded/collapsed parameters and values.
NOTE: The entire Model Region must be within the Find Region to be found.
The FindPatterns Property Sheet includes the following:
- Image – Reference to target image cell
- Fixture – Where tool should fixture itself
- Model Region – Region specifying model features
- Model Settings – Advanced settings (more…)
- Find Region – Region specifying searching area
- Number to Find – How many instances to look for
- Angle Range – ± rotation tolerance
- Scale Tolerance – Enable +/-10% size variations
- Thresh: Accept – Minimum score needed for match
- Thresh: Confuse – Minimum score needed for speed match
- Timeout – Max amount of time spent
- Show – Graphic options for display
TIP: Property Sheet parameters support drag-and-drop placement of their controls into the
spreadsheet. By selecting a parameter from the Property Sheet and then dragging that parameter to
the spreadsheet, the parameter’s label and edit control will be automatically created and referenced
in the spreadsheet.
Section 3 | Slide 11 Section 3 | Slide 12
How to Set Model or Find Region FindPatterns: Model Settings
1.
2.
3.
4.
To set the Model or Find Region: The Model Settings specifies the model training parameters.
The Area Model training creates patterns based on a uniform sampling of greyscale pixel values The Edge Model training creates patterns based on a sampling of points biased to the immediate
from the Model Region. The area model similarity metric is based on the normalized greyscale vicinity of the greyscale discontinuities typically found on object’s boundaries. The edge model
correlation coefficient. similarity metric is based on the normalized comparison of greyscale derivatives.
The Area Model may be used when: The Edge Model may be used when:
- Model Region is small - Back lighting
- No well-defined edges - Non-linear lighting changes, such as shiny metal parts
- Speed is critical - Significant changes in focus or background
NOTE: The maximum number of points that may be used to define a model is 4096. NOTE: The maximum number of points that may be used to define a model is 4096.
Coarseness specifies size of the smallest features in trained model. Accuracy specifies the tradeoff between accuracy/reliability and execution speed.
- 0 = Fine – Smallest features are approximately 4 pixels in size - 0 = Accurate – Specifies higher accuracy/reliability and slower execution speed
- 1 = Medium (default) – Smallest features are approximately 4 to 8 pixels in size - 1 = Medium (default) – Moderate accuracy and speed
- 2 = Coarse – Smallest features are larger than 8 pixels in size - 2 = Fast – Specifies lower accuracy/reliability and faster execution speed
Edge Tolerance – the maximum amount that an edge can move (± in pixels) from the trained
location and still give a strong response. If this tolerance is small, the pattern can be located more
accurately, but small steps must be taken during run time to ensure that the pattern is found, resulting
in a slower search. If this tolerance is large, the pattern can't be located as accurately, but larger
steps can be taken during run time, resulting in a faster search.
For Area Models, the Edge Tolerance numbers derived from Coarseness and Accuracy are stored
during training so that coarser or finer searching can be done, resulting in a small difference in search
speed.
For both types of models, an Accuracy setting of Fast causes the run-time to evaluate fewer similar
looking possible results than an Accuracy setting of Accurate. This also results in a small speed
difference.
0 100
No Yes
Phase 1: Phase 2:
- Uses lower resolution - Only runs if Phase 1 returns an insufficient number of valid results
- Fast - Uses full resolution
- Ignores candidates below Accept threshold - Evaluates only those candidates that scored between the Accept and Confusion thresholds
- Considers candidates above Confusion threshold to be matches
Fastest
This slide depicts how to make the tool use only Phase One or Phase Two. Results would only deal In this section we will find a model using the PatMax Tool.
with the accuracy of the ‘found’ pattern.
FindPatMaxPatterns
- Finds features in an image based
on the trained pattern
TrainPatMaxPattern
PatMax: Feature-based FindPatterns: Pixel grid-based
- Extracts and trains a pattern: a
trained geometric description of an
object you wish to find
- PatMax is a pattern-based search technology.
- The models in PatMax use geometric (feature based) representation;
FindPatterns is pixel grid-based. Feature based is more accurate.
A PatMax pattern is a collection of geometric features where each feature is a point on the boundary PatMax offers three key features that distinguish it from other pattern-location technologies in
between two regions of dissimilar pixel values. PatMax calculates geometric representations of machine vision:
curves. This makes it more accurate than FindPatterns, which approximates curves with line - High-speed location of objects whose appearance is rotated, scaled and/or stretched
segments that follow the pixel grid. PatMax can be more accurately rotated or scaled. - Location technology that is based on object shape, not greyscale values
- Very high accuracy
Situations that require the accuracy and reliability of the PatMax Patterns Tool: The TrainPatMaxPattern Property Sheet includes the following:
- When variations in lighting and reflections are difficult to control. For example, metal parts - Image – Reference to target image cell
can reflect light in random directions. - Fixture – Where tool should fixture itself
- When the pattern being inspected is similarly shaped or shaded in comparison to something - Pattern Region – Region specifying features to train
in the background, or the pattern is being overlapped or partially hidden by other objects in - External Region – Allows use of custom region
the image. - Pattern Origin – Location within pattern to report
- When you want to accurately recognize one type of pattern from other, similar, patterns. - Pattern Settings – Specialized settings
- When the conditions of your deployment environment are too demanding for the Patterns - Algorithm – Choose from PatQuick or PatMax
Location Tool to consistently and reliably. - Elasticity – Specifies allowed perimeter deviation
- Ignore Polarity – Toggles to check for opposite polarity
- Sensitivity – Disabled
- Coarse Granularity – Granularity used to find large features
- Fine Granularity – Granularity used to find small features
- Reuse Training Image – Model image is saved for later use (retrain)
- Timeout – Milliseconds before tool gives up
- Show – Graphic options to display
NOTE: Any time you make region changes to the TrainPatMaxPattern tool (directly or through an
EditRegion), the model is retrained on whatever is inside the training region at that time – even with
the “Reuse Training Image” selected. If that is not selected, any changes to TrainPatMaxPattern will
retrain the model.
PatQuick PatMax
Choose from two algorithms: Elasticity specifies the amount of non-linear perimeter deviation (0 to 10; 0 = default).
- 0 = No Tolerance
PatQuick - 1 or Greater = Flexible Boundary
- Optimized for speed
- Uses coarse granularity (details later) A linear change in a pattern is where the whole pattern changes in the same way. For example, if a
cross were twice as large all over, as the model, that would be a linear change, called Scale. But in
PatMax the cross at the right, only part of the cross has changed. For example, it could be made of rubber,
- Optimized for accuracy and the top section was pulled out at the corners. This is a non-linear change.
- Uses fine granularity (details later)
Increasing the elasticity parameter makes it more likely to find a part such as this. But if you increase
NOTE: The color of a trained feature (Green, Yellow and Red) represents the quality of the feature the elasticity value too much, you run the risk of a wrong feature being selected.
candidate for matching. Green represents high quality; red represents low. Yellow is intermediate.
The Granularity parameters should be adjusted until all trained features are green or you should
choose a better image to train. Distinct features (edges) and good image contrast will yield the best
results.
The Ignore Polarity defines whether or not found patterns can contain color-inverted match features, The FindPatMaxPatterns Property Sheet includes the following:
in respect to the Model pattern (default = Off, unchecked). When Ignore Polarity is applied (On,
Checked), detected patterns with color inverted features, such as black/white vs. white/black in the - Image –Reference to target image cell
Model pattern, for example, will be classified as matching the Model pattern. - Fixture – Where tool should fixture itself
- Find Region – Region specifying searching area
Ignore Polarity instructs PatMax to check for original polarity identified at training and opposite - External Region –Allows use of custom region
polarity as well. This may increase the execution time for PatMax. - Pattern – Reference to a TrainPatMaxPattern
- Number to Find –Number of match to search for
- Accept –Minimum score necessary
- Contrast – Specifies lowest contrast necessary
- Clutter in Score – Toggles consideration of extra features
- Outside Region –Percentage allowed out of region
- Find Tolerances – Sets rotation, scale & aspect ratio range
- Find Overlapping –Sets allowable overlapping
- Timeout – Milliseconds before tool gives up
- Show – Graphic options to display
Allowable rotation of
match relative to model
Accept Threshold defines the degree of similarity that must exist between the model pattern and Find Tolerances – Specifies the settings for finding patterns that are rotated or scaled with respect
the found pattern. The minimum acceptable score will vary depending upon the Location Tool to the trained pattern.
selected. Increasing the Accept value reduces the time required for the search.
- Angle Start – Specifies the angle at which to begin searching for matches, in degrees
PatMax uses this to determine if the match represents a valid instance of the model within the search counter-clockwise (-180 to 180, default = -15).
region. Increasing the Accept value reduces the time required for search. - Angle End – Specifies the angle at which to stop searching for matches, in degrees
counter-clockwise (-180 to 180; default = 15).
- Scale Start – Specifies the scale at which to begin searching for matches (1 to 10,000;
default = 100).
- Scale End – Specifies the scale at which to stop searching for matches (1 to 10,000;
default = 100).
- Aspect Ratio – Limits match-finding to uniform scale only or allows aspect ratio in X or Y, in
addition to a uniform scale.
NOTE: The Aspect Start and Aspect End parameters are disabled if Uniform Scale
Change is selected.
- 0 = Uniform Scale Change
- 1 = Uniform and X Change
- 2 = Uniform and Y Change
- Aspect Start – Specifies the minimum scale change, at which to begin searching for
matches, as a percentage of the trained pattern (1 to 10,000; default = 100).
- Aspect End – Specifies the maximum scale change, at which to stop searching for
matches, as a percentage of the trained pattern (1 to 10,000; default = 100).
A B
Structure
GetRow($A$2,B2)
GetCol($A$2,B2)
Cross GetAngle($A$2,B2)
Tool Structure Auto-inserted Functions
GetScale($A$2,B2)
GetScore($A$2,B2)
Results: This slide shows the formula that is entered into each cell when the tool is added to the spreadsheet.
- Tool Structure – Holds all the result information returned by the tool.
- Cross – Indicates the center of the model found.
- Auto-inserted Functions – Are the most commonly needed information, they pull the data
out of the Structure.
In this section we will find a model using the PatMax RedLine Tool. PatMax Redline has been designed from the ground up to be optimized for speed on the newer In-
Sight 5 megapixel models. It is not based on PatMax. PatMax Redline is typically 3 – 7X faster than
PatMax, and is sometimes even faster. It allows for both speed in pattern matching and high
resolution (accuracy).
PatMax RedLine 150 ms (3.2x faster) PatMax RedLine 111 ms (4.9x faster) PatMax RedLine 31 ms (6.8x faster)
Newer In-Sight models, such as:
Requirements for PatMax RedLine: PatMax Redline is generally 4-7 times faster than PatMax, but may be even faster. The speeds
- Newer In-Sight models , such as 57xx, 78xx, and 8xxx series depend on the In-Sight camera model, the nature of the part itself, and the settings in the Property
- Spreadsheet Mode: In-Sight Explorer and firmware version 5.1.0 or higher Sheet, including Model Region, Find Region, and Angle of Rotation.
- EasyBuilder Mode: In-Sight Explorer and firmware version 5.2.0 or higher
NOTE: These timings were obtained on an In-Sight 5705.
PatMax RedLine is not supported by the following:
- In-Sight 51xx, 54xx, 56xx models
- In-Sight 74xx models
- In-Sight Micro 1xxx models
Demo TrainPatMaxRedLine
FindPatMaxRedLine
The demo In-Sight job (PatMaxRedLine_demo) allows you to compare timings between PatMax and PatMax RedLine consists of two functions: TrainPatMaxRedLine and FindPatMaxRedLine.
PatMax RedLine, using an In-Sight camera that has both tools. (The comparison of timings is not NOTE: These functions are only available on In-Sight vision systems running In-Sight firmware 5.1.0
useful on the Emulator, since relative behavior between the two tools might not represent and later.
comparisons on actual cameras.)
- TrainPatMaxRedLine – Extracts and trains a pattern from an image for use with the
To run the demo: FindPatMaxRedLine function.
1. Use a camera that has PatMax RedLine. Position a part under the camera and adjust NOTE: A trained pattern consumes approximately 1Mb when using the default pattern Region
exposure and focus to get a good image. Open the demo job and go into its Custom View. (320 X 440), although the pattern size varies greatly depending on the size of the trained
2. Set up the Pattern Region. This will be used for both PatMax and PatMax RedLine. pattern.
3. Click on Train Pattern. This will create separate models for PatMax and PatMax RedLine,
since they use different algorithms for their models. - FindPatMaxRedLine – Finds objects in an image based on a trained pattern.
4. Set up the Find Region. This will be used for both PatMax and PatMax RedLine. FindPatMaxRedLine finds patterns within an image based on a trained pattern from a Patterns
5. Run the PatMax portion of the spreadsheet by clicking on the PATMAX logo in the Custom structure.
View, then click on the Acquire button. This will display the job time for PatMax.
6. Click on the PATMAX REDLINE logo in the Custom View, then click on the Acquire
button. This will display the job time for PatMax RedLine. Compare with PatMax.
7. Try changing other parameters in the Custom View and repeating steps 5 and 6 to
compare times. For example, increasing the Angle should increase the ratio of speed
between the two tools.
8. Note that if you change a model’s parameter for PATMAX REDLINE, its first timing after
that will be much higher due to an optimization that is done at that point.
Moving from PatMax to PatMax RedLine : PatMax RedLine matches tend to score higher than PatMax. Therefore, it may be necessary to use
a higher Accept Threshold for PatMax RedLine compared to PatMax.
- Existing job files using PatMax can be converted to PatMax RedLine
- Load job on supported 5 MP system (5705, 5705C or Micro 8405) PatMax RedLine contrast is a relative measure of contrast change between features in the trained
- Replace TrainPatMaxPattern function with TrainPatMaxRedLine pattern versus target features in the search image. PatMax contrast is an absolute measure of
- Retrain the pattern using PatMax RedLine (PatMax RedLine can not use PatMax contrast for target features in the search image.
patterns)
- Replace FindPatMaxPatterns with FindPatMaxRedLine In PatMax RedLine, an automatic (re)optimization occurs following any change to Angle/Scale Find
Tolerances or search ROI. This includes optimization at every trigger when PatMax RedLine is
- However, there are some differences in the Property Sheet parameters and how they work. fixtured. This can make the tool run slow if it is fixtured. However, PatMax RedLine is so much
- Some parameters are not present in both tools faster than PatMax for a 5MP image that a fixture is no longer needed in most cases.
- Some parameters are in both tools, but behave differently
PatMax RedLine Find Tolerances are strict, meaning the tool will fail if a limit is exceeded. PatMax
Find Tolerances are permissive, meaning there is some allowance beyond the limits. Therefore, if
PatMax RedLine does not find matches that PatMax finds, it may be necessary to widen the Angle
and Scale Find Tolerances in PatMax RedLine.
PatMax
FIndPatterns PatMax
RedLine
~$750
Additional Price included ~$500
(includes PatMax)
Differences: Take a moment to review the differences between the Pattern Matching tools that are available in the
various In-Sight vision systems. Comparisons of speed are general, and may not apply to every
- Fixtured search region – In versions 5.4 and earlier, PatMax RedLine’s automatic search situation. The best way to compare speeds is to measure them for your setup.
optimization may cause it to run slower than PatMax if the search region is fixtured,
especially when searching for multiple targets on a confusing image background.
- Variable aspect ratio, perspective or skew – PatMax RedLine does not yet support X/Y
aspect ratio changes, perspective changes, or skew in target compared to trained pattern
- Non-linear deviations along perimeter – PatMax RedLine does not yet have an Elasticity
parameter, and may be less tolerant of deviations along the target perimeter compared to
PatMax
- Filtering match results at approximately the same X/Y location by Angle and Scale
Overlap – PatMax RedLine cannot filter results based on angle and/or scale differences
between two or more matches found at the same X/Y location (as allowed by XY Overlap
setting).
Logic Tools
In this section we will review many of the Mathematical Logic Tools. In-Sight has an extensive set of Vision, Mathematical, and other types of functions. The ultimate goal
is to make a decision about an object being inspected using the information returned by the vision
tools.
- Logic
- Lookup
- Math
- Statistics
- Trigonometry
Logic functions provide Boolean algebra and conditional testing capabilities in the In-Sight
spreadsheet. All bitwise functions perform the corresponding logical operation on each bit position in
the binary representation of the arguments and return the resulting binary number in decimal form.
Example: Example:
A1 = 23, B1 = 10, C1 = 22 B7 = 92
F1 = InRange (C1, B1, A1) D1 = If(Not(B7>95), 1, 0)
Ø F1 contains 1 Ø D1 contains 1
Example: Example:
A1 = 23, B1 = 10, C1 = 22 B7 = 92
F1 = InRange (C1, B1, A1) D1 = If(Not(B7>95), 1, 0)
- F1 contains 1 - D1 contains 1
A2 = 6
B2 = 244.5
What is in these cells?
F1 = Not(InRange(A2, 0, B2))