Fusion Testing
Maximizing Test Execution
By: James Tischart
About Me
Director - Product Delivery SaaS at McAfee
15+ years experience in testing and engineering
Multiple certifications in Agile and Testing
Passionate about Testing as an eng discipline, a
science and an art
Continue to challenge the status quo, test new
approaches and always strive to improve
Support the legitimacy of the testing professional in
the broader engineering world
What is Fusion Testing?
An occurrence that involves the production of a union
Organization of structured testing
Freedom of exploratory testing
Rigors of automated testing
Combined into one test methodology
To achieve
Maximize code execution
Increased test coverage
Reduced test artifact and documentation
Higher quality for users
Improved data for organization
Fusion
Focus start your day with 15 minutes of thought
Usage how will your users work with the system
Scope decide on the scope of everything
Initiate just go and explore
Organize create a plan & be ready to deviate from it
Note keep track of your exploration to retrace steps
Fusion Testing Guidance
Maximize test execution with Fusion by:
1.
Identify Goals for Testing
2.
Choose the right mix of methodologies
3.
Utilize Tests Lists to guide exploration
4.
Automate the Right Things
5.
Document at the Right Detail
Fusion Implementation
1. Guide your team through the change
1. Create a Change Message
1. Mentor your team members
2. Train the team in testing and engineering
3. Create Metrics to measure success & failure
Fusion & Planning
Daily Plan
Release
Iterative
Test
TestPlan
Plan
Very
Set
Individually
the
effective
goals created
for
in the
Agile
overall
each
implementations
morning
release
Break
Plan
Provides
the
uptest
guidance
theenvironment
testingand
into
goals
iterations
for the day
Low
Organize
metrics
performance
Details
level
thethe
details
execution
for and
the
plantime
frame
Keep to two weeks or less
Detail
Plan
a budget,
and a time frame
Test Lists
the resources
testing to accomplish
A grouping
Detail the
of test
performance
ideas
tests
Plan
List
yourwhat
automation
will notand
be tested
tool set
Nospecifics
Identify risks
in thetotest
thelist
plan
Plan
Plan
at a high
whatlevel
Regression Testing
Priority,
Identify
variations
what test-lists
& guide need
for exploration
to be written
The Power of Many
Utilize different users of your system to bring fresh perspectives
- Support personnel
- Developers
- User groups
- Sales staff
- Training
- Product Management
Help guide the testing by providing checklists, environments and goals
Dont give exact details or specific steps this minimizes innovation
Make the event fun and you will have many people continuing to help
Have testing experts available to help the volunteers for problems
Identify Defect Traits
Next
For new
analyze
products
Sev 1use
internally
historical
found
traitsand
from
fixed
similar
issues
projects
and determine the traits
Analyze
Inform
Severity
the developers
1 Customer
& testers
reported
of incidents
the traits
Create
Determine
test lists
the trends
or testtoideas
those
todefects
cover these
in coding
traits & testing
Repeat
Constrain
thethe
process
review
with
to aeach
calendar
severity
yearlevel
of defects
you use
Fusion & Automation
Determine your layers for automation:
Server
Databases, server processes
Machine interaction points
Process interaction with other
layers
Middleware/API layers
User Interface
Optimize your test artifacts
Dont duplicate test-cases
Use automation for
traceability.
Test Results & Metrics
Since you cant test everything, here are some ideas of results
to report:
Performance/Benchmark
#There
of Test
will
Ideas
always
Executed
be interest
based
comparisons
in
onthe
Priority
test
by build/iteration/release
data no matter the test structure.
Team
#Q:ofHow
Test
Quality
can
Cases
you
Satisfaction
passed/failed
accuratelyRating
assess
versus
quality
get
total
thecoverage
when
gut feeling
thethat
testing
ofthe
thetests
combinations
team
represent
exceed the particles of the universe?
Open
# of Automated
defects to Tests
highlight
Passed/Failed
potential issues
by # of
that
executed
your customers
times may find
A: You cant! We need to present what was tested, what has not been tested
Go
and to
support
fusiontesting.blogspot.com
the assessment used to for
make
more
this details
prioritization.
on these metrics
Traceability Matrices
Do
Positives
Dont
Negatives
Can
tracetest
tests
to requirement
Tie your
lists
to requirements or
coverage
stories
Time
to create
and
Traceconsuming
to the test case
or test
idea
maintain
Constrain your traceability to the
Displays
what has been executed
feature level
Try to trace to the functional level
Often out of date and misleading
Build traceability into the automated
Shows relationship between tests and
tests
features
Duplicate information in many
Duplicates information from test
documents
cases and requirements
Map
automation
to test lists to
or test
Provides
defect traceability
specs
features
Over complicate
maintenance
Dedication
to frequent
updatesof
multiple sources
needed
Implementation Challenges
Challenge: Management
Identify the
Respond
to the
Challenges:
Challenges:
Provide
Reliancebetter
on historical
metrics metrics
Show
Understand
the current
how new
processprocesses
improves and
quality
practices
Prove how decision can be made faster
Decision-making timelines
Review traceability needs and support them
Need traceability to feel confident of data
Challenge: PMO
Identify the
Respond
to the
Challenges:
Challenges:
Debunk
Reliancemisleading
on historical
metrics
metrics
Show
Struggle
how
with
more
thetesting
detailsprovides
of testingbetter data
Prove
Need predictability
reduced costsfor
with
costs
efficiency
and schedules
gains
Track
Require
lost
improved
opportunity
timecost
to market
on test artifacts
Challenge: Engineering
Identify the
Challenges:
Respond
to the
Challenges:
Takes
time
awayincreases
from coding
Test first
design
new coding time
Testing
can &
beexploratory
tedious
Automated
tests are less tedious
Not
their specialization
Understanding
testing improves code writing
Rely
on a serial
testing
Less defects
willapproach
be loggedtowith
up-front tests
Challenge: Regulation
Identify the
Respond
to the
Challenges:
Challenges:
Lean
Documentation
towards more
Requirements
structured testing
Use
Formal
exploratory
or Standards
but document
Approval results
Plan
Full traceability
for and work in shorter iterations
Automate
Rigorous Automation
more, document
& detailed
more test results
Challenge: Outsourcing
Identify the
Respond
to the
Challenges:
Challenges:
Phase
Require
in more
Fusion
instruction
and train and
on exploratory
detail
Provide
Lack expertise
more details
in the in
product
documentation
and markets
Implement
Multi-cultural
multiple
communication
communication paths
Support
Formal process
and train
& new
document
processes
to support work
Ensuring Success
Implement
in on
stages
Monitor
Adjust based
the needs of the project
1.Measure
the
impact
to in
customers
(before
& after)
1.Increases
1.One project
or may
decreases
require
more
execution
precise
time
documentation
2.Collect
a project
for
execution the
timetest
before
implementation
2.Review
2.Anothermetrics
documentation
projectfor
may
require
size closely
lesstest
structure
reviewing
artifacts
3.Introduce
Fusion
to cover
about
3.Measure
3.Identify these
the
amount
requirements
of direct
at automation
the25%
startofoftesting
projects and adjust
4.Increase
Fusion
to 50%
to further
time
4.Review
4.Reduce the
with
amount
theusage
team
of the
time
amount
spent
on
of artifact
timeimprove
spent
management
onexecution
maintenance
5.Continue
to phase
inimprove
until
75exploration
to 90%
testing
is Fusion
5.Continuously
5.Inform people
try
of to
the
time impacts
of of
data
time
collection
and
increase efficiency
Five Keys to Fusion
1. Detail the right level of detail at the right time
2. Planning consistently plan, execute and adjust
3. Automate - spend time on specifics in automation
4. Report measure what truly represents the team and customer
5. POM get more people involved to expand the breadth of test
Conclusion
Rick Craig
James Bach
My family, friends, colleagues
who have supported me!
QA&Test
Hung Nyugen
Cem Kaner
