BPT 1
BPT 1
"Computers are useless. They can only give you answers." – Pablo Picasso
(1881—1973)
Performance testing is the discipline concerned with determining and reporting the
current performance of a software application under various parameters. My series
"User Experience, Not Metrics" discusses and demonstrates how to do it, with the
help of the Rational Suite® TestStudio® system testing tool. Computers excel at this
stuff, crunching numbers and displaying answers in neat ways. But there comes a
time after the tests are run when someone who’s reviewing the results asks the
deceptively simple question, "So what, exactly, does all this mean?!?" This point
beyond performance testing is where the capabilities of the human brain come in
handy.
"Computers are good at swift, accurate computation and at storing great masses of
information. The brain, on the other hand, is not as efficient a number cruncher and
its memory is often highly fallible; a basic inexactness is built into its design. The
brain’s strong point is its flexibility. It is unsurpassed at making shrewd guesses and
at grasping the total meaning of information presented to it," writes British journalist
Jeremy Campbell in Chapter 16 of Grammatical Man: Information, Entropy,
Language, and Life (1982).
"Beyond Performance Testing" will address what happens after initial test results are
collected, the part it takes a human brain to accomplish. We’ll explore what
performance test results mean and what can be done to improve them. We’ll focus on
performance tuning, the subset of performance engineering that complements
performance testing. We’ll examine the process by which software is iteratively tested
using Rational Suite TestStudio and tuned with the intent of achieving desired
performance, by following an industry-leading performance engineering methodology
that complements the Rational Unified Process® approach.
• Speed – Does the application respond quickly enough for the intended users?
• Scalability – Will the application handle the expected user load and beyond?
• Stability – Is the application stable under expected and unexpected user loads?
The engineering part comes in as soon as the first measurement is taken and you start trying to figure out how
to get from that measurement to the desired performance. That’s really what engineering is all about: solving a
problem to achieve a desired and beneficial outcome. As a civil engineering student at Virginia Tech in the early
1990s, I was taught an approach to solving engineering problems that can be summarized as follows:
Or as my father used to say, "What have you got? What do you want? How do you get there?" In civil
engineering, the implied problem was always "build or modify a structure to satisfy the given need." In
performance engineering the implied problem may seem different, but it’s fundamentally the same – that is,
"build or modify a computer system to satisfy the given performance need."
With that said, I suspect you would agree that performance is the trait of the system that we wish to engineer,
thus performance engineering.
• Part 5: Determining the Root Cause of Script Failures – The obvious symptom is very rarely the
actual cause of a script failure. It’s imperative to determine if the failure is due to the script or the
application and be able to quantify the root cause, not just the symptoms.
• Part 6: Interpreting Scatter Charts – Scatter charts are the most powerful evaluation tool at a
performance engineer’s disposal and must be used wisely to pinpoint performance issues. This article
expands on discussions of the scatter chart in the "User Experience, Not Metrics" series.
Advanced Topics
Parts 11 through 14 are "clean-up" topics. These are areas that are referred to in one or more of the previous
articles but not done justice due to topic and length constraints. These articles expand on topics that are
applicable throughout the engineering process.
• Part 11: Collaborative Tuning – Once you can exploit the failure or bottleneck, you need to know how
to work with the rest of the team to resolve it.
• Part 12: Testing and Tuning on Common Tiers – Web, application, and database servers are
common tiers related to testing and tuning. Here we discuss specific issues, methods, and resolutions
for these areas.
• Part 13: Testing and Tuning Load Balancers and Networks – These components require a
significantly different approach from that called for by common tiers. This article discusses issues
specific to these areas and approaches for handling them.
• Part 14: Testing and Tuning Security – Security is the most performance-intensive activity of an
application. Tuning security issues often involves taking a risk-based approach to balancing necessary
security with desired performance.
Summing It Up
Our job isn’t done when we accurately report the current performance of an application; in fact, it’s just begun.
The "Beyond Performance Engineering" series is designed to discuss and demonstrate how to take the step
from simply reporting results to achieving desired performance with Rational TestStudio and a proven
performance engineering methodology. The articles will focus on the topics that are hardest to find information
about and point you to where you can find more information. I hope you’re looking forward to reading this series
as much as I’m looking forward to writing it!
Copyright, 2003