Chapter 3 The Realities of Software Testing

The Realities of Software
Testing
Chapter 3

Highlights
No perfect process
Trade-offs and concessions
Why software can never be perfect
Why software testing isn’t just a technical
problem
The terms commonly used by software
testers
Testing Axiom

It’s impossible to Test a Program
Completely
Four key reasons
The number of possible inputs is very large
The number of possible outputs is very large
The number of paths through the software is
very large
The software specification is subjective. You
might say that a bug is in the eye of beholder

Software Testing Is a Risk-Based
Exercise
Not to test every possible test scenario
Customer will eventually find it someday
Dilemma
Testing vs. Release deadline
Stop testing vs. Costly bug
Key concept
Reduce the huge domain of possible tests
into a manageable set
Make wise risk-based decision on what’s
important to test and what’s not

Optimal Amount of Testing
Number of
Missed Bugs
Cost of Testing
Optimal Amount
of Testing
Under Testing
Over Testing
Quantity
Amount of Testing

Testing Can’t Show That Bugs Don’t
Exist
Bugs in the House
Testing
Can show that bugs exist
Can’t show that bugs don’t exist
What can you do?
Test more and find more

The More Bugs You Find, the More
Bugs There Are
The nature of real bug and software bug:
Both types tend to come into groups
“bugs follow bugs”
Several reasons
Programmers have bad days
Programmers often make the same mistake
Some bugs are really just the tip of the iceberg

The Pesticide Paradox
The more you test software, the more
immune it becomes to your test
Spiral model example
How to overcome
Write new and different tests to exercise
different parts of the program and find more
bugs

Not All Bugs You Find Will Be Fixed
Reasons
There’s not enough time
It’s really not a bug
Misunderstanding, test errors or spec changes
Its’ too risky to fix
Make a bug fix that causes other bugs to appear
It’s just not worth it

When a Bug’s a Bug Is Difficult to say
There’s problem in the software but no one
ever discover it
Is It a bug????
Obey the rules to define bugs in Chapter 1
You can’t claim that a bug exists if you didn’t
see it
Bugs that are undiscovered or haven’t yet
been observed are often referred to as latent
bugs

Product Specification Are never Final
Constantly changing product specification
Fast moving industry
Longer development schedule
The only invariance is the variance of the
requirement (specification)

Software Testers Aren’t the Most
Popular Members of a Project Team
Goal of software tester
Find bugs, find them as early as possible, and
make sure they get fixed
Keep peace with your fellow team member
Find bug early
Temper your enthusiasm
Don’t just report bad news

Software Testing Is a Disciplined
Technical Profession
Professional software tester are mandatory
Core, vital members of the staff
Career choice
Software Testing Terms and
Definitions

Precision and Accuracy
Neither accurate nor precise
Precise, but not Accurate
Accurate, but not Precse
Accurate and Precise

Verification and Validation
Verification
Process confirming that something-softwaremeets
its specification
Validation
The process confirming that it meets the
user’s requirement
Example: Hubble telescope

Quality and Reliability
Quality
The customer feel that the product is excellent
and superior to his other choices
Reliability
One aspect of quality
How often the product crashes or trashes his
data

Testing and Quality Assurance (QA)
The goal of software tester is to find bugs,
find them as early as possible, and make
sure they get fixed
A software quality assurance person’s main
responsibility is to create and enforce
standards and methods to improve the
development process and to prevent bugs
from ever changing