Black-Box Testing

Click here to load reader

  • date post

    02-Jan-2016
  • Category

    Documents

  • view

    60
  • download

    4

Embed Size (px)

description

Black-Box Testing. ( “FÅP”: First-year Project Course, ITU, Denmark ). Claus Brabrand [ [email protected] ]. Reminder: Learning & Exam Goals. ”Product”: ”Oral Exam”:. Outline. Introduction: ”Bugs” and ”Black-box testing” Testing the Specification: [ ”static black-box testing” ] - PowerPoint PPT Presentation

Transcript of Black-Box Testing

Projekt og Kommunikation[ * ]
Other techniques:
Model-checking & Static-analysis
[ * ]
Date: 1622
1 a: an insect or other creeping or crawling invertebrate (as a spider or centipede)
b: any of several insects (as the bedbug or cockroach) commonly considered obnoxious
c: any of an order (Hemiptera and especially its suborder Heteroptera) of insects that have sucking mouthparts, forewings thickened at the base, and incomplete metamorphosis and are often economic pests —called also true bug
2: an unexpected defect, fault, flaw, or imperfection <the software was full of bugs>
3 a: a germ or microorganism especially when causing disease b: an unspecified or nonspecific sickness usually presumed due to a bug
4: a sudden enthusiasm
6: a prominent person
7: a crazy person
9: a weight allowance given apprentice jockeys
[ * ]
A propos test og datoberegninger:
Fredag morgen mente min PDA kalender (Microsoft Windows CE) at vi skrev lørdag 1. marts. Jeg stillede selvfølgelig tålmodigt uret tilbage til fredag 29. februar 2008 (dette fandt sted på et møde hos Microsoft i Vedbæk ;-) Lørdag formiddag startede den så med at vise min (ret tomme) kalender for august 2049. Det indbyggede ur stod på 1. januar 1601. Ahem.
Peter
Typically imply blame; as in:
- ”it was his fault”
[ * ]
Photo of first
actual ”bug”:
“The first documented computer bug was a moth found trapped between
points at Relay # 70, Panel F, of the Mark II Aiken Relay Calculator
while it was being tested”
Harvard University, Sep. 9, 1947
[ * ]
Specification &
Implementation
Explicit or Implicit
Written or Oral
Formal or Informal
implementation
Should do!
Should do!
Unspecified?
S
[ * ]
1. Impl doesn’t do sth spec says it should:
2. Impl does sth spec says it shouldn’t:
3. Impl does sth spec doesn’t mention:
4. Spec doesn’t mention sth, but should:
5. Impl is hard to understand/use by user(s):
(or does it incorrectly)
~
~
?!
(but wasn’t supposed to)
”Easter egg”?
e.g., hit [Alt+Shift+2] in Solitaire to win game
…is a bug !
…is a bug !
trivially no bugs in impl !
(according to [own] definition #1,#2,#3)
B) More sensible to interpret as ”no spec”:
entire impl is essentially one big bug !
(…according to definition #3+#4 [p.15])
”Because the software is already complete, you have the perfect specification
– the product itself” [R.Patton, ”Software Testing”, p.31]
”an impl w/o a spec” ?
bug
…is the specification:
Often due to:
(NB: no percentages given)
log
spec
design
code
test
release
$1
$100
$10,000
$1,000,000
$10,000,000
[ * ]
The goal of a software tester is:
Testers (therefore) often aren’t the most popular members of a project team
- 1) “to find bugs”;
- 3) “make sure they get fixed”
[R.Patton, “Software Testing”, p. 19]
[cf. ”Software Testing”, R.Patton, p.19]
[ * ]
the more immune it becomes to your tests”
B.Beizer, “Software Testing Techniques”, 1990
[ * ]
If you find one, chances are there are more nearby
Not all bugs will be fixed
Too costly?
Too risky?
Not ”cost-effective”?
Bug unknown :-)
Often not a good idea to ”test” your own code :-(
Destructive thinking:
Advice:
[ * ]
Make sure impl solves problem it is supposed to:
i.e.,
testing w/o insights of code
impl ~ spec
Other techniques:
Model-checking & Static-analysis
[ * ]
Try to violate (i.e., find exemptions from rule)!
Unconditionals (never):
Try to violate (i.e., find exemptions from rule)!
Tautologies:
Check assumptions (that nothing’s swept under the rug)!
[cf. ”Software Testing”, R.Patton, p.61]
[ * ]
Continuations:
Check that spec is comprehensively unambiguous?
Examples:
Is example representative (what about other examples)?
[cf. ”Software Testing”, R.Patton, p.61]
[ * ]
Subjective (needs objectification if to be used for testing)!
Alegedly completed:
Is something hidden?
[ * ]
Check what happens in the ”Else-case”
[cf. ”Software Testing”, R.Patton, p.61]
IF …
THEN …
ELSE … ?!
A) white-box testing ; black-box testing ;
(i.e., white-box testing first)
(i.e., black-box testing first)
Before zooming in on the impl [as in white-box testing]
…or…
Other techniques:
Model-checking & Static-analysis
[ * ]
e.g. testing legal inputs (time, in hours):
e.g. testing legal inputs (dates, in April):
Property
Minimum-1
Minimum
e.g. #game-spectators (assume held in a 16-bit word):
Property
Minimum-1
Minimum
e.g., ’any program’:
e.g., calculator:
(i.e., ”testing can’t prove absence of bugs”)
There are infinitely many possible inputs:
(hence, testing will take an infinite amount of time)

Example2: “equals” relation:
Example3: “dist-btwn” relation:
‘~’ A A
Reflexive:
Symmetric:
Transitive:
x,y,zA: x ~ y y ~ z x ~ z


[ * ]
…and which are not (and why not)?:
a) The "less-than-or-equal-to" relation: ' '
b) The ”almost-total-relation-on-integers”,
(relating all numbers except 42, but relating 42 with 42):
{ (n,m) | n,m ( Z \ {42} ) } { (42,42) }
c) The "is-congruent-modulo-three" relation:
d) The "have-talked-together" relation:
e) The "is-in-the-same-group-as" relation:
[ * ]
A ~ P, B ~ Y, P ~ Z
Partition ’~’:
A
B
P
X
Y
Z
Capture the same information;
i.e., notion of ”equivalence”
(i.e., ”testing can’t prove absence of bugs”)
There are infinitely many possible inputs:
(hence, testing will take an infinite amount of time)

Here 3: { ”zero”, ”pos”, ”neg” }
We can now test all equivalence classes
Using representative elements from each category
neg
pos
0
zero
Using representative input from each category
Sum (testing all equivalence classes):
Property
Lists:
- partition into qualitatively different
[ * ]
Other techniques:
Model-checking & Static-analysis
Claus Brabrand, ITU, Denmark
(modeling lang)
If models are “finite”, we can have a computer test all possibilities.
“Never send a human to do a machine’s job”
-- A.Smith (’99)
e.g., can program P have a type error (when run):
Compilers use approximations
[ * ]
that do not have a self-reference
(i.e. don't contain their title inside)
Finitely many books; i.e.:
We can sit down & figure out whether to include or not...
Q: What about "The Book-of-all-Books";
Should it be included or not?
"Self-referential paradox" (many guises):
"The Bible"
Hence: "Termination is undecidable"
bool halts(Program p) { ... }
Program p0 = read_program("P0.java");
Type error <EXP> evaluates to true
void f() {
var b;
if (<EXP>) {
b = 42;
[ * ]
And have constant type (throughout the program)
void f() {
bool b;
/* some code */
if (<EXP>) {
b = 42;
Other techniques:
Model-checking & Static-analysis