Assignment 2

[Model answers]

1. Derive a set of black-box test cases for EACH of the following components:

(a) A function that takes a line of text as input and counts the number of non-blank characters in the line. [25 pts]

Cases should include following equivalence partitions [5 for each]

Regular line with many non-blank
Regular with one non-blank
Regular with zero non-blank
Single character line (blank)
Single character line (non-blank)
Embedded special characters ("text" is ambiguous - just printable characters?)
Zero length line

Deduct up to five for:

Test cases that consistently aren't concrete
Don't meet question (e.g. no expected output)
Non-string input
Wrong expected output

(b) A function that examines a line of text and replaces sequences of spaces with a single space. [25 pts]

Cases should include following equivalence partitions [4 for each]

Regular line with several space sequences
Regular with one space sequence
Regular with no spaces at all
Regular with no space sequences but some single spaces
Regular whose spaces are all in a sequence
Line consisting only of spaces
Single character line (space)
Single character line (non-space)
Embedded special characters ("text" is ambiguous - just printable characters?)
Zero length line

Deduct up to five for:

Test cases that consistently aren't concrete
Don't meet question (e.g. no expected output)
Non-string input
Wrong expected output

Each test case in your answer should consist of an input, the expected output for that input, and a phrase describing the equivalence class or boundary value that the test case represents. You may assume that the functions are written in a strongly-typed language and that their inputs are guaranteed to be strings of characters. For invalid input strings, make assumptions about what the function should do in those cases and base your test cases on those assumptions.

2. Suppose that the functions above are among a larger number of editing functions provided by a TextLine abstract data type. Two testers, A and B, working independently design and apply their test cases to the TextLine ADT. Tester A finds 10 faults in the ADT, and tester B finds 12. Seven of the identified faults are common to Tester A and B.

(a) What is your best estimate of the number of faults remaining in the ADT? [20 pts]

a = 10, b = 12, q = 7

Ea = a/n = q/b = 7/12 Eb = b/n = q/a = 7/10

n = q/(Ea*Eb) = (a*b)/q = 120/7 = approx 17

(b) Would you be satisfied to release the code in its current state? Justify your answer. If you would not release the code yet, when would you? [10 pts]

EITHER No.

Because:

Defect density is too high. (ADT would have to be several KLOCs (v. big ))
There are SOME known defects. (Assuming non-trivial and no documentable workaround)

OR Maybe.

If answer justified by result of faulty calculation - i.e. if n came out <= 5 and assuming all identified defects are trivial and have documentable workarounds)
Needs strong justification for full marks.

(c) Under what circumstances do you think it would be good professional practice to use these metrics to assess the quality of programmers and testers? [10 pts]

Evidence of thinking about problem. FOR EXAMPLE:

+ Testers try their hardest to break code (to increase their 'a' or 'b')
- Easy to defeat these statistics if you know you're being assessed. (Could oncentrate on minor, easy-to-detect failures; could pad your code or lay it out so that defect density is lower than colleagues')
- Overemphasis on numbers. Not all defects ARE the same
- In programmers' interest to categorize many failures arising from one defect, so need very strict definitions of what a defect is and how to classify

IGNORE argument that quality of program improves. That would be true without assessment of staff.

(d) Explain in no more than one paragraph what you would do if your project manager insisted that the ADT be included in the current software release once the known defects had been corrected. (You may assume that correcting the known defects did not result in any new ones.) [10 pts]

REASONING

How many defects were found (10 + 12 - 7) = 15
How many probably remain: 17 - 15 = 2 [3 pts]
Assuming non are introduced in correcting, defect density would now be 2N/KLOC for an N KLOC ADT. [DENSITY is impt. 2pts]

ACCEPTABLE CONCLUSIONS [5 pts]

OK, do it if 2N is acceptably low for that industry/criticality of software
OK, but document concerns and reasoning, if 2N is too high and you have professional concerns about potential defects being serious. Whistleblowing if 2N is too high and you have professional concerns about potential defects. [This is ethically questionable, but we haven't discussed employee responsibilities in class, so let it go]

UNACCEPTABLE ANSWERS/CONCLUSIONS [0 pts]

OK, because he/she is the boss.
Object because software should always be defect-free.