This document defines the defect Severity scale for determining defect criticality and the associated defect Priority levels to be assigned to errors found in software. It is a scale which can be easily adapted to other automated test management tools.
ANSI/IEEE Std 729-1983 Glossary of Software Engineering Terminology defines Criticality as,
"A classification of a software error or fault based on an evaluation of the degree of impact that error or fault on the development or operation of a system (often used to determine whether or when a fault will be corrected)."
The severity framework for assigning defect criticality that has proven most useful in actual testing practice is a five level scale. The criticality associated with each level is based on the answers to several questions.
First, it must be determined if the defect resulted in a system failure. ANSI/IEEE Std 729-1983 defines a failure as,
"The termination of the ability of a functional unit to perform its required function."
Second, the probability of failure recovery must be determined. ANSI/IEEE 729-1983 defines failure recovery as,
"The return of a system to a reliable operating state after failure."
Third, it must be determined if the system can do this on its own or if remedial measures must be implemented in order to return the system to reliable operation.
Fourth, it must be determined if the system can operate reliably with the defect present if it is not manifested as a failure.
Fifth, it must be determined if the defect should or should not be repaired.
The following five level scale of defect criticality addresses the these questions.
The five Levels are:
1. Critical
2. Major
3. Average
4. Minor
5. Exception
1. Critical - The defect results in the failure of the complete software system, of a subsystem, or of a software unit (program or module) within the system.
2. Major - The defect results in the failure of the complete software system, of a subsystem, or of a software unit (program or module) within the system. There is no way to make the failed component(s), however, there are acceptable processing alternatives which will yield the desired result.
3. Average - The defect does not result in a failure, but causes the system to produce incorrect, incomplete, or inconsistent results, or the defect impairs the systems usability.
4. Minor - The defect does not cause a failure, does not impair usability, and the desired processing results are easily obtained by working around the defect.
5. Exception - The defect is the result of non-conformance to a standard, is related to the aesthetics of the system, or is a request for an enhancement. Defects at this level may be deferred or even ignored.
In addition to the defect severity level defined above, defect priority level can be used with severity categories to determine the immediacy of repair. A five repair priority scale has also be used in common testing practice. The levels are:
1. Resolve Immediately
2. Give High Attention
3. Normal Queue
4. Low Priority
5. Defer
1. Resolve Immediately - Further development and/or testing cannot occur until the defect has been repaired. The system cannot be used until the repair has been effected.
2. Give High Attention - The defect must be resolved as soon as possible because it is impairing development/and or testing activities. System use will be severely affected until the defect is fixed.
3. Normal Queue - The defect should be resolved in the normal course of development activities. It can wait until a new build or version is created.
4. Low Priority - The defect is an irritant which should be repaired but which can be repaired after more serious defect have been fixed.
5. Defer - The defect repair can be put of indefinitely. It can be resolved in a future major system revision or not resolved at all.