CSTE CBOK Domain 9 – The Defect Management Process
1) What fields would you include in creating a new defect-tracking program (used by QA, developers, etc)?
Fields to include at new defect-tracking program are:
· Defect Prevention
· Deliverable base-lining
· Defect discovery/defect naming
· Defect resolution
· Process improvement
· Management reporting
Defect Prevention
· Identify critical Risks
o Missing a key requirement
o Application does not function properly
o Performance is unacceptably poor
o User’s ability to actively participate in project etc.
· Estimate expected impact
o Risk should be prioritized by the expected impact and the degree to which the expected impact can be reduced
· Minimize expected impact
Minimizing expected impact involves a combination of the following three strategies
o Eliminate the risk (not always possible)
o Reduce the probability of a risk becoming a problem
o Reduce the impact if there is a problem
Deliverable Base lining
Deliverable base lining involves the following activities
· Identify key deliverables
· Define standards for each deliverable
Defect Discovery
· Find defect
o Intended to uncover defects or in effect by accident
o Techniques to find defects are
- Static techniques (reviews, walkthroughs etc.,)
- Dynamic techniques (Testing)
- Operational techniques (defect found as a result of a failure – customer, users etc.,)
· Report defect
o Defect once found to be quickly brought to the developers attention and formally controlled
· Acknowledge defect
o Whether are not the defect is valid (developer / customer determines & a senior manager of SDD is selected to resolve the dispute)
Defect Naming
· Name the defect
· Developmental activity in which the defect occurred
· Categorize defect
o Missing
o Inaccurate
o Incomplete
o Inconsistent
Defect Resolution
Once the developers have acknowledged that a reported defect is a valid defect the defect resolution process begins. The steps involved in it are:
· Prioritize fix
o Critical (would have serious impact on organization business)
o Major (would cause an output of the software to be incorrect or stop)
o Minor (something wrong, but not directly affect the user )
· Schedule fix
o Bases on priority, schedule the fix
· Fix defect
o Remove the defect from the system
· Report resolution
o Verifying the fix (automation can help a lot)
Process Improvement
· Improve the process and prevent potentially major failures
· While the defect itself may not be a big deal, the fact that there was a defect is a big deal
· Not only is the process strengthened to prevent defects, but also have to strengthen to find defects which have been created but not yet discovered
Defect Reporting
Recording the defects identified at each stage of the test process is an integral part of a successful life cycle testing approach. The purpose of this activity is to create a complete record of the discrepancies identified during testing. The information captured is used in multiple ways throughout the project, and forms the basis for quality measurement.
A defect can be defined in one of two ways.
From the producer’s view point –
A defect is a deviation from specifications, whether missing, wrong, or extra.
From the Customer’s view point –
A defect is anything that causes customer dissatisfaction, whether in the requirements or not; this view is known as “fit for use.” It is critical that defects identified at each stage of the project life cycle be tracked to resolution.
Defects are recorded for four major purposes:
· To correct the defect
· To report status of the application
· To gather statistics used to develop defect expectations in future applications
· To improve the software development process
Defect Tracking Tools
At a minimum, the tool selected should support the recording and communication of all significant information about the defect. For example, a defect log could include:
· Defect ID number
· Descriptive defect name and type
· Source of defect – test case or other source
· Defect severity
· Defect priority
· Defect status (e.g. open, fixed, closed, user error, design, and so on) – more robust tools provide a status history for the defect
· Date and time tracking for either the most recent status change or for each change in the status history
· Detailed description, including the steps necessary to reproduce the defect
· Component or program where defect was found
· Screen prints, logs, etc. that will aid the developer in resolution process
· Stage of origination
· People assigned to research and / or correct the defect
Severity vs. Priority
· Severity is the impact of defect on the system.
· Priority is assigned to determine the order in which defects should be fixed.
· The priority assigned to a defect is usually more subjective based upon the project (user says which is defect is more important to him to be fixed)
· It is recommended that severity levels be defined at the start of the project so that they are consistently assigned and understood by the team. This foresight can help test teams avoid the common disagreements with development teams about the criticality of a defect.
A sample Process
The steps below describe a simple defect tracking process. Depending on the size of the project or project ream, this process may be substantially more complex.
1. Execute the test and compare the actual results to the documented expected results. If a discrepancy exists, log the discrepancy with a status of “open”. Supplementary documentation, such as screen prints or program traces should be attached if available.
2. The test manager or tester should review the problem log with the appropriate member of the development team to determine if the discrepancy is truly a defect.
3. Assign the defect to a developer for correction. Once the defect is corrected, the developer will usually enter a description of the fix applied and update the defect status to “Fixed” or “Retest.”
4. The defect is routed back to the test team for retesting. Additional regression testing is performed as needed based on the severity and impact of the fix applied
5. If the retest results match the expected results, the defect status is updated to “closed”. If the test results indicate that the defect still not fixed, the status is changed to “open” and sent back to the developer.
Steps 3-5 should be repeated until the problem is resolved. Test reports are issued periodically throughout the testing process to communicate the test status to the rest of the team and management. These reports usually include a summary of the open defects, severity. Additional graphs and metrics can also be provided to further describe the status of the application.
Notes by Pavan