Orthogonal defect classification
Orthogonal defect classification (ODC)[1] turns semantic information in the software defect stream into a measurement on the process.[2] The ideas were developed in the late 1980s and early 1990s by Ram Chillarege[3] at IBM Research. This has led to the development of new analytical methods used for software development and test process analysis. ODC is process model, language and domain independent. Applications of ODC have been reported by several corporations on a variety of platforms and development processes, ranging from waterfall, spiral, gated, and agile[4] [5] development processes. One of the popular applications of ODC is software root cause analysis.
ODC is known to reduce the time taken to perform root cause analysis by over a factor of 10. The gains come primarily from a different approach to root cause analysis, where the ODC data is generated rapidly (in minutes, as opposed to hours per defect) and analytics used for the cause and effect analysis. This shifts the burden of analysis from a purely human method to one that is more data intensive.[6]
ODC as proposed in its original papers have specific attribute-value sets that create measurements on the development process. Two of the five more well known categories are the defect type and defect trigger. The defect type captures the changes made in the code as a result of the defect. There are seven values for defect type and they have been empirically established to provide a measurement of the product through the process through their distribution. The concept is that changes in the defect type distribution is a function of the development process model, and thus provides an intrinsic measurement of progress of the product through the process.
The defect trigger, similarly provides a measurement of the Testing process. The concept of the trigger is a key contribution that came through ODC and is now fairly widely used in technical and research publications.[7] The software trigger is defined as the force that surfaced the Fault to create the failure. The full set of triggers is available in ODC Documentation.
The defect type and trigger collectively provide a large amount of causal information on defects. Additional information from the defect that is captured in standard ODC implementations includes "impact", "source" and "age". ODC training courses report that, once trained, an individual can categorize a defect via ODC in less than 3 minutes when performing the task retrospectively.[8] The time taken is far lower when done in-flight, or in-process. The categorization cannot be directly compared to root-cause-analysis, since ODC data is about "what-is", not "why". However, root cause analysis is very commonly performed using ODC. The analysis that studies ODC data is performing the first pass of root cause analysis, which is confirmed by discussing the results with the development team. This approach has five primary differences between the classical method and the ODC method.[9]
Root cause analysis is just one of the applications of ODC. The original design of ODC was to create a measurement system for software engineering using the defect stream as a source of intrinsic measurements. Thus, the attributes, either singularly, or in conjunction with one of the others provides specific measurements on certain aspects of the engineering process. These measurements can be used for one or more analytical methods, since they were designed with general measurement principles in mind. Todate, several research papers have applied these for a variety of purposes. More recently, there have been research articles that use ODC to assess the methods used for security evaluation, and expanded the scope of ODC.[10]
References
[edit]- ^ Orthogonal Defect Classification-A Concept for In-Process Measurements, IEEE Transactions on Software Engineering, November 1992 (vol. 18 no. 11). http://www.chillarege.com/articles/odc-concept.html
- ^ What is ODC? https://www.youtube.com/watch?v=mno4pQMqtBM
- ^ IEEE Computer Society, 2002 Technical Achievement Award https://www.computer.org/profiles/ram-chillarege
- ^ Orthogonal Defect Classification (ODC) in Agile Development. M. Jagia, S. Meena, IEEE ISSRE 2009 Supplemental Proceedings, Nov. 2009.
- ^ Orthogonal Defect Classification: An Agile Test/QA Primer, Agile Development Conference, Nov. 2012
- ^ "ODC - a 10x for Root Cause Analysis", R. Chillarege 2006
- ^ Software Defects and their Impact on System Availability - A Study of Field Failures in Operating Systems. M.Sullivan and R. Chillarege, IEEE 21st Fault-Tolerant Computing Systems, 1991.
- ^ Diamonds from Defects, LADC Keynote, http://www.unicauca.edu.co/ladc2016/?q=node/22
- ^ "5 Differences between classical root cause analysis and ODC root cause analysis. https://www.youtube.com/watch?v=fTJr2Pgnxco
- ^ P. J. Morrison, R. Pandita, X. Xiao, R. Chillarege, and L. Williams, “Are vulnerabilities discovered and resolved like other defects?,” Empir Software Eng, vol. 23, no. 3, pp. 1383–1421, Jun. 2018, doi: 10.1007/s10664-017-9541-1.