User:Atulnk11/Orthogonal array testing
Orthogonal array testing is a black box testing technique which is a systematic, statistical way of software testing.[1][2]It is used when the number of inputs to the system is relatively small, but too large to allow for exhaustive testing of every possible input to the systems.[1] It is particularly effective in finding errors associated with faulty logic within computer software systems.[1]Orthogonal arrays can be applied in user interface testing, system testing, regression testing, configuration testing and performance testing. The permutations of factor levels comprising a single treatment are so chosen that their responses are uncorrelated & hence each treatment gives unique piece of information. The net effect of organizing the experiment in such treatments is that the same piece of information is gathered in the minimum number of experiments.
Background
[edit]Orthogonal Vector
[edit]All orthogonal vectors exhibit orthogonality. Orthogonal vectors exhibit the following properties:
- Each of the vectors conveys information different from that of any other vector in the sequence, i.e., each vector conveys unique information therefore avoiding redundancy.
- On a linear addition, the signals may be separated easily.
- Each of the vectors is statistically independent of the others, i.e., the correlation between them is nil.
- When linearly added, the resultant is the arithmetic sum of the individual components.
Orthogonal Array testing vs. Pairwise testing
[edit]Pairwise testing describes a process for defining test vectors that ensure that all two parameter combinations are covered and it is often confused with Orthogonal Array testing. They both have a key similarity in that they ensure that all pairs of two parameters are covered, but they also have a key difference.
The difference is that Orthogonal Array test cases are uniformly distributed over the test domain and this provides significant cost advantages when you look at the end-to-end testing process. Pairwise testing can sometimes yield fewer test points than Orthogonal Array testing; however, the additional cost of defect analysis can far outweigh the potential reduction in execution cost. Since test cases created by the pairwise method can be unbalanced, it requires significantly more time to isolate the root causes of defects. In addition, it becomes very challenging to effectively assess performance in terms of the statistical properties like mean and variance. In fact, to effectively conduct analysis of faults and results, users have to run several additional test cases if they originally ran Pairwise tests. Orthogonal Arrays distribute test cases uniformly in the multidimensional test domain and defect analysis and performance assessments are very straightforward. [3] This significantly reduces the expert labor effort required to analyze test results
Technique
[edit]Consider a system which has 3 parameters and each of them has 3 values. To test the all possible combination (i.e. exhaustive testing) we will need a set of 27 test cases.But instead of testing the system for each combination of parameters, orthogonal array select the combination effectively and uniformly. With orthogonal array technique we can maximize the test coverage while minimum number of test cases.[4].We here assume that the pair that the test that maximizes interaction between the parameters, will have more defects and the technique works.[4]
| Test case ↓ | Parameter 1 | Parameter 2 | Parameter 3 |
|---|---|---|---|
| 1 | 1 | 1 | 3 |
| 2 | 1 | 2 | 2 |
| 3 | 1 | 3 | 1 |
| 4 | 2 | 1 | 2 |
| 5 | 2 | 2 | 1 |
| 6 | 2 | 3 | 3 |
| 7 | 3 | 1 | 1 |
| 8 | 3 | 2 | 3 |
| 9 | 3 | 3 | 2 |
Given that assumption, the table shows the set of nine combination of parameters which are sufficient to catch the fault considering the interaction of the input parameters, which is very effective and economical. The array is orthogonal because all possible pair-wise combinations between parameters occurs only once.[4]
The given L9 Orthogonal Array assess result of test cases as follows,[1]
Single Mode Faults - Single mode faults occur only due to one parameter. For example, in above Orthogonal array if test cases 7, 8 and 9 show error, we can expect that value 3 of parameter 1 is causing the error.Likewise we can detect as well as isolate the error.
Double Mode Fault - Double mode fault is caused by the two specific parameters values interacting together. Such a interaction is a harmful interaction between interacting parameters.
Multimode Faults - More than two interacting components if produce the consistent erroneous output, then it is a multimode fault. Orthogonal array detects the multimode faults.
Benefits
[edit]- Reduces testing cycle time and analysis is simpler.
- Balanced test cases ensure straightforward defect isolation and performance assessments. This provides a significant cost savings over pair-wise testing
Important Notes
[edit]- The only means to guarantee coverage of all defects is to test all possible combinations. As this is not feasible within budget and schedule, testers generally select a "sample: of the total test domain. Orthogonal arrays provide a means to effectively generate high coverage samples to validate the test domain.
References
[edit]- ^ a b c d Pressman, Roger S (2005). Software Engineering: A Practitioner's Approach (6th ed.). McGraw-Hill. ISBN 0-07-285318-2.
- ^ Phadke, Madhav S. "Planning Efficient Software Tests". Phadke Associates, Inc.
Numerous articles on utilizing Orthogonal Arrays for Software and System Testing.
- ^ Phadke, Kedar M. "Utilizing Design of Experiments to Reduce IT System Testing Cost" (PDF). Phadke Associates, Inc. and Crosstalk Journal.
Article Detailing Empirical Study on Orthogonal Array Testing.
- ^ a b c Dustin, Elfriede. "Orthogonally Speaking" (PDF). (subscription required)
External links
[edit]- Rao, Calyampudi Radhakrishna (2009). "Orthogonal arrays". Scholarpedia. 4 (7): 9076. doi:10.4249/scholarpedia.9076.
- Delius, Gustav W (May 2004). "Orthogonal Arrays (Taguchi Designs)". University of York.
{{cite web}}: CS1 maint: date and year (link) - Kuhfeld, Warren F. "Orthogonal Arrays". SAS Institute Inc.
SAS provides a catalog of over 117,000 orthogonal arrays.
- Phadke, Madhav S. "Planning Efficient Software Tests". Phadke Associates, Inc.
Numerous articles on utilizing Orthogonal Arrays for Software and System Testing.
- "rdExpert Software for Orthogonal Array Testing". Phadke Associates, Inc.
Commercial toolset for Orthogonal Array Testing.
Category:Survival analysis Category:Design of experiments
 | This statistics-related article is a stub. You can help Wikipedia by expanding it. |