Orthogonal array testing

Orthogonal array testing is a black box testing technique that is a systematic, statistical way of software testing.^[1]^[2] It is used when the number of inputs to the system is relatively small, but too large to allow for exhaustive testing of every possible input to the systems.^[1] It is particularly effective in finding errors associated with faulty logic within computer software systems.^[1] Orthogonal arrays can be applied in user interface testing, system testing, regression testing, configuration testing and performance testing. The permutations of factor levels comprising a single treatment are so chosen that their responses are uncorrelated and therefore each treatment gives a unique piece of information. The net effects of organizing the experiment in such treatments is that the same piece of information is gathered in the minimum number of experiments.

Background[edit]

Orthogonal vectors exhibit orthogonality. Orthogonal vectors exhibit the following properties:

Each of the vectors conveys information different from that of any other vector in the sequence, i.e., each vector conveys unique information therefore avoiding redundancy.
On a linear addition, the signals may be separated easily.
Each of the vectors is statistically independent of the others, i.e., the correlation between them is nil.
When linearly added, the resultant is the arithmetic sum of the individual components.

Testing cycle time is reduced and analysis is simpler.
Test cases are balanced, so it's straightforward to isolate defects and assess performance. This provides a significant cost savings over pair-wise testing.

^ ^a ^b ^c Pressman, Roger S (2005). Software Engineering: A Practitioner's Approach (6th ed.). McGraw-Hill. ISBN 0-07-285318-2.
^ Phadke, Madhav S. "Planning Efficient Software Tests". Phadke Associates, Inc. Numerous articles on utilizing Orthogonal Arrays for Software and System Testing.
^ Dustin, Elfriede. "Orthogonally Speaking" (PDF). (subscription required)^{[dead link]}

Rao, Calyampudi Radhakrishna (2009). "Orthogonal arrays". Scholarpedia. 4 (7): 9076. Bibcode:2009SchpJ...4.9076R. doi:10.4249/scholarpedia.9076.
Delius, Gustav W (May 2004). "Orthogonal Arrays (Taguchi Designs)". University of York.
Kuhfeld, Warren F. "Orthogonal Arrays". SAS Institute Inc. SAS provides a catalog of over 117,000 orthogonal arrays.
Phadke, Madhav S. "Planning Efficient Software Tests". Phadke Associates, Inc. Numerous articles on utilizing Orthogonal Arrays for Software and System Testing.
"rdExpert Software for Orthogonal Array Testing". Phadke Associates, Inc. Archived from the original on 2011-01-14. Commercial toolset for Orthogonal Array Testing.

v t e Software testing
The "box" approach	Black-box testing All-pairs testing Exploratory testing Fuzz testing Model-based testing Scenario testing Grey-box testing White-box testing API testing Mutation testing Static testing
Testing levels	Acceptance testing Integration testing System testing Unit testing
Testing types, techniques, and tactics	A/B testing Benchmark Compatibility testing Concolic testing Concurrent testing Conformance testing Continuous testing Destructive testing Development testing Dynamic program analysis Installation testing Random testing Regression testing Security testing Smoke testing (software) Software performance testing Symbolic execution Test automation Usability testing
See also	Graphical user interface testing Manual testing Orthogonal array testing Pair testing Soak testing Software reliability testing Stress testing Web testing