Effective Testing using Taguchi's Orthogonal Arrays (OA)
Objectives of Effective Testing
- Reduce cycle time of the test phase.
- Offer maximum coverage with minimum test cases.
- Find maximum defects with minimal no of test cases.
- Goal is to obtain reproducible results incurring the least possible expenses.
Dr Genichi Taguchi's Orthogonal Array (OA)
- Based on Combinatorial and Orthogonality theories
- Allows the designer to execute only a subset of all possible combinations.
- Orthogonal Array test cases can be customized based on available time and known problems.
- Independent of platforms and domains.
Advantages of using OA
- Results in smaller number of experiments.
- More efficient in handling a large number of input variables.
- Helps to determine the contribution of each input that influences quality of the product.
- Allows easy interpretation of experiments.
How to apply??
-
Step 1: Brainstorming
- First and most important step in experimental design.
- Mostly performed by a team rather than an individual.
- Output of Brainstorming session
- Define the objective: Reduce the number of Test Cases and choose the one that best suits the problem in hand.
- Create a problem statement: Convert the engineering situation into an experimental design problem.
- Identify the factors: A factor is a variable under study –an input that can be controlled.
- Identify the levels: A level is a value that a factor can assume when used in experiment.
-
Step 2: Designing the experiments
- Select an appropriate orthogonal array from a standard set of orthogonal array defined by Taguchi.
|
OA |
No. of Experiments |
No. of Factors |
Max No. of factors at these levels(Level 2) |
|
L-4 |
4 |
3 |
3 |
|
L-8 |
8 |
7 |
7 |
|
L-32 |
32 |
10 |
1 |
- Assign Factors: Once OA is finalized, the factors are assigned to the columns of the array and the integers are translated to the different levels the factors can take.
- Each row corresponds to the particular experiment
- Unassigned columns can be deleted from the array.
-
Step 3: Running the experiments
- Execute the tests as designed.
-
Step 4: Analyzing the results
- Uses statistical treatment
- Analysis of Means (ANOM)-which determines the effects of the various parameters.
- Analysis of Variance (ANOVA)-the variation contributed by each factor can be determined.
-
Step 5: Running confirmation experiments.
- Optimize the final product.
- Running the experiments at the optimum conditions.
Where to apply?
- Configuration Testing
- User Interface Testing
- Performance Tuning
- Regression Testing.
Verification Suite Development Using OA
- Defining the Parameters and the respective Levels is the crucial activity in the entire process.
- Optimize combination of test cases.
- Help of the tool called Minitab can be used for this purpose.
- For Minitab the input is the Parameters and levels and the output is the optimized test case combination.
- Generally the code coverage is >90%, if the parameters and levels selection is effective.
- Thus this method can give a very high code coverage which would ensure that the testing of all functionality in a minimum time and cost.
- A good analysis is required while identifying the parameters and levels.
- If the possibility of identifying parameters and levels is tough –it is better to look for alternatives rather than using OA technique.
The Way Ahead
- Future of testing lies in statistics!
- These methods doesn’t address all aspects of system testing
- Tester’s intuition is still a much needed asset!
