Performance Testing Core Activity: Analyze -- PerfTestPlus

Core Performance Testing Principle:

Analyze

Overview

Managers and stakeholders need more than just the results from various tests — they need conclusions, and consolidated data that supports those conclusions. Technical team members also need more than just results — they need analysis, comparisons, and details behind how the results were obtained. Before results can be reported, they data must be analyzed.

Keys to Quality Analysis

Consider the following while analyzing performance test data:

Analyze the data both individually and as part of a collaborative, cross-functional technical team.
Analyze the captured data and compare the results against the metric’s acceptable or expected level to determine whether the performance of the application being tested shows a trend toward or away from the performance objectives.
If the test fails, a diagnosis and tuning activity are generally warranted.
If you fix any bottlenecks, repeat the test to validate the fix.
Performance testing results can enable the team to analyze deep into components and correlate the information back to the real world with proper test design and usage analysis.
Performance test results should enable informed architecture and business decisions.
Frequently, the analysis will reveal that to completely understand the results of a particular test, additional metrics will need to be captured during subsequent test execution cycles.
Immediately share test results and make raw data available to your entire team.
Talk to the consumers of the data to validate that the test achieved the desired results and that the data means what you think it means.
Modify the test to get new, better, or different information if the results do not represent what the test was defined to determine.
Use current results to set priorities for the next test.
Collecting metrics frequently produces very large volumes of data. While it is tempting to reduce the amount of data, always exercise caution when using data reduction techniques as valuable data can be lost.