A software metrics based approach to enterprise software beta testing design

Abstract

Today, major manufacturers of enterprise software have shifted their focus to integrating the influence of the user community in the validation of software. This relationship is established through the corporate beta testing programs to confirm to the customer base that the software was properly reviewed prior to releasing to the target market. However, there are no industry standards for beta testing software and no structured software beta testing processes that may be used for the common enterprise application. In addition, beta testing models used today lack clear testing objectives. An additional problem in software beta test processes used by most manufacturers, is that there are no organized procedures for developing an effective beta test. Beta testing models strictly focus on error feedback, time of testing, and the number of beta testing sites participating in the program. ^ This research addresses the fundamental problems in poor beta testing design by contributing a software metrics based beta testing design model that uses weakness in the current product to plan an effective beta test. A set of formal beta testing objectives is initiated and essential software attributes are suggested to support each objective. Each objective is quantitatively measured with a set of metric functions used to predict risk levels in a software product. The predicted risk levels are used to assist in prioritizing tasks for the pending beta test. A learning process is provided to demonstrate the effects of the metric functions when the process matures. ^ The metric functions were tested on three real-world enterprise applications to validate the effectiveness of the formulas when employed on a live product. The experiment demonstrated close prediction to the actual risk value for each product and demonstrated how the predictions improved with experience. ^