Software Testing Fundamentals-Methods and Metrics

metric = a measure.

metric system = a set or system of measures.

· The only type of metrics used regularly has to do with counting bugs and ranking them by severity. Only a small percentage of respondents measure the bug find rate or the bug fix rate. No other metrics are widely used in development or testing, even among the best-educated and seemingly most competent testers. It can also be inferred from these results that the companies for which these testers work do not have a tradition of measuring their software development or test processes.

· Inspection or structured analysis = some documented structured or systematic method of analyzing the test needs of a system.

· Automated test tools are used today, but test automation is also voted as the most difficult test technique to implement and maintain in the test effort.

· Within months, every major hardware and software vendor had a support presence on the Web. The bug fix process became far more efficient because it was no longer necessary to ship fixes to everyone who purchased the product-only those who noticed the problem came looking for a solution. Thanks to the Internet, the cost of distributing a bug fix fell to almost nothing as more and more users downloaded the fixes from the Web. The customer support Web site provided a single source of information and updates for customers and customer service, and the time required to make a fix available to the users shrank to insignificance. The cost of implementing these support Web sites is very small and the savings were huge; customer satisfaction and profit margins go up. It has become common practice to distribute bug-fix releases (put the patches and fixes on the Web site) within a few weeks of the initial release-after the market has been captured. Consequently, reliability metrics are not currently considered to be crucial to commercial success of the product.

· The way to develop a good cost-benefit statement, and add real credibility to software testing, is to use formal methods and good metrics.

· Regardless of the cause, once a software maker has decided to use formal methods, it must address the question of which formal methods and metrics to adopt. Once methods or a course toward methods has been determined, everyone must be educated in the new methods. Moving an established culture from an informal method of doing something to a formal method of doing the same thing takes time, determination, and a good cost-benefit ratio. It amounts to a cultural change, and introducing culture changes is risky business. Once the new methods are established, it still takes a continuing commitment from management to keep them alive and in use.