1. Strategic Approaches to Software Testing:
A number of software testing strategies provide the software developer with a template for testing and all have the following generic characteristics:
ü Testing begins at the component level and works “outward” toward the integration of the entire computer-based system.
ü Different testing techniques are appropriate at different points in time.
ü Testing is conducted by the developer of the software and an independent test group.
ü Testing and debugging are different activities, but debugging must be accommodated in any testing strategy.
1.1. Verification and Validation
Verification refers to the set of activities that ensure that software correctly implements a specific function. Validation refers to a different set of activities that ensure that the software that has been built is traceable to customer requirements.
Verification: “Are we building the product right?”
Validation: “Are we building the right product?”
1.2. Organizing for Software Testing
There are often a number of misconceptions that can be erroneously inferred from the preceding discussion:
(1) That the developer of software should do no testing at all
(2) That the software should be “tossed over the wall” to strangers who will test it mercilessly.
(3) That testers get involved with the project only when the testing steps are about to begin.
The software developer is always responsible for testing the individual units of the program, ensuring that each performs the function for which it was designed. In many cases the developer also conducts integration testing – a testing step that leads to the construction of the complete program structure. Only after the software architecture is complete does an independent test group become involved.
Independent test group is to remove the inherent problems. It also removes the conflict of interest that may otherwise be present.
2. Strategic Issues
The following issues must be addressed if a successful software testing strategy is to be implements.
ü Specify product requirements in a quantifiable manner long before testing commences.
ü State testing objectives explicitly.
ü Understand the users of the software and develop a profile for each user category.
ü Develop a testing plan that emphasizes “rapid cycle testing”.
ü Build “robust” software that is designed to test itself.
ü Use effective formal technical reviews as a filter prior to testing.
ü Conduct formal technical reviews to assess the test strategy and test cases themselves.
ü Develop a continuous improvement approach for the testing.
3. Unit Testing
Unit testing focuses verification effort on the smallest unit of software design – the software component or module. Using the component-level design description as a guide, important control paths are tested to uncover errors within the boundary of the module. The relative complexity of tests and uncovered errors is limited by the constrained scope established for unit testing. Unit testing is white-box oriented.
3.1. Unit Test considerations:
The tests that occur as part of unit tests are illustrated with modules and programs. The module interface is tested to ensure that information properly flows into and out of the program unit under test. All independent paths through the control structure are exercised to ensure that all statements in a module have been executed at least once. Boundary conditions are tested to ensure that the module operates properly at boundaries established to limit or restrict processing.
The common errors in computations are
(1) misunderstood or incorrect arithmetic precedence,
(2) mixed mode operations,
(3) incorrect initialization,
(4) precision inaccuracy,
(5) Incorrect symbolic representation of an expression.
Comparison and control flow are closely coupled to one another. Test cases should uncover errors such as
(1) comparison of different date types,
(2) incorrect logical operators or precedence,
(3) expectation of equality when precision error makes equality unlikely,
(4) incorrect comparison of variables,
(5) improper or nonexistent loop termination,
(6) failure to exit when divergent iteration is encountered, and
(7) Improperly modified loop variables.
Among the potential errors that should be tested when error handling is evaluated are
(1) Error description is unintelligible.
(2) Error noted does not correspond to error encountered.
(3) Error condition causes system intervention prior to error handling.
(4) Exception – condition processing is incorrect.
(5) Error description does not provide enough information to assist in the location of the cause of the error.
3.2 Unit Test Procedures:
Unit testing is normally considered as an adjunct to the coding step. After source level code has been developed, reviewed and verified for correspondence to component level design, unit test case design begins. Each test case should be coupled with a set of expected result. A component is not a stand alone program; driver and/or stub software must be developed for each unit test.
Unit testing is simplified w hen a component with high cohesion is designed. When only one function is addressed by a component, the number of test cases is reduced and errors can be easily predicted and uncovered.
4. Integration Testing
Integration testing is a systematic technique for constructing the program structure while at the same time conduction tests to uncover errors associated with interfacing. The objective is to take unit tested components and build a program structure that has been dictated by design. All components are combined in advance and the entire program is tested as a whole.
The Program is constructed and tested in small increments, where errors are easier to isolate and correct; interfaces are more likely to be tested completely; and a systematic test approach may be applied.
4.1. Top-down Integration
Top-down integration testing is an incremental approach to construction of program structure. Modules are integrated by moving downward through the control hierarchy, beginning with the main control module. Modules subordinate to the main control module are incorporated into the structure in either a depth-first or breadth-first manner.
The integration process is performed in a series of five steps:
(1) The main control module is used as a test driver and stubs are substituted for all components directly subordinate to the main control module.
(2) Depending on the integration approach selected, subordinate stubs are replaced as each component is integrated.
(3) Tests are conducted as each component is integrated.
(4) On completion of each set of tests, another stub is replaced with the real component.
(5) Regression testing may be conducted to ensure that new errors have not been introduced.
The top-down integration strategy verifies major control or decision points early in the test process. In a well-factored program structure, decision making occurs at upper level in the hierarchy and is therefore encountered first. Top-down strategy sounds relatively uncomplicated, but in practice, logistical problems can arise. The most common of these problems occurs when processing at low levels in the hierarchy is required to adequately test upper levels.
4.2. Bottom-up Integration
Bottom-up integration begins construction and testing with atomic modules. Because components are integrated from the bottom up, processing required for components subordinate to a given level is always available and the need for stubs is eliminated. A bottom-up integration strategy may be implemented with the following steps:
(1) Low level components are combined into clusters that perform a specific software sub function.
(2) A driver is written to coordinate test case input and output.
(3) The cluster is tested.
(4) Drivers are removed and clusters are combined moving upward in the program structure.
4.3 Regression Testing
Each time a new module is added as part of integration testing, the software changes. New data flow paths are established, new I/O may occur, and new control logic is invoked. These changes may cause problems with functions that previously worked flawlessly. Regression testing is the re-execution of some subset of tests that have already been conducted to ensure that changes have not propagated unintended side effects.
Regression testing may be conducted manually, by re-executing a subset of all test cases or using automated capture/playback tools. Capture/playback tools enable the software engineer to capture test cases and results for subsequent playback and comparison. The regression test suite contains three different classes of test cases:
ü A representative sample of tests that will exercise all software functions.
ü Additional tests that focus on software functions that are likely to be affected by the change.
ü Tests that focus on the software components that have been changed.
4.4 Smoke Testing
Smoke testing is an integration testing a approach that is commonly used when “shrink wrapped” software products are being developed. It is designed as a pacing mechanism for time-critical projects, allowing the software team to assess its project on a frequent basis; the smoke testing approach encompasses the following activities:
ü Software components that have been translated into code are integrated into a “build”. A build includes all data files, libraries, reusable models and engineered components that are required to implement one or more product functions.
ü A series of tests is designed to expose errors that will keep the build from properly performing its function.
ü The build is integrated with other builds and the entire product is smoke tested daily.
Smoke testing provides a number of benefits when it is applied on complex, time-critical software engineering projects:
ü Integration risk is minimized.
ü The quality of the end-product is improved.
ü Error diagnosis and correction are simplified.
ü Progress is easier to assess.
Integration test plan describes the overall strategy for integration. Testing is divided into phases and builds that address specific functional and behavioral characteristics of the software. This integration testing includes, “User interaction”, “Data manipulation and analysis”, “Display process and generation” and “Database management”
A history of actual test results, problem, or peculiarities is recorded in the Test Specification. This information can be vital during software maintenance.
5. Validation Testing
Software is completely assembled as a package, interfacing errors have been uncovered and corrected, and a final series of software tests – validation testing will begin. Validation can be defined in many ways, but a simple definition is that validation succeeds when software functions in a manner that can be reasonably expected by customer.
5.1 Validation Test Criteria
Software validation is achieved through a series of black-box tests that demonstrate conformity with requirements. The plan and procedure are designed to ensure that all functional requirements are satisfied, all behavioral characteristics are achieved, all performance requirements are attained, documentation is correct, and human engineered and other requirements are met. This validation tests conducted when (1) the function or performance characteristics conform to specification and are accepted or (2) a deviation from specification is uncovered and a deficiency list is created.
5.2. Alpha and Beta Testing
It is virtually impossible for a software developer to foresee how the customer will really use a program.
The Alpha test is conducted at the developer’s site by a customer. The software is used in a natural setting with the developer “looking over the shoulder” of the user and recording errors and usage problems. Alpha tests are conducted in a controlled environment.
The Beta test is conducted at one or more customer sites by the end-user of the software. Unlike alpha testing, the developer is generally not present. Therefore, the beta test is a “live” application of the software in an environment that cannot be controlled by the developer. The customer records all problems that are encountered during beta testing and reports these to the developer at regular intervals.
6. System Testing
Software is only one element of a larger computer-based system. Software is incorporated with other system elements and a series of system integration and validation testes are conducted. These tests fall outside the scope of the software process and are not conducted solely by software engineers. A classic system testing problem is “finger-pointing”. This occurs when an error is uncovered and each system element developer blames the other for the problem. The software engineer should anticipate potential interfacing problems and
(1) deign error-handling paths that test all information coming from other elements of the system,
(2) conduct series of tests that simulate bad data or other potential errors at the software interface,
(3) record the results of tests to use as “evidence” if finger-pointing does occur, and
(4) participate in planning and design of system tests to ensure that software is adequately tested.
System testing is actually a series of different tests whose primary purpose is to fully exercise the computer-based system.
6.1 security Testing
Security testing attempts to verify that protection mechanisms built into a system will, in fact, protect it from improper penetration. During security testing, the tester plays the roles of the individual who desires to penetrate the system. The role of the system engineer is to make penetration cost more than the value of the information that will be conducted.
6.2 Stress Testing
Stress testing executes a system in a manner that demands resources in abnormal quality, frequency, or volume. A variation of stress testing is a technique called sensitivity testing. A very small range of data contained with the bounds of valid data for a program may cause extreme and even erroneous processing or profound performance degradation.
6.3 Performance Testing
Performance testing is designed to test the run time performance of the software with the context of an integrated system. Performance testing occurs throughout all steps in the testing process. Even at the unit level, the performance of an individual module may be assessed as white box tests are conducted. Performance tests are often coupled with stress testing and usually required both hardware and software instrumentation.
No comments yet.