@(#)ivvstds.htm 3.3 03/10/01
The Generic UWF Maintenance Process (GUMP) 1994, 1995, 1996 The University of West Florida. All rights reserved.
Permission is granted to reproduce and adapt this document provided credit is given to the University of West Florida. This documentation is provided "as is" and no warranty of fitness for any particular purpose is made or implied.
This document is the Independent Verification and Validation (IV&V) Standards document for the generic maintenance process architecture for the University of West Florida. The IV&V Standards document describes the basic procedures and standards applied to IV&V tasks. The IV&V group is responsible for the system testing performed on the software product each time a set of deficiencies is processed in a TISK. This document describes standards for the test cases developed, the implementation procedures required, and the resulting reports generated for testing. This document should be referred to by the IV&V group when performing product system testing.
ABSTRACT
LIST OF FIGURES
1.0 INTRODUCTION
1.1 BACKGROUND
1.2. PURPOSE
2.0 PROJECT TESTING ACTIVITIES
2.1 IV&V PURPOSE
2.2 FILE MANAGEMENT
2.2.1 Directory Structure
2.2.2 File Documentation
2.3 GENERAL PROCEDURES
2.3.1 Regression Testing
2.3.2 Test Case Development
2.3.3 Incident Forms
2.3.4 IV&V Completion Forms
REVISION HISTORY
The IV&V group of a software development team is responsible for final system testing. The system testing is performed to ensure the integrity of the overall product and to decide whether or not a new product version is ready for release to the users. System testing will test the software product as a whole to ensure that all features produce the desired results. Each implementation of system testing will run the complete baseline of system test cases, including any tests created during the tisk to test new features and verify bug fixes. The IV&V team performs testing each time a set of deficiencies is processed and changes are made to the software product.
The IV&V Standards document identifies standards to be used in developing and administering system testing during the software maintenance process. This document contains details on the creation, storage, documentation, development, and implementation of product system testing. In addition, forms used during testing are identified.
The IV&V group is responsible for performing system testing and developing new system tests as needed to ensure the integrity of the software product. The IV&V individuals perform testing each time enhancements are made to the software product.
The IV&V group will provide their time and resource estimates for inclusion in each Analysis Report. It is recommended that the IV&V team use the Delphi Estimation Process to arrive at these estimates. When this process is used, all members of the IV&V team should participate. The Delphi Estimation Process is also used to estimate resources for activities unrelated to IV&V. In this case, at least one member of the IV&V team should serve as an estimator.
As soon as the Analysis Report for a tisk is available, the IV&V team shall begin developing tests to revalidate the software.
A recommended directory would be /usr/u/course/proj_name/ivv/tests/tisk#/(OPERATING SYSTEM, i.e. UNIX). The test directory should contain all tests used to test the integrity of the product for the most current version. As changes are made to the product and new test cases are developed, these new test cases are added to the testing baseline for all subsequent system testing. Groups of test cases which test specific product features may be grouped together in separate directories. Appendix 'A' of the Software Configuration Management Document establishes the standard directory structure. Because the tests will be maintained it is recommended each test case filename have a unique name. Furthermore, it is recommended the test cases' filenames exhibit the following structure: tiskid.filename (e.g., t001.r2src.lst). For some operating systems, a different naming convention may be required, which should be documented in the README file.
A related working directory structure should be maintained on all supported platform operating systems (e.g., UNIX and DOS). Due to DOS naming convention limitations, UNIX IV&V directory and file structures may not be exactly replicated.
Each set of test cases should include at least one README text file which identifies the following items:
The IV&V group uses a regression test suite to ensure system integrity is maintained and there were no changes to the product's performance. These test cases must be maintained as a group for continuous regression testing after a set of changes are made to the software which do not change the functionality of the product.
If new features are added to the current system or existing functions require major changes which affect the overall performance of the product, new test cases may need to be developed to test the enhancements. Furthermore, if previously untested features exist, new test cases may be developed to test these features. The IV&V group will use the tisk's Analysis Report and updated documentation to determine the new functionality to be tested. These test cases are developed while the SE is implementing the product changes.
As events occur during testing which require additional investigation and/or produce unexpected results, the tester will fill out an Incident Report Form to document the problems encountered (see Figure 1).
DATE: 950615
INCIDENT FORM #: t001.01.irf
TISK:
COMPLETED BY: Joe Tester
PROBLEM DESCRIPTION (include which platform error
occurred on): MAIN.C has a division by zero
error on line 133 when 0 entered as input. Input seemed
valid from input prompt ("Enter a number:"). While
although 0 is a number, program will not gracefully
degrade. Problem occurred on both UNIX and DOS
platforms.
ACTION(S) TAKEN: Verified MAIN.C will always crash when
input is 0.
Figure 1. Incident Report Form
(Example responses in bold)
The Incident Report Form is created as a text file in a central project tisk directory identified by CM for the team's access. All Incident Report Forms should be combined into one report and placed after the IV&V Completion Form (see 2.3.4 below). Each Incident Report Form should have a designation consisting of the current tisk identifier, a testing sequence number to identify the latest occurrences, and a "irf" (i.e., tiskid.nn.irf).
After all test cases in the baseline and any new test cases have been implemented, and any Incident Report Forms have been generated, the tester fills out an IV&V Completion Form to record the testing results (see Figure 2).
DATE: 950630
IV&V REPORT#: t001.950630.icf
TISK: t001
COMPLETED BY: Joe Tester
PASS / FAIL Fail
TEST ADMINISTRATOR(S): Joe and Jane Tester
INCIDENT REPORT(S) GENERATED: t001.01.irf, t001.02.irf
COMMENTS: Created test case t007 to test new
requirements. Recommend regression to cell 600 due to
major bug introduced.
Figure 2. IV&V Completion Form
(Example responses in bold)
The IV&V Completion Form identifies whether or not the product passed the testing phase and is suitable for delivery to the customer. The IV&V Completion Form should be included at the beginning of the text file created for any Incident Report Forms. The naming convention for the IV&V Completion Form is a combination of the tisk identification, year-month-date group, followed by the suffix "icf" (e.g., t001.950630.icf). If no Incident Report Forms were generated, a new text file is created with the same naming conventions.