The Generic UWF Maintenance Process (GUMP) 1994, 1995, 1996, 1997, 1998 The University of West Florida. All rights reserved.
Permission is granted to reproduce and adapt this document provided credit is given to the University of West Florida. This documentation is provided "as is" and no warranty of fitness for any particular purpose is made or implied.
The class will begin their year with the identification of goals for the Metrics Program. Once the project's goals are determined, the team will identify the metrics that will best provide an insight to the project. The Metrics Program goals of product quality and/or development team productivity improvement should be described in accordance with the terminology of the specific project team's environment. The goals should be derived from and consistent with the overall goals of the project team as outlined in Sections 1.2, 1.3, & 1.4 of the Management Document. Quantitative goals of the expected improvements for a specific time period should be identified. These provide the basis for adjusting projected schedules and forecasts as each cell outcome is analyzed for deviation from the defined goals.
2.2 METRICS INFORMATION TO GATHER
Metrics information will include Timecard Information, Software Quality Assurance (SQA) Defect Information, Independent Verification and Validation (IV&V) Results, TISK Completion Information, and TISK Summary Information.
2.3 POTENTIAL PROBLEMS IDENTIFICATION
2.3.1 Team Productivity - The results of computed metrics should help to highlight team members who are being over- or underutilized.
2.3.2 Process Productivity - The results of computed metrics should help to highlight team cells, which require more or less resources.
2.3.3 Training Requirements - The results of computed metrics should help to highlight areas where additional training is needed.
2.3.4 Project Estimation - Historical metrics data should provide insight and a foundation for future project effort and duration.
This section will explain the data to be collected by the Metrics officer during the project class.
Timecard information is collected to identify process trends and ultimately improve the process. It is intended to assist future Software Engineering Project classes with their scheduling of resources.
3.1.1 Information To Be Collected
The Metrics officer will enter timecard data in the Timecard Database by amount of time spent on each cell, by each team member, by TISK. Time is measured in tenth-hour increments daily (i.e., 0.1) and the amount entered will be the total time for the week spent in a specific process cell within the TISK. If work is performed in more than one process cell for a TISK, then a separate entry is made for each additional process cell. TISK T000 (for the Fort Walton Beach Class) and TISK T900 (for the Pensacola Class) will be used as the TISK Number for all other activities not specified by the cells in the Process Architecture. Time that is not specific to a TISK will not be recorded (e.g., class time, papers, quiz preparation, etc.). The metrics officer will verify the appropriate process cell assignment for time entry upon receipt of timecards from team members. See Table 1.
The timecard period is Monday through Sunday weekly. The timecard will be completed weekly by midnight each Sunday for the previous week. Per the General Policy of supporting each other outlined in Section 4.0 of the Management Document, each team should decide by consensus if a grace period for completing timecards will be extended to team members (i.e., 30 minutes prior to class time). Timecards will be submitted electronically to the metrics officer using the Timecard form, which is provided, by the metrics officer.
3.1.3 Compliance Monitoring Responsibility
The Metrics officer should check the validity of the timecard data (i.e., if an inspection was conducted: Did the inspection team members enter their times for the appropriate TISK and cell? Is the time entered consistent from inspector to inspector for the same inspection?). The metrics officer will enter inspection data into the Metrics Database using the Inspection Defect Log Form.
3.2 SQA DEFECT DATA COLLECTION
SQA Defect Information will help to identify where problems may be occurring in a TISK. It will help identify any process problems within the phase of the TISK being reviewed.
3.2.1 Information To Be Collected
The SQA data will consist of type of material inspected, size of materials inspected, number of defects, defects by type, class, and severity, and estimates of rework effort. This information is collected and maintained in the Inspection Reports in the TISK directory, as defined in Appendix A of the SQA Document, and in each TISK Checklist. See table 2.
SQA defect results will be collected within 24 hours of completion of each inspection.
3.2.3 Compliance Monitoring Responsibility
The Metrics officer will review the TISK Process Checklist prior to the scheduled class period for team compliance of SQA Defect Information submitted for the prior week if an inspection was conducted. The PC will check with SQA to ensure that the collected data is maintained in the TISK directory. The metrics officer will enter inspection data into the Metrics Database using the Inspection Defect Log Form.
IV&V Results are needed to track the number of problems found during the Regression Testing phase of each TISK. The results will identify implementation problems and aid in tracking these problems to a source (requirements, design, coding, etc.) to facilitate process improvement.
3.3.1 Information To Be Collected
IV&V Results Information will consist of the number of tests, count of defects, and time estimates for every TISK. See Table 3.
The IV&V Tester will collect IV&V results within 24 hours of completion of the Regression Test Set. The IV&V Tester will then post the results to the TISK Checklist for review by the Metrics officer and the PC.
3.3.3 Compliance Monitoring Responsibility
If Regression Testing was completed, the Metrics officer will review the TISK Process Checklist prior to the scheduled class period for IV&V Defect Information submitted for the prior week. The PC will review the IV&V Report and check with the IV&V Team to clarify the nature of the problems.
3.4 TISK COMPLETION DATA COLLECTION
TISK Completion Information will summarize the code and/or documentation that was changed, as well as the total time spent. It will provide data for metrics analysis to develop productivity factors.
3.4.1 Information To Be Collected
The TISK completion data to be collected are the estimated and actual lines of code (LOC), lines of documentation (LOD), person-hours expended, and time spent in each cell. Estimated information will be recorded in the Analysis Report. Actual information will be recorded in the TISK Process Checklist or compiled from the Timecard Data for each TISK. See Table 4.
The TISK Completion Information will be collected as soon as possible (ASAP) by the SQA Manager, but no later than one week after completion of the Regression Test Set and will be posted to the TISK Checklist for review by the Metrics officer and the PC.
3.4.3 Compliance Monitoring Responsibility
The Metrics officer will review the TISK Process Checklist prior to the scheduled class period for team compliance of TISK Completion Information submitted for the prior week. The PC will also review the TISK Process Checklist and if necessary, will advise any team member to update his/her section (ASAP).
3.5 TISK SUMMARY DATA COLLECTION
TISK Summary Collection data will provide a brief summary of metrics information for every TISK. Metrics data of previous project classes provide an overall picture of metrics data collection, analysis, and reporting. Metrics data of similar TISKs also provide a basis for estimate and resource allocation for the future project classes. The metrics officer should overview the metrics data from previous software engineering classes before starting the GUMP++ process on any TISK, and report their findings to the project class for reference.
3.5.1 Information To Be Collected
The TISK summary collection data to be collected will be a brief summary of the TISK, total LOC/LOD for the TISK, and total man-hours. See Table 5.
The TISK summary collection data will be collected after completion of the metrics report.
3.5.3 Compliance Monitoring Responsibility
The metrics officer will review the TISK metrics report and record the needed information on the TISK Summary Estimation Report.
The metrics officer will provide to the class weekly reports and a final report at the end of the semester.
During the weekly reports the metrics officer will provide to the class any updates to metrics that have been recorded for every TISK being executed. This will be in the form of an Excel spreadsheet. A template for this Graphical Analysis Excel spreadsheet is available in the metric directory in the MetricsTemplates folder. This weekly report will also be used to inform the team of any developments in metrics and any late timesheets.
At the end of each semester, the metrics officer will report to the class the metrics for each TISK. It is beneficial to the client, the PC, and to all project members to know how long a TISK stayed in each phase, the number of man-hours required by each TISK, and how many defects are being found during inspections and IV&V. It is also beneficial to the future project classes for the efficient allocation of resources and more accurate TISK estimates. The SQA, IV&V, and metric report will all be compiled and discussed at the end of each TISK to complete cell 700 of the Process Architecture. A template for the final metrics report is available in the metric directory in the MetricsTemplates folder named TiskMetricsTemplate.xls. See Section 3.2, 3.3, and 3.4 for more information on what is contained in the final report.
5.0 METRICS DATABASE MAINTENANCE
The metrics database shall be maintained under the metric directory. The current method to record and store metrics data is in the format of an Excel spreadsheet. The metrics database prior to the school calendar year 2002-2003 was stored in different formats and, different structures. It has been moved to the archive folder in the metric directory. The metrics officer shall make the current address of the database known to the team at the beginning of each term or as necessary.
The metrics officer shall devise an alternative method by which time card information is gathered for use when the primary method is unavailable. This location shall be announced in class with instructions as to how and when to use this alternative method.
All modifications to the metrics database or timecard will be subject to the SEPG change proposal process outlined in the SEPG Document.
All the collected data should be uploaded to the UWF UNIX server and stored in the metric directory in the correct location. At the end of each semester, the metrics officer shall fill out the TISK metrics report and save it under the TISK’s folder in the Metrics directory.
5.1 METRICS DATABASE STRUCTURE
The metric database is composed of four sections, which are separated into three folders and one subfolder in the metric directory. The sections are MetricsTemplates, TiskSummaries, Tisks, and Archive.
The MetricsTemplates folder, in the metric directory, contains empty templates that are used to collect, analyze, and report metrics data. These templates have been created for future project class.
The TiskSummaries folder contains summaries for each TISK separated by each school year. The summaries are in the form of Excel spreadsheets. The TiskSummary spreadsheet is a “cheat sheet” for easy look up of metrics information. The TiskSummaries should have a brief description of the TISK, total LOC/LOD for the TISK, and total man-hours. This will aid in helping engineers find TISK similar to theirs. See section 3.5 for more information.
The Tisks folder contains a folder for every TISK that has metrics data. Every folder will be named after the TISK it represents. For example, T065 folder contains all metrics information for TISK 065.
5.1.3.1 TISK Folder - subfolder
This folder will contain a metrics report for each TISK and a weekly timecard, which is a compilation of weekly timecards for every TISK. The WeeklyTimeCards subfolder will include weekly timecards that combine all member times for the duration of the TISK.
The Archive folder contains all metrics information prior to the school year 2002 –2003. This folder never has to be modified and should be used only for historical purposes.
The metrics database is used to store all metric data collected for team members during the duration of a TISK. Many of the files in the database are in Excel format. The files will have to be FTPed or moved to a type of portable storage media and opened in a Windows operating system. During the semester the metric directory may be maintained on a Windows operating system, but the files and folder on the UNIX directory should be updated monthly, for the monthly backup. Text files should be FTPed in ASCII and Excel files should be FTPed in binary.
5.2.1 Data Collection Procedure
Metrics data should be collected weekly, after every inspection, and after IV&V testing is completed.
When starting a new TISK a new folder should be created to store the metric report and the weekly time card for that TISK. In the Tisks folder in the metric directory create a new folder named after the TISK’s number (i.e., T065). Then copy the WeeklyTimeCardTemplate.xls and TiskMetricsTemplate.xls into the newly created folder. Modify the name of the WeeklyTimeCardTemplate.xls to WeeklyTimeCardXXXX.xls (i.e., WeeklyTimeCardT065.xls) and modify the name of the TiskMetrcisTemplate.xls to TISKMetricsXXXX.xls. Input the new metrics information into the modified versions of the templates.
Metrics for TISK 900 is recorded differently than other TISKs. When a new school calendar year begins go to the folder titled 900 located in the Tisks folder in the Metrics directory. Copy the Tisk_Metrics_Template_900.xls and save it as TISK900_XXXX.xls in the 900 folder. The XXXX stands for the semester year (i.e. TISK900_0203 equals 900 metrics for school calendar year 2002-2003)
Download the "TimeCardTemplate.xls" into the MetricsTemplates folder, and send it to each team member or post it on the web for each team member to download a copy. This form is used weekly by team members to collect their metrics data. On Monday morning collect timesheets and record time. For team members who did not send a timecard send them a reminder email requesting a timecard or notification that they have no time to record. If no timecard or notification is received by the day of class, remind them once again in the metrics weekly status brief. The individual team member's timecards are not required to be saved, but can be for the duration of the school calendar year for reference purposes. All individual timesheets should be deleted at the end of the school calendar year.
After metrics data has been collected from each team member, calculate total hours by TISK and record all data received in the correct week, for the particular TISK, in the WeeklyTimeCardXXXX.xls.
Download “GraphicalAnalysis.xls “ from the MetricsTemplates folder. Enter the calculated total hours by TISK into the "Weekly Hours by TISK" table on WeeklyData tab and enter the estimated and actual hours by TISK into the "Estimated Compared to Actual Hours by Cell" for each cell. The graphs on EstvsAct and Total Hours by TISK tabs are linked to these tables and will automatically show the development trend of hours by TISK in each week and the visual comparison among all the TISKs. The GraphicalAnalysis.xls is for the purpose of the Weekly Reports only and does not need to be saved. The estimated vs. actual graph should be saved on the TISK metrics report in the TISKMetricsXXXX.xls. See Section 4 Metrics Reporting.
Complete the TISKMetricsXXXX.xls Excel spreadsheet that includes tables for IV&V, SQA, TISK Metrics Report, and Weekly time. See also Section 4 Metrics Reporting.
Download “TiskSummaryTemplate.xls” from the MetricsTemplates folder. After TISK is complete, record the required information and save the file as TiskSummaryXXXX.xls (where XXXX is the year it applies to. i.e. TiskSummary0203.xls for the 2002-2003 class) in the TiskSummaries folder. There will be only one TiskSummaryXXXX.xls file for each school year that contains all TISK summaries for that year. See also Section 5.1.2 TiskSummaries Folder.
5.2 DATABASE BACK-UP PROCEDURE
Upon completion of time entry each week, the Metrics officer will create a back-up copy of the database to be maintained on a zip disk, CD, or other portable storage media that can be accessed by any UWF PC.
Once a month, table data will be saved and uploaded to the UWF UNIX server, to be stored in the metric's directory. The files should be named appropriately, using the table name and back-up date in the filename. The metrics officer should provide a status report to the project class confirming completion of the monthly back-up procedures (similar to the SCM status report).
5.3 SEMESTER CLOSE-OUT PROCEDURE
At the end of the second project semester, the Metrics officer will provide two copies of the database to the client along with a status report. The status report should list the database file names, text file names and locations and any additional information that may prove beneficial to subsequent project classes.
| ASAP | As Soon As Possible |
| IV&V | Independent Verification and Validation |
| LOC | Lines of Code |
| LOD | Lines of Documentation |
| PC | Project Coordinator |
| SCM | Software Configuration Management |
| SEPG | Software Engineering Process Group |
| SQA | Software Quality Assurance |