CM AND DEVELOPMENT LABORATORY - DTIC · 2018. 11. 8. · linear regression analysis. An emphasis on...

LIBRARY TECHNICAL REPORT SECTION NAVAL POSTGRADUATE SCHO; MONTEREY. CALIFORNIA 039

fiD WS*}

PERSONNEL CM AND DEVELOPMENT

LABORATORY WTR 73-47 June 1973

AN ANALYSIS OF A FLEET POST-TRAINING PERFORMANCE

MEASUREMENT TECHNIQUE

Bernard A. Rafacz Paul P. Foley

APPROVED FOR PUBLIC RELEASE *,

DISTRIBUTION UNLIMITED

WASHINGTON NAVY YARD» WASHINGTON, D. C. 20390

- f

DEPARTMENT OF THE NAVY NAVAL PERSONNEL RESEARCH AND DEVELOPMENT LABORATORY

WASHINGTON NAVY YARD WASHINGTON. O. C. ' 20374

NPRDL:71:PPF:sjh 3930 Ser 341

19 June 1973

From: Commanding Officer To: Distribution List

Subj: Personnel Technology: Relating Individual Performance Effective- ness to Unit and Ship Effectiveness

End: (1) "An Analysis of a Fleet Post-Training Performance Measurement Technique," WTR 73-47, June 1973

1. Enclosure (1) is an evaluation of four different performance estimators relative to a criterion measure of absolute technician performance. The technicians were involved in fleet electronic maintenance activities in one of the ratings - EM, ET, FT, IC, RD, RM, ST, and TM.

2. The report is forwarded for information. Comments and recommendations are invited.

A. L. BLANKS

DISTRIBUTION LIST See report

UNCLASSIFIED Security Classification

DOCUMENT CONTROL DATA -R&D (Security classification ol title, body of abstract and indexing annotation must be entered when the overail report is classitied)

I. ORIGINATING ACTIVITY (Corporate author)

Naval Personnel Research and Development Laboratory Washington Navy Yard Washington, DC 2037*+

Ze. REPORT SECURI TY CLASSIFICATION

2b. GROUP

3. REPORT TITLE

An Analysis of a Fleet Post-Training Performance Measurement Technique

4. DESCRIPTIVE NOTES (Type ol report and inclusive datea)

a- AUTHORIS) (First name, middle Initial, last name)


6- REPORT DATE

June 1973

7«. TOTAL NO. OF PAGES

180 7b. NO. OF REFS

21 ia. CONTRACT OR GRANT NO.

b. PROJEC T NO.

PO 2-00U6 NR 150-336

»a. ORIGINATOR'S REPORT NUMBERIS)

WTR 73-^7 9b. OTHER REPORT NO(S) (Any other numbers that may be assigned

this report)

None 10. DISTRIBUTION STATEMENT

Approved for public release; distribution unlimited

II. SUPPLEMENTARY NOTES

N/A

12. SPONSORING MILITARY ACTIVITY

Personnel and Training Research Programs Office Office of the Naval Research

13. ABSTRACT

The purpose of this research effort is to validate the utility and effectiveness of a unique human performance measurement technique developed under ONR contract (N0001*t67C0107)■ Performance data on eight Navy ratings was collected from ships of LANTFLT and PACFLT. This report is the last in a series of technical reports on the statistical analysis of that data. A statistical analysis is provided on performance related data for electronic maintenance personnel sampled from 21 ships. Four different performance estimators, as functions of critical incidents, were evaluated. A detailed explanation of the distributional properties of the performance estimators is presented and an explanation of the factors that lead to the adoption of a curvilinear regression analysis for analysis of the data is discussed.

The results of the statistical analysis indicated that a certain combination of the performance data possessed moderate validity for appraising the absolute level of technician on-the-job performance in the EM, ET, FT, and IC ratings. Application of the technique to technicians in the RM, ST, and TM ratings was tenuous, but still appropriate, while none of the performance estimators seemed to be applicable to technicians in the RD rating. For this reason it would seem that the appropriateness of application of this technique to other ratings warrants investigation, perhaps by the approach employed in this report. In any event it has been observed that the technique possesses sufficient merit to be recommended for more widespread use within the U. S. Navy.

DD FORM 1 NOV es

O1O2-O1H-6S00 1473 (PAGE 1)

ITTJnT.ARST-FT-FT) Security Classification

UNCLASSIFIED Security Classification

KEY WO HDS

Personnel System Performance

Magnitude Estimation

Technician Reliability

Personnel System Model

Posttraining Performance Evaluation

Performance Prediction

Personnel Subsystem

DD ,FN0oRvM„1473 (BACK,

(PAGE 2) UNCLASSIFIED

"Security Classification

AD

WTR 73-47 June 1973

AN ANALYSIS OF A FLEET POST-TRAINING PERFORMANCE

MEASUREMENT TECHNIQUE


Principal Investigator: Paul P. Foley

Supported by Personnel and Training Research Programs,

Office of Naval Research, under Project Order Number

2-0046, NR 150-336

APPROVED FOR PUBLIC RELEASE

DISTRIBUTION UNLIMITED

NAVAL PERSONNEL RESEARCH AND DEVELOPMENT LABORATORY WASHINGTON, D.C. 20374

A LABORATORY OF THE BUREAU OF NAVAL PERSONNEL

FOREWORD

This research effort is being performed with the support of the Personnel and Train- ing Research Programs Office, Office of Naval Research, under Project Order No. 2-0046, Work Unit Number NR 150-336 entitled PERSONNEL TECHNOLOGY: Relating Individual Perform- ance Effectiveness to Unit and Ship Effectiveness. This is a final report on the analysis of the performance data collected in the course of the project.

Appreciation is expressed for the cooperation and assistance provided by Commander, Cruiser-Destroyer Force, Atlantic Fleet, Commander, Cruiser-Destroyer Force, Pacific Fleet, and Commander, Cruiser-Destroyer Flotilla NINE for providing the ships and men that took part in the data collection effort of this project.

The authors are indebted to Mr. William A. Sands of the Naval Personnel Research and Development Laboratory for his critical reading of the manuscript and invaluable suggestions on presenting the results of the data analysis.

The assistance of Dr. Arthur I. Siegel of Applied Psychological Services, Inc., Wayne, Pennsylvania, is also appreciated for providing some of the computer programs which were employed in analyzing the performance related data.

SUBMITTED BY

Paul P. Foley

Director, Personnel Measurement Research Division

APPROVED BY

E. M. RAMRAS

Acting Director, Psychological Research Department

ALVA L. BLANKS EUGENE M. RAMRAS Captain, U. S. Navy Technical Director Commanding Officer

n

SUMMARY AND CONCLUSIONS

Problem

The advent of a more streamlined Navy operating under reduced manning levels and heightened operational requirements imposes the need for accurate human-performance evaluation of ship personnel systems. On the personnel systems level, electronic technician reliability measurement is a necessary and integral part in the evaluation of particular combat systems of which technicians are components. The objective is then to develop and evaluate human-performance reliability estimates so as to be able to effectively alter the personnel system in order to maximize the overall performance of the system. The purpose of this project is to validate the utility and effectiveness of a unique human performance measurement procedure developed under a prior Office of Naval Research project and designed to improve upon existing performance measuring techniques in a systems environment.

Background and Requirements

Human reliability performance estimation can be accomplished by considering the Individuals being evaluated as components of a personnel system. This consideration allows the use of much of the theory already applied to equipment reliability estimation to be modified to human-performance estimation. After this theory is applied to evaluate the performance of human components in a personnel system, appropriate combinations of the individual performance estimators will provide a performance or reliability estimate of the personnel system itself.

In order to improve upon existing performance estimators of the human component in a personnel system, Dr. Arthur I. Siegel and his associates of Applied Psychological Services, Inc., Wayne, Pennsylvania, developed fleet post-training performance criteria for electronic maintenance personnel with the support of the Office of Naval Research. The cumulation of these efforts resulted in the development of unique human performance measurement techniques, closely allied with equipment reliability estimation techniques. Siegel also developed procedures for combining the technician performance estimates in appropriate ways in order to estimate team, ship or squadron performance.

An outgrowth of the prior research effort was the suggestion that the techniques be Introduced on a limited basis to determine how they may be modified or elaborated upon. The Naval Personnel Research and Development Laboratory has been validating the utility and effectiveness of these techniques. The main technical objectives of this validation effort are to determine the validity of the performance measurement techniques, Identify modifications required to maximize their possibility for implementation and to comment on the statistical properties of those techniques as related to their effectiveness in an operational context.

Approach

In order to realize an efficient and timely data collection effort, optical scanning instruments were utilized similar to those employed by Applied Psychological Services in prior research efforts.

The main data collection instruments were:

1. Job Performance Questionnaire (JPQ) ANSWER SHEET - this form records supervisory estimates of the total number of a technician's uncommonly effective (EUE) and uncommonly Ineffective performance (EUI) that the supervisor has observed during a specified time period.

111

2. Technical Proficiency Checkout Form (TPCF) - this form records the level of technical complexity at which a man is able to perform without direct supervision.

3. Personnel Identification Information Form (PIIF) - this form records demographic data on the technician being evaluated.

On each of the above instruments an individual in one of the electronic maintenance ratings EM, ET, FT, IC, RD, RM, ST, and TM was evaluated by his supervisor. On the basis of the total number of uncommonly effective (£UE) and the total number of uncommonly ineffective (EUI) incidents of performance recorded on the Job Performance Ques- tionnaire (JPQ), three different performance estimators were developed previously by Applied Psychological Services. These estimators are functions of the total number of uncommonly effective (ZUE) and the total number of uncommonly ineffective (ZUI) incidents of performance observed by the supervisor on each of eight job dimensions characteristic of electronic maintenance activities. The three estimators of human reliability are:

1. Series Reliability Estimate (SRE)

2. Series-Parallel Reliability Estimate (PRE)

3. Geometric Mean Reliability Estimate (GRE)

In addition, a fourth measure of technician on-the-job performance developed in the course of this research effort was:

4. Weighted-Average Reliability Estimate (WRE).

By adopting the Technical Proficiency Checkout Form (TPCF) as a criterion measure of technician on-the-job performance, the degree of association between each of the four performance estimators and criterion measure was developed for various locations and each rating. Furthermore, a curvilinear regression analysis was applied to determine the best linear relationship between those variables.

While the purpose of this project was not to evaluate the criterion measure (TPCF), conditional and joint frequencies of the job tasks by rating revealed the modifications needed on the TPCF to make it more current. Furthermore, from the conditional and joint frequencies it was possible to develop a competency level for each rating and permit an in-depth analysis of the job task structure for those ratings.

Comparisons between ships and ratings were made by employing the WRE since it was identified as a more promising performance estimator. Initially a two-way Analysis of Variance was employed on the sample data. However, because of the significant interaction that was found to exist between ships and ratings, comparisons between ships (ratings) were made for a fixed rating (ship).

Finally, product-moment correlation coefficients were developed between various performance variables and several demographic variables. In particular, Basic Test Battery scores (GCT, ARI, MECH, and CLER scores), usually employed to predict actual school success, were investigated in order to determine the degree of association between those scores and the measures of on-the-job performance developed in this research effort.

Findings

In order to.determine the appropriateness of standard statistical techniques or tests employed for various purposes in this research effort, initial findings were concerned with the results of an analysis of the distributional properties of the predictor and criterion variables. From an application of appropriate goodness-of-fit tests for

IV

normality to the sample data, only the predictor variable WRE could be termed normally distributed. This result was also generally true when individual ratings were similarly studied. These characteristics of the sample data necessitated the use of a curvilinear regression analysis. An emphasis on only the least-squares analysis resulting from an application of that technique was employed in order to determine the relative merits of each of the predictor variables.

A comparison of the multiple correlation coefficients resulting from the curvilinear regression analysis applied across all ratings revealed that a straight-line was the best fit to the sample data. The product-moment correlations between the predictor variables (SRE, PRE, GRE, and WRE) and the selected criterion variable suggested moderate promise on the part of the WRE for appraising the on-the-job performance of individuals involved in electronics maintenance activities. However, the relatively low multiple correlation coefficients suggested a significant degree of unexplained criterion variance. As such the analysis was applied by rating. Relevant findings by rating revealed that in almost e^jery rating the WRE demonstrated more promise for appraising on-the-job technician performance than the other estimators. The WRE was considered a more promising estimator in the sense that the sample product-moment correlation coefficients were generally of a larger magnitude and, by some fit to the data, the WRE seemed to account for more criterion variance. For example, the WRE indicated product-moment correlations of .492, .445, .430, and .434 for the EM, ET, FT, and IC ratings, respectively, while the next more promising estimator (GRE) demonstrated product-moment correlations of .301, .508, .374, and .387 for the EM, ET, FT, and IC ratings, respectively. It is to be noted that while the WRE may not necessarily be a statistically significantly different estimator in terms of producing higher correlations with the criterion variable and explaining more criterion variance, the sample data results did tend to demonstrate that the WRE was the more promising estimator in that consistently the sample results did produce higher product-moment and multiple correlation coefficients for the WRE. Altogether the WRE demonstrated very promising results on the EM, ET, FT, and IC ratings and fair results on the RM, ST, and TM ratings. None of the performance estimators were at all promising in the RD rating.

Relevant findings from an analysis of the Technical Proficiency Checkout Form revealed that it was a very instructive instrument for determining technician proficiency In one job task in relation to another. Only one job task - the calibrating of the equipment used by the technicians - seemed to be out of place in the hierarchial order of the job tasks represented by this instrument.

Findings of the multiple comparison of ships and ratings included the result that significant interaction exists between ship-rating combinations. This resulted in the development of multiple comparisons of ships (ratings) for a fixed rating (ship). From this analysis it was found that no pairwise significant difference exists between ships. On some ships a pairwise significant difference exists between some ratings, but no pattern emerged across ships as to which rating(s) demonstrated a higher or lower mean performance level.

The product-moment correlation coefficients developed between the demographic variables and the performance variables were of the same magnitude as those which are usually found to exist between predictor variables and measures of on-the-job performance. Prom- ising results were found in this research effort on the relationship of demographic variables to on-the-job performance.

Conclusions

Employing the TPCF as a criterion measure of technician on-the-job performance it may be stated that the following list represents an accurate portrayal of the performance measurement technique that was researched :

1. The distribution of the predictor variables SRE, PRE, and GRE are generally skewed in one direction or another, while the criterion variable derived from the TPCF is negatively skewed. The WRE is normally distributed. These conclusions are for each rating and across ratings.

2. The WRE is a more promising type of performance estimator. It has greatest utility when applied to technicians in the EM, ET, FT, and IC rating and fair promise for application in the RM, ST, and TM ratings.

3. None of the performance estimators is appropriate for use upon technicians in the RD rating.

4. The Technical Proficiency Checkout Form (TPCF) demonstrated significant promise for appraising the job task structure and proficiency of electronics maintenance personnel without restriction to a particular rating.

5. In all ratings a more current factor analytic task analysis would be desirable before implementation of the technique. This would involve a revision of the job activity factors on the JPQ ANSWER SHEET and job task descriptions on the TPCF by rating.

6. It is recommended that a validation of the performance variables (SRE, PRE, GRE and WRE) be completed before the technique is applied to other than those ratings researched in this report.

In conclusion it is to be noted that the performance measurement technique that was researched is of significant merit to be considered for practical application in the U.S. Navy (particularly in the EM, ET, FT, and IC ratings). At the present stage in the art of developing performance measurement techniques, the technique that was researched is probably the best performance measurement procedure presently available for application within the U. S. Navy.

VI

REPORT USE AND EVALUATION

Feedback from consumers concerning the utilization of reports is a vital element in improving products so that they better respond to specific needs. To assist the Chief of Naval Personnel in future planning, it is requested that the use and evaluation form on the reverse of this page be completed and returned. The page is preaddressed and franked; fold in thirds, seal with tape, and mail.

Department of the Navy

Official Business

POSTAGE AND FEES PAID

DEPARTMENT OF THE NAVY DoD-316

Commanding Officer Naval Personnel Research and Development

Laboratory Building 200 Washington Navy Yard Washington, D. C. 20374

Report Title & No.^ Analysis of a Fleet p0St-Training Performance Measurement Technique WTR 73-47

1. EVALUATION OF REPORT. Please check appropriate column.

FACTORS RATING

LOW AVE HIGH COMMENTS

USEFULNESS OF DATA

TIMELINESS

COMPLETENESS

TECHNICAL ACCURACY

VALIDITY OF RECOMMENDATIONS

SOUNDNESS OF APPROACH

PRESENTATION AND STYLE

OTHER (Please Explain)

2. USE OF REPORT. Please fill in answers as appropriate. Use continuation pages as necessary.

A WHAT ARE YOUR MAIN USES FOR THE MATERIAL CONTAINED IN THE REPORT?

B. ARE THERE ANY SPECIFICS OF THE REPORT THAT YOU FIND ESPECIALLY BENEFICIAL (OR THE REVERSE) TO YOUR AREA OF RESPONSIBILITY? IF SO, PLEASE AMPLIFY

C. WHAT CHANGES WOULD YOU RECOMMEND IN REPORT FORMAT TO MAKE IT MORE USEFUL?

D. WHAT TYPES OF RESEARCH WOULD BE MOST USEFUL TO YOU FOR THE CHIEF OF NAVAL

PERSONNEL TO CONDUCT?

E. DO YOU WISH TO REMAIN ON THE DISTRIBUTION LIST? YES. NO

F. PLEASE MAKE ANY GENERAL COMMENTS YOU FEEL WOULD BE HELPFUL IN PLANNING THE RESEARCH PROGRAM.

NAME: CODE:

ORGANIZATION: ADDRESS:

G-PO 861-277

TABLE OF CONTENTS

Page

FOREWORD SUMMARY AND CONCLUSIONS USE AND EVALUATION FORM (AUTHORIZED TEAR-OUT FORM) LIST OF TABLES LIST OF FIGURES

INTRODUCTION

BACKGROUND

Personnel Performance Estimation Job Performance Questionnaire Estimation of Technician Reliability Validation of Performance Estimators

Technical Proficiency (TP) Score Technical Proficiency Checkout (TPC) Level . . .

Main Results of Prior Studies

PURPOSE . . ■

DATA COLLECTION

Data Collection Instruments Analyses Based on the Data Collection Effort

Analyses Based on the Job Performance Questionnaire . . . . Problems 1n Calculating Performance Estimates A Convention for Estimating Performance 1n Certain Job Activities

Analyses Based on the Technldal Proficiency Checkout Form.

APPROACH

RESULTS AND DISCUSSION

Validity of the Performance Estimators Tr1serial Correlation Analyses Curvilinear Regression Analysis Within Rating Analyses

Technician Job Competency Evaluation Job Task Analyses Conditional Frequencies for Job Tasks Joint Frequencies for Job Tasks

Multiple Comparisons of Ships and Ratings Distributional Properties of the Performance Estimators Choice of a Variable An Analysis of Variance with Interaction Testing for Significant Ship (Rating) Effects for a Particular Rating (Ship)

Association Between Demographic and Performance Variables Utility of Demographic Variables in Performance Prediction . . .

Demographic Variable Prediction on Combined Locations . . . Demographic Variable Prediction in the EM, ET, FT, and IC Ratings

11 111

x1 X11

1

1

2 2 3 4 5 5 6

7 7 7

14

14 14

18

19

19 19 22 24 29 29 32 33 34 35 35 36

37 39 39 39

39

ix

TABLE OF CONTENTS (Contd)

Page

SUMMARY AND CONCLUSIONS 42

Summary of Data Analyses 42 Evaluation of the Performance Estimators 42 The Criterion Measure of On-the-Job Performance 42 Comparisons Between Ships and Ratings 43 Analysis of Demographic Data 44

Observations on the Use of Composite Reliability Values 44 Effect of the Convention on the Performance Estimates SRE, PRE, and GRE 45

The Convention in the RD Rating 45 The Convention in Other than the RD Rating 46

Observations on the Use of Composite Reliability Values. ... 46 Conclusions 46

APPENDIX A - Performance Evaluation Instruments A-l

APPENDIX B - Data Collection Procedures. . B-l

APPENDIX C - Instructions for Ship Liaison Officer C-l

APPENDIX D - Instructions for Supervisor D-l

APPENDIX E - Problems 1n Calculating Reliability Ratios E-l

APPENDIX F - The Adoption of a Convention for Each Job Activity and Rating F-l

APPENDIX G - A Few Remarks on Curvilinear Regression G-l

APPENDIX H - Results of the Curvilinear Regression on 949 Technicians Evaluated H-l

APPENDIX I - Results of the Curvilinear Regression Analysis by Rating. . 1-1

APPENDIX J - Job Task Conditional and Joint Frequencies by Rating. ... J-l

APPENDIX K - The Utility of the WRE K-l

REFERENCES

DISTRIBUTION LIST

LIST OF TABLES

Page

1. NUMBER OF MEN IN EACH RATING AND SHIP 8

2. CLASS INTERVALS FOR HISTOGRAMS 13

3. NUMBER OF MEN AT EACH TECHNICAL PROFICIENCY CHECKOUT (TPC) LEVEL LOCATION NO. 1 15

4. NUMBER OF MEN AT EACH TECHNICAL PROFICIENCY CHECKOUT (TPC) LEVEL LOCATION NO. 2 16

5. MEAN RELIABILITIES 20

6. TRISERIAL CORRELATION ANALYSES 21

7. CURVILINEAR REGRESSION ANALYSES BY LOCATION 23

8. EM CURVILINEAR REGRESSION ANALYSES (N = 97) 25

9. ET CURVILINEAR REGRESSION ANALYSES (N = 173) 25

10. FT CURVILINEAR REGRESSION ANALYSES (N = 154) 26

11. IC CURVILINEAR REGRESSION ANALYSES (N = 58) 27

12. RD CURVILINEAR REGRESSION ANALYSES (N = 139) 27

13. RM, ST, AND TM CURVILINEAR REGRESSION ANALYSES 28

14. JOB TASK ANALYSES 31

15. EM RATING CONDITIONAL FREQUENCIES (N = 97) 32

16. EM RATING JOINT FREQUENCIES (N = 97) 33

17. g1 TEST STATISTIC VALUES FOR THE PERFORMANCE ESTIMATORS 35

18. AN ANALYSIS OF VARIANCE FOR 21 SHIPS AND 8 RATINGS 36

19. AN ANALYSIS OF VARIANCE FOR 21 SHIPS AND 4 RATINGS 37

20. MULTIPLE COMPARISON OF FOUR RATINGS ON NINE SHIPS 38

21. DEMOGRAPHIC CORRELATIONS ON EIGHT RATINGS 40

22. DEMOGRAPHIC CORRELATIONS ON FOUR RATINGS 41

E-l NUMBER AND PROPORTION OF TECHNICIANS IN PROBLEM AREAS. . E-4

F-l NUMBER AND PROPORTION OF TECHNICIANS IN PROBLEM AREAS ON A PARTICULAR SHIP F-5

F-2 COMPOSITE RELIABILITY VALUES . F-6

K-l NUMBER AND PROPORTION OF TECHNICIANS IN EACH PROBLEM AREA K-4

xi

LIST OF FIGURES

Page

1. Histogram of Series Reliability Estimates (SRE) 9

2. Histogram of Series-Parallel Reliability Estimates (PRE) 10

3. Histogram of Geometric Mean Reliability Estimates (GRE) 11

4. Histogram of Weighted-Average Reliability Estimates (WRE) 12

5. Histogram of Technical Proficiency Scores (N = 949) 17

xn

INTRODUCTION

Current and expanded commitments of a modern sophisticated Navy will require greater operational effectiveness of fleet personnel systems. The Navy will have to operate fleet personnel systems at optimal effectiveness levels and maintain fleet readiness. Together with these requirements, the advent of an all-volunteer force and smaller ship systems with reduced manning levels make the problem of optimizing, and evaluating, personnel system effectiveness a complex and critical problem.

While performance assessment serves a multitude of purposes, it has been seen as an especially valuable tool when applied to the areas of optimizing personnel system performance, providing feedback on naval school effectiveness, the interpretation of man-machine interaction, and as a factor in the optimal assignment of men to jobs. It is in these applications that personnel systems performance measurement will be able to address some of the more critical present-day Navy problems. For example, many ship systems are operating with Increasingly sophisticated and complex equipment. However, 1s it necessarily true that technicians of comparable sophistication in training and mental ability need to be employed to operate, maintain, and repair that equipment 1n order that the ship complete its mission? While this report does not address that specific question, future research employing individual, and system, performance measurement may reveal that the Navy could effectively utilize personnel in those positions who may now be rejected for some reason related to their projected on-the- job performance in those positions. As such there is a definite need for valid and reliable individual system performance assessment.

In order to achieve a performance appraisal of personnel systems, recent research has been directed towards viewing personnel system performance estimation as analogous to equipment reliability estimation. This particular approach views individuals (in the system) as "components" in the personnel system. This viewpoint, and an application of the techniques already employed In equipment performance estimation, reduces personnel system performance estimation to an evaluation of the performance of the Individuals in the system. Once the individual performance estimates have been made, meaningful combinations of these estimates can provide estimates of personnel system performance. As such the Initial problem reduces to that of finding accurate and valid measurements of Individual performance.

Much recent research 1n the area of Individual performance measurement has been directed towards examining Individual performance as a function of the extremes of behavior (critical Incidents) of an Individual's performance. These functions of behavior can provide estimates of Individual performance. By appropriate combinations of the Individual performance estimates, estimates of personnel system performance can be developed. This report will address the validity, application, and Implications of a particular procedure that employs the critical incidents technique to estimate the performance of electronics maintenance personnel.

BACKGROUND

The purpose of this section is to give the reader a logical development of the performance measurement techniques employed by Applied Psychological Services and others. Fundamentally these researchers employed a critical incidents technique in deriving estimates of human performance.

Generally the main approach is to estimate the performance of a particular personnel system as a function of the performance of individuals that are a part of the system. This necessarily reduces personnel system performance estimation to a discussion of estimators of individual performance where individuals are the components of the system. Combining the individual estimates will provide estimates of personnel system performance.

Personnel Performance Estimation

Let UE (UI) represent an uncommonly effective (uncommonly Ineffective) Incident of performance observed by a rater 1n a certain time period on some Individual under observation. Furthermore, let £UE (£UI) represent the total number of uncommonly effective (uncommonly Ineffective) incidents.of performance observed. Using these functions of critical incidents, Whltlock [21] demonstrated that there is a definite straight line or curvilinear relationship between EUE (or the ratio ZUE/EUI) and corresponding performance evaluations. Prior results such as this provided significant evidence that the application of a critical Incidents technique to performance evaluation is a valid and useful approach.

Following upon the results of Whltlock, for example, Applied Psychological Services further developed and applied the above mentioned techniques to the post-training performance evaluation of Individuals 1n various avlonlc or electronic ratings 1n the U. S. Navy. In particular Siegel and Pfeiffer [18], utilizing estimates of uncommonly effective and uncommonly ineffective performances, showed that these estimates possess merit as useful Indicators of overall personnel proficiency. The researchers employed magnitude estimates of the number of uncommonly effective and uncommonly ineffective performances relative to a short prior period for avlonlc personnel. They derived a performance Index from the ratio of the sum of uncommonly effective performance (SUE) to the sum of uncommonly effective plus the sum of uncommonly Ineffective performance (ZUI), namely (ZUE/[£UE + ZUI]). Siegel and Pfeiffer [18] concluded that: (1) magnitude estimates of uncommonly effective and Ineffective performance yielded useful data which could form the basis for a personnel subsystem reliability index; (2) the . ratio of the sum of uncommonly effective to the sum of uncommonly effective plus the sum of uncommonly ineffective performance yields an index which discriminates in the anticipated direction; and, (3) the obtained avionic personnel subsystem index could be utilized for post-training performance appraisal, personnel placement, and squadron evaluation purposes.

Job Performance Questionnaire

Eight job activity factors descriptive of naval avionic electronic maintenance jobs were isolated by Siegel and Schultz [19] and are shown in Appendix A, page A-7, along with their definitions. These factors formed the basis of the Job Performance Questionnaire (JPQ), an instrument for recording the frequency of critical incidents for each of the job activity factors. Siegel and Federman [16] demonstrated the utility and practicality of a Job Performance Questionnaire (Appendix A, page A-3) for technicians in the eight electronic maintenance ratings EM (electrician's mate), ET (electronics technician), FT (fire control technician), IC (interior communications electrician), RD (radarman), RM (radioman), ST (sonar technician), and TM (torpedoman's mate). From an evaluation of 499 technician in those ratings, the researchers found that the JPQ yields an estimate of the total number of uncommonly effective and uncommonly ineffective Incidents of behavior on the eight identified job activity factors.

1 AT 1 numbers enclosed 1n brackets refer to corresponding numbers of documents and publications listed under REFERENCES.

Specifically, for each job activity the reliability ratio (ZUE/CzUE + zui]) yields an estimate of the probability of effective performance for the individual technician on the particular job activity considered. Then the reliability ratios are compounded to provide estimates of individual effectiveness or reliability of on-the-job performance across the job activities. It was reported by Siegel and Federman [i6] that estimates of uncommonly effective and of uncommonly ineffective behavior along all eight dimensions of job activities could be combined into a meaningful measure of technician effectiveness. Moreover, they indicated that the individual technician effectiveness values can be further treated to form effectiveness values for ratings, ships, and squadrons.

Estimation of Technician Reliability

Employing the reliability ratio concept (ZUE/[zUE-+ ZUI]), Siegel and Federman [16] have developed the following three reliability estimates:

1) Series Reliability Estimate (SRE) The series reliability measure of total effectiveness for

an individual 1s derived by multiplying individual job activity reliability ratios to yield a total reliability score, i.e.,

Rs = r-| x r2 x ... x r%

where R. = series reliability^, and

r^ = (ZUE/[ZUE + ZUI]) is the reliability ratio for the itn job activity.

It is to be noted that use of the series reliability estimate requires the assumption that performance reliability on each job activity is independent of performance reliability on other job activities.

2) Series Parallel Reliability Estimate (PRE) Siegel and Federman [16] reported that "... the series and

series-parallel reliabilities provide measures of personnel proficiency relative to performance on the entire job (i.e., all eight job activities).", (p. 46). The series-parallel estimate of individual proficiency is defined as:

Rp = Rs x(2 - r^ x ... x(2 - rQ)

where Rs and r^(1 = 1 8) are defined in 1) above.

2 It 1s helpful to note that the series reliability estimate possesses the fol lowing properties:

a) for each of the 1 = 1 8 job activities

0 1 r1 1 1 • anc'» therefore,

b) 0 ^R < 1

c) Rs i smallest r...

This particular estimate tends to provide a more optimistic estimate of individual performance. However, the content validity and derivation of this estimate deserves further development.

3) Geometric Mean Reliability Estimate (GRE) Let r*. r*. r*. and r* be the four highest job activity reliability

ratios of the eight reliability ratios for a technician being evaluated. The geometric mean reliability for the technician is defined as:

Rg = 4/r* x r* x r* x r*.

This particular estimate is an estimate of individual performance that stresses the strong points of an individual's performance. However, it also tends to ignore his weak points and, therefore, should be used with caution.

In addition to the three performance estimators (SRE, PRE, and GRE) previously introduced, this report will also discuss an estimate that weights the importance of each job activity in determining a technician's overall performance.

4) Weighted Average Reliability Estimate (WRE) To develop this estimate let:

NJ = number of job activities on which the technician actually worked;

i = index for the sum over the job activities on which the technician actually worked;

ri = the reliability ratio for the ith job activity;

wj = weight denoting the importance of the itn job activity in estimating the technician's overall performance.3

The Weighted-Average Reliability Estimate (WRE) of technician effectiveness 1s then defined as:

NJ

1=1 n 1

Validation of Performance Estimators

In addition to performance data collected on the JPQ, performance data were collected by Siegel and Federman [16] by means of an evaluative Instrument called the Technical Proficiency Checkout Form (TPCF) (Appendix A, page A-5). The TPCF consists of eight job tasks listed in hierarchlal order from easiest to most difficult. The eight tasks meet the Guttman requirements for scalability (see for example, Guttman [11]). Siegel, Schultz, and Lanterman [20], 1964, employed the scale underlying the eight tasks to determine the cutting points for placing avionic petty officers, third class and strikers 1n one of three levels of technical proficiency. The procedure for placing a technician in one of three levels of technical proficiency is accomplished by means of a Technical Proficiency (TP) score developed from the TPCF.

Hhe procedure for deriving the weights is given in Appendix K.

Technical Proficiency (TP) Score

Define the function F^i = 1, ..., 8) as:

Fi =

1 if the technician is CHECKED OUT on the ith

task of the TPCF

0 if the technician is NOT CHECKED OUT on the ith task of the TPCF.

The TECHNICAL PROFICIENCY (TP) score for a technician 1s then defined as:

8 TP score = ]Z F^.

Technical Proficiency Checkout (TPC) Level

Three TECHNICAL PROFICIENCY CHECKOUT (TPC) levels are:

Level 1: above desirable

Level 2: below desirable but at least minimally acceptable

Level 3: below minimally acceptable.

Siegel, Schultz, and Laterman [20] based this trlchotomous division of the TPCF on supervisor's judgments of the performance level required for achieving the objectives given in Appendix A, page A-8.

The procedure for determining the TPC level was reported on 1n the previously mentioned report and is given by:

a) add 0.5 to TP score for an Individual. Let TP* be the resultant score.

b) 1f TP* < 3.92, the TPC level = 3

c) 1f 3.92 < TP* < 5.63, the TPC level = 2

d) 1f TP* > 5.63, then TPC level = 1.

Siegel and Flschl [17] correlated technicians TPC levels with the technicians total scores on a performance test. Employing a trlserlal correlation coefficient (see, for example, Jaspen [13]) as an estimate of the product-moment correlation, they found a trlserlal correlation of .40. When corrected for the lack of perfect reliability in the performance test criterion, the correlation became .74. On the basis of their investigation of the then concurrent validity of the TPCF, they concluded that the Technical Proficiency Checkout Form, "... previously shown to be reliable and practical, may now be considered to possess a substantial degree of validity for appraising the absolute proficiency level of avionics technicians in the fleet.", (p. 46). Finally, Siegel and Federman [16] recorded a trlserlal correlation of .38 between the TPC level of the technicians evaluated and their Series Reliability Estimate (SRE), concluding "... there is some basis to believe that the JPQ results correlate with on-the-job performance.", (p. 62).

Main Results of Prior Studies

Important conclusions of prior reports relative to the merits of the Series Re- liability Estimate (SRE), Series-Parallel Reliability Estimate (PRE), Geometric Mean Reliability Estimate (GRE), and the Technical Proficiency Checkout Form (TPCF) are as follows:

1. Reliability ratios of the form 2UE/(2U.E + IUI) indicate the probability of effective performance on a particular job activity for the technician being evaluated.

2. The JPQ is an instrument for providing magnitude estimates of £UE and £UI for each man being evaluated by his immediate supervisor.

3. The TPCF possesses a substantial degree of promise for appraising the absolute level of avlonic technician proficiency.

4. There is some basis (triserial correlation of .38 with TPC level) to believe that the SRE is a reasonably good estimator of on-the-job performance.

PURPOSE

The purpose of this research effort is to report on the data collection effort and data reduction methods and analyses that have been performed for the Office of Naval Research (ONR) under the project entitled Personnel Technology: Relating Individual Performance Effectiveness to Unit and Ship Effectiveness (Project Order Number: PO 2-0046 NR 150-336). The goal of this research project is to provide an empirical basis for assessing the utility to the Navy of a performance measurement technique developed under a prior ONR contract. Under that contract Dr. Arthur I. Siegel, Philip J. Federman, and their associates of Applied Psychological Services, Inc., Wayne, Pa., developed fleet post-training performance evaluative measures which have potential value for eventual widespread implementation within the U. S. Navy. The results of their study were contained within the report - Development of Performance Evaluative Measures: Investigation into and Application of a Fleet Post-Training Performance Evaluative System [16^ An outgrowth of that effort was the suggestion that the technique be employed" on a limited basis by a Navy laboratory to identify areas of modification upon operational testing. In response to that recommendation the Naval Personnel Research and Development Laboratory submitted a proposal to ONR to accomplish that task. Es- sentially the research effort undertaken by NAVPERSRANDLAB was accomplished by replicating the efforts of Siegel and Federman [16],

A second major objective of this research effort was to further develop the performance measurement techniques of Siegel and Federman [16]. Furthermore, similarly related performance measurement techniques were researched with a view towards possible implementation of those techniques within the U. S. Navy.

DATA COLLECTION

The procedures employed in data collection for this project closely paralleled those employed by Siegel and Federman [16], with some modifications in the research instruments. This procedure was adopted so that a similar statistical analysis on the same type of population would permit some comparisons to be made between the results of this research effort and the results obtained by Siegel and Federman [16].

Every effort had been made to minimize interfering with normal shipboard duties. For this reason the data collection procedure centered upon the efforts of ship liaison officers conducting the data collection aboard each ship. Appendix B contains a discussion on the procedures for the data collection reflecting various aspects of the effort that resulted in the orderly and successful completion of the task.4

Data Collection Instruments

An example of the performance evaluation forms that were completed by each supervisor for each technician evaluated are given in Appendix A. The instruments are optical scanning forms, thus, making them machine-readable and more capable of being placed 1n an operational mode. In particular the forms were:

1) Job Performance Questionnaire (JPQ) ANSWER^SHEET This form serves the same purpose as the JPQ discussed earlier,

I.e., to record estimates of the total number of uncommonly effective (SUE) and uncommonly ineffective (£111) performances the supervisor has observed on each of the eight job activities for each man he 1s evaluating.

2) Technical Proficiency Checkout Form (TPCF) This form 1s essentially identical to the TPCF used by Siegel

and Federman [16].

3) Personnel Identification Information Form (PHF) This form was concerned with the background data of the man

being evaluated. It was completed 1n part by his supervisor with the Administrative Officer providing the remaining informa- ti on.

Analyses Based on the Data Collection Effort

Employing the data collection Instruments discussed in the previous section, performance data were collected with the assistance of men and ships of Commander; Cruiser- Destroyer Flotilla NINE (located at San Diego, California) and men and ships of Com- mander; Cruiser-Destroyer Force Atlantic Fleet (located at Newport, Rhode Island, and Boston, Massachusetts).5 The participating ships and type are shown in Table 1 along with the number of men evaluated by rating and ship for each location.

Analyses Based on the Job Performance Questionnaire

A descriptive analysis of each of the performance estimators (SRE, PRE, GRE, and WRE) derived from the Job Performance Questionnaire (JPQ) 1s presented in Figures 1, 2, 3, and 4 and 1n the form of histograms. Each of these histograms was developed on 949 technicians and based on the performance estimators which are continuous over the range of 0.0 to 1.0. Class intervals are numbered from 1 through 21 where a given class interval 1s of length .05. Class intervals corresponding to each of the numbered intervals are provided in Table 2.

4 For the Interested reader Appendices C and D contain the instructions for the ship

liaison officer and the technician supervisor, respectively.

5Henceforth 1n this report Location No. 1 will refer to ships at San Diego, Calif., and Location No. 2 will refer to ships at either Newport, R.I., or Boston, Mass.

TABLE 1

NUMBER OF MEN IN EACH RATING AND SHIP

CRUDESFLOT NINE (Location No. 1)

Ship Type EM ET FT IC RD RM ST TM TOTAL

USS AGERHOLM i DD-826 6 7 7 4 5 5 5 3 42

USS BROOKE DEG-1 6 8 16 3 10 8 9 1 61

USS GRAY DE-1054 5 TO 5 4 9 7 9 1 50

USS HORNE DLG-30 4 12 11 6 5 5 8 2 53

USS HULL DD-945 3 9 8 3 9 9 8 1 50

USS JOUETT DLG-29 5 TO 12 5 7 6 9 2 57

USS PRAIRIE AD-15 6 29 12 6 0 10 0 13 76

USS RUPERTUS DD-851 5 11 3 2 9 9 10 1 50

USS SHELTON DD-790 4 5 5 1 5 5 5 0 30

USS SOUTHERLAND DD-743 5 8 9 5 11 11 9 2 60

USS HENRY W. TUCKER DD-875 5 9 0 3 13 11 11 2 54

TOTAL 54 na 88 42 83 86 83 28 582

CRUDESLANT (Location No. 2)

Ship Type EM ET FT IC RD RM ST TM TOTAL

USS BASILONE DD-824 4 4 5 0 0 5 5 0 23

USS DEALEY DE-1006 3 6 5 2 3 4 6 1 30

USS DEWEY DLG-14 6 6 6 4 6 5 6 2 41

USS FISKE (Boston) DD-842 5 4 5 0 5 5 6 0 30

USS J. A. FURER DEG-6 4 5 14 4 5 8 5 2 47

USS GARCIA (Boston) DE-1040 4 6 6 2 6 8 10 2 44

USS GLOVER (Boston) AGDE-1 4 7 3 0 6 6 12 1 39

USS HUGH PURVIS DD-709 6 7 10 0 15 0 9 2 49

USS TALBOT DEG-4 5 5 9 2 5 5 5 0 36

USS JOHN WILLIS DE-1027 2 5 3 2 5 5 5 1 28

TOTAL 43 55 66 16 56 51 69 11 367

i TOTAL (BOTH LOCATIONS) 97 173 154 58 139 137 152 39 949

FREQUENCY 38 18 25 37 21 29 29 26 25 27 50 31 35 UO 38 3S 66 3't 27 21 297

205 *»* 290 *•* 205 »** 280 **• 275 »*» 270 *•• 265 *•* 260 »»» 255 »»* 250 **# 21*5 **» 2it0 * ## 235 * *» 230 # # * 225 **• 220 • •* 215 * ** 210 • •• 205 *»• 200 *»* 195 *** 190 *** 185 »»* 180 »** 175 »•* 170 *** 165 »»* 160 • •• 155 *** 150 »*» 1U5 » »* ldO *»* 135 *** 130 »»* 125 »♦*

120 • »» US »*» 110 *** 105 • •» 100 **» 95 *** 90 *»* 85 »•• 80 **» 75 *•• 70 »* * 65 »•* »»* 60 ♦ *« *♦*

55 »** ♦ *«

50 *»« «»• «»» it 5 **« ««* »** UO *•* *** • •* «*»_ 35 *** »** *** *** »»» *•« * *» »«* *** 30 *»• • •• **» * ** *** »** * ** ♦ *» »* » » »» *** 25 • »♦ *** * »4 A* * *** **» * * * **« **« »** #»* * * * * ** *** * ** *»» * »* *** 20 • ** *** »** *** * * 4 • ** • *♦ **♦ *** **» ♦ ** * »# **• ** * • # * »*» * * • *** * *• *** IS * ** »»♦ *** »*» *** **» »*» * ** *** * »• * ** **« *** **» *»»' **.* *»* ** # « »» • •♦ *»* 10 *«* *»« »*» *»« *** «« » »*• *»» * * * *»* * * » **» »»* »*» *•« »** *♦* 444 *** »»* »»* 5 ♦ ** *** * »* **» • ** »4» »* * * » * *» * '**«' »* # » * » »»* »»« *»» »** *** * * • »*• *** • •*

CLASS INTERVAL 1 2 3 1» 5 6 7 8 9 10 11 12 13 lit 15 16 17 18 19 20 21

F1g. 1.—Histogram of Series Reliability Estimates (SRE)

FREQUENCY 178 8K 81* 39 38 68 <iO 27 15 2fi 6 11 6 3 8 2 0 8 ?. 3 292

290 " 285 280 275 270 265 260 255 250 2U5 2i»0 235 230 225 220 215 210 205 200 195 190 18S 180 175 *** 170 *** 165 *** 160 *** 155 *** 150 *»• 1U5 **» 1UO *»* 135 «»* 130 **» 125 *•* 120 ♦** 115 **♦ 110 *•* 105 *** 100 *** 95 *** 90 *** 85 *** 80 *** *** *** 75 *** *** *** 70 *** *** *** 65 *** *** *** *** 60 »»* »*• »»* *** 55 *** *** *** *** 50 *** *** »#* *»# («5 *** *** »** *** *** I|0 *«* »»* **• *** *** 35 *»* *** *** *** *** *** *** 30 *** *** *** *** *** *♦* *** 25 *** *** *** *** *** *** *** *** *** 20 *** *** *** **« »»» *»* *** *** *** 15 *** *** *** **# *** *** *** *** *** *** 10 *** *•* *** *** **» »** *** *♦* *** *** *»» 5 *»* *** *** »** »*» *•*. »** •*** *** *♦♦ *** **» **» »** *»»

CLASS INTERVAL 1 2 3 I» 5 6 7 8 9 10 11 12 13 It» 15 IF 17 18 19 20 21

Fig. 2.—Histogram of Series-Parallel Reliability Estimates (PRE)

10

FREQUENCY 519 15't 122 70 HO 12 2 2 IS

513 501. 1*95

'i 7 7 1*68 1)50 1*50 i*Ul i»52 i*23 Uli» 1*05 310 337 378 369 3fi0 351 3'* 2 333 321* 315 306 297 2SS 279 270 261 252 21*3 23U 22S 216 207 198 189 ISO

* • * **» *** »«» ** » *** *** *#* * »* ** * *** *** *** *** * »» *»* *** *»» *•• »»* *•* *** »** **» *»» *** *** *** *** *** *** *** **• *** *** **♦ ***

171 *»* 152 • *• 153 *** • •• , li*i» **« *»• 135 *** ♦ *»

126 *»« *** 1.17 * * * *** **• 10.1 • *• *** ** * 99 • *• ** * ** * 90 • *• **• •** 81 ** * *** *** 72 ** * * * * *** 63 »»» » » » * * • *»* 51* * * * *ft* •** *** US *** *»* **• *** 3fi • •» *•» *•* •ft* *** 27 • •* »•• •** *.* * * * * 1« *•* •** *** *•• *•*

9 • •* »*• *** ********* ***

CLASS INTERVAL 1 2 3

Fig. 3

k 5 6 7 8 9 10 11 12 13 1U 15 16 17 18 19

.—Histogram of Geometric Mean Reliability Estimates (GRE)

11

20 21

FREQUENCY 21» 25 H6 62 78 70 05 91» 92 85 70 51» Uli 28 IB 27 II» 11

91» #** *** 92 *** ** * *** 90 »#* *** **# 88 * ** ** * **• 86 *** *** »*• *** SI» *** »** *** • *» 82 *#* ** * **» *** 80 #** *»» *** *** 78 ** * *** »•* *** »** *** 76 ♦ ♦* *#« *** * #* * ** *** 71» *** *** *** * ** »»» **» 72 *** 4** **# * * * *»* **• 70 *** * ** *♦♦ *** »** *** *** 68 *** #** ♦ ♦» **» *** **• *** 66 *** *** *** *** **# *** *** 61» *»* *** ♦ »* *** »** *** **• 62 *** *** *** *** *** *»» *** »** 60 • ** • •• #** *** *** *** *** *** 58 *** *♦* *** *** *** *** *** **» 56 **♦ **» **# **» »** * ** • ** *** 51» »** *** *** *»* *** *** *** • ** **• 52 ♦ ♦♦ *** *## *** *** *** *** *♦* *** SO ♦ ♦♦ *** *** *** ** * ** * *** *** *** I»8 *** *** *** *** *»* **♦ • »• *»* *** l»6 • *» ♦ ♦* *** **# *** *** ** * **• **» *** l»l» *** *** *** *** *** **♦ *** ** * *** *** I»2 *** *»* *** ** + *** *** *** • ft * *** *** 1*0 *** *** * * * *** •*• ♦ «* *** *** • ** *** *•» 38 *** »»# *** *# * **» ** * **» *** *** • ** **• 36 *»* »*• *»# *** • • * *»* *** «** »** *** *•• 31» 32 ♦ •» *** *** **# #•« »** *• * *** **» • ** • »» 30 *** ** » »** *** *** *♦ • **• *•* *** *** *♦*

28 **• * ** ** * *** *** *** *** *•* • ** **• *•* *** 26 **• **« #* » *** *♦* * # * »*» *** *** »•• . »** *** *»* 21« *•* • ** *«* »** »** *** »*# *** *** *** *** »•* »** **» *** 22 *• * * * * *•• **» * * * _** *** *♦♦ *** *** ** * *•* »•« *** • •• 20 • * * ** * • * * ** * *** *** * ** *** ** * • ** ** * ** * *** * ** *** 18 #** *** *•* • »» **» *** *»* *** • *» *•* *** *** »** *** »** 16 *** *** **• *** *** *** *** ** * «** *»* *** *** *** **• **» • *• li» **« *** *** *** • *« *** *** *♦* *•* *** *** *** *** *** »*• *** **• 12 * •» **« . •*» • •* #** *#* *•* *** *** *** *** * * * • ** ** * *»* ♦ »» **» 10 • •» ** * * * * • ** **» • ** *»• # * # ** * ** * ** * *** »ft* »** ** * • ** *** **» 8 *»» *•« *** **« *»» *** *** **» • ** *«* ft*« *** *** *** **» »** *** *** 6 »*« *** »** ** * * * * * ♦* • ** ♦ ♦♦ *** *** *** ** * ** * ♦ ♦♦ *** *** *** • *• i» ** * # * * *** *** ♦ ** *** *** * * * *»* *** * • * * * * ♦ * * ♦ ♦♦ ** * *** *** **• *»* 2 ** * *** **» *** ##♦ *#* »»* *•• *** *•* *** *** *•* *** *** *** *»* »*• «**

CLASS NTERVAL 1 2 3 I» 5 6 7 8 9 10 11 12 13 11» 15 16 17 18 19 20 21

Fig. 4.—Histogram of Weighted-Average Reliability Estimates (WRE)

12

TABLE 2

CLASS INTERVALS FOR HISTOGRAMS

Class Interval Number

Lower Boundary

Upper Boundary

1 0.96 1.00

2 0.91 0.96

3 0.86 0.91

4 0.81 0.86

5 0.76 0.81

6 0.71 0.76

7 0.66 0.71

8 0.61 0.66

9 0.56 0.61

10 0.51 0.56

11 0.46 0.51

12 0.41 0.46

13 0.36 0.41

14 0.31 0.36

15 0.26 0.31

16 0.21 0.26

17 0.16 0.21

18 0.11 0.16

19 0.06 0.11

20 0.01 0.06

21 0.00 0.01

13

Further analysis of results on the JPQ ANSWER SHEET Indicated a high frequency of nonresponse in some job activities and ratings, the type of nonresponse that resulted was for the case 1n which the man being evaluated did not work at the particular job activity under consideration. Furthermore, there was also a significantly high proportion of men who, while they worked at the job activity being considered, received SUE = 0 and EUI = 0 from their supervisors. These observations required the consideration of two important areas relative to the JPQ.

Problems In Calculating Performance Estimates. As discussed 1n the Background section of this report, reliability ratios of the form (EUE/[EUE + EUI]) were derived for each man on each of eight job activities and these ratios were combined to form the SRE, PRE, and GRE. However, the following two cases require the adoption of some convention 1n order to calculate the reliability ratios:

1) the technician did not work at that job activity, or

2) the technician received EUE = 0 and EUI = 0 by the supervisor, Implying that the reliability ratio 0 1s undefined.

0"+T

By observing the frequency with which such cases occur across all 21 ships participating 1n the project, 1t 1s possible to determine the extent.to which any convention for estimating performance 1n those cases would effect individual SRE, PRE, and GRE values. A complete discussion of this effect 1s given 1n Appendix E. Summarizing, the above two cases can have a dramatic affect upon the individual performance estimates and these estimates will be greatly Influenced by the convention that 1s adopted.

A Convention for Estimating Performance 1n Certain Job Activities. Siegel and Federman L16J employed "... the average value for his rating on his ship ...", (p. 28), on those job activities which the technician did not work at or received SUE a 0 and EUI = 0 by his supervisor. Unfortunately the results of the data collection effort at Location No. 1 (Destroyer Flotilla NINE) and at Location No. 2 (Cruiser-Destroyer Force Atlantic Fleet) Indicated that this technique was not feasible.6

In order to overcome this problem, the convention adopted 1n this report was to employ a composite reliability value across all ships at a location for each job activity and rating. Appendix F discusses the procedure for deriving the composite reliability values, as employed 1n this report.

Analyses Based on the Technical Proficiency Checkout Form

Table 3 represents the numbers of men at each of the three TPC levels by rating and ship and across each rating and ship at Location No. 1. Table 4 reflects the same information for Location No. 2. It will be remembered that level 1 reflects an "above desirable" proficiency level while level 3 reflects a "below minimally acceptable" proficiency level.

In addition to the TPC levels, TP scores were developed. A histogram of the resulting TP scores for the 949 technicians evaluated is presented in Figure 5. Almost identical histograms of TP scores were obtained for data collected at the two locations. Hence, only one histogram is presented.

60n every ship sampled at those locations there were ratings for which in some job activities those two cases occurred for all men 1n that rating. Appendix F provides a detailed account of this problem for the interested reader.

14

TABLE 3

NUMBER OF MEN AT EACH TECHNICAL PROFICIENCY CHECKOUT (TPC) LEVEL

Location No. 1

SHIP TPC EACH

RATING LEVEL 1 2 3 i» 5 6 7 8 9 10 11 RATING

1 k 5 k 2 1 I» 5 3 2 5 3 38 EM 2 1 1 0 1 1 1 1 2 2 0 2 12

3 1 0 1 1 1 0 0 0 0 0 0 1»

1 1» 6 7 9 7 8 17 8 5 8 7 86 ET 2 2 2 3 1 2 1 9 2 0 0 1 23

3 1 0 0 2 0 1 3 1 0 0 1 9

1 5 10 3 10 8 8 6 2 2 5 0 59 FT 2 2 5 0 0 0 3 3 0 3 0 0 16

3 0 1 2 1 0 1 3 1 0 k 0 13

1 2 2 2 5 1 3 k 2 1 2 2 26 IC 2 1 0 1 1 1 2 1 0 0 3 0 10

3 1 1 1 0 1 0 1 0 0 0 1 6

1 0 0 0 0 0 0 0 0 0 1 0 1 RD 2 1 1 1 0 1 0 0 8 0 3 0 15

3 it 9 8 5 8 7 0 1 5 7 13 67

1 1 3 7 0 0 u k 1 5 0 0 25 RM 2 1 U 0 0 8 2 3 5 0 2 8 33

3 3 1 0 5 1 0 3 3 0 9 3 28

1 k 7 7 6 3 6 0 t» 5 5 5 52 ST 2 1 2 0 1 2 0 0 6 0 k l» 20

3 0 0 2 1 3 3 0 0 0 0 2 11

1 2 1 0 2 1 2 1» 1 0 2 2 17 TM 2 1 0 0 0 0 0 3 0 0 0 0 I*

3 0 0 1 0 0 0 6 0 0 0 0 7

EACH 1 22 3«t 30 31* 21 35 UO 21 20 28 19 30«» SHIP 2 10 15 5 I* 17 9 20 23 5 12 15 133

3 10 12 15 15 11» n 16 6 5 20 20 1U5

15

TABLE 4 •

NUMBER OF MEN AT EACH TECHNICAL PROFICIENCY CHECKOUT (TPC) LEVEL

Location No. 2

TPC SHIP

EACH RATING LEVEL 1 2 3 k 5 6 7 8 9 10 RATING

1 2 2 3 k 3 0 2 5 I» 1 26 EM 2 2 0 1 1 1 2 1 1 1 1 11

3 0 1 2 0 0 2 1 0 0 0 6

1 3 5 5 3 5 5 7 7 2 3 l»5 ET 2 1 1 1 0 f) 1 0 0 2 1 7

3 0 0 0 1 0 0 0 0 1 1 3

1 2 It U 1» 12 1» 1 7 8 2 «»8 FT 2 1 0 2 1 2 2 2 3 1 1 15

3 2 1 0 0 0 0 0 0 0 0 3

1 0 0 2 0 1 0 0 0 2 1 6 IC 2 0 1 2 0 1 0 0 0 0 0 k

3 0 1 0 0 2 2 0 0 0 1 6

1 0 0 0 0 0 1 2 0 0 0 3 RD 2 0 0 6 h 0 1 l» 6 0 5 26

3 0 3 0 1 5 k 0 9 5 0 27

1 3 3 k 1 7 2 0 0 0 0 20 RM 2 2 1 1 k 1 6 3 0 k 0 22

3 0 0 0 0 0 0 3 0 1 5 9

1 5 2 I» 3 1» 5 2 9 U 2 1*0 ST 2 0 2 0 2 0 1 0 0 1 1 7

3 0 2 2 1 1 I» 10 0 0 2 22

1 0 0 0 0 2 1 0 1 0 1 5 TM 2 0 1 1 0 0 0 1 1 0 0 U

3 0 0 1 0 0 1 0 0 0 0 2

EACH 1 15 16 22 15 31» 18 It» 29 20 10 193 SHIP 2 R 6 1U 12 5 13 11 11 9 9 96

3 2 8 5 3 8 13 11» 9 7 9 78

16

F'&EOUENCV 1« >>Q 69 76 97 132 133 107 257

?SS • ••» 2S0 • ••• ?45 • ••• 24f> • ••• 235 • ••• 230 • ••• 225 • ••• 220 • ••• 21« • ••• 210 • ••• 20 b • ••• 200 • ••• 195 • ••• no • ••• IPS • ••• 1^0 • ••• 175 • ••• 170 • ••• 165 • ••• 160 • ••• 155 «••• 150 •••• US • ••• 140 • «•• 135 • ••• 130 • ••• • ••• •••• 125 •••• •••• • ••• 120 • ••• • ••• • ••• 115 • ••• • ••• •••• 110 • ••• •••• • ••• 105 •••• •••• •••• • ••• 100 • ••• •••• •••• • ••• 95 • ••• •••• • ••• •••• • ••• 90 • ••• •••• • ••• •••• • ••• 8S «••• •••• •••« • ••• • ••• 80 • »•• • ••• •••• •••• •••• 75 «»•• • ••« ••»• • ••• • ••• •••• 70 ••«• • ••« •••• • ••• • ••• •••• 65 • ••• ••«• • ••• • ••• • ••• • ••• • ••« 60 • ••» • ••« «««• 0«0« • ••« • ••• • ••• • ••• 55 • ••« • ••» •«•• • ••« • ««• »«•• • «•• •••• SO • ••• • ••• •••• • ••• •*•• • ••• • ••• • ••• 45 • ••« • ••• 0««« • «00 •••• • ••• • ••• • ••• 40 «««» • «•• ««ft« 0000 •••• • ••• • ••• • ••• 35 • ««• «o«< • ««• 000« «»«* • ••• • ••• • ••• 30 • »»» • «»« • •»• • «•» «««« • 000 • ••• • ••• ?S «••• • ••« «•«« • «0» «•»« 0000 • ••• • ••• 20 • •«• 0»«« »•«« «••• • ••• • 000 • ••• • ••• 15 • 0«« «»«» ««00 «««« «•«• »•«• • 000 • ••• • ••• 10 «««* • «•• *««« • ««« 0000 • «•* • ••• • ••• • ••• 5 «••• • ••• • 00« • •«» • ••• • ••• • ••• • ••• ••••

TP SCOW* 0 1 2 3 4 5 6 7 8

Fig. 5.—Histogram of Technical Proficiency Scores (N = 949)

17

APPROACH .

An approach to the statistical analysis of the data Involved the selection of appropriate analyses within four general areas:

1. The validity of the performance estimators SRE, PRE» GRE, and WRE with respect to a selected criterion measure.

In order to determine the validity of the four performance estimators (SRE, PRE, GRE, and WRE) for predicting on-the-job performance of the electronics maintenance personnel involved in the research effort, the results of the TPCF were used as a criterion measure of absolute technician proficiency. The belief that the TPCF reflects the on-the-job performance of electronics maintenance personnel must rest to a large degree upon related results of prior research efforts. In particular, from references that were cited in the section Validation of Performance Estimators.

Initially, 1n order to determine the degree of association between the performance estimators (SRE, PRE, GRE, and WRE) and the selected criterion variable (TPC level), trlserlal correlation coefficients were developed by location. Due to the extreme skewness of the distribution of the underlying continuum (represented by TP score, see Figure 5), a test of normality of TP score was executed to determine the appropriateness of trlserlal correlation. This resulted 1n the choice of a curvilinear regression analysis as a better approach for validating the performance estimators by location and subsequently by rating with TP score as the continuous criterion measure.

2. An evaluation of technician job competency as determined or implied by the TPCF.

The appropriateness of the job tasks represented on the TPCF was approached by developing a frequency table of men CHECKED OUT and NOT CHECKED OUT by rating on each job task. The agreement of each job task to the hlerarchlal classification of the tasks provided some Indication of the extent to which 1t was still applicable to electronic maintenance activities. A more detailed analysis of the TPCF that Included the development of sample conditional and joint frequency tables allowed the development of a procedure for determining technician job competency within a rating. Furthermore, these analyses revealed areas of suggested modifications of the TPCF prior to Its implementation.

3. Multiple comparisons between ratings or ships with respect to their average performance levels.

The approach employed 1n this report to develop comparisons between ratings and between ships was an additive model suggested by a two-way fixed effects analysis of variance with Interaction. This required the selection of the appropriate variable, from among SRE, PRE, GRE, WRE, and TP score, upon which to base the comparisons. The variable selected was the one which best met the statistical requirements, I.e., normality of the variable, homo- genlety of variances over the main effects, and Independence of ship, rating, and cell observations.

4. Degree of association between various demographic variables and the performance variables.

18

From the demographic Information collected on the Personnel Identification Information Form (PIIF), product-moment correlations between the demographic variables, and the performance estimators (SRE, PRE, GRE, WRE, and TP score) were developed. This same approach was applied to each of the eight job activities 1n order to determine 1f any job activity related to a particular demographic variable.

These areas are the most rewarding 1n the sense that they would provide some In- sight Into the merits of the performance measurement technique being researched and of the implications for its application within the U. S. Navy.

RESULTS AND DISCUSSION

Validity of the Performance Estimators

Tr1serial Correlation Analyses

Initially sample mean reliabilities were developed for each of the four performance estimators (SRE, PRE, GRE, and WRE) for each TPC level. The mean reliability values are the average values of the performance estimators 1n the TPC levels, therefore, the mean values would be expected to be smaller for a lower proficiency level. The results of this phase of the statistical analysis are presented in Table 5 for technicians at those ships sampled at Location No. 1, Location No. 2, and combined locations.

Employing the results of Table 5, tr1serial correlation coefficients were developed between each of the performance estimators (SRE, PRE, GRE, and WRE) and Technical Pro- ficiency Checkout (TPC) level for technicians sampled at each location and combined locations. Table 6 presents the resulting triserlal correlations.

Comparing the locations, particularly with respect to the SRE, PRE, and GRE, only the triserlal correlation coefficients for data collected at Location No. 2 agree with Siegel and Federman's prior results [16], This observation required a consideration of the appropriateness of triserlal correlation to the data collected in this project.

Application of the triserlal correlation coefficient involves the following requirements:

a) the segmented variable 1s basically continuous and normally distributed; and,

b) all the segments which together would form a whole normal distribution are present.

Consider again the histogram 1n Figure 5. Recall that the variable, Technical Proficiency score (TP score), was segmented into one of three levels of technician proficiency. This histogram represents the entire distribution of the segmented variable, which may be taken as continuous and is clearly negatively skewed. A

19

TABLE 5

MEAN RELIABILITIES

Location No. 1

Mean Reliabilities 1n Each TPC Level

TPC LEVEL N SRE PRE GRE WRE

1 303 .351 .597 .931 .657

2 134 .331 .553 .919 .566

3 145 .427 .625 .923 .557

Location No. 2



1 193 .361 .629 .947 .642

2 96 .213 .380 .928 .557

3 78 .141 .338 .878 .502

Combined Locations



1 496 .355 .608 .937 .651

2 230 .283 .483 .923 .562

3 223 .328 .524 .907 .538

20

TABLE 6

TRISERIAL CORRELATION ANALYSES

N

Performance Estimators

SRE PRE 6RE WRE

Location No. 1 582 -0.090* -0.015 0.031 0.256*

Location No. 2 367 0.340* 0.335* 0.235* 0.335*

Combined Locations 949 0.061 0.1.22* 0.099* 0.283*

♦Significantly different from zero at the a = .05 level.

goodness of fit test for normality [5 ] was applied to the distribution of TP scores of the 949 technicians. This test statistic will be called the g-| test statistic.7

When the g-| test statistic was applied to the sample data of TP scores, the resulting test statistic values were

g-l = -.5123, implying 2 = 6.4632.

Therefore, the assumption of normality for TP scores must be rejected for the sample data collected on the population of electronics maintenance personnel (949 technicians in the sample).

7The g1 test statistic 1s given by g = ^N"SX - Y)3

[HX - T)2]3/2

where X represents an observation, Y the sample mean, and N is the sample size. If the null hypothesis 1s that the underlying distribution is normal,

then it has been shown [5 ] that z = g. (N + 1) (N + 3) 6(N - 2)

1s approximately normal with mean zero and variance one. In fact a test of the hypothesis that the underlying distribution 1s normal (at the er = .05 level of signfi- cance) is given by:

reject the null hypothesis of normality if z is greater than 1.96 or less than -1.96.

This particular goodness of fit test has several advantages over the usually applied Kolmogorov-Smlrnov tests or the well known Chi-square tests in that, 1n particular, the population mean and standard deviation need not be known and the test need not be applied just to large samples. Furthermore, this test is more sensitive to departures from normality due solely to skewness than the other two tests [5 ].

21

The g, test statistic was also applied to the distribution of TP scores at either location. The test statistic values were g-| = -.4930, z = -4.8810 and g^ = -.5473, z = -4.3156 at Location No. 1 and Location No. 2 respectively. Therefore, at both locations the assumption of normality of TP scores must be rejected. These results were verified by the histograms of TP scores for those locations. In both cases these histograms demonstrated the negative skewness of the distribution of TP scores.

Curvilinear Regression Analysis

Essentially due to the non-normality of the TP scores, an alternate analysis was employed 1n order to determine the degree of association between the predictor variables (SRE, PRE, 6RE, and WRE) and criterion variable (TP score). The particular procedure to be employed to achieve this end was a curvilinear multiple regression procedure outlined 1n Cooley and Lohnes [3]. A few remar-ks on this subject for the purposes of this report have been provided in Appendix G.

Appendix H provides the results of the analyses of these predictor and criterion variables for the total of 949 technicians sampled. Similar results as found in Ap- pendix H were also developed for Location No. 1 and Location No. 2. Although those printouts are not presented 1n this report, the essential information from those printouts 1s given in Table 78 along with the essential information from Appendix H.

Consider Table 7 and the evaluation of SRE as a predictor of TP score for the two locations combined. The product-moment correlation between SRE and TP score is .055 (not significantly different from zero at the a = .05 level). In attempting to fit a linear, quadractlc, and cubic model to the data of SRE values and TP scores, the multiple correlation coefficient (R2) values were .003, .007, and .043 respectively. However, in view of the fact that the residual mean square does not change from the linear to cubic model, 1t would be just as well to chose the linear model (particularly since R2 for the cubic equation 1s not significantly larger than .007). Therefore, from Appendix H, the best regression equation 1s

TP score = 5.219 + 0.397 SRE.

Because SRE and TP score are essentially independent, the best estimate of SRE will always be the mean of the observed TP scores, regardless of the observed SRE. This result 1s further reflected 1n noticing that the sample mean TP score is 5.350, approximately equal to 5.219 - the TP score Intercept of the regression line.

Observing the results of Table 7 for the predictor variables PRE and GRE on the two locations combined, only minimal Improvement can be made with these estimates over the SRE. In fact the GRE 1s almost identical to the SRE and for practical purposes cannot be held to possess significant merit. The PRE is modestly better with a correlation of .1. However, for the PRE, the highest Rz value does not even reach .05, a long way from a perfect fit with an R2 value of 1.0. Based upon this analysis, PRE must be termed only slightly better than the SRE and GRE.

The WRE provides the most promising and consistent (over locations) estimator of the four predictors considered. It 1s most promising in the sense that it provides

°A11 of the computer printouts on the curvilinear regression procedure employed 1n this report are in terms of "centered data." This technique Improves the computation of the printout values by minimizing roundoff errors. Therefore, when reviewing the results of Table 7, one must be concerned with the relative magnitude of the residual mean square in attempting to fit a linear model versus fitting a higher order model.

22

TABLE 7

CURVILINEAR REGRESSION ANALYSES BY LOCATIONt

Location No. 1

rxy

Type of Curve

Predictor L1 near Quadratic

R2 s2 Cubi c

Variable R2 s2 R2 s2

SRE -.069 .005 .002 .007. .002 .072 .002

PRE -.009 .000 .002 .043 .002 .047 .002

GRE .024 .001 .002 .003 .002 .004 .002

WRE .242* .058 .002 .058 .002 .062 .002

Location No. 2

r xy

Type of Curve


R2 s2

Cubi c Variable

R2 s2 R2 s2

SRE .288* .083 .003 .083 .003 .090 .003

PRE .277* .077 .003 .078 .003 .100 .002

GRE .212* .045 .003 .047 .003 .052 .003

WRE .292* .085 .003 .085 .003 .089 .003

Locations Combined

r xy

Type of Curve


R2 s2 Cubi c

Variable R2 s2 R2 2

S '

SRE .055 .003 .001 .007 .001 .043 .001

PRE .100* .010 .001 .036 .001 .036 .001

GRE .087* .008 .001 .008 .001 .009 .001

WRE .257* .066 .001 .066 .001 .071 .001

2 2 tR = rYU 1n the linear case.

*S1gn1f1cantly different from zero at the a = .05 level.

rXy = product-moment correlation coefficient between the predictor variable x and the criterion variable y (TP score).

R2 = multiple correlation coefficient.

s2 = residual mean square

23

the highest product-moment correlation coefficient with the selected criterion variable, TP score. However, it cannot be said that it better fits the data than any of the other three estimators with respect to the three types of curves considered.

The curvilinear regression for the predictor variable WRE, Appendix H, page H-6_, indicates that a linear curve is the best fit to the data. The regression equation is given by

TP score = 3.571 + 2.95 WRE.

In comparison to the other estimators the WRE can be said to possess moderate validity at best for appraising an absolute level of technician performance. As such its application to the population of electronics maintenance personnel is tenuous.

Analyzing the results at either location again points out the differences between the results obtained at either location. However, this is due mainly to a difference 1n rxy values and not to R^ values for goodness of fit of the linear, quadratic, and cubic models from one location to the next. This may be due to the high degree of unexplained criterion variance 1n the data at either location and with respect to the locations combined. Scatterplots of the data in those three cases with respect to each predictor variable verified the high degree of dispersion in the data and the lack of any obvious pattern or functional relationship in those plots.

Within Rating Analyses

A factor that may influence the frequency with which a population does not work at a particular job activity is, of course, the appropriateness of the job activity to present-day electronic maintenance activities. The most homogeneous type of subpopulation that would reflect most members of the subpopulation working at the same job activities should be rating. It is mainly for this reason that the subpopulations considered in this report are ratings, and not, for example, ships within locations which should (and did) reflect results similar to those found in Appendix H. In order to perform the most general type of analysis to determine the validity of the performance estimates, a curvilinear regression analysis as previously discussed and employed in Appendix H was applied per rating for each of the four predictor variables - SRE, PRE, GRE, and WRE. Appendix I gives the resulting 32 printouts of the Cooley and Lohnes [3] curvilinear regression analysis. The object is to select for each rating, the best of three possible curves - linear, quadratic, and cubic - for each performance estimator which best fits the data in terms of significantly larger multiple correlation coefficients for smaller residual mean squares. Comparisons between the performance estimators may then be made by performing an appropriate test of hypothesis of the equality of two correlation coefficients (or multiple correlation coefficients). The test that is usually applied is the "Fisher's z" test which employes the asymptotic distribution of the sample correlation coefficient. However this test requires that the distributions of the underlying populations are bivarlate normal (see Anderson [1], page 78). Because the distribution of TP scores and the SRE, PRE, and GRE were not normally distributed with respect to the individual ratings, the appropriateness of employing this test is in question. Furthermore the literature seems to be vacant of a discussion of the robustness of the test. Therefore the approach must be in terms of comparing the observed sample product-moment correlations, and, in particular, on the amount of criterion variance explained by the variables. This approach will not necessarily produce a statistically significantly different performance estimator but one which is a more promising estimator, in terms of the sample information.

The following outline represents the essential results of the curvilinear regression analyses presented 1n Appendix I. The results are presented by rating in Tables 8 through 13 together with observations and recommendations.

24

TABLE 8

EM CURVILINEAR REGRESSION ANALYSES (N = 97)

rxy

Type of Curve

Predictor Linear

R2 s2 Quadratic

R2 s2

Cubic Variable

R2 s2

SRE .365* .133 .009 .136 .009 .176 .009

PRE .247* .061 .010 .133 .009 .113 .009

GRE .301* .090 .010 .154 .009 .221 .008

URE .492* .242 .008 .254 .008 .300 .008

< Significantly different from zero at the a = .05 level.

1. EM rating - Clearly the estimator WRE demonstrates the best fit to the data. The product-moment correlation of .492 indicates at least a fair degree of association of the WRE with the technician's absolute performance level. (Although the Rz value is significantly better for the cubic curve over the other two curves, the value of .300 1n that case can only indicate a moderate fit of the cubic curve to the data.

TP score = 4.849 - 15.571 WRE + 44.319 WRE2 - 26.561 WRE3.

The other estimators do not demonstrate a better fit to the data than the WRE. In view of this and other considerations (e.g., the examination of the scatterplots), the WRE is recommended for application in this rating for those situations in which the probability of effective performance of an individual in the EM rating is required.

TABLE 9

ET CURVILINEAR REGRESSION ANALYSES (N = 173)

rxy

Type of Curve

Predictor Linear Quadratic 2 2

R^ s*

Cubi c Variable R2 S2 R2 2

s

SRE .318* .101 .005 .142 .005 .142 .005

PRE .366* .134 .005 .135 .005 .135 .005

GRE .508* .258 .004 .266 .004 .271 .004

WRE .445* .198 .005 .209 .005 .210 .005

♦Significantly different from zero at the a = .05 level

25

2. ET rating - For this rating two significant estimators arise, the GRE and WRE. The VIRE is still a promising estimator even though it has a lower product-moment correlation than the GRE (.445 versus .508) and does not fit the data as well. The GRE values may be spurious observations for this rating for 1t 1s not significantly higher in any other rating. However, without evidence of this fact 1t is recommended that either the GRE or WRE be employed in this rating. Further- more, the values of the product-moment correlations for those estimators reflect at least a moderate degree of association with TPCF results.

Because the R2 values do not change appreciably from one type of curve to the next, it is recommended that the linear model be employed for prediction purposes in either case. The linear curves are:

TP score = 2.99 + 4.08 GRE, and

TP score ■ 4.344 + 3.79 WRE.

TABLE 10

FT CURVILINEAR REGRESSION ANALYSES (N = 154)

rxy

Type of Curve

Predictor Linear

R2 2

s

Quadratic

R2 s2 Cubic

Variable D2 2 R s

SRE .346* .119 .006 .120 .006 .120 .006

PRE .265* .070 .006 .100 .006 .104 .006

GRE .374* .140 .006 .148 .006 .165 .006

WRE .430* .185 .005 .200 .005 .204 .005


3. FT rating - This rating again illustrates that the WRE is the most promising estimator. In fact it 1s almost identical to the results obtained for the ET rating and the WRE. The product-moment correlation of .43 reflects a moderate degree of association with the criterion variable. The R^ value of near .2 still Illustrates a significant degree of unexplained criterion variance 1n the data. It 1s again suggested that the linear model:

TP score = 3.366 + 4.635 WRE

be employed 1n this rating. The linear model fit is modest (R = .185) but the WRE possesses sufficient promise to be called a good estimator of FT technician proficiency.

26

TABLE 11

IC CURVILINEAR REGRESSION ANALYSES (N = 58)

r xy

Type of Curve

Predictor Variable

Linear

R*~ s2 Quadratic

R2 s2 Cubic

R2 s2

SRE .361* .131 .016 .131 .016 .133 .016

PRE .322* .104 .016 .131 .016 .134 .016

GRE .387* .149 .015 .154 .015 .169 .015

WRE .434* .188 .014 .241 .014 .292 .013


4. IC rating - Unfortunately the sample size (N = 58) for this rating 1s too low to place confidence in the obtained correlation coefficient. However, the WRE must be selected as the most valid estimator on all counts. The cubic model

TP score = -7.57 + 68.26 WRE - 113.95 WRE2 + 63.03 WRE3

1s suggested for use 1n this rating and is a moderate estimate of TP score. The residual mean square 1s high for this rating only because of the small sample size. In any event this rating reflects the general trend that we have been witnessing and, therefore, the WRE may be employed 1n this rating.

TABLE 12

RD CURVILINEAR REGRESSION ANALYSES (N = 139)

r xy

Type of Curve

Predictor Linear Quadratic

R2 S2

Cubic Variable R2 s2 R2 S2

SRE -.168 .028 .007 .093 .007 .131 .006

PRE -.215* .046 .007 .046 .007 .093 .007

GRE .021 .000 .007 .013 .007 .047 .007

WRE -.051 .003 .007 .021 .007 .041 .007


27

5. RD rating - None of the four estimators that have been discussed in this report should be employed 1n the RD rating. In part the failure of the estimators 1n this case must be attributed to the significantly high frequency with which Individuals 1n this rating do not work at the job activities that were used on the evaluation forms.

TABLE 13

RM, ST, AND TM CURVILINEAR REGRESSION ANALYSES

rxy

Type of Curve

Predictor Linear

R2 s2 Quadratic

R2 s2

Cub" lc Variable

R2 s2

RM Rating (N = 137)

SRE .211* .044 .007 .057 .007 .068 .007

PRE .156 .024 .007 .028 .007 .031 .007

GRE .041 .002 .007 .140 .006 .139 .006

WRE .366* .134 .006 .153 .006 .154 .006

ST Rating (N = 152)

SRE .199* .040 .006 .069 .006 .069 .006

PRE .058 .003 .007 .003 .007 .102 .006

GRE .099 .010 .007 .134 .006 .178 .006

WRE .234* .055 .006 .080 .006 .123 .006

TM Rating (N = 39)

SRE .198 .039 .026 .129 .024 .217 .022

PRE .243 .059 .025 .094 .025 .153 .024

GRE .186 .035 .026 .035 .027 .085 .026

WRE .403* .162 .023 .177 .023 .224 .022


28

6. Rating RM, ST, and TM - The results for these ratings are almost Identical and can be discussed as a group. In every case the WRE 1s the best estimator of TP score. The product-moment correlations are low for the RM and ST ratings, and moderate for TM's. All of the Rz values demonstrate a poor fit to the data, Indicating an even higher degree of unexplained criterion variance than that found 1n the EM, ET, FT, and IC ratings. It 1s recommended that the cubic curve be employed 1n the ST and TM rating and a linear curve for the RM rating. The corresponding equations may be found from the computer printout of the curvilinear regression for those ratings (pages 1-26, 1-30, and 1-34 respectively). However, the utility of their use as prediction Instruments has not been convincingly demonstrated.

Technician Job Competency Evaluation

As hypothesized 1n the discussion on the validity of the performance estimators SRE, PRE, GRE, and WRE, the TPCF could be employed as a criterion measure of the absolute proficiency level of electronics maintenance personnel. That the TPCF possesses that characteristic was evaluated by Siegel and Flschl [17] for avionics technicians and at that time the TPCF possessed a substantial degree of promise. Siegel and Federman [16] applied the TPCF when appraising the proficiency level of electronics maintenance personnel and 1n evaluating the SRE, PRE, and GRE.

Due to the high degree of reliance on the TPCF as a criterion measure, it is well to Investigate the TPCF and consider whether some of Its properties seem to hold up for the data collected 1n this project. The appropriateness of the job tasks on the TPCF for present-day electronic maintenance activities can be considered, by rating. Further- more, are the job tasks listed 1n the same hlerarchlal order as when originally developed? While 1t may not be possible to completely answer questions of this type, 1t is possible to give at least a partial answer based on the present format of the TPCF. By using the properties of the TPCF 1t 1s possible to develop a competency level for technicians 1n each rating and determine whether a particular rating may be performing at a certain proficiency level.

Job Task Analyses

The reader will recall that the job tasks listed on the TPCF are ordered with Job Task No. 1 being the easiest and Job Task No. 8 the most difficult. The following is a 11st of the job tasks as found on the TPCF:

Task Description

(easiest) 1. Capable of employing safety precautions.

2. Capable of replacing most of unit's equipment.

3. Capable of removing most of unit's equipment.

4. Capable of following block diagrams.

5. Capable of knowing relationship of equipment to other related equipment.

29

6. Capable of calibrating most of unit's equipment.

7. Capable of trouble-shootlng/lsolated malfunction(s).

(most 8. Capable of employing electronic principles Involved 1n difficult) maintenance.

Table 14 1s the frequency of occurrence of men CHECKED OUT and NOT CHECKED OUT by rating over the eight job tasks without reference to their performance on other job tasks. For example, of the 97 men 1n the EM rating, 95 were CHECKED OUT on Job Task No. 1, 87 were CHECKED OUT on Job Task No. 2, .... and 55 were CHECKED OUT on Job Task No. 8. Similarly, for the other ratings.

From Table 14 the hlerarchlal difficulty of tasks 1s generally represented 1n the EM, ET, FT, IC, ST, and TM ratings. This 1s evidenced by the fact that one would expect progressively fewer men to complete a more difficult task. Except for the ST rating, Job Task No. 6 - Capable of calibrating most of this unit's equipment with which Ms rating 1s concerned - seems out of place 1n that fewer Individuals are CHECKED OUT on this task than the next more difficult task, Job Task No. 7. In fact the better position for Job Task No. 6 would be after Job Task No. 8. In that case the job tasks would follow more closely a hlerarchlal classification.9

In any event it can be said that Job Task No. 6 1s no longer of such prominence as originally thought. This 1s a result of the Introduction of more sophisticated electronic equipment aboard ships 1n recent years. This equipment frequently consists of Integrated circuitry and compact electronic modules requiring little if any calibration. Furthermore, test instrumentation calibration 1s also 1n demise because less test equipment is being employed. Normally a defective component is replaced en toto without regard aboard ship for finding the particular fault 1n the component. Those functions are beyond normal shipboard electronic maintenance activities.

In the RD rating there are a significant number of individuals NOT CHECKED OUT on many of the tasks. This 1s probably due more to a nonapplicability of those tasks to that rating than to a lack of sufficient training. This is to a degree verified by the findings in Appendix E where 1t was demonstrated that for most job activities there are a very high proportion of individuals 1n that rating who either do not work at the job activities or received EUE ■ 0 and ZUI = 0 from their supervisor. It is possible that the nonapplicability of many of the tasks to the RD rating did give erroneous TP scores for this rating and depressed the validity coefficients (see Table 12). A more detailed study than the present one would be required to come to a definite conclusion on this issue.

As shown 1n Table 14 and the RM rating, Job Tasks 2 and 4 are not in agreement with the underlying order of job tasks, but Job Task No. 6 1s 1n a proper position for this rating. It seems that a reordering of the first five job tasks on the TPCF would allow this Instrument to be more applicable to this rating. It would probably require a minor research effort to revise the tasks in the RM rating.

Q It 1s Important to note that none of the conclusions of previous sections would be

altered with this modification because the TP score (or TPC level) is independent of the hlerarchlal classification. Those variables are dependent only upon the number of job tasks an Individual was CHECKED OUT on, and not upon the order of the CHECKED OUT tasks.

30

TABLE 14

JOB TASK ANALYSES

Job Task No. 1

Checked Out 95

Not Checked Out* 2

Job Task No.

Checked Out

Not Checked Out

Job Task No.

Checked Out

Not Checked Out

Job Task No.

Checked Out

Not Checked Out

1

168

5

1

148

6

1

57

1

Job Task No. 1

Checked Out 136

Not Checked Out 3

Job Task No. 1

Checked Out 135

Not Checked Out 2

Job Task No. 1

Checked Out 144

Not Checked Out 8

Job Task No. 1

Checked Out 38

Not Checked Out 1

EH Frequencies (N • 97)

2 3 4 5

87 87 77 75

10 10 20 22

ET Frequencies (N " 173)

2 3 4 5

147 164 165 126

26 9 8 47

FT Frequencies (N = 154)

2 3 4 5

125 134 143 116

29 20 11 38

IC Frequencies (N • 58)

2 3 4 5

45 49 44 41

13 9 14 17

RD Frequencies (N ■ 139)

2 3 4 5

27 38 50 97

112 101 89 42

RM Frequencies (N = 137)

2 3 4 5

88 100 88 113

49 37 49 24

ST Frequencies (N ■ 152)

2 3 4 5

118 119 117 121

34 33 35 31

TM Frequencies (N » 39)

2 3 4 5

31 32 31 34

8 7 8 5

6 7 8

43 63 55

54 34 42

6 7 8

96 129 132

77 44 41

6 7 8

93 100 102

61 54 52

6 7 8

26 33 27

32 25 31

6 7 8

8 8 26

131 131 113

6 7 8

46 27 13

91 110 124

6 7 8

103 77 80

49 75 72

6 7 8

14 18 8

25 21 31

*The reader Is cautioned that a score of NOT CHECKED OUT on a task does not dif- ferentiate between whether a technician really cannot be trusted with doing the task on his own without direct supervision or whether he was given a score of NOT CHECKED OUT because he does not work at that task.

31

Conditional Frequencies for Job Tasks

Appendix J provides tables of a more detailed analysis of some observations on the TPCF. In particular the Job Task Conditional Frequency 1s given by rating; i.e., for a given task, the number of Individuals who were CHECKED OUT on each easier task given they were at most CHECKED OUT on the given task. Those tables should provide more In- sight Into the hlerarchlal classification of the job tasks and also a relative level of technician competency by rating.

To illustrate the use of the tables prepared 1n Appendix J, consider Table 15. This table represents the Job Task Conditional Frequencies for the EM rating as can be found in Appendix J, page J-4.

TABLE 15

EM RATING CONDITIONAL FREQUENCIES (N = 97)

55 46

17

33

Job Task

5 4 3 2 1

50 50 50 52 55

13 14 17 17 17

6 5 5 4 6

6 5 6 6 6

3 3 3 3

6 5

0

6

0

2

The number of men that were not checked out on any job task is 2.

From the first row of Table 15, 55 of the 97 EM's were CHECKED OUT at most on Job Task No. 8. Of th9.s.g 55_men, 46 were also CHECKED OUT on Job Task No. 7 (the next easier task), 33 were CHECKED OUT on Job Task No. 6, ..., and finally all 55 men were CHECKED OUT on Job Task No. 1 (the easiest task). Continuing, from the second row of the Conditional Frequency table, 17 of the 97 EM's were CHECKED OUT at most on Job Task No. 7 (I.e., none of those 17 men were CHECKED OUT on Job Task No. 8). Again, of those 17 men, 4, 13, 14, 17, 17, and 17 were also CHECKED OUT on Job Task No.'s 6, 5, 4, 3, 2, and 1 respectively. Finally, only 2 men were CHECKED OUT on Job Task No. 1, I.e., none of those 2 men were CHECKED OUT on a more difficult task. Also, there were 2 men 1n the EM rating that were not CHECKED OUT on any of the eight tasks. Clearly then one would expect any row of a Conditional Frequency table to contain almost identical entries, i.e., if the job tasks are truly hlerarchial and representative of tasks in that rating.

32

Consider now Appendix J, page J-4, and the Conditional Frequency table. From the first two rows of that table, most of the EM's CHECKED OUT at most on Job Tasks 7 and 8 were also able to complete the easier tasks» except for Job Task No. 6. Again, that task 1s very much out of place 1n the order of job tasks. There 1s a low degree of In- competence present for 13 percent of the EM's could be CHECKED OUT at most on no more than Job Task No. 4. However, 1t must be realized that this result could also be due to the possibility that many of those Individuals simply do not work at a more difficult task than Job Task No. 4.

The ET, FT, ST, and TM ratings (Appendix J, pages J-5, J-6, J-10, and J-ll respectively) demonstrate practically the same results for the Conditional Frequency tables as. the EM's. For those ratings 8, 10, 9, and 13 percent were CHECKED OUT at most on no more than Job Task No. 4 respectively. The IC, RD, and RM ratings demonstrated 21„ 30, and 17 percent. These remarks illustrate the practicality of the TPCF as an instrument for evaluating the competency level of particular ratings and for isolating those job tasks requiring further training by electronic maintenance technicians.

It 1s to be noted that as in the EM rating, Job Task No. 6 is inconsistent in most of the other ratings with respect to Its proper order in the eight job tasks.

Joint Frequencies for Job Tasks

Appendix J also provides the Job Task Joint Frequencies by rating. For each rating this 1s the number of individuals who were CHECKED OUT on any two given tasks. Use of the Joint Frequency tables is illustrated by Table 16.

TABLE 16

EM RATING JOINT FREQUENCIES (N = 97)

95 87 87 77 75 43 63 55

87 84 73 70 40 63 52

87 71 69 40 62 50

11 69 40 56 50

75 42

43

56

35

63

50

33

46

55

From the first row of Table 16, 95 of the 97 EM's were CHECKED OUT on Job Task No. 1, 87 (of the 97 EM's) were CHECKED OUT on both Job Task No.'s 1 and 2, 87 were CHECKED OUT on Job Task No.'s 1 and 3, .... and 55 (Of the 97 EM's) were CHECKED OUT on Job Task No.'s 1 and 8. Similarly, from row 2 of the Joint Frequency table, 87 (of the 97 EM's) were CHECKED OUT on Job Task No. 2, while 84, 73, 70, 40, 63, and 52 (of the 97

33

EM's) were CHECKED OUT on both Job Task No. 2 and Job Task No. 3, 4. 5, 6, 7, and 8 respectively. It Is well to note that the diagonal entries of this table corresponds exactly to the frequency entries for the CHECKED OUT case given 1n Table 14.

Just as 1n Table 15 on the Conditional Frequencies one would expect that any row of that table to have almost identical frequencies, 1n a Joint Frequency table one would expect any column to contain Identical frequencies. For example, completion of Job Task No. 5 should also Insure completion of Job Tasks 1, 2, 3, and 4. Likewise, the rows of the Joint Frequency tables should Illustrate decreasing frequencies for being CHECKED OUT on a task does not Insure being CHECKED OUT on a more difficult task.

From the Joint Frequency tables (Appendix J, pages J-4, J-5, J-6, J-7, and J-ll) for the EM, ET, FT, IC, and TM ratings respectively, the previously mentioned patterns that are Illustrated by a Joint Frequency table are demonstrated except for Job Task No. 6. This 1s consistent with the analysis on the Conditional Frequency tables. The ST rating conforms almost exactly to the expected patterns for a Joint Frequency table. Unfortunately there are a few Instances 1n which the Conditional Frequency table for that rating deviates from the expected pattern. In any event the task descriptions on the TPCF are very descriptive of the ST rating job tasks.

The RD rating Joint Frequencies (page J-8) Illustrate the general lack of patterns characteristic of a Joint Frequency table. The RM rating Joint Frequencies (page J-9) more closely conform to the desired patterns but stm leave something to be desired.

Multiple Comparisons of Ships and Ratings

From either a research or applied point of view 1t 1s often necessary to compare the performance levels of various ships or ratings. Such comparisons may be employed to compare ships (or ratings) relative to mean performance levels. However, such comparisons can also offer a basis upon which to formulate a decision as to whether a particular ship configuration of men and/or equipment 1s, or is not, detrimental to ship/rating performance. In a similar fashion, policy making decisions also can be evaluated with respect to their influence on the performance of ships or ratings affected by that policy.

In this section the main subject of concern will be the development and justification of a technique for performing palrwise comparisons between ratings and between ships on the basis of mean performance levels with respect to some variable. The variables to be considered are the SRE, PRE, 6RE, WRE, and the selected criterion variable TP score. In general this subject falls under the area of multiple comparisons, arising from an Analysis of Variance (AOV) of ship/rating effects. The particular model10 to be considered will be a two-way fixed effects design with an unequal number of observations for the ship/rating

'^T+ 4c aeenma/4 that aarh nkcnmifiAn «/• .■ nn tha l/tn A nA4 wi #4na 1 In +hti 4Ll\ l0It 1s assumed that each observation y-M^ on the ktn individual in the ith ship in the jtn rating 1s due to an overall effect (n), a ship effect (c^), and a rating effect (ßj). Furthermore, a general model addresses the question of the existence of significant row-column Interaction (&u) up- ~~L ~L "J~~ T~ —--•»---■'-- -•-■•- -

JJ the additive model of the form row-column Interaction Uij) upon each observation. In particular this report addresses

;he fi

Y1jk " v + «i + 0j + «u + e1jk

where e^j. is an error term associated with the above model. In this situation it is a two-way fixed effects design with an unequal number of observations in the i-jtn cell.

34

combinations (see Table 1). Choice of this model requires the verification of normality of the selected performance variable over the main effects, homogeneity of the variances of the main effects and Independence across and between the main effects.11 The main effects are "ships" and "ratings." That performance variable to be selected for performing the multiple comparisons will, most naturally, be the one which best conforms to the requirements for use of the model being considered and 1s most associated with technician on-the-job performance.

Distributional Properties of the Performance Estimators

The g, test statistic for normality was applied to the distribution of SRE, PRE, GRE, and WRE values for the 949 technicians 1n the sample (combined locations). Table 17 illustrates the results of this analysis.

TABLE 17

91 TEST STATISTIC VALUES FOR THE PERFORMANCE ESTIMATORS

SRE PRE GRE WRE

9l .5807 -.4565 -4.7092 .7939

2 7.3265 -5.7599 -59.4129 -.5823*

♦Because WRE scores (Figure 4) are almost normally distributed, a test statistic [4] similar to the g, test statistic but more sensitive to departures from normality due to kurtosls was applied to the WRE scores.

Recall that rejection of normality results if | z | >J.96.

Clearly then the only estimator which appears to be normally distributed is the WRE. This same analysis was applied by location and rating with practically the same results. In each case only WRE scores proved to be normally distributed, except for isolated Incidents of the other estimators. This characteristic of WRE scores is not surprising as this situation 1s essentially an application of the Central Limit Theorem as found in probability theory, see [12]. The lack of normality on the part of the other estimators is not necessarily an undesirable feature, but it is true that this exercise does point out yet one more desirable feature of the WRE, namely, its general normality.

Choice of a Variable

As previously noted 1t 1s necessary to choose an appropriate variable upon which to base the multiple comparisons of ships and ratings. From the previous section - Distributional Properties of the Performance Estimators - it was observed that WRE scores possessed the unique characteristic among the performance variables (SRE, PRE, GRE, and TP score) of being normally distributed. This characteristic of WRE scores is most helpful particularly since homogeneity of variances in the ships, ratings, and

"A search of the literature failed to locate a particular test to employ for Independence 1n the case of unequal numbers for the ship/rating combinations. See Anderson [1] for the case of equal numbers.

35

of the ship/rating combinations 1s a prerequisite12 for the particular analysis of variance design employed 1n this report. The usual test for homogeneity of variances (Bartlett [2]) 1s extremely sensitive to non-normality. Therefore, because the SRE, PRE, GRE, and even the selected criterion variable (TP score) are not normally distributed, this characteristic decreases their utility to be employed within the present AOV model. Furthermore, it was also shown 1n the prior section - Validity of the Performance Estimators - that the WRE 1s the most promising type of performance estimator of technician on-the-job performance. The reader 1s also referred to Appendix K which presents in more detail some considerations on the greater utility of the WRE with respect to the other performance estimators. In view of these considerations the WRE is certainly the most viable variable upon which to base the multiple comparisons and, therefore, this report will consider only sh1p/rat1ng comparisons on the basis of technician Weighted-Average Reliability Estimates (WRE).

An Analysis of Variance with Interaction

Because WRE scores are normally distributed, Bartlett's test (as found in [6 ]) for evaluating the homogeneity of variances across ships and across ratings was applied to the WRE scores. When all eight ratings were considered (21 ships), equality of rating variances was accepted while equality of ship variances was rejected. Re- vising the situation to include only the four more promising ratings (EM, ET, FT, and IC) to which the WRE most applied, Identical results were obtained. Clearly the non- verification of homogeneity of ship variances would make the model being considered less applicable to the data, however, because 1n practice such may not be the case for the situation 1n which the comparisons are being made, the analysis will continue here under the additional assumption of homogeneity of ship variances. Subsequent results 1n this report can be appropriately weighed by the reader.

As noted previously, the model being considered is a two-way fixed effects design with interaction. Upon observing Table 1 the reader will realize that it also must include the characteristic of unequal cell numbers. Scheffe [15, pp. 112-116] has outlined in detail the procedure for performing multiple comparisons under this particular situation. Employing this procedure for the main effects - 8 ratings and 21 ships (N = 949, the total number of observations) - Table 18 provides the essential information resulting from this analysis.

TABLE 18

AN ANALYSIS OF VARIANCE FOR 21 SHIPS AND 8 RATINGS

Source Degrees Freedom

of Sum of Squares

Mean Square F Value

Total 949 384.2407

Between Ships 20 5.0938 .2547 7.0904*

Between Rating 7 .7155 .1022 2.8458*

Ship by Rating Interaction 140 12.3940 .0885 3.3420*

Residual 781 20.6885 .0265

♦Indicates significance at the a = .05 level.

'2Scheffe [15] has noted that non-normality has little effect on inferences about means but serious effects on Inferences about variances. Furthermore, inequality of variances in the cells of a layout has serious effects on inferences about means in lay- outs with unequal cell numbers (see Table 1).

36

An Identical AOV was executed as above but for the four ratings (EM, ET, FT, and IC) for which the performance measurement technique was most promising. Table 19 reflects the results of this AOV.

TABLE 19

AN ANALYSIS OF VARIANCE FOR 21 SHIPS AND 4 RATINGS

Source Degrees Freedom

Of Sum of Squares

Mean Square F Value

Total 482 192.0419

Between Ships 20 2.669 .1334 3.884*

Between Rating 3 .5621 .1874 5.4598*

Ship by Rating Interaction 60 4.0437 .0674 2.2975*

Residual 398 11.6747 .0294

♦Indicates significance at the a = .05 level.

In either of the above two AOV tables there exists significant ship/rating interaction. This situation will not permit the comparing of any two of the main effects under a two-way design because of the confounding of the main effects with the interaction terms. The only recourse is to test for significant differences in each ship, across ratings, or in each rating, across ships. If the Interaction terms were not significant, it would be an easy matter to compare ratings and to compare ships by employing the entire set of data represented in the AOV's of Table 18 or Table 19, see again Scheffe [15].

Testing for Significant Ship (Rating) Effects for a Particular Rating (Ship)

In order to develop comparisons between ships (for a fixed rating) and to develop comparisons between ratings (for a fixed ship), this report will employ an AOV model'3

resulting from a one-way fixed effects design with unequal numbers of observations across the main effects. Homogeneity of the variances across the main effects must be assumed 1n this case for the usual tests are large sample tests and inapplicable for ship/rating numbers less than ten (see Layard [14] and Table 1).

For present purposes only the AOV's, and subsequent multiple comparisons, will be developed on the 21 ships participating in the project and the four ratings EM, ET, FT, and IC. This will involve 25 different AOV's to be constructed and subsequently developed for multiple comparison of the main effects if the AOV indicates significant main effects.

13This report will employ a model of the form

Yijk = v + a1 + eijk , 21) (j is fixed and i=l,

for testing for significant ship effects, and a model of the form

Yij|< = P + ßj + e^jk (1 is fixed and j=l, ..., 4)

subscript k is an index representing for the

testing for significant rating effects. The kth man 1n that main effect.

37

Initally the hypothesis of equality of mean WRE scores was tested for the four ratings EM ,ET, FT, and IC for each of the 21 ships. This hypothesis was accepted for 12 of the 21 ships. On the remaining 9 ships Scheffe's multiple comparison technique was applied 1n searching for a possible significant difference between any two (of the four) ratings on the basis of mean WRE scores. Table 20 reflects the results of these multiple comparisons, each derived from a separate Analysis of Variance. (The resulting 21 AOV tables are too extensive to be presented in this report).

Finally the hypothesis of equality of mean WRE scores was tested for the 21 ships for each of the four ratings EM, ET, FT, and IC. This hypothesis was rejected for each of the four ratings. No significant differences were found between any pair of the ratings using Scheffe's multiple comparison technique. This 1s not an inconsistent result, for this could be a result of trying to perform too many multiple comparisons (therefore, Increasing the comparison error rate) and/or the effect referenced 1n the footnote of Table 20.

TABLE 20

MULTIPLE COMPARISON OF FOUR RATINGS ON NINE SHIPS*

Highest Lowest Ship Mean WRE Mean WRE

a EM FT ET IC

FT ET EM IC

EM ET IC FT

ET FT EM IC

JC EM ET FT**

JX ET FT EM**

FT IC EM ET

EM ET IC

EM ET FT

*For each ship a line under two ratings signifies no significant difference between the ratings.

**It is important to note that while no differences 1n rating means were detected, the rejection of the null hypothesis of the equality of rating means is not In error. This only Implies that some contrast other than those contrasts testing rating differences lead to that rejection, see Ferguson [8, pp. 279-283].

38

Association Between Demographic and Performance Variables

Usually it is of interest from either a research or applied point of view to estimate school success, final grade, etc. of an individual in some training program or school by employing such predictor variables as the Basic Test Battery (BTB) scores - GCT, ARI, MECH, and CLER scores or combinations of those scores. Typically BTB scores are used to predict actual school success which 1n turn is viewed as being the best available measure of potential job success. Seldom, however, are direct measures of on-the-job performance available as those developed 1n the current study. Accordingly the relationship between the BTB scores - GCT, ARI, and Skill (GCT + ARI) - and measures of on-the-job performance - SRE, PRE, GRE, WRE, and TP score - was investigated.

Besides the BTB scores, other available predictor, variables were also considered under the general heading of demographic variables. In this report the demographic variables to be considered are: months known by supervisor, months on current job assignment, GCT score, ARI score, SKILL (GCT + ARI) score, and A, B, and C school final grade. The performance variables that will be employed to relate to on-the-job technician performance are the eight reliability ratios relative to the job activity factors on the JPQ ANSWER SHEET, and the variables SRE, PRE, GRE, WRE, and TP score.

Utility of Demographic Variables in Performance Prediction

In order to determine the utility or effectiveness of the demographic variables previously introduced to relate or be associated with on-the-job technician performance, product-moment correlation coefficients were developed between those demographic variables and the performance variables. From this analysis it is possible to infer the extent which the demographic variables may be viewed as linear predictors of the demographic variables.

Demographic Variable Prediction on Combined Locations

Table 21 reflects the product-moment correlation coefficients that resulted when the 949 technicians were considered. In that table there are many product-moment correlation coefficients significantly different from zero, but they are of such low magnitude. Now from a linear prediction standpoint, the multiple correlation coefficient (Rz) is related to the product-moment correlation coefficient (rxy) by R2 = r&y. (x 1s one of the demographic variables and y is a performance variable.) Therefore, from Table 21, even for the largest r„„ value observed,'^ namely 0.217, the Rz value for this case is still a very low 0.Ü47 for the degree of fit of the linear model to the data (x 1s SKILL score and y 1s TP score). However, even with the low correlations represented by the BTB scores, those scores do offer some hope of predicting on-the-job performance as measured by the TPCF. The correlation coefficients reflect the degree of association that is typically found between predictor variables and on-the-job performance. However, these results show considerable promise for further research in the area of the development of on-the-job performance measures.

Demographic Variable Prediction in the EM, ET, FT, and IC Ratings

In a similar manner product-moment correlation coefficients were developed between the demographic and performance variables for the technicians sampled from the four more promising ratings EM, ET, FT, and IC. In Table 22 are the results of this analysis. From this table the same observations as above can be made for this case since no product-moment correlation 1s greater than 0.217, the largest previously observed. Also, as in the above, some hope is offered by BTB scores for predicting on-the-job performance as measured by the TPCF. The correlation coefficients, though suppressed, are still of sufficient magnitude to warrant some promise for further research.

^Excluding B school final score which has such a low sample size represented.

39

TABLE 21

DEMOGRAPHIC CORRELATIONS ON EIGHT RATINGS

Mo known by super

Mo on current assign

GCT scores

ARI scores

SKILL scores

A school final grade

B school final grade

C school final grade

N Ref Mat

Equip Op

C1r Anal

Pers Rel

Elec Safe Inst

Elec Rep

Elec Cog

948 0.032 0.068* -0.006 -0.020 0.038 0.001 -0.069* -0.036

94B 0.029 0.106* -0.005 -0.034 0.057 -0.013 0.012 -0.009

909 -0.027 -0.034 -0.048 -0.078* -0.014 -0.096* -0.050 -0.076'

909 0.018 -0.014 -0.030 -0.086* -0.031 -0.076* -0.039 -0.022

909 -0.006 -0.029 -0.046 -0.095* -0.027 -0.100* -0.052 -0.057

774 0.089* 0.090* 0.046 0.038 0.017 0.082* 0.104* 0.101*

16 -0.022 0.379 0.227 0.231 0.178 0.323 -0.044 0.300

264 0.016 0.077 0.006 0.000 -0.054 -0.009 0.119 0.079


Mo known by super

Mo on current assign

GCT scores

ARI scores

SKILL scores

A school final grade

B school final grade

C school final grade

N SRE PRE GRE WRE TP Score

948 . -0.029 -0.038 0.055 0.011 0.086*

948 0.067* 0.036 0.044 0.070* 0.191*

909 -0.119* -0.100* -0.040 0.018 0.186*

909 -0.076* -0.066* -0.013 0.037 0.193*

909 -0.111* -0.095* -0.034 0.032 0.217*

774 0.142* 0.096* 0.084* 0.098* 0.070*

16 0.339. 0.387 0.167 0.265 -0.173

264 0.041 0.027 0.047 0.092 0.091


40

TABLE 22

DEMOGRAPHIC CORRELATIONS ON FOUR RATINGS

N Ref Mat

Equip Op

C1r Anal

Pers Rel

Elec Safe Inst

Elec Rep

Elec Cog

Mo known by super 481 -0.043 -0.021 -0.097* -0.055 0.025 -0.085 -0.062 -0.084

Mo on current assign 482 -0.035 0.093* -0.034 -0.044 0.064 0.032 0.081 -0.019

GCT scores 459 -0.045 -0.001 0.026 -0.072 -0.047 -0.013 0.008 -0.007

ARI scores 459 -0.002 0.014 0.026 -0.094* -0.035 0.006 -0.008 0.032

SKILL scores 459 -0.026 0.007 0.030 -0.093* -0.046 . -0.003 0.001 0.014

A school final grade 383 0.086 0.107* 0.047 0.009 0.057 0.063 0.067 0.088

B school final grade 10 -0.342 0.322 0.103 0.129 -0.201 0.328 -0.225 0.250

C school final grade 164 0.004 0.051 -0.057 0.016 -0.103 -0.097 -0.018 0.085


N

Mo known by rater 481

Mo on current assign 482

GCT scores 459

ARI scores 459

SKILL scores 459

A school final grade 383

B school final grade 10

C school final grade 164

SRE PRE GRE WRE TP Score

-0.083 -0.096* -0.005 -0.040 0.089*

0.017 0.014 0.088* 0.062 0.214*

-0.050 -0.081 0.008 0.065 0.172*

-0.033 -0.056 0.016 0.048 0.144*

-0.047 -0.077 0.014 0.064 0.178*

0.071 0.049 0.112* 0.125* 0.167*

0.028 0.213 -0.010 0.027 -0.265

-0.003 -0.045 -0.019 0.080 0.135


41

SUMMARY AND CONCLUSIONS

Summary of Data Analyses

Evaluation of the Performance Estimators

When evaluating a performance estimator 1t is initially of Interest to consider the degree to which that estimator doas in fact measure an individual's on-the-job performance. In order to address this particular aspect of the analyses it was necessary to choose an appropriate criterion measure of on-the-job performance. The variable selected as the criterion measure in this report was Technical Proficiency (TP) score (or Tech- nical Proficiency Checkout (TPC) level) as derived from the Technical Proficiency Check- out Form (TPCF). Prior research efforts demonstrated that this variable was a viable criterion variable. Initially triserial correlation coefficients were developed between the TPC level and each of the performance estimators (SRE, PRE, GRE, and WRE) being evaluated. Essentially due to the lack of normality of the continuous variable TP score, triserial correlation had to be considered Inappropriate for the data collected in the course of this project. Finally a curvilinear regression analysis was executed between each of the performance estimators and TP score with an emphasis upon a least-squares interpretation of the corresponding results.

When the entire sample of 949 technicians was considered, relatively low multiple correlation coefficients corresponding to a linear, quadratic, and cubic model were obtained. Furthermore, low product-moment correlation coefficients were obtained for some estimators. Due to these results it was decided that a more appropriate approach to analyzing the data would be an analysis by rating. That is also a more appropriate approach for job tasks or job activities would be more homogeneous within a particular rating.

When the individual ratings were considered it could be concluded that of the four performance estimators - SRE, PRES GRE, and WRE - the WRE possesses the greatest degree of promise in all ratings. The o^ly exceptions occur for the ET and RD ratings in which the GRE is slightly better and none of the estimators apply, respectively. The WRE is a moderate estimate of absolute technician performance 1n the EM, ET, FT, and IC ratings and a modest estimate of absolute performance in the RM, ST, and TM ratings. It is recommended that the WRE be applied, in particular, upon the EM, ET, FT, and IC ratings for purposes of comparing Individuals or groups of Individuals in those ratings. In fact, 1t is suggested that in most analyses where the performance of an Individual, or group of individuals in those ratings is needed, (I.e., an estimate of the probability of effective performance is required) that the WRE be employed. Usually these analyses would Involve a classification or discrimination of such Individuals on the basis of their present-day on-the-job performance. However, use of the WRE in the RM, ST, and TM ratings 1s not as justified and, therefore, 1f used in those ratings, it should be only with utmost caution with regard to Its low validity, as determined by the TPCF.

The Criterion Measure of On-the-Job Performance

Because the TPCF was employed as a criterion measure of technician on-the-job performance, an analysis of the extent to which its properties were verified by the data collected 1n this project was considered. To achieve this end Conditional and Joint Frequency tables were developed by rating to determine whether the hierarchial classification of the job tasks was in effect and whether the job tasks were appropriate

42

to present-day electronic maintenance activities. Results from those tables revealed that only one job task - capable of calibrating most of this unit's equipment with which his rating 1s concerned - was more dated than the other .job tasks. With this exception the job tasks seemed to follow a hlerarchial classification in all but the RM rating, and to a lesser degree 1n the RD rating. From these analyses 1t is suggested that the following alterations be completed by a potential user before the TPCF is employed either in an operational or research context:

1. An updating of the job task descriptions by rating. It is not likely that a lirt of task descriptions can be developed that are characteristic of the population of electronics maintenance personnel without being too general and not specific enough for their Intended use in any rating.

2. A verification of the hlerarchial classification of the up- dated task descriptions by rating.

3. Revision of the TPCF to include an answer column that would reflect whether the technician actually works at the task description being considered.

4. Development of alternative test statistics that would reflect the hlerarchial classification of the job tasks which TP score does not do. Suggested random variables to consider are:

X = the highest job task the technician 1s CHECKED OUT on, or

Y = (X + TP score )/2.

Use of X 1n a rating is most warranted 1f the hierarchial classification 1s correct; if not, use of Y will "average" the effects of X and TP score.

5. A validation by rating that the TPCF is highly predictive of on-the-job performance for that rating.

In conclusion 1t can be said that the TPCF represents the most promising of the performance Instruments evaluated. It 1s certainly the most easily administered and offers the least amount of confusion to the evaluator. Furthermore, the possibility of deriving competency levels from the Conditional Frequency tables offers an alternative procedure for comparing ratings or groups of technicians within a rating on the basis of the proportion of Individuals CHECKED OUT at most on a certain task. Although such comparisons were not attempted 1n this report, it certainly is an appropriate topic for future research on the TPCF.

Comparisons Between Ships and Ratings

Multiple comparisons between ratings and between ships was accomplished in this report by means of appropriate fixed-effect additive Analysis of Variance (AOV) models. Use of these models required the homogeneity of variances across the main effects (ships and/or ratings), normality of the variable upon which the comparisons are to be based, and independence of the observations across ships and ratings. The performance variable selected, from among SRE, PRE, 6RE, WRE, and TP score, was that one which best conformed to the above requirements and could be most associated with technician on-the-job performance. Initially this involved the Investigation of the distributional properties of each of the performance variables. The distributions of the variables SRE, PRE, GRE, and TP score were all shown to be non-normal for the subpopulations considered. This

43

characteristic of those variables is only detrimental in AOV analyses for the case of unequal number of observations for the ship-rating combinations, which was the case for the data collected in this report (see Table 1). Essentially the difficulty arises when one attempts to apply a test of homogeneity of variances for the variable considered. The standard tests for homogeneity of variance require normality of the variable considered.

Only the variable WRE could be termed normally distributed for all the subpopulations considered. Furthermore, it was shown to be the most promising type of performance estimator, particularly 1n the EM, ET, FT, and IC rating. In view of these and other related considerations it was the most optimal choice of a performance variable upon which to base the multiple comparisons. Unfortunately, not all the requirements for use of the AOV models to be employed were satisfied by the WRE, however, the techniques employed to that end demonstrate the procedures that one should go through in order to validly apply the AOV models.

Initially a two-way fixed effects design was applied to the data. However, because significant Interaction was found to exist between ships and ratings, this necessarily forced multiple comparisions between ships (ratings) to be performed for a fixed rating (ship). Twenty-five AOV's were executed in that event in order to detect significant mean WRE scores between the four more promising ratings (EM, ET, FT, and IC) for each of the 21 ships 1n the project and between the 21 ships for each of these four ratings. No particular pattern seemed to emerge across ships as to which rating is more proficient (in terms of mean WRE scores). Furthermore, no pairwise significant difference was detected between any two ships (for any of the four ratings).

In conclusion 1t would seem that the WRE is the most appropriate performance variable upon which to base the multiple comparison of ships and ratings. However, it would be a valuable exercise for a user to also Include the other candidate performance variables and choose that variable most appropriate to the characteristics of the data collected. Likewise, the application of the appropriate variable for particular ship- rating configurations may yield insignificant ship-rating interaction, thus eliminating some of the difficulties encountered in this report. In particular it is the user's responsibility to test the requirements and various configurations before deciding on a particular technique for performing multiple comparisons. It is hoped that the procedure employed in this report offers some guidelines to the potential user for application of the performance measurement technique being researched.

Analysis of Demographic Data

In addition to the performance data collected on the TPCF and JPQ ANSWER SHEET, demographic data were collected by means of the Personnel Identification Information Form (PIIF). The demographic variables that were subsequently studied were: months known by rater„ months on current job assignment, GCT and ARI scores, and A, B, and C school final grades. Product-moment correlation coefficients were developed between those variables and the SRE, PRE, 6RE, WRE, TP score and each of the eight job activity reliability ratios. Although the resulting correlations were low, some hope was offered by GCT and ARI scores for predicitng on-the-job performance as measured by TP scores. It 1s felt that further research 1n this area may reveal that Basic Test Battery scores have some utility for predicting job performance levels.

Observations on the Use of Composite Reliability Values

It will be observed that many of the analyses chosen for analyzing the data rest upon the particular characteristics observed 1n the data, e.g., distributional properties

44

of the data. This is a reasonable and preferred procedure to follow in order to form an opinion based on the data collected. However, there was one necessary, and arti- ficial, revision of the fundamental characteristics of the original data that was required 1n order to arrive at a numeric estimate of performance based on the estimators SRE, PRE, GRE, and WRE. That revision, of course, Involved the adoption of a convention to estimate technician performance for those cases in which the technician either does not work at a job activity or received SUE = ZUI ■ 0. The choice of a composite reliability value (Appendix F) to estimate technician performance 1n those cases cannot be viewed as the most optimal choice. However, it 1s certainly a reasonable and efficient means of employing the original data 1n order to achieve performance estimates 1n those cases. Reasonable 1n the sense that this estimate is derived as the individual job activity reliability ratios are derived, i.e., by totaling the number of UE's and UI's observed and forming the ratio ZUE/[ZUE + ZUI]. Therefore, 1t should be no less objectional than are the reliability ratios. Furthermore, it is efficient in the sense that no complicated mathematical procedure is required in order to derive the composite reliability values.

The composite reliability values were derived by location (Table F-2) and it was those values which were subsequently employed throughout the remainder of the analyses. A better procedure would have been to derive a composite reliability value for the total of N = 949 technicians and employ those values for subsequent analyses rather than by the use of "per location" composite reliability values. However, as can be observed in Table F-2, the composite reliability values do not differ significantly by location except 1n two cases: Job Activity No. 7 in the RD rating and Job Activity No. 3 1n the TM rating.

Effect of the Convention on the Performance Estimates SRE, PRE, and GRE

The Convention in the RD Rating

For the RD rating the composite reliability value 1s 0.0 for Job Activity No. 7 at Location No. 2 (Table F-2). However, at Location No. 2 the RD's received SRE = PRE = 0.0 (see p. 3 and footnote 2) because all (but one) RD at that location either did not work at Job Activity No. 7 or received £UE = EUI = 0 (see Table E-l). In addition to the low SRE and PRE scores resulting from the use of this convention at Location No. 2, it must be pointed out that low scores were likewise recorded for the TPCF at that location (Table 4). This effect would result 1n a high degree of association between the predictor and criterion variables for the RD rating at Location No. 2 because low predictor scores are corresponding to low criterion scores. This situation, however, is unique to the RD rating at Location No. 2.

At Location No. 1 the situation of a high degree of association between the predictor and criterion variables was not illustrated for the RD rating. As at Location No. 2, most of the RD technicians are 1n the lowest TPC levels (Table 3) and every RD at Loca- tion No. 1 did not work at Job Activity No. 7 or received EUE = ZUI = 0 (Table E-l). However, all the composite reliability values are high for Location No. 1 (see Table F-2) Implying that most likely the SRE and PRE are different from zero and of a large magnitude. This characteristic of the SRE and PRE would imply an inconsistency with the low TPC levels recorded at Location No. 1. Namely, an inverse relationship has been created at Location No. 1 essentially due to the convention that was adopted. That is, high predictor scores correspond to low TPCF scores in the RD rating, whereas at Location No. 2, low predictor scores are associated with low TPCF scores. It may be said that this effect at Location No. 1 has significantly contributed to the low correlation values that were subsequently calculated (see Tables 6 and 7). However, in terms of composite reliability values, the correlations are most correct at Location No. 1. Only at Location No. 2, where as previously noted, there is an inconsistency in the composite reliability values, were the performance estimators, SRE, PRE, and GRE able to correspond to the low TPC levels and, thus, Incidentally, produce higher correlations. Therefore, the correlations obtained at Location No. 1 probably reflect the most accurate portrayal of the use of composite reliability values.

45

The Convention in Other than the RD Rating

Now except for the extreme case mentioned above in the RD rating, all other composite reliability values in Table F-2 are high. This 1s in close agreement with the TPCF results (Tables 3 and 4) for the EM, ET, FT, and IC ratings. The distribution of TPC levels for those ratings demonstrate a high concentration of individuals in the highest level of technical proficiency. On the other hand, in the RM, ST, and TM ratings, the distribution of TPC levels is uniform or demonstrate a higher concentration of technicians in the lowest level of technical proficiency, but the composite reliability values are high.

Now it 1s difficult to generalize as to how the above observations are reflected in the estimators SRE, PRE, GRE, and WRE for association with TPCF results. This de- pends on the frequency with which the composite reliability values are used (Table E-l). However, it seems true that the greater the frequency of use of the composite reliability value the less agreement of the TPCF with the reliability estimators.

Observations on the Use of Composite Reliability Values

As with any convention one employs, there seems to be a degree of artificality in the composite reliability values, for it would probably be easy to select a convention, by rating perhaps, which would allow the estimators to have a significant degree of promise in all ratings. This, of course, is a most inappropriate means by which to evaluate an estimator. However, results of this research effort seem to indicate that 1t is not the use of any_ convention that is posing difficulty, but rather its overuse. As discussed in Appendix E, there were 42 (out of 164) rating-job activity combinations where the convention had to be employed on at least one-third of the technicians in some job activity and rating. Such a high use of a convention can only force the individual performance estimates (in particular the SRE, PRE, and GRE) to more reflect the effect of the convention rather than individual performance.

This observation 1s further reinforced with the WRE by the frequency with which it must employ the convention. This estimator only employs the convention for the case in which the technician received EUE = IUI = 0. The case in which the technician did not work at the job activity is ignored by this estimator. Table K-l reflects the improvement in nonuse of the convention. It 1s believed that in no small way does considering only the job activities a technician works at and then appropriately weighting those to form a reliability value permit a greater reflection of individual performance and subsequently a greater degree of association with the TPCF.

Conclusions

In conclusion the more promising estimator of electronic maintenance performance as a function of uncommonly effective and uncommonly ineffective performance incidents is a performance estimator similar to the Weighted-Average Reliability Estimate (WRE). This conclusion is made under the assumption that composite reliability values are employed 1n those job activities in which a man does not work at or received ZUE = ZUI = However, even this estimator cannot be recommended for general use within the U. S. Navy at the present time. Only in the EM, ET, FT, and IC ratings is Its use presently warranted, while application 1n the RM, ST, and TM ratings is tenuous. It is suggested that before this estimator 1s employed 1n other ratings that the appropriate job task analysis and Isolation of job activity factors be completed before a validation effort 1s attempted in that rating, perhaps similar to the procedures conducted in this report. That 1t is necessary to perform a validation effort was illustrated 1n this research effort by the results obtained 1n the RD rating. It would seem that 1t is too much to assume that the technique can be automatically applied to other ratings with even similar job activities.

46

Finally, the results of this research effort do demonstrate that the performance measurement technique being researched does have a large degree of potential for practical application on specific ratings within the U. S. Navy. In particular, the detailed analyses of the Technical Proficiency Checkout Form (TPCF) demonstrated the potential of that instrument to isolate job tasks which may require further training on the part of some technicians. Likewise the Job Performance Questionnaire can also be employed to Isolate those Individuals of potential detrimental performance levels in terms of ex- cessive uncommonly Ineffective Incidents of performance. Even though such instruments are completed by the technician's immediate supervisor, as opposed to an unbiased evaluator, those positive qualities of the performance measurement Instruments can prove to be most valuable even 1f one 1s not Interested 1n deriving an estimate of the probability of effective performance for the technician evaluated. Therefore, short of a procedure which allows for a completely unbiased evaluator, the technique must be given credit for being perhaps the most viable performance measurement technique presently available and of being of greatest potentltal value to the U. S. Navy.

47

THIS PAGE IS BLANK

48

APPENDIX A

PERFORMANCE EVALUATION INSTRUMENTS

A-l

THIS PAGE IS BLANK

A-2

JOB PERFORMANCE QUESTIONNAIRE

Name of Supervisor Rating Ship or Unit_

Instructions to Supes visor. The purpose of this form is to determine the number of effective and ineffective performances you have observed amongyour mpn durine the oast two months. We are only interested in the uncommonly effective and the uncommonly inef feet i ue performances.

List below the names of all the men under your supervision who are currently striking for, or in any of the following ratings: DS, EM, ET, FT, IC, MT, RD, RM, ST, TD, TM (AE, AT, AQ, AX). If you supervise more than one of these ratings, please use a separate form for each rating.

Now, considering the fleet electronic maintenance objectives, enter your estimate of the number of uncommonly effective (UE) and uncommonly ineffective (UI) performances during the past two months for each man being rated. Please refer to the definitions lists for the meanings of the JOB ACTIVITIES and of the OBJECTIVES involved.

The first line has been filled in as an example. The supervisor completing the example felt that Peter Smith had ten unusually effective performances and two unusually ineffective performances while performing Electronic Circuit Anal- ses when considered against the objectives of fleet electronic maintenance. He also felt that Smith showed two uncommonly effective performances in the area of Electrosafety and four uncommonly ineffective performances in Instruct i on .

If a man has not had an opportunity to perform in a particular area, enter a dash (-); if he has had an opportunity but has not shown any uncommonly effective or ineffective performances, enter a zero (0).

Name and rating UE UI UF UI UE UI UE UI UE UI UE UI UE UI UE UI

A-3

REMINDER: WHEN TYPING LAYOUT, PLEASE KAKli SURE THE RISt?GN IS IN COCO CONDITION SO THAT YOU WILL HAVE A SHARP, DARK COPY FOR BETTER <\ EF RCOUCTION.

LU

co

Cd

UJ

3c CO

<c

Q_

<

«a: a; ■=} o _i c/> < *-^ :> >• LU Q:

LU S?T Q. < = i. to

U- u_ o o LU LU V «r < <

/ Z ZZ3Z vl

<TJ s. 0) Q. O

7" o —• CM m

J3 •-"

03 LU

z_ o^HCNjfOj"tr)^DP* CP cyi

o H oj w 4- no io [*•*• co en

OHNfoj-mioNooo^

o -H NW J i/i ^ N a) cn

zr z. z ~~^

Z—7" o« (M n

<0 LU >— =3

OrtNrttflfiUJNCOO'

o ~+ <\j fr> j to ip r- ep en

o H N n j u"> uj r» co a>

z z z z I O <— CVJ CD

•^^

O —IN <n j ir> to Is- 00 'IT*

O «-1 CM CO J- LD <JD JN GO <Ti

o H (sj w j u> iß fôocr^

o H fv m j i/> LO f^coffi

z: 2: o ■-• «>J m

10 LU

HCMWJl/lülNtOO

o H N m ^ into r* co en

o.HCMOJ-mLD<>»cri

OrtN 1 J U)lC N 00 oi

o •p- > <u Q.

I/O

^cMco-a-mior-cDcr»

^* c\j rn j" ui iß N co oi

^cMroj-mvor^cocn

r-icMroj- in a MO m

H N PI j- m CD r* co <r»

-• M m j- tftvorâocr»

CM CO .3- in to NCO gi'

OrtNmjIftiONfflm

Ortnjroj-inijoiôocn

«■Hcsioj-mvör^coo

o^cMmj-mißr^cDcr»

OHNnjuiioMom

Ortrgroj-iftdsrôoo»

o^tMfoj-iniDrvoocn

o HN n *,.in.* Mom

Or-tcjcoj-miptôocr*

PRINTCD IN U.S.*. „ OPTICAL SCAMHIHO CGtPCMUTIOH C "•••*- -

A-4

TECHNICAL FRO F*0 IE N GY< 'OH EORO U T FOR

NAME OF SUPERVISOR RATING/RATE

FULL NAME OF MAN EVALUATED,

SHIP OR UNIT LOCATION DATE

TASK DESCRIPTION

NOT CHECKED CHECKED

OUT OUT

1.

2.

3.

5.

6.

7.

8.

Capable of employing safety precautions on most of this unit's equipment with which his rating is concerned.

Capable of replacing most of this unit's equipment with which his rating is concerned. .

Capable of removing most of this unit's equipment with which his rating is concerned.

Capable of following block diagrams for most of this unit's equipment with which his rating is concerned.

Capable of knowing relationship of equipment to up other related equipment with wh is concerned.

ich his rating

Capable of calibrating most of this unit's equipment with which his rating is concerned.

Capable of trouble-shooting/isolated malfunction^) in most of this unit's equipment with which his rating is concerned.

Capable of employing electronic principles involved in maintenance of most of this unit's equipment with which his rating is concerned.

D □ D D D

□ D □

D D □

□ D D □ □

MAKE CERTAIN THERE IS AN "X" IN A BOX OPPOSITE EACH TASK DESCRIPTION

A*5

—

■■

■■'■

-

■

'■',

•_-

>

DIRE

CTIO

NS

Use

a #2 pencil only.

Blacken

the

answer

j rectangle

completely.

Clea

nly

erase

answers

j yo

u wa

nt to ch

ange

. Er

ase

any

stray

marks.

j 'T

ECHN

ICIA

N' is th

e man

bein

g ev

alua

ted.

j~

In th

e bo

xes

just below, print

as much as r

possible of th

e TECHNICIAN'S NAME.

Start with h

his

last na

me,

prin

t on

e le

tter to a

box.

LeavS

one

box

blank, then print

his

firs

t na

me;

leave"

J another

box

blank, then hi

s middle na

me,

etc.

Z

\

_9

YOU WILL HAVE A SHARP, DARK COPY FOR BETTER REPRODUCTION.

10 11 1? 11 U 15 16 17 II 1».. »0_2t 11_1) Ji_«.J» IL?'..-?♦—yj-1 .?-- ,5-J4. 'S.

05X)ü'Oa)«Hb0Ä-iH »-3^ H BCOfttf!*»)^ 3 > > X >> H

«IP yfl ID<H WÄ-H --a,X HBGOPiO'hw-p 3 > > x >> N

aJXi u-ti 4)"« bOÄ-H *-3 J«S HaC0p4O,hW+5Dp->X>»N

cdxtO'ÖOJvihOÄ-H --3,X HeßOftC^w-P 3 >■ > X >» N

eaxi ox) <U<M bOrö-rH >OAS HfiCOftCtmiP p > > X >> N

05X1 U-d (U4H bOXI-H »3*1 HSÖOCHO<VlOT4J 3 >• > X >» N

dx) ox) <U<H toxi-H --a*! HBüOfto'Viw+j 3 >• > x >> N

(ÖJ3 OT) <U<H bDXI'H --a.* HgCOftO,^[0+> 3 > > X >» N

aJfQOTj(U<t-itJDX)-H^*!HecOP(O,>HC0+> 3 > > X >> N

BipO'ÖJJÎiDÄ-H'jXHaßOftO'milp 3 > > X >> N

BjXtt)'d(U<H«)J3'H»-3j«!HaÖ0P<O,L(tn4J 3 > > X >> N

ajjau-d 0)<H Mx:-H--3*iH e c oao'L.to-p 3 > > x >> w

Technician's Social Security N

o.

z

:

o^4CMcoj-mtoNcoen

Oi-iCMfo^mtocôocn

o^4CMco^miorôocn

o »-»cMcoj-tn ôr^cocn

oîcMcoj-uivaCvcocn

OHNrtj-mvDNoo en

/ / / / /■ CD

to L. • r : ;-.,■• " .'

I r

t

en = >> O ro - O.

r— CMCO«3-LOVOr>.000> r i i i i i i i r

LULULULULULULULULU

J

si—i- oos: r-Z LU uju_ >-4 ac ai oo (—

//////////////

'/

Tech

ni

bati

ng

an's

.

ite

Ot~t oj *n j- in ID r*> co en

o »-H cMm-^^^t-^cocr»

nths

urre

nt

ob

gnment

0 l

/

/ / / / / / O X •i- -C c +■

J i— O <_> "3 T- s: to O -» 0J

3 C to to o <=c / / / / tO -r- CD

- +J o .

(PIIF

G/RAT

ION 1— csojcjftcöaps.fuoooj

c «J ■ j_

•r- >> O . o ja to _ •r- -r- ° — C C > .

J= 3 t- u 5 <u _

N m 41 l/l ifl N CO ffl

N m j in j r^ co en echnician

iths of A

c )uty Se

rvi on o^cMcoj-mior^cocn

O^^CMcoj-myor^cocn

S £ <_> o <=c o wwvv

o

C3^4CMmJ-mtDl^'COCT>

1—^3 00 O 00 S 73

NEL

IDEN

TIFI

CATION INFORMATION

1

1—4

V o^^cMcoj-tnLorscocn

XAAAAAX JZ o 00

o o-4tNfoj-mvor> oo oi

o

o HM n j m tc r»

O >-4 CM CO J" Ullfl N

o rt N ro j ui <x> r»

co en

co en

co en

LU M

00

to to

o

O -4 CM co j-

o -4 CM co J-

O -4 CM CO j-

in IJO r^ oo &

m ^ p*. oo <r>

in ip r«- oo oi

■o LU

to s- >-

OrHCMcojinujtvcoen

OHN« J i/>uDr»co en

0^4 CM n .a- m to c* co en

cc LU -J

o HW n * mu>pêoen

0£ MARK

001

z 1—1 a < r- oo oo

O ^H CM <»> .J- Ul l& t**

o r-.rgn^' m tf r^

co en

co en

co en

O —1 CM CO J-

O *< CM CO j-

O H N fl J

m 0 N oo ^

in io t"* co cr»

in »jp r* co en

o —* CM co j- ir>u3C^coen O

to

< z tx

JE j (J : u> i

. CQ

©^cMcoîntor^cocn

o -H CM m j- m to f* co G~>

o LU

Or-4CMcoj- in ID h- co ffi

ÄÎCMCOJ- iriiocôocn

PERS

ON

WE OF SUPERVISOR

O O OHM« JU) il so»

O^CMfOjtlTluOt^CDOl

o *H CM co J m u) Mt er

OHN"i*mus

o ~* CJ co -* i/> to f">

©•-icMco^mtDr»

oo en

co en

oo en

O -4 CM CO .»

OF4 N n j

o H (M n*

in vP ^ co cr*

m U3 P** co c*

m \S> r> co cr>

I "

Or4 N m j invDf^cocn

o ^ W Kl J /• d MS Jl JE u

r- O

OHNnjI/lioMSCl

//// '///

// o^^cMcoj-tnor^cocn

Z Q / / / / / / ■/

/ / // /. •/ / / / /

X

i'Tli.l #RIMTKO IM U.»>A. DS 2966 „ OrncAL »CAHnma COBPOKATION O -..-. .

A-6

DESCRIPTION OF JOB ACTIVITIES

JOB ACTIVITY DESCRIPTION

1. Using Reference Materials—includes the following type of activities:

a. use of supporting reference' materials b. making out reports

2. Equipment Operation—includes the following type of activity:

a. operating equipment, electrical and electronics test equipment, and other electronic equipments

3. Electronic Circuit Analysis—includes the following type of activities:

a. understanding the principles of electronic circuitry

b. making out failure reports c. keeping records of maintenance

usage data

A. Personnel Relationships—includes the following type of activity:

a. supervising the operation, inspection, and maintenance of electronic equipments

5. Electro-safety—includes the following type of activity:

a. using safety precautions on self and equipment

6. Instruction-—includes the following type of activity:

a. teaching others how to inspect, operate, and maintain electronic equipments

7. Electro-repair—includes the following type of activity:

a. equipment repair in the shop

8. Electro-cognition—includes the following type of activities:

a. maintenance and troubleshooting of electronic equipments

b. use of electronic maintenance reference materials

A-7

MEANINGS OF FLEET ELECTRONICS MAINTENANCE OBJECTIVES

1. Readiness

To maintain efficiently self, subordinate personnel, equipment, and systems in state of readiness consistent with fleet requirements.

2. Performance

To complete any given mission in minimum time with appropriate level of accuracy and reliability.

3. Operation

To obtain optimum system output when equipment is operated, i.e., output characterized by precision and variability appropriate to mission.

4. Safety

To carry out duties with maximum protection for men and equipment consistent with mission.

5. Preparation

To prepare for personnel requirements of present and future equipment, systems, and situations through use of training programs, maintenance of high morale, etc.

A-8

APPENDIX B

DATA COLLECTION PROCEDURES

B-l

THIS PAGE IS BLANK

B-2

PROCEDURE EMPLOYED IN DATA COLLECTION

The data collection effort originated by requesting of CINCPACFLT and CINCLANTFLT that ships (Destroyer-type vessels) and men (in the electronic maintenance ratings - EM, ET, FT, IC, RD, RM, ST, and TM) be considered for participation in this project. It was requested that initially a project coordinator at an appropriate fleet command be assigned in order to coordinate the efforts of the project researchers from NAVPERSRANDLAB and ship liaison officers. Favorable response from both fleet commands resulted in the assignment of the following project coordinators:

LCDR G. W. Dunne Flag Secretary Cruiser-Destroyer Flotilla Nine FPO San Francisco, CA 96601 (located at the U.S. Naval Station, San Diego, CA)

RDCS C. J. Masterson COMCRUDESLANT Staff CRUDESLANTFLT FPO New York, N. Y. (located at the U.S. Naval Station, Newport, RI)

The duties and responsibilities of the project coordinator may be outlined as:

1. identifying the ships available for participating in the project.

2. seeking the assignment of a liaison officer (usually the senior electronics materials officer) aboard each available ship to coordinate the data collection aboard that ship.

3. scheduling the meetings between the ship liaison officers and the project researchers.

4. making appropriate checks on the progress of the data collection effort aboard each ship.

5. collection and reviewing for completeness of the data forms before mailing all forms to NAVPERSRANDLAB.

Once the project researchers had established thee ships and liaison officers of those ships that were participating in the project, an initial

arranged between the ship liaison officers and the project At this meeting the researchers explained the purpose of

and outlined the duties and responsibilities of a ship liaison officer for completing the data collection effort aboard his ship. It was emphasized that it would be the ship liaison officers responsibility to perform all phases of the data collection effort aboard ship.

meeting was researchers the project

B-3

Each liaison officer was given a set of instructions (Appendix C) which outlines in detail the steps he was to perform in order to complete the data collection effort aboard his ship. Essentially it was required that the liaison officer select immediate supervisors of men involved in electronic maintenance activities in the EM, ET, FT, IC, RD, RM, ST, and TM ratings who may participate in the project. Each liaison officer was also given a set of Roster Forms (a sample of a Roster Form can be found in Appendix C, page C-8). One Roster Form was completed by the liaison officer for each supervisor participating in the project and it includes a list of the men the supervisor is to evaluate. A technician occurred on this list if that supervisor had been his immediate supervisor for at least the past two months, length of the evaluation period, and the technician at that time was at most a petty officer second class in one of the aforementioned ratings. In order to achieve the most reliable information and not to place too much of a burden on any one supervisor, the supervisor was limited to evaluating no more than seven men - the seven, or less men, he would be most knowledgable about with respect to their on-the-job performance.

At the second and final meeting, the Roster Forms were returned to the project researchers who gave a Xerox copy of these forms to the ship liaison officer and the project coordinator. Using the Roster Forms the project researchers completed packages of Performance Evaluation Forms, one such package for each supervisor. Each package contained a set of Instructions for the Supervisor for Completing Performance Forms (Appendix D), a Xerox copy of the appropriate Roster Form listing the men that the supervisor is to evaluate, and the appropriate number of sets of Performance Evaluation Forms (discussed in the section, Data Collection Instruments), one set for each technician the supervisor was to evaluate.

Upon returning to his ship, the liaison officer held a meeting with the supervisors to explain the purpose of the project and to describe how they were to complete the sets of Performance Evaluation Forms. Once the supervisors had completed their evaluations, the liaison officer submitted all forms to the Administrative Officer aboard his ship who provided the additional demographic information on the men evaluated. After this exercise the ship liaison officer returned all forms to the project coordinator. The project coordinator, once he received the completed forms from all the ships, mailed all forms to NAVPERSRANDLAB.

B-4

APPENDIX C

INSTRUCTIONS FOR SHIP LIAISON OFFICER

NOTE: The instructions on pages C-3 thru C-12 (pages -1 thru -5 of the text of the instructions and pages -1 and -1 thru -4 for enclosures (1) and (2) respectively) of this report are paginated for publication. For economy of reproduction, the page numbers were not changed for this report.

C-l

THIS PAGE IS BLANK

C-2

NAVAL PERSONNEL RESEARCH AND DEVELOPMENT LABORATORY Washington, D. C.

Performance Evaluation Measures Information to Ship Liaison Officers

Background

The general purpose of the present study is to further investigate an economical and practical method for providing feedback information regarding the readiness of Naval electronically oriented technical personnel (specifically EM, ET, FT, IC, RD, RM, ST, and TM ratings) for completing their assigned mission. Essentially, a result of the study may include the application of a performance evaluative technique which will allow continuous and quantitative answers to questions such as: What is the current level of effectiveness of the maintenance personnel in a given rating, ship, or squadron? How does the maintenance personnel effectiveness level of a given rating, ship, or squadron compare with that of other ratings, ships, or squadrons?

In a previous Office of Naval Research study, a unique performance measurement technique was developed which the Naval Personnel Research and Development Laboratory is now concerned with replicating in order to establish its validity and to determine the practicality of this concept. Basically, eight dimensions of the electronics maintenance ratings were defined: (1) using reference materials, (2) equipment operation, (3) electronic circuit analysis, (4) personnel relationships, (5) electrosafety, (6) instruction, (7) electro-repair, and, (8) electro-cognition. On the basis of a supervisor observing his men performing on the job, he indicates the number of times that he saw unusually effective or unusually ineffective performance demonstrated on each of the eight job dimensions. Based upon the estimates of unusually effective/ineffective behavior, meaningful measures of technician effectiveness can be established. Further, the individual technician effectiveness values can be further treated to form effectiveness indices for ratings, ships, and squadrons.

Because this effort is primarily concerned with replicating a prior study, any information provided by ship personnel will be used only for research purposes by the Naval Personnel Research and Development Lab- oratory and only to serve as a statistical data base for validating the results of that prior study. Furthermore, any comparisons made between ratings, ships, or squadrons will be used only for research purposes.

Suggested Ship Liaison Officer Assistance Plan

The steps outlined below are those as envisioned by the project researchers which you may execute in order to insure an efficient data collection effort aboard your ship.

Completion of the Roster Forms

Either a project researcher or the project coordinator at CRTTDKSLANT has given you a set of Roster Forms (enclosure (1)). To complete the Roster Forms you are to select men involved in electronic maintenance activities'in the EM, ET. FT. IC. RD. RM. ST. and TM ratines and immediate supervisors of those men aboard your ship. Generally the men being evaluated will be petty officers second class and below. The supervisors may be either enlisted personnel or officers but must be the immediate supervisors of the men they are to evaluate. They should have known the men they are evaluating for at least the past two months. Not all supervisors of men in the previously mentioned ratings will qualify for participation in this project, however,, it is desired that all individuals in the above ratings participate. Once you have completed the Roster Forms, go to each supervisor to participate in this project and have him verify that he is the immediate supervisor of the men that you have selected for him on the respective Roster Form.

Once the Roster Forms have been verified by the supervisors, send a copy of these forms to:

Mr. Bernard Rafacz Naval Personnel Research

and Development Laboratory Room 3315, Bldg. 200 Washington Navy Yard Washington, D. C. 20390

He is your contact man at the Naval Personnel Research and Development Laboratory. If you have any problems or questions concerned with the implementation of the instructions, it may be possible for you to contact him (AUTOVON 288-4457) or the project coordinator.

He will prepare the packages of Performance Evaluation Forms for mailing to you. One such package is a blue folder containing a set of Instructions to the Supervisor on the "left inside cover and sets of performance evaluation forms on the right. On the front of each package is a label with the name of the supervisor who is to complete the sets of performance evaluation forms contained therein on the men for whom he is their immediate supervisor.

Briefing Each Supervisor

Once you have received the Packages of Performance Evaluation Forms from the Naval Personnel Research and Development Laboratory, you are now ready to distribute these forms to the supervisors participating in this project. This may be done in one of two ways: 1) calling a meeting of all supervisors so that you may distribute the packages and brief them as to their responsibilities all at one time, or, 2) meet individually with each supervisor and instruct him on a personal basis as to his responsibilities in completing the experimental forms.

It is your choice as to which procedure to follow, however, for ships with only several supervisors participating, the latter method may be the most convenient.

When meeting with the supervisor you should instruct him on the same subject material as you x^ere originally briefed either by the project researcher or the project coordinator at CRUDESLANT. In particular, you should guide the supervisor through his set of Instructions, going over the three SAMPLE pages of performance evaluation forms. Important things to mention to the supervisor about the three forms in each set of Performance Evaluation Forms are outlined below.

1. The Personnel Identification Information Form (PIIF)

The purpose of this form is to provide various background information on the man being evaluated. The supervisors should first of all supply the information at the top of this form. Only the background information to the right of the cross-hatched area is to be supplied by the supervisor. Please tell him that he must do this to facilitate the job of the Administrative Officer aboard his ship. Because he is normally in everyday contact with the men he is evaluating, it should not be too difficult to obtain this information directly from the man being evaluated. The information to the left of the cross-hatched area will be supplied by the Administrative Officer aboard your ship and so the supervisor should not concern himself with this section of the PIIF. If for some reason he cannot obtain some of the background information from the man being evaluated, please have the Administrative Officer supply this information at a later date. Particular instructions for completing this form are given in the Supervisor's Instruction Package. Please be sure that he understands these instructions and the correct procedure for entering the requested information on the blank PIIF Op-Scan Sheet in each set of experimental forms.

2. The Technical Proficiency Checkout Form (TPCF)

This form is designed to estimate the level of technical complexity at which a man is able to work without direct supervision. The supervisor should provide the information at the top of the TPCF. The eight task descriptions are listed in ascending order of difficulty with task No. 8 identified as the most difficult task. The supervisor is simply to record an "X" in the CHECKED OUT box for each task which he feels the ratee is capable of performing "on his own without direct supervision." Otherwise he should record an "X" in the NOT CHECKED OUT box. The supervisor must be certain there is an "X" in a box opposite each task description.

3. The Job Performance Questionnaire (JPQ) ANSWER SHEET

The JPQ ANSWER SHEET is undoubtedly the most difficult form we are asking the supervisor to complete. However, the researchers have tried to facilitate the understanding of what information is requested by giving explicit samples to each question to be answered, In particular the JPQ ANSWER SHEET records estimates of the total number of uncommonly effective (UE) and uncommonly ineffective (UI) incidents of performance the supervisor has observed over a prior two month period on the man being evaluated. Read page 7 of the Supervisor's Instructions aloud to every supervisor and be sure that he understands what he is expected to record on the JPQ ANSWER SHEET. At the top of page 9 is a definition of what is generally meant as an uncommonly effective or uncommonly ineffective performance. However, the supervisor should not adhere strictly to this definition« He would best know for his particular working area what such performances are. He should use the researchers definition only as a guide to conceptualize upon the extremes of performance.

Page 13 of the Supervisor's Instructions is a SAMPLE of a JPQ ANSWER SHEET found in each set of performance evaluation forms. The supervisor must review the table on page 9 together with the sample in order to understand where and how to record his estimates of the total number of UE and UI incidents of performance on the JPQ ANSWER SHEET. Please review this table with him. Also remark on the NOTE at the bottom of page 9.

QUESTION (c) on page 11 of the Supervisor's Instructions must be answered in column (c) for each job activity. Please point this out to the supervisor.

Three remaining questions given on pages 11 and 12 of the Supervisor's Instructions are to be answered in the lower corner provided on the JPQ ANSWER SHEET. Finally the supervisor is to place his social security account number in the space provided on each JPQ ANSWER SHEET he completes.

Instruct the supervisor that he will have approximately one week to complete all forms for the men he is to evaluate. It is important that he realizes that the Op-Scan sheets (the PIIF and JPQ ANSWER SHEET) will be machine processed and therefore it is essential that they remain as much in their original condition as is possible. Once the supervisor completes the forms, he must return them to you in the original folder, as you submitted them to him. You should review the forms as they are submitted and check that all of the information that is requested has been provided. Please be sure that each form of each set contains the supervisors name and the technician's name in the proper locations.

4. Briefing the Administrative Officer

It is now necessary that you submit all packages to the Administrative Officer aboard your ship in order that the Personal Identification Information Form (PIIF) be completed for each man evaluated. Enclosure (2) is a set of Instructions for the Administrative Officer. You will have to request his assistance in order that the information to the left of the cross-hatched area on each PIIF be completed. All of this information may be found in the personnel jacket of the man being evaluated. Com- plete details for providing this information are contained within the Instructions to the Administrative Officer. To insure that the forms are properly completed, you should review these instructions with him.

Once the Administrative Officer has completed all the PIIFs and returned all packages to you, from each set of Performance Evaluation Forms remove the paper clip. (For Op-Scan purposes it is imperative that the forms are not bent.) All completed packages and any extra forms are to be given to the project coordinator. He will send them back to Mr. Rafacz at the Naval Personnel Research and Development Laboratory.

SAMPLE

ROSTER FORM

Name of Liaison Officer LT George Merklin

USS ROAN

Date 2 Jan 72

Ship_ Homeport Location Newport

Immediate Supervisory Abner Smith _Rating/Rate_ EMC

Names of Men to be Evaluated Tom Henry Jones

George William Klack

Robert Larry Lane

etc.

Rating/Rate EM3

RMSN

ETN2

etc.

Enclosure (1)

NAVAL PERSONNEL RESEARCH AND DEVELOPMENT LABORATORY WASHINGTON, D.C.

Instructions for the Administrative Officer Completing the Personnel Identification

Information Form (PIIF)

Recently your ship and some supervisors of men in the electronic or related ratings - EM, FT, ET, IC, RD, RM, ST, TM - aboard your ship have been selected to participate in a performance evaluation program being conducted by the Naval Personnel Research and Development Laboratory. The men selected are immediate supervisors of personnel in those ratings. The liaison officer aboard your ship in charge of coordinating the activities between the researchers and your ship for this program has given each such supervisor a package of materials containing a Supervisor's Instruction Package and sets of Performance Evaluation Forms; one set for each man the supervisor is evaluating. Every set of Performance Evalua- tion Forms is made up of the following forms:

1. Personnel Identification Information Form (PIIF) 2. Technical Proficiency Checkout Form (TPCF) 3. Job Performance Questionnaire (JPQ) ANSWER SHEET

The supervisor will complete in part a set of Performance Evaluation Forms for each man he is evaluating. Only the Personnel Identification Information Form (PIIF) found in each set is to be completed by the Admin- istrative Officer. In essence, this form records various background information on the man being evaluated. Once the supervisor has completed supplying the information requested on the set of Performance Evaluation Forms, he will return them to the liaison officer who in turn will give them to you, the Administrative Officer, in order that the PIIF in each set be completed. For each PIIF you receive, the supervisor was to have supplied the information to the right of the cross-hatched area on this form. The first thing you should do is to check over this section of the form and see that all of the following information was provided for the man being evaluated:

a. Technician's Name b. c. d. e. f. g-

Social Security Number Birthdate Months Known by Supervisor Months on Current Job Assignment Months of Active Duty Service (USN* and USNR**) Rating and Paygrade

*Record 99 months for USN if the Technician's Months of USN Active Duty Service equals or exceeds 99 months.

**Record 99 months for USNR if the Technician's Months of USNR Active Duty Service equals or exceeds 99, months.

Enclosure (2)

If some information in this section is omitted, please provide it. Use only a #2 pencil when supplying any information on this form. To aid you in completing the PIIFs, page 4 of these instructions contains a sample of a completed PIIF. From this sample you will see that the following information is obtained from the section of the form to the right of the cross-hatched area:

1. The supervisor is EMC Abner Smith located at Newport aboard the USS ROAN and completed this form on 7 January 1972.

2. This supervisor was evaluating EM3 Tom Henry Jones whose social security account number is 123-45-6789 and he was born in March, 1948. He was known for 13 months by the supervisor, and had been 15 months on his current job assignment. Jones had served no USNR active duty time but had been on USN active duty for 29 months.

One you have reviewed the sample on page 4 for that section, turn your attention to the section of the PIIF to the left of the cross- hatched area. The following table represents all of the information for this section that is to be provided by you from the service record for the man being evaluated.

Item

1 A School Final Mark B ii

1 C n

A II

Class Standing B II

C it

A II

Class Size B II

C it

Location in Service Record

NAVPERS 601-4

Years of Civilian Education (Yrs. Ed.) NAVPERS 601-3

GCT Score ARI MECH " CLER "

In all cases the most recently completed A, B, or C school information is to be provided. If for some reason a technician did not attend some of the schools, leave that section of the PIIF blank for that school. A Final Mark such as 54.23 is recorded as 54.23 on the PIIF. Final Marks of

the form, 90, 82, 77, etc., (i.e., two digit scores) are recorded as 90.00, 82.00, 77.00, etc., respectively. If some schools gave a Final Mark of "satisfactory" or "unsatisfactory", record 99.99 for a satisfactory Final Mark and 00.01 for an unsatisfactory Final Mark.

Once again turn to page 4 of these instructions which contains the previously mentioned sample of a PIIF and to the section to the left of the cross-hatched area. From that section of the sample form, the following table represents the information the Administrative Officer aboard the USS ROAN provided on EM3 Tom Henry Jones:

Item As Recorded in Service Record

Final Mark

Class Standing

Class Size"

A School B ii

C ii

A ti

B it

C II

A it

B it

C it

Years of Civilian Education (Yrs. Ed.)

77.86 (did not attend B School)

90

57 th

26th

122

27

12

GCT Score ARI " MECH " CLER "

50 49 56 47

If you have any questions regarding the type or the recording procedure of any of the desired information, please consult the liaison officer for this project. Once all PIIFs in all sets of Performance Evaluations Forms have been completed by you, return all packages to the liaison officer. Thank you for your cooperation and efforts in completing these forms.

u . oi a»

o to 13 . Ill] td.o> >> to

ca cti a 0) Cd >-l r-l ■c M *j cd 4J 0) 10 >

0) R >> >» «H fl «C M d cd d O «d T(

OT «J 41 V 01 as i-i i-i w 43

i. .M1 g r-l iH .CU M a 0) 0) X! Q O 4J 00 ti

oi a r-t H CO OJ •H ft. 43 •*

§ O O S5 a u a 0) H

CM r-l u u

c <d a cd cd » S

•u o 0) Ü a w tu «I OH

fa «o H S

2 U» M o g H

H H

8 Si

ä I pxxxxxxx/zz

2 O M

H fa H Ä H -W

CO fa W fa

fa O

fa s to

O £

-»11

ES cv

I CO

01 > 01 ■ &1

43 « > 4-1 01 Cd • •H hJ 0) CJ > iH 4J On

0) CO 4J . ••» «8 M M OJ «

cd o a oi * 4J 43 8 a o to a c3 2 cd a B 4J 5;

• o to oi »HtJ KH « X -H X)

3 H >*-l TJ o> UZ « <H a u to g

1-1 CO 44 fl m - oi j3 to Pi fa»g H -H

< « 43 • H ii a

&s §t s ■-•a &* 01 S 4J 4J xt 0 a a to

Cd i-l 01 « 4J H >J 43 4«! to fa *J Q s oi 8 i*i i-> 43 » » r-l

*J 0) 4« 43

«)<u 1 S X > x o d i-i o O 43 43

43 01 4-1 H tO X U 0|

01 43 (d O 0) 43 -H H 43 43 4J CO 4J

to to 0> O i-a a o -H d d H o..d o cd

iJflUTjoiHMJ'n'n^nBaoaB'UMUS > * X >* N

J «j ufl ojiHDOÄ'H'FiiHe a o a cf M w 4J d > 5» x >» N

K*t)'d4l|w»*'H'n^HBaoao'M»tJ3 > I* X >% N

I cd JO wa onw eo ja •W"-iî-4BdOo.o<rJco4JS > :* x ts N

I I

W 43 O-d 01 IH 6043 «4'n^HSdO ftd'MaiJd^ll K >i N

— dAU'Cti|Hetj:'4Ti^HgaoaD'hioua > ;* X >% N

I cd 43 cj-o oi «w 60JS -H T-)^ HBao atrhnug > gt x »s N

I Cd 43 O -d 0J >H 6043 ft i-i4tf H S d O ACh »U'3 > * X !»- N

I cd43O'O0)>4-i6043'Hi-14dr-iadOO<cr»Jt04Jd>»X>.N

I cd 43 CJ no 0) M-l 6043 -H «-l4»( H 0 d O fa cr M co 4-> g > St X >■» N

I ed 43 U'B »im 6043 f* "-I44 r-l fl C O ftgH B U 3 > > X >-H

to

"d oi **, cd 4J

•H <d o> CJ T3 i-l

•H 43 0 4J

43 H CJ H

O HN CO ■* Ift * N CO 9>

I O HtMW^lfUQtsOO»

I .oi« d43MM!r>tii-i6oa44p>a H coaicdo.cddd9o)uooi

a o s o

43 U

I" M 0Q fa

I OHNfOstirilONOSOl

Or-iCN|ro~a-iri\or»ooc^

OHNCI-JlOlOh-OOm

OHCMf1-»U110IN»0'

©r-l cv»c0N3-u~ivor^00 o\

c\ic»>sj-irivor»oo <j\

ICMOO-tfl/lvOrÔO <Ti

i<Mco>a-m^Dr^co o>

I I

icNco«*iri\orôo o\

\" ' ieMn«*ir>vorôo a\

I itNO'd-irivOi^co txi

////////

CO

Is I I —> CO OHN CIÛHO NM ff»

jj lEZZZZZZ^ o

•H >, M d 43 O

43 CO Ö d-H©'~,{N'(y,,<''ir>v<3'>"COON 0» S f» ■

d 0) • . «2 ftOr-icMcn-a-irivofôoaN

3|

o a

I-I

o CJ

00

o o

CO

c 43 Ü 01 H

z

I I

I OT-»cMc*i-tfu-ivop*cocy\

I O i-i N M <r i * r> ooo>

I I

I ©I-I CNfO>a-irivorôoo\

I Or-4 CNf>-*U1vO rÔOON

I © IH CNI cn •* m \J3 rôo o>

0) •o

to cd u

C 60

.3 %

£ 60 cj d 0J i-l

Z2

l r-i es en -tr m vo r* oocri i i i i i i i i i UUUHUHH fact]

I

01 > CO i-l 01 - 4J CJ d CJ <H

.2^ ß | o <4-i eu 52 •H o co P a

43 tn !* CJ 43 4J 0) 4J 3 H d Q

//////

/

I OHMCO< in »o r» CO <T\

I OH NfO-J mvoiôo»

Ü

OrlfMn -JinvONCOOl

I OHNfO-»IAvONCOCi

o.-icNfO'a-Lrivofôoo>

Or-icNc*ivi'in\j3i<eoo\

OHN n«*IOvONC0 01

QHN^'j'IfHONCOai

OHN ro >» io* r«. oo cr.

I OHNrO«*IOvOI>COO\

Or-ieNjfO<tinv43r>.00ON

I OHNn-tmiotscooi

I ©r-icNro-*mvor^co<j\

I OH cNinsj- mvorôoo»

I OHcNjm^firi«3rôocri

y-AAAAv\Annn?

OHNCKiOior>eoffi

OHNCn>» IflvO SOOCTl

o H Nn-Jinvc r»oo_o\ 1 T "-* OHN(»l<flO*NOOffi

I I

©r-ievio-5fiover>.ooefN 7—j— j

/ /

/

1—1

%

I I

OHN Wî^M]|N COON

I OHN m«* in \or> ooo\

I OHN fo«a- u*i vo r~» co g\

I OHNfo^s-mvor^cocr»

I I

I Jp. .rLtSLcTJ ^f in yp f» to o>

I E-H o HNCi^mioNcocfi

3 I O ft tN CO «-T IT) vo r-*oo ON

'/ 7 / / / / / /

APPENDIX D

INSTRUCTIONS FOR SUPERVISOR

NOTE: The instructions on pages D-3 thru D-15 (pages -] thru -13) of this report are paginated for publication. For economy of reproduction, the page numbers were not changed for this report.

D-l

THIS PAGE IS BLANK

D-2

NAVAL PERSONNEL RESEARCH AND DEVELOPMENT LABORATORY WASHINGTON, D.C.

Instructions for the Supervisor for Completing Performance Evaluation Forms

You, as a supervisor, have been selected to participate in a performance evaluation program being conducted by the Naval Personnel Research and Development Laboratory. Performance evaluation in the U.S. Navy has served as a guideline for the optimal positioning of manpower and for feedback on Naval school effectiveness. It is felt that you will be the most qualified to give accurate and meaningful information on the performance of the men you supervise. For that reason it is necessary that you make an earnest effort at providing the information requested of you. Without your support and honest efforts, the time and funds put into this research endeavor will have been negated.

You are asked to complete a set of three performance evaluation forms for each petty officer and designated striker - in the EM, ET, FT, IC, RD, RM, ST, and TM ratings only - over whom you have immediate supervision. The titles and descriptions of the three experimental forms are:

1. Personnel Identification Information Form (PIIF) - this form is concerned with the background data of the man you are evaluating. You will complete part of it and the Administrative Officer aboard your ship will supply the remaining information.

2. Technical Proficiency Checkout Form (TPCF) - this form records the level of technical complexity at which a man is able to perform without direct supervision.

3. Job Performance Questionnaire (JPQ) ANSWER SHEET - this form records your estimates of the number of a man's uncommonly effective and ineffective performances that you have observed during a specified time period.

From your ship's liaison officer, who is coordinating this project, you have received a packet of materials containing these instructions and a number of blank copies of the three forms just described. Ad- ditional blank forms may also be obtained from the liaison officer if you need them. Together, these forms - the PIIF, TPCF, and JPQ ANSWER SHEET - constitute a full set of Performance Evaluation Forms. You are to fill out one of each type of form - one complete set of three forms - for each man whose performance you evaluate. You will have approximately one week in which to complete the sets of Performance Evaluation Forms for all the men who have been designated for you to evaluate. Please remember to use only a #2 pencil in filling out the Performance Evaluation Forms,

Turn now to pages 4, 6, and 13 of these instructions. On those pages are samples of the three Performance Evaluation Forms. Notice that each of the three sample forms is preceded by a page of instructions. These particular sample forms were completed for the following typical, naval situation.

EMC Abner Smith is the immediate supervisor of seven men aboard the USS ROAN located at Newport. He had been given seven sets of Performance Evaluation Forms, with a Super- visor's Instruction Package, by the liaison officer aboard the USS ROAN. During the week of 7 January 1972, he completed all Performance Evaluation Forms. One of the men he evaluated was EM3 Tom Henry Jones. The samples on pages 4, 6, and 13 are therefore the PIIF, TPCF, and JPQ ANSWER SHEET containing information on EM3 Tom Henry Jones furnished by his supervisor, EMC Abner Smith.

Before starting on the Performance Evaluation Forms, study the samples that are included in these instructions. Please do not let your answers be influenced by the information given on the sample Performance Evaluation Forms. The answers given on the sample forms are meant only to be guides in aiding you to Complete your forms on the men you evaluate.

All responses to questions are completely CONFIDENTIAL. Any information you provide will be used only by the researchers of the Naval Personnel Research and Development Laboratory, and only to serve as a statistical data base for arriving at performance estimates of enlisted naval personnel in general.

Your help and cooperation in participating in this project are greatly appreciated by the project researchers at the Naval Personnel Research and Development Laboratory.

Instructions for Completing the PIIF

The Personnel Identification Information Form (PIIF) records various background information on the man you are evaluating. Refer to the sample PIIF on page 4 of these Instructions and the DIRECTIONS given there. Note that you need furnish only the information requested to the right of the cross-hatched area, namely:

a. Technician's Name b. c. d. e. f. g.

Social Security Number Birthdate Months Known by Supervisor Months on Current Job Assignment Months of Active Duty Service (USN* and USNR**) Rating and Paygrade

It is understood that you will obtain most of this information directly from the man you are evaluating. It is important that this background information be as accurate as is possible. If you doubt the accuracy of any of the information you have obtained from the technician, or if you are unable to obtain it, please ask the Administrative Officer aboard your ship to provide you with the correct information. In addition, you as the supervisor will also fill out the two upper left lines of the PIIF. This includes your name, rating and rate, date, ship and its current homeport location.

From the sample PIIF on page 4, notice that the following information is obtained:

1. The supervisor is EMC Abner Smith located at Newport aboard the USS ROAN and completed this form on 7 January 1972.

2. This supervisor was evaluating EM3 Tom Henry Jones whose social security account number is 123-45-6789, and he was born in March, 1948. He was known for 13 months by the supervisor, and had been 15 months on his current job assignment. Jones had served no USNR active duty time but had been on USN active duty for 29 months.

*Record 99 months for USN if the Technician's Months of USN Active Duty Service equals or exceeds 99 months.

**Record 99 months for USNR active time if the Technician's Months of USNR Active Duty Service equals or exceeds 99 months.

0) <u til 1 1

43 cd > U 01 « ■ cd 43 O T3 0IM-I 6043i-l i-i4«S i-l S C O fa CM CO 4J 3 > 3 X S^ N • OHMCl^ IOvOI~-00<7\

03 ■H U 0) ü

1 O 1 01 CO a 1-1 «J

01 ft: 25

U » j«S (0 4-1 . •- Cd43OT3 01lWÖ043i-l i-l-«! i-l S d O fa cr I-I U) 4-1 3 > » X >. N >~, OHNn-r invo rscoov At «3 J-l • cd )-■ X d) « 44

1 * s s -o cd o g oi •H OO Cd 0 I-l c 4J o co S 3 Cd43 OT3 0C4-I 0043i-l i-l4<i rH S d O fa D*V4 CO 4-1 3 > S X >. N 3 OHtsn-J insor^ooos <d «j j*, cd 3 cd d

1 o

1 so cd a a w 3» 01 01 « >-i i-l • O CO 01 CO

43 u U cd CO U 4J H i-l ■ W43 OT3 0»<4-l 6043T4 i-l4«! HQ C O a cru CO 44 3 > ^ X >< N OI-ICNPI-* invor^ooffv u oi m > cd S i-i "O

1 1—1

1 0» < M "H -O O id d 5» >> 4J a 01 -H 1-4 0) r-4 g

JA d 3 60 d 4J ca B Cd43 OT3 0)14-1 6043-H i-|4<! 1-4BÖO fa cr u CO 4-1 3 > 3 X >> N u Oi-I CMrt-3-irivOt^-OOOl d T4 CO 44 -H

1 o U ctj •H M ► 0) 43 CD e-i CO 1 co cd cu oi a) ass i-1 -H

Z HH B 43 3! 4J 43 W43 ÜT3 CUM-4 6043-1-1 i-i4«! i-l 0 d O o. ty u CO 4J 3 > S X >> N CO OH r-i n-* m*Noo 01 onu eg -H a) d 1 M H d !» o d -H d d

H W cd o M o M a) w U • • fl Hg p.J= — CO 43 UTI a) *4-J 60 43 i-l •i-l4<! r-4 0 c o fa cr u CO 44 3 > a x >> N •H ©i-tcNO-a-u-ivor^coas 05 i-l i-l •

o) 53 4-1 4J oi 43 o d d to 1 o

i-l 1 M C 0) 0) 43 W -rJ 0) « c Q O 4J 60 4J 4J H U 43 A! W43 0-Ö «)iw ö043T4 --)4«i .-H 0 d O fa cr H CO 4-1 3 > & X >. N 43 OHNrt-t mvoi^cooi

0) C to ft« (3

1 o 1 HH If co 3 01 cd fej 0) ■H P.J3 i-l i-i 43 «• • i-4 H

penc

con

to c 4-1 0) A! 43

co g d 25- oj in 3 cd x < x o d t-4 o MO 43 43

fe cd 43 U -O 0) «W 6043 T4 -nJ«: i-4 S d O

! td 43 O TJ 0) <4-l 6043 -H i->4*i iH 0 C O

fa cr u

fa cr I-I

CO 44 3

CO 4-1 3

> u X >v N

5> 3 X >> N

OHCM01-* irjvo NCOcr,

' / / / S 0)

0) NH *i t_> 43 0) 44

i •a co cd

Use a

# rectang

you wan

M i-l CO X M z ai 43 cd o at

o 1 ij 43 -H i-l 43 43 J U III u W co co 01 O H ö o T4 d d

i I r\ r* r\ ni

"-» C8 43 O "O 01

Cd 43 O 13 01

M-l 60 43 i-l i-)4<! i-l S C O

1 ftcrii

fa tri-i

7

CO 4-1 3

CO 4J 3

> 3 X >> N

> S X >s N

d 60 cd >,

T4 Cd

i-icNC>«a-irivor-~ooo> 1 1 1 1 1 1 1 1 1 UHUHMUUUH

" •* r*-4 ■H d

43 60 o a cu i-i H 4J

3

1 2HHUQ2HS H W fa M pa K cn P

^ /' / ' / / X /

*-\ 1 co 1

CO

43 4J 4J 44 c d do) oi I /

fa "■ is d « ~i

Cd 4J ™ i-i cd o\ U TJ i-l

OHNn>Jlrt*NCOOv

1 OH CMOO-a-irivO P» 00 CT>

O U 0 O r-4 CN

3 43 60 1 CO O Oi-I » •

ci •* m \o t^ oo o\

co -* m vo r~« oo o\ 01 >

CO 1-4 44

d o cd •<

/ / / / ■w ^ co

d d co 1 « O < |

Or-I CM cu u

■H 05 > 25

§ g^ d 4J I 43 M ■ O -H

' X / <■ 1 1 fa « ss

u OH MCO-JITHO NCOOl O)PQ ßfl n i4 ^CH Man H Cd0lcdp,cd333a>cj

> o •H I-l 1 •H M CO 1 *> o O 01 d >. o O <4-l 0) 3 z o M >-)fa 33 << S>Tl<WO z o 43 43 CO •H O CO o a H O -rl O i-l CN

a ££ i O 0) ■

f3 «* in so l"~ on CT> d 43 CO 0 43 01 4-1

Oi—i <Nc")<fu"ivor^cocn 3 O K \X \A \X >s

44 3 1

>J 1 \ d ftOH NM-*in*NMW H d Q OH(Mf1^lfl»0M»O\

o fa 25 M

2 o

« 3 CO ^

O 25 CO 1

\ / \ OHcvirO^fiOsONOOüv

A\, ̂ vy/n M

o (— «si cq CD u H co 3= >- tu Ö CQ

S>

H M ~ a

rf ^ § ^ LU QC

65 O

1— LU

CD o CO fa ! *= _J

GO as 1-4 CD 1 ,

w 33 Q_ fa CO. > OS" CNJ | t- S cc: S ts CD CO C_> LU

i «-> o_ CO ZD

a- CO ^ LU CO

S- fe c\ CQ CO <c ° i

LU f/x fxl Cd

3 -a! I = o rc 2 Q 1 — h— r—

Instructions for Completing the TPCF

A sample of a completed Technical Proficiency Checkout Form is shown on page 6, completed by EMC Abner Smith for EM3 Tom Henry Jones. Eight tasks are listed in descending order of difficulty. It is felt that an individual's overall proficiency is directly related to the highest level of task in the set which he can perform without direct supervision.

Beginning with task description NO. 1. (Capable of employing safety precautions ...), place an "X" in the "CHECKED OUT" box if you feel that the man you are evaluating is capable of performing the task on his own without direct supervision. If you feel he is not able to perform the task on his own, place an "X" in the "NOT CHECKED OUT" box. Complete this form by going through the eight tasks described. Be sure to provide the information at the top of the TPCF form, to include your name, rating and rate, full name of the man evaluated, ship and its current homeport location, and the date. Refer to the sample TPCF given on page 6 of these instructions. You will notice that EMC Abner Smith marked EM3 Tom Henry Jones, as being "CHECKED OUT" on tasks 1 through 6, but felt that he was not able to perform tasks 7 and 8 without direct supervision.

SAMPLE

TECHNICAL PROFICIENCY CHECKOUT FOR

NAME OF SUPERVISOR Abner Smith RATING/RATE EMC

Tom Henry Jones FULL NAME OF MAN EVALUATED

SHIP OR UNIT USS ROAN LOCATION Newport DATE 7 January 72

TASK DESCRIPTION

1. Capable of employing safety precautions on most of this unit's equipment with which his rating is concerned.

2. Capable of replacing most of this unit's equipment with which his rating is concerned.

3. Capable of removing most of this unit's equipment with which his rating is concerned.

4. Capable of following block diagrams for most of this unit's equipment with which his rating is concerned.

5. Capable of knowing relationship of equipment to other related equipment with which his rating is concerned.

6. Capable of calibrating most of this unit's equipment with which his rating is concerned.

7. Capable of trouble-shooting/isolated malfunction(s) in most of this unit's equipment with which his rating is concerned.

8. Capable of employing electronic principles involved in maintenance of most of this unit's equipment with which his rating is concerned.

NOT CHECKED CHECKED OUT OUT

m m m El

D

D

D

D D D

D

D

M

MAKE CERTAIN THERE IS AN "X" IN A BOX OPPOSITE EACH TASK DESCRIPTION

Instructions for Completing the JPQ ANSWER SHEET

The purpose of the Job Performance Questionnaire (JPQ) ANSWER SHEET is to record, for a given individual, your estimates of the total number of uncommonly effective and uncommonly ineffective incidents of performance you have observed during the last two months on each of eight job activities (see page 8).

In previous efforts at evaluating the performance of an individual, such as the reports of Enlisted Performance Evaluation, you mentally "aver- aged" your observations of excellent and poor incidents of performance for that man in order to arrive at an overall performance estimate. However, in this project we are asking you to focus your attention on only the extremes of performance — uncommonly effective and uncommonly ineffective performances. In addition, we want you to disregard an individual's overall performance and to record for each of the eight job activities the total number of times you have observed the occurrence of uncommonly effective and uncommonly ineffective incidents of performance.

As a general example, on a particular day a man has demonstrated two incidents of uncommonly effective performance while involved in the job activity Equipment Operation and no incident of uncommonly ineffective behavior for that job activity on that day. Sometime later, on perhaps another day, he has demonstrated one uncommonly effective performance and one uncommonly ineffective performance for the job activity Equipment Operation. If in the past two months no other of his duties involved Equipment Operation, then your estimate of the total number of uncommonly effective and uncommonly ineffective incidents of performance, for that job activity, is three uncommonly effective and one uncommonly ineffective incident of performance.

In order to determine exactly what specific types of activities you should consider in estimating whether the man has shown an "uncommonly effective" or an "uncommonly ineffective" performance, refer to the DESCRIPTION of that JOB ACTIVITY as given in the following list:

DESCRIPTION OF JOB ACTIVITIES

JOB ACTIVITY DESCRIPTION

1. Using Reference Materials—includes the following type of activities:

a. use of supporting reference materials b. making out reports

2. Equipment Operation—includes the following type of activity:

a. operating equipment, electrical and electronics test equipment, and other electronic equipments

3. Electronic Circuit Analysis—includes the following type of activities

a. understanding the principles of electronic circuitry

b. making out failure reports c. keeping records of maintenance

usage data

4. Personnel Relationships—includes the following type of activity:

a. supervising the operation, inspection, and maintenance of electronic equipments

5. Electro-safety—includes the following type of activity:

a. using safety precautions on self and equipment

6. Instruction-—includes the following type of activity:

a. teaching others how to inspect, operate, and maintain electronic equipments

7. Electro-repair—includes the following type of activity:

a. equipment, repair in the shop

8. Electro-cognition—includes the following type of activities:

a. maintenance and troubleshooting of electronic equipments

b. use of electronic maintenance reference materials

8

Because you are the most knowledgeable of the area you supervise, you will know what incidents of performance can be labeled as uncommonly effective or uncommonly ineffective. It is your standard that is to be used for this trial. However, to further assist you the researchers have provided the following general definitions.

An uncommonly effective performance in a specific job activity is an impressive and/or decisive incident of performance qualitatively above those usually observed. Likewise, an uncommonly ineffective performance in a specific job activity is an impressive and/or decisive incident of performance qualitatively below those usually observed.

With this in mind, turn to the sample of a JPQ ANSWER SHEET on page 13, Under each JOB ACTIVITY are three columns - (a), (b), and (c). Notice that under "Using Reference Materials", column (a), the estimate of the total number of uncommonly effective (UE) performances observed during the past two months for EM3 Tom Henry Jones was two. That is, the supervisor, EMC Abner Smith, estimated that altogether he-observed two impressive incidents of performance for EM3 Tom Henry Jones which were qualitatively above those usually observed. Similarly for column (b), the supervisor's estimate of the total number of uncommonly ineffective (UI) performance is one. Dis- regard part (c) for the moment.

The estimates recorded on the JPQ ANSWER SHEET will be machine processed and must be accurately recorded. The sample on page 13 together with the following table, illustrates the correct procedure for recording one-digit numbers (numbers 0 through 9), and for recording two-digit numbers (numbers 10 through 99), in various positions on the JPQ ANSWER SHEET. Review the table in conjunction with the page 13 sample.

JOB ACTIVITY

Using Reference Materials Equipment Operation Electronic Circuit Analysis Personnel Relationships Electro-safety Instruction Electro-repair Electro-cognition

NOTE: If the man has not had an opportunity to perform a particular activity, leave that job activity unmarked (as shown for "Instruction" on the sample JPQ ANSWER SHEET).

If he has had an opportunity to perform a particular activity, but has not shown any uncommonly effective or ineffective performances, enter a zero (0) for both UE and UI (as shown for "Personnel Relationships" on the sample JPQ ANSWER SHEET).

No. of

2 10 14

0 0

2 1

UE No. of UI

1 5

12 0 2

1 1

While observing a technician perform any of the eight JOB ACTIVITIES, you, as his supervisor, may have had many objectives in mind which you felt he should be striving for. However, for the purposes of this trial, we ask you to limit your attention to only the following fleet electronic maintenance objectives:

MEANINGS OF FLEET ELECTRONICS MAINTENANCE OBJECTIVES

1. Readiness

To maintain efficiently self, subordinate personnel, equipment, and systems in state of readiness consistent with fleet requirements.

2. Performance

To complete any given mission in minimum time with appropriate level of accuracy and reliability.

3. Operation

To obtain optimum system output when equipment is operated, i.e., output characterized by precision and variability appropriate to mission.

4. Safety

To carry out duties with maximum protection for men and equipment consistent with mission.

5. Preparation

To prepare for personnel requirements of present and future equipment, systems, and situations through use of training programs, maintenance of high morale, etc.

After recording the estimates of the number of UE and UI performances for each of the eight JOB ACTIVITIES, complete the JPQ ANSWER SHEET by placing, in column (c) for each job activity, your reply to the following question:

10

QUESTION (c) Considering this man's overall performance, it is your opinion that the importance of this job activity, as a factor in determining the overall performance of this man, is best described as being:

3. of central and primary importance 2. a significant factor, but of secondary importance 1. of only moderate importance in estimating overall

performance 0. of little or no importance

On the sample JPQ ANSWER SHEET on page 13, QUESTION (c) was answered as 2, 3, 3, 2, 1,0, 2, 3, respectively for each of the eight JOB ACTIVITIES. For this particular man, his supervisor felt that the job activity, "Equipment Operation," was of primary importance as a factor in determining the man's overall job performance.

Answer the following questions in the block marked QUESTIONS in the lower corner provided on the JPQ ANSWER SHEET.

QUESTION (d) You were asked to recall the number of UE and UI performances that you have observed for this man during the past two months. You feel that a reasonable time span for evaluation

1. two months 2. six weeks 3. four weeks 4. two weeks 5. one week

QUESTION (e) What degree of confidence have you that your estimated number of UE and UI performances for this man are very close to the actual number of such performances that occurred during this time period?

1. My estimates are probably very close to the actual numbers.

2. There may have been a few UE or UI performances more or less than my estimates.

3. Cannot be too sure about my estimates. It was very difficult to recall UE and UI performances.

4. The actual number of UE and UI performances could be very different from my estimates. Recalling these UE and UI incidents is too difficult to have any confidence in such estimates.

11

QUESTION (f) Aside from his performance, to what extent are this man's efforts on the job devoted to tasks and activities directly related to his rating? Compare this amount of rating-related activity to the average for men in his rating and paygrade.

1. Involved in definitely more rating-related activities than is usual in his rating and paygrade.

2. Involved in about the same rating-related activities as usual in his rating and paygrade.

3. Involved in less rating-related activities than is usual in his rating and paygrade.

The above questions were answered as 2, 1, and 3 respectively on the sample of the JPQ ANSWER SHEET of page 13 of these Instructions.

Finally, place your social security account number in the block provided. Also be sure to record your name, rating and rate, ship and its current homeport location, date, and the name of the man you are evaluating at the top of the JPQ ANSWER SHEET.

12

GP 0 923.819

13

THIS PAGE IS BLANK

D-16

APPENDIX E

PROBLEMS IN CALCULATING RELIABILITY RATIOS

E-l

THIS PAGE IS BLANK

E-2

PROBLEMS IN CALCULATING RELIABILITY RATIOS

The purpose of this section is to examine the frequency with which the two cases:

1. a technician did not work at a job activity, and

2. a technician received SUE = 0 and SUI = 0,

occurred for each rating and job activity across all 21 ships participating in the project. From this one can infer on the extent which any convention for estimating performance in those cases would affect individual SRE, PRE, and GRE values.

Refer to Table E-l on page E-4. Each square in the table represents the number and proportion of technicians by rating who did not work at a particular job activity or received SUE = 0 and EUI = 0 by their supervisor. Therefore, on job activity Number 1, 25 of the EM's (or 25.8% of the EM's) evaluated either do not work at that job activity (Using Refer- ence Materials) or received SUE = 0 and ZUI = 0. This may seem a toler- able level of occurrence of such cases, but when the proportion of such cases exceeds .33, one should begin to consider whether the performance of some individuals is due more to the convention that must be adopted rather than to the individual's own job effectiveness. Of the 64 squares in the table, 42 squares had one-third or more of the men in some rating falling into the two cases for some job activity, 25 squares had one-half or more of the men in some rating falling into the two cases, and most critically, 6 squares had at least 75% of the men in those cases. The RD and RM ratings were particularly notorious for this type of situation occurring. No rating seems to be entirely free of this situation for some job activities. However, some ratings significantly demonstrate this effect for many job activities.

E-3

TABLE E-l

NUMBER AND PROPORTION OF TECHNICIANS IN PROBLEM AREAS

Job Activity

1

2

3

EM ET FT Rating

IC RD RM

Number of Men Each Rating

97 173 154

ST TM 25

0,258

38

0,220

96

0.J64

9

0,155

50

0,360

53

0.387

45

0,296

6

0,154

23

0,237

47

0,272

49

0.318

7

0,121

53

0,381

42

0.307

45

0,296

13

0,333

42

0,433

45

0.260

75

0,487

18

0,310

133

0,957

104

0.759

69

0,454

26

0,667

43

0,443

63

0.480

93

0,604

27

0,466

75

0,540

77

0,562

76

0,500

10

0,256

83

0,237

83

0,460

60

0,519

7

0,121

76

0,547

90

0,365

62

0,408

11

0,282

90

0,515

116

0,671

115

0,747

39

0,603

100

0.719

68

0.642

90

0,592

20

0,513

18

'0,186

48

0.277

83

0,539

9

0,086

137

0.993

128

0.934

76

0,900

25

0,641

35

0,361

48

0.277

64

0,416

12

0,207

132

0,950 ■

114

0.832

61

0,401

22

0,564

58 139 137 152 39

E-4

APPENDIX F

THE ADOPTION OF A CONVENTION FOR EACH JOB ACTIVITY AND RATING

F-l

THIS PAGE IS BLANK

F-2

THE ADOPTION OF A CONVENTION FOR EACH JOB ACTIVITY AND RATING

This section discusses the finding that no convention can be adopted per ship that will account for those cases in. which a technician either does not work at a particular job activity or received ZUE = 0 and EUI = 0 from his supervisor. As an example, observe Table F-l. This table is of the same type as that previously reported on for all 949 men participating in the project (Appendix E, Table E-l). However, Table F-l is reporting on only one typical ship out of 21 ships in the project. For this ship there were 8 (out of 64) instances where one of those two cases occurred for all men in some job activity and rating. The other ten ships at Location No. 1 demonstrated 16, 7, 8, 10, 13, 12, 5, 5, 5, and 16 (out of 64) instances. The 10 ships at Location No. 2 demonstrated 1, 16, 5, 15, 7, 8, 16, 14, 10, and 15 (out of 64) such instances. Therefore, it is impossible to form an average estimate (or some composite value) per ship for each rating and job activity because, in some ratings and job activities on all ships, there are no technicians who received either EUE or EUI different from zero.

Derivation of Composite Reliability Values

In order to overcome the aforementioned problem, the convention employed in the report was to develop a composite reliability score to estimate technician performance on those job activities in which the technician received EUE = £UI = 0 or did not work at that particular job activity.

Let ZUE(i,j) be the sum across all ships at a location of all EUE's over all men in the itn rating and jth job activity. Similarily the sum of all EUI's is calculated; denote this sum by ZUI(i,j). The composite reliability score for the i™ rating and j™ j0D activity is defined as:

R(i,j) = £UE(i,j)/[EUE(i,j) +ZUI(r,j)].

This particular estimate of job performance provides an "expected" level of effectiveness for a technician in the itn rating and jth job activity (for ships at a particular location). Table F-2 gives the resulting composite reliability values (R(i,j)) for each rating and job activity at each location. For example, from Table F-2 and men at Location No. 1 (Cruiser-Destroyer Flottilla NINE), R(l,3) is the composite reliability value for EM's on job activity number 3 - Electronic Circuit Analysis - and is given by R(l,3) = .8465. Reiterating, it may be said that for all EM's at Location No. 1 who have not worked at job activity number 3 or who received EUE = 0 and ZUI = 0 for that job activity, their reliability ratio for that job activity is expected to be r$ = R(1,3) = .8465. Simi- larly this procedure is employed on the other ratings and job activities.

F-3

The composite reliability score is an estimate that always can be derived when many ships are involved. However, it does not overcome the implications of the results discussed in Appendix E and their subsequent effect on the estimates SRE, PRE, and GRE.

F-4

TABLE F-l

NUMBER AND PROPORTION OF TECHNICIANS IN PROBLEM AREAS ON A PARTICULAR SHIP

Job Act. EM ET FT

Rating

IC RD RM


6 29 12

ST TM

4 9 5 0 0 9 0 0

0.667 0.310 0.417 0.0 0.0 0.900 0.0 0.0

6 13 7 0 0 9 0 1

! . ftii'l o .4« a 0.583 0.0 0.0 0.900 0.0 0.077

5 14 7 1 0 10 0 9

0.833 0.483 0.S83 0.167 0.0 1 .000 0.0 0.692

6 18 10 3 o 9 0 5

,.-,„ 0.621 0.833 0.500 0.0 0.9G0 0.0 0.385

6 21 8 0 0 8 0 1

1 . r,'., r, 0.724 II .667 0.0 0.0 0.600 0.0 0.077

5 ?1 9 o 0 10 0 5

0.833 0.724 0.750 0.0 0.0 1.000 0.0 0.3:35

5 11 12 1 0 9 0 9

0.833 0.379 l.ooo 0.167 0.0 • 0 . ') 0 0 0.0 0.692

6 16 8 1 0 10 0 6

1.000 0.552 0.667 0.167 0.0 1.000 0.0 0.46?

10 13

Fr5

TABLE F-2

COMPOSITE RELIABILITY VALUES

Location No. 1

Job ;tivi

l ty EM

0.8770 ET

0.6831 FT

0.71*50

Rating IC RD.

0.7466 0.9257

RM 0.9310

ST 0.8537

TM 0.7333

2 0.9050 0.7733 0.7802 0.8217 0.9110 0.9497 0.8899 0.8165

3 0.8U65 0.6932 0.73i»0 0.7981 o.nooo 1.0000 0.8662 0.8667

l» 0.8639 0.5987 0.7890 0.7692 0.9300 0.9671 0.8930 0.7733

5 0.9001» 0.7706 0.9107 0.7865 0.9012 0.9712 0.9161 0.7281»

6 0.9333 0.8481 0.8I»35 0.8039 0.9677 0.9877 0.8571 0.8971»

7 0.8981 0.7872 0.8701 0.8163 1.0000 1.0000 0.9078 0.8269

8 0.87UI» 0.7097 0.7771 0.8105 1.0000 0.9949 0.8571 0.7I»19

Location No. 2

Job Acti vi ty EM ET FT

Rating IC RD RM ST TM

l 0.9101 0.7203 0.8552 0.7879 0.8352 0.8351 0.851*7 0.5806

2 0.8962 0.7110 0.8808 0.8673 0.9017 0.8673 0.8567 0.8095

3 0.83i»2 0.7803 0.8673 0.7719 1.0000 0.R402 0.8191» 0.0909

1» 0.9586 0.6729 0.8167 0.9333 0.8211» 0.821*0 0.7950 0.7317

5 0.8987 0.82i»l» 0.9032 0.9718 0.8649 0.9667 0.9135 0.6000

6 0.9530 0.7368 0.9231 0.8095 0.7975 1.0000 0.6759 0.8333

7 0.9441 0.7866 0.8819 0.8240 0.0000 1.0000 0.8148 0.5714

8 0.9120 0.7918 0.8827 0.8267 0.7059 1.0000 0.8357 0.4167

F-6

APPENDIX 6

A FEW REMARKS ON CURVILINEAR REGRESSION

G-l

THIS PAGE IS BLANK

G-2

A FEW REMARKS ON CURVILINEAR REGRESSION

Let Y be a criterion variable and let X be a predictor variable. If N is the number of observations on each of X and Y, then Y' » [Y., ..., Yy] is the row vector of N observations on Y. For a given matrix A, A* will be the transpose of A. In particular one wishes to establish a regression equation for a particular response Y in terms of the variables X, X , X ; i.e. it is desirable to establish which of the three power curves (linear, quadratic, or cubic):

P i Y = 3n+ H ß, X1 p = 1, 2, or 3

1*1 1 "~

best fits the observations obtained on X and Y. The above equations, in terms of the sample observation vectors, can be expressed in matrix nota- tion as:

I = Xß + E

where X was defined above. The.matrix X. - [J, X., x9, xA where J' = [1, .... l]lxN and X« = [X], ..., XjJ,] for i ± 1/2, 6V 3. There-

fore ^ is an N x 1 column vector of observations on the predictor variable. ß' = [B0, .... ß_] is the vector of p + 1 regression parameters. £"s[e,,...teN] is the vector orerrors due to lack of fit in the particular model. One wishes to estimate $ such that the error sum of squares is minimized. In particular a least squares estimate ß*of ß is given by

£ = {X'X) X'Y

provided the square matrix X'X is nonsingular and the regression problem has been properly expressed. The usual assumption one makes is that £' is distributed with mean [0, ..., 0], N and variance-covariance matrix aM

where I is the identity matrix. The term a^ is called the common error variance of the observations. The assumption of normality of the error vector is not required in order to obtain the least squares estimates for any of the parameters in the regression equation. Because any assumption of normality for E implies that the observations on X or Y are normally distributed, this report will only be concerned with least squares estimates. One cannot in general discuss normality on X or Y because of some results in this report where it is shown that the distribution of TP scores, SRE, PRE, and GRE are not normally distributed (see Table 17 and page 21). Therefore it is imperative that the reader be aware that while the assumption of E being normally distributed is not required in order to obtain IT» it is required in order to make tests of hypotheses, as contained in an Analysis of Variance Table. These tests are the usual t- or F-tests and they cannot be applied validly to the sample data collected at either location nor on the combined sample consisting of 949 technicians for the variables SRE, PRE, GRE, and TP score (see, for example, Draper and Smith [7], page 59).

G-3

A Least Squares Analysis of the Sample Data

It is possible that a least squares analysis can be attempted in- dependently of the distributional properties of the criterion and predictor variables. For a particular predictor variable (X) and criterion variable (Y), the multiple correlation coefficient (R2) provides a measure of the proportion of the total variance about the sample mean for the criterion variable explained by a particular regression model. The term R2 is defined as:

Sum of squares due to regression - Sum of squares due to 3 R2 = —

Total (corrected) sum of squares

= (SVJ-IJI) / Wi-^r) It should be clear that the larger R2 is, the better the fitted equation explains variance in the criterion variable. Furthermore, 0 S R2 < 1, and therefore R - 1 implies a perfect fit. However there are a few problems with this approach (see, for example Draper and Smith [7], page 63). One must weigh the value of R2 with the least squares estimate (s2) of the common error variance (a2) where

s2 = residual mean square

= U'i-^p) / (N - p - 1).

Of course, the smaller s2 is for a particular model under consideration the better the model fits the data. Therefore the approach is to weigh increases in R2 with decreases in sz in order to arrive at the best least squares model for the data.

G-4

APPENDIX H

RESULTS OF THE CURVILINEAR REGRESSION ON 949 TECHNICIANS EVALUATED

H-l

THIS PAGE IS BLANK

H-2

0.323 0.288 0.265 2,310

0.993 0,887 0,055 1.000 0,983 0,032 0.983 1,000 0,034 0,032 0,034 1,000 8S3SS3S ::II::::;I>3: ;:>:nii3:>>3a3:>i:iii31

3:a::33a31)TJlU]3311;3|l3::33ll9a3II3S33llS133i:::;i;4|i:333G8t3333gi13i:l HOLVNDMIAL FITTING FOR 2 VAH! AOLES-SRfc AND TP SCORE. . Y<1>« OUSERVATJ ?NS,

THE PREnlClflR VAR1 AF)LE(X) IS SRE, A*IU THK Cnl?EB|9.N VARI Aein'm: IS TP'5C9RET.

TEST HEANS AND STANDARD DEVIATIONS 1 X 0.331 2 SQUAW 0.214 3 X CU6E 0.159 4 t 5.35(3

CORRELATlGN MATRIX 1 X 1,000 2 SOUAR 0.953 3 X CUBE 0,887 4 Y 0.05S

MUST. SfcCENC, AND THIRD DEGREE P0LYN6MI ACS, t:::::::::::::!-::s:::s:::::t:3:3:::::Ji3:«i:i:;;:;;t:3ji3:::!r«333i::i333

MULTIPLE R SCUARE (J 0,003 MULTIPLE R * 0.055 N.D.F.l a 1 N.D.F.2 a 947 f FOR ANALYSIS 0F VARIANCE 8N R 9 23850 UETA WEIGHTS

0.055 CONTRIBUTIONS T8 MULTIPLE C9RRELAT18N

AND REGRESSION FACT9R LOADINGS, 2ND COLUMN) 1 X 0.003 1-000

SQUARED BETA WEIGHTS 0.003

B WEIGHTS 0.397

INTERCEPT CONSTANT « 9.219 .... ::::::::3:3:::3:::3:::«3::::s:3:3::3::3S3:33i:::3:;:3:;i::33:>3iaisi:::iaa MULTIPLE R SQUARE ■ 0,007 MULTIPLE R * 0.086 N.Ü.F.1 a 2 N.D.F.2 a 946 f FOR ANALYSIS 0F VARIANCE SN R 1 3,537 BETA WEIGHTS

0.264 -0,220 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTSR LOADINGS, 2ND C9LUMN) 1 X 0.01* (J.636 2 SQLAR »0.007 0.372

SQUARED BETA WEIGHTS 0.070 0.048

B WEIGHTS 1.912 -1.783

INTERCEPT C0NSTANT a 5,098 .....

:3::s::3:::3::::::33::i33:::i:3:3C33::3*3333333333;::3l3:r:3:33t3>9i:t:ai> MULTIPLE R SQUARE ? 0,043" MULTIPLE R = 0.207 N.D.F.l a 3 N.D.F.2 a 945 F FOR ANALYSIS OF VARIANCE 8N R.a .14,091 ...._...._ . . . BETA WEIGHTS

1,492 -3,627 2,276 CONTRIBUTIONS TO MULTIPLE CSRRELATtflM

AND REGRESSION FACTOR LOADINGS, 2ND C8LUNN) 1 X g.082 n.265 2 SQL'AR »0,116 5.155 3 X CUBE 0,077 0.164

SQUARED BETA WEIGHTS 2.226 13.157 9,179

B WEIGHTS 10.803 .29.497 20,101' INTERCEPT CONSTANT » 4.869

ANOVA rABLE FOR P0LYN6MULS :33:::3:a;Be533:3333;:|3S33R3=l3aS3B:33ll3S33313:3;ii:S13B:S3B3l&33l33ta33 REDUCTION OLE TO LINEAR FIT, WITH 1 OF i 0,003 RESIDUAL S.S. a Q.V97 DF ■ 947 RESIDUAL M.S. • 0,001

F FOR LINEAR FIT ■ 2,890 3::3:s3:l:333:3:33l3:3BB>3:3icB:BS333l>3IS<<lti3:i]33Bti:BiB3B3iaa3iB3SBBB

REDUCTION DLE TO GENERAL QUADRATIC FIT WITH OF 2, ■ 0.007 REDUCTION M.S. a 0,004 RESIDUAL S.S, a Q.993 DF ■ 946 RESIDUAL M.S. a 0,001

F FOR QUACHATIC FIT « 3,937 REDUCTION CUE TO QUADRATIC TERM ALONE, WITH 1 DF, | 0,004 F FOR QUADRATIC TERM ALONE • 4,2i5 3:33S333B33:l3BBa3Bas:|3B8BSBSflBBBS3333BI3BSSe3333)B,3IB3B33333|3BlflBBBBBI

REDUCTION DUE TO GENERAL CUBIC FIT WITH OF 3, * 01043 REDUCTION M.S. a 0,014 RESIDUAL S.S, « 0,957 DF » 949 RE3IDUAL H.3, » 0,001

F FOR GENERAL CUBIC FIT a 14,091 REDUCTION. CUE TO CU81C TERM AL9NE HITH 1 DF • , 0.035 H-3 F FOR CUBIC TERM ALONE a 34,913 DC3<a>>iai>mil>>3)BlIM3>ai3333B3IB:3B,t33BB)333SllIIM3>!"IH3BS,B|BBS3

POLYNOMIAL F t T T ( NG F0R 'i VARIA3LES-PRE AND TP SCORE» 949 flß-SEHVAt HNS,

THE PdEDtC T0R VARIABLE^) IS PRE, AND THE CRITERION VARl ABLE (Y) IS TP SCORE!

TEST MEANS AND STANDARD OEVIATHNS 1 X 0.558 0,407 2 SQUAR 0.477 0.388 3 X CU6E 0.420 0.37/ 4 Y 5,350 2,340

C0HRELATUN MATRIX 1 X 1,000 0.976 0,934 0,100 2 SQUAR 0,97« 1,000 0,989 0,063 3 X CUBE 0.934 0,989 1,000 0,036 4 Y 0.100 0.063 0,036 1,000

:::3s:::::::3:a:j:i3::i33::i3:i:]:3g::s:33S:i::s3:;:::ti::i3:i:i9t9t:33iia

FIRST, SECONC, AND THIRD DEGREE POLYNOMIALS, 353:::3:333S3:8:3S83:3l333:3a:B3B8 3«S383:3l333«3S3338E9lS933S38l33IB39883a MULTIPLE R SCUARE « 0,010 MULTIPLE R * 0.100 N.D.F .1 " 1 N.D.F.2 a 947 f F9R ANALYSIS OF VARIANCE ON R n 9,976 BETA WEIGHTS

0.100 CONTRIBUTES TO HULTIPLE C0RRELATI8N

AND REGRESSION FACTOR LOADINGS', 2ND COLUMN)

1 X 0.010 1.000 SQUARED BETA HEIGHTS

0.010 B WEIGHTS

0.576 INTERCEPT CONSTANT • 5,028 i:3:33::i:a333333 388::a33:3:t3B:i:3l3333>83a3;8333;38:]a3S:3333l33>g:]3333 MULTIPLE R SQUARE » 0,036" MULTIPLE R a 0.189 N.D.F.I » 2 N.D.F,2 s 946 F FOR ANALYSIS OF VARIANCE ON R « 17,908 BETA WEIGHTS 0.817 -0,734

CONTRIBUTIONS TO MULTIPLE CORRELATION AND REGRESSION FACTOR LOADINGS. 2ND COLUMN)

1 X 0-082 0.530 2 SQLAR tO.046 0.332

SQUARED SETA WEIGHTS 0.667 0.539

B WEIGHTS 4,700 -4.431

INTERCEPT CONSTANT a 4,838 ...... 833:::::33 33t33S33tlS3|3t3:3II|3333a333;ttB9383338j:lt33333333l|8SI8:3>3 8l

MULTIPLE R SQUARE ? 0,036' MULTIPLE R a 0,189 N.D.F.I s 3 N.D.F.2 a 945 F FBR ANALYSIS 0F VARIANCE »N R » 11,669 BETA WEIGHS

0.754 -0,579 -0,099 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTOR LOADINGS', 2ND COLUMN) 1 X 0.075 n.529 2 SQLAR »0,036 0,332 3 X CUBE «0,003 0.192

SQUARED 8EU WEIGHTS 0.569 0.335 0.009

B WEIGHTS 4.341 -3.495 -0,590

INTERCEPT CONSTANT ■ 4,841

ANOVA TABLE FOR POLYNOMIALS :33333338:33a3a333S8:a3asa838383BB8a33SaB38BBa8S88g383SB3aSE3838BsasS3388B REDUCTION DUE TO LINEAR FIT, WITH 1 Of ' 0,010

RESIDUAL S.S, • 0'"0 ^ " 947 "ESIDUAL M.S. » OiOOl V FOR LINEAR FIT » 9,576 >3 3::B:sS:3 33 38883B3::i383:33:l3SB3i:3i8B33glictt8t38<|B8t:38lBBlBl838BBB8

REDUCTION DUE TO GENERAL QUADRATIC FIT WITH OF 2, ■ 0.036 «EDUCTION M.S. a 0.018 RESIDUAL S.S, ■ 0.964 DF » 946 RESIDUAL H.9, * 0,001

F FOR QUADRATIC FIT « 17.508 REDUCTION CUE TO QUADRATIC TERN ALONE, WITH 1 DF, 5 0,02« F FOR QUADRATIC TERM ALONE « 25,1.95 :3:3:3:ilaa3::8E3333c:B33::8B8B3BI3l83l333IS8]B8t3)aSS|a3a:BI8BBlBBB8a8aaS

REDUCTION DUE TO GENERAL CUBIC FIT WITH DP 3, ■ 07036 REDUCTION M.S. » 0,012 RESIDUAL S.S, • 0,964 OF « 945 RESIDUAL M.S, ■ 0,001

F FOR GENERAL CUBIC FIT » 11,669 H-4 REDUCTION CUE TO CUBIC TERM ALONE WITH 1 DF « , 0,000 > FflH CUBIC TERM ALONE » 0,016 ■i3:ii:3iii:i3ii3iq3,:|]D:iiiil33iia3>ii>]liM,i33|]BCi,i<iiiai,)aaa3illil

0.145 0.477 0.210 2.340

0.956 0,693 1.000 0,986 0.986 1,000 0 .OBI 0,077 I3a::>: ::mis>:i

POLYNOMIAL FITTING FOR 2 VARI A8LES-GRE AND Tf> SClMlE-. 94'J OUSERVATIPNS,

IHE PREDIC'OR VARIABLE(X) IS ORE, AND THE CRITERION VARIABLES) |S TP SC0R?;

TEST MEANS AND STANDARD DEVIATIONS 1 X 0.927 2 SQUAR 0.679 3 X CUBE 0.839 4 Y 5.390

CORRELATE* MATRIX 1 X 1,000 0.956 0,695 0,067 2 SQUAR 0,956 1.000 0,966 O.Ofll 3 X CUEE 0.895 0.986 1,000 0.077 4 Y 0.067 0.0B1 0,077 1,000

c:QS:s::::3::5as3:3:::i::::::3i3as:ss?:gS3Sas:::s:;B:::i:3:3:3attsii:i393t

FIRST, SECENC, AND THIRD DEGREE POLYNOMIALS, ::I:::::::::::::;:S:"3:::::E:3:333]::3:3::I:::3;SJ::::I:::::3:I>I33:II3:3

MULTIPLE P SCUARE a 0,008 MULTIPLE R = 0.087 N.D.F.l « 1 N.D.F.2 » 947 F FOR ANALYSIS BF VARIANCE 6N R < 7,161 BETA WEIGHTS

0.087 CONTRIBUTIGNS TO MULTIPLE CORRELATION

AND REGRESSION FACTOR LOADINGS', 2ND COLUMN) 1 X O.OQ8 1.000


B WEIGHTS 1.400

INTERCEPT CONSTANT » 4,033 c:;:::::::3:::3r::3:::3:::::3r3t8c:i::3:£333333:::i:::J8:::i:3s»J333:3:3E3 MULTJPLE R SQUARE « 0,008' MULTIPLE R « 0.087 N.D.F.l ? 2 N.0.F.2 = 946 F FOR ANALYSIS OF VARIANCE ON R 1 3,989 BETA WEIGHTS

0.103 .0,017 CONTRIBUTES TO MULTIPLE CORRELATION

AND REGRESSION FACTOR LOADINGS', 2ND COLUMN) 1 X 0.00' 0.998 2 SOLAR »0,001 0.937

SQUARED SETA WEIGHTS 0.011 0,000

B WEIGHTS 1.663 -0,225

INTERCEPT CONSTANT o 4,006

MULTIPLE R SCUARE f 0,009" MULTIPLE R si 0.094 N.D.F.l ? -3 N.D.F.2 « 949 V FOR ANALYSIS.OF VARIANCE 6N. R ■ 2,79» BETA WEIGHTS

0,661 '1,942 1,009 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTOR LOADINGS, 2ND COLUMN) 1 X 0.037 0.923 2 SOLAR «0.125 0.867 3 X CUBE 0.077 0.816

SQUARED BETA WEIGHTS 0.438 2,378 I.010

B WEIGHTS 10.690 '20,332 11,213 INTERCEPT CONSTANT • 3,9*4

ANOV.A TABLE FOR POLYNOMIALS ...... E:33S3S333333333l33833l3SS39B:illC33B53at3S38S8a33g8I3l|3l3I3tlllllB8ial3a REDUCTION DUE T0 LINEAR FIT, WITH 1 DF 8 0,008 RESIDUAL S.S, » (1.992 DF » 947 RESIDUAL M.S. » 0«001

F FOR LINEAR FIT » 7,161 a3833:3B333S33a3S338S:a3B3S38:i3BB38S33aa33aia>3.asi333ia3a33:8a3BBaa3B>>3a

REDUCTION DUE TO GENERAL QUADRATIC FIT WITH DF 2, ■ 0.008 REDUCTION M.S. a 0,004 RESIDUAL S.S, a 0,992 DF a 946 RESIDUAL M.S, * 0,001

F FOR QUADRATIC FIT « 3,589 REDUCTION CUE TO QUADRATIC TERM ALONE, WITH I DF, « 0,000 F FOR QUADRATIC TERM ALONE • 0.024 3 33:33 33 8:3 3338:8888 33 33 83338333838338333388383883^83 313:8:338888888838838

REDUCTION DUE TO GENERAL CUBIC FIT WITH DF 3, a 0^009 REDUCTION M.S, a 0,003 RESIDUAL S.S, a 0.991 DF s 945 RESIDUAL M.S, a 0,001

F FOR GENERAL CUBIC FIT • 2,798 H-5 REDUCTION DUE TO CUBIC TERM AL8NB M|TH 1 OF » , 0.001 F FOR CUBIC TERM AL0NE a 1,215 833033338333883•a3f8B=88883383B388388>:BBSBa83883B|33S38S*3aS8aa338s3«3*S3

0.204 0.231 0.227 2.340

0.970 1.000 0.9S6 0.247

0,920 0,9fi6 1,000 0,229

:iss::3::: :3::>i:a

[:::i:Jt::«:j:i;::5i::ul-.--l :iiiE:i::H3:<ti::i s j t r:i.it3eii]iiiii9gtita

POLYNOMIAL FITTING FOR 2 VAHIABLES-HRE AND TP SC9RB.I 949 01SFRVA t J9N9,

THE PREDICTOR VARIABLE(X) IS WRE , AND THE CRITERION VARIAGLBtY) 15 TP SC@R&;

TEST MEANS AND STANDARD DEVIATIONS 1 X 0,603 2 SQUAR 0,409 3 X CUBE 0,290 4 Y 5.350

CORRELATUN MATRIX 1 X 1.000 0.970 0,920 0.297 2 SQUAR 0.970 1.000 0,9fi6 0,247

. 3 X CUBE 0.920 0.9S6 1,000 0.229 4 Y 0.257 0.247 0,229 1,000 ::::::::::::3::c::::::g3g:i:3:i::::3::::i::gi:i::c;i:::::i:::l:a3ili:i]|II

FIRST, SEC6NC, AND THIRD DEGREE POLYNOMIALS, ::::::::3:s:::>:sjssr::s3:3s:>:i:Bt3B:sa:»i|«3jc3i::;i93Il3is3isisttsi>31iK) MULTIPLE R SCUARE f 0,066 MULTIPLE fl B 0.257 N.D.F.I « 1 N.D.F.2 » 947 F FOR ANALYSIS 0F VARIANCE 6N R =t 67,017 BETA WEIGHTS

0.297 CONTRIBUTIONS 19 MULTIPLE CORRELATION

AND REGRESSION MCT9R LOADINGS', 2ND COLUMN) 1 X 0.0*4 I.000

SQUARED BETA WEIGHTS 0,066

B WEIGHTS 2.950


MULTIPLE R SCUARE * '0,066 MULTIPLE R o 0.2.57 » N.D.F.I s 2 N.D.F.2 » 946 F FOR ANALYSIS OF VARIANCE 9N R I 33,919 BETA WEIGHS

0.294 -0,038 CONTRIBUTIONS TO MULTIPLE C8RRELATI9N

AND REGRESSION FACTOR LSAPINGS, 2ND C91UMN) 1 X 0-076 0.999 2 SOLAR «0,009 0,961

SQUARED BETA WEIGHTS 0.066 0,001

B WEIGHTS 3.371 -0.384

INTERCEPT CONSTANT • 3,473 .... :r3:3:3:3:3 3333333SS333S33ls:t3:3<3>>i:33333;o33S333333 333lia3:33<ll MULTIPLE R SCUARE * 0,071 MULTIPLE R = 0.266 N.D.F.I a 3 N.D.F.2 i 945 F FOR ANALYSIS OF VARIANCE ON R » 23i?99 BETA WEIGHTS • -0.337 1.572 -1,012

CONTRIBUTIONS TO MULTIPLE C9RRELAT19N AND REGRESSION FACTOR LOADINGS, 2ND COLUMN) 1 X »0.087 o.'** 2 SOLAR 0,389 0.929 3 X CUBE «0.231 0.859

SQUARED BETA WEIGHTS 0.113 2.472 1,024

B WEIGHTS -3.865 19.953 .10.493 INTERCEPT CONSTANT » 4.248 .

AN0VA TAELE FOR POLYNOMIALS ..... 3:::3::3 33 3333333333:3l3B333I3l33:3l313SI33IB3SI33i3|:BS33I333 3l33ll33>lll REDUCTION DUE TO LINEAR FIT, WITH 1 DF I 0,066 RESIDUAL S.S, 3 o<*34 DF 4 947 RESIDUAL M.S. ■ 0.001

F FOR LINEAR FIT » 67,017 B33:33S:S3 3333333:l>::iII33333l3asll33l3l3>333tl3Bt3l33l33333ll|ll||333ISI

REDUCTION DUE TO GENERAL QUADRATIC FIT WITH DF 2. ■ 0.066 REDUCTION M.S. a 0,033 RESIDUAL S.S. * 0.934 DF • 946 RESIDUAL M.S. ■ 0,001

F FOR QUACRATIC FIT » 33,519 REDUCTION CUE TO QUADRATIC TERM ALONE, WITH 1 DF, I 0,000 F FOR OUACRATIC TERM AL8NE ■ 0>085 33133 3:lli3I>::|li:ilBCIMI>l3IIIIIIIII3BI3il3 1l3IIS>llIBSIBB3IIBIB

REDUCTION CUE TO GENERAL CUBIC Fit WITH of 3, « 0,071 REDUCTION M.S. » 0.024 RESIDUAL S.S, ■ 9,929 DF * 949 RESIDUAL H.S, * 0,001

F FOR GENERAL CUBIC FIT I 23,999 H~6 REDUCTION CUE TO CUBIC TERM ALSNE HtTH 1 DF ■ , 0,009 F ^R CUBIC TERM ALONE ■ 4,698 .... I11.1-.1.1III13I3IIII11III IXIlll. IIIIIIIIIJlIlKBlllllMlllllllillll

APPENDIX I

RESULTS OF THE CURVILINEAR REGRESSION ANALYSIS BY RATING

1-1

THIS PAGE IS BLANK

1-2

POLYNOMIAL FITTING FOR 2 VARIABLES-SHE ANÜ TP SCORE- 9? OBSERVATIONS.

EM RATING THE PREDICTOR VARIABLE(X) IS SRE, AND THE CRITERION VARIABLE(Y) IS TP SCORE,

TEST MEANS AND STANDARO OEVIATIONS X

SOUAR x CUBE

Y

0,413 0.?60 o.i as 6,000

0.302 0.27« 0.258 1.953

0.943 1.000 0.98o 0.364

lazaaaaasaaaaaaaaaaaassaaBasaaaaaaaaaaaa

CORRELATION MATRIX 1 X 1.000 0.945 0.867 0.365 2 SQUAR 0.945 1.000 0.980 Q.364 3 X CUBE 0.R67 0.98o 1,000 0.328 4 Y 0.365 0.364 0.328 1.000

Bia«S8333S3lB3B=<S.B:S3S:aai

FIRST, SECOND, ANO THIRD ÖEGREE POLYNOMIALS. asasaaaasaaaaaaaaaaaasassaaaaanaaaaaBaaasaaaassaaaaaaaaaaaaaaaaaaaanaaaaa« MULTIPLE R SQUARE ■ 0.133 MULTIPLE R ■ 0.365 N.D.F.I ■ T N.O.F.2 ■ 95 F FOR ANALYSTS OF VARIANCE ON R a 14.560 BETA WEIGHTS

0,365 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTOR LOADINGS, 2ND COLUMN» T X 0,133 1.000

SOUÄREO BETA WEIGHTS 0.133

B WEIOHTS 2.358

INTERCEPT CONSTANT ■ 5.027 aaasaaaaaaaaaaaaaaaaaKaaaaaaaaaaaaaasaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaBi MULTIPLE R SQUARE a 0.136 MULTIPLE R ■ 6.369 N.O.F.I ■ 2 N.O.F.2 ■ 94 F FOR ANALYSIS OF VARIANCE ON R ■ 7.420 BETA WEIGHTS

0.196 0.179 CONTRIBUTIONS TO MULTIPLE CORRELATION

ANO REGRESSION FACTCR LOAOINGSI 2N0 COLUMN) T X 0.071 0.987 2 SOUAR 0.065 O.98S


8 WEIGHTS 1.265 1.266

INTERCEPT CONSTANT ■ 5.148 asaBasssaaaaaaaaBaaaaBasaa'aaBaaaaaasaasaaaaaaaaaaaaaaaaBBaaaaaaaaaaaaaaaa MULTIPLE R SQUARE ■ 0.1(b MULTIPLE R ■ Ö.419 N.D.F.I ■ 3 N.O.F.2 B 93 F FoR ANALYSTS OF VARIANCE ON R ■ 6.61S BETA WEIGHTS •1.120 3.778 -2.404

CONTRIBUTIONS TO MULTIPLE CORRELATION ANO REGRESSION FACTOR LOADlNGSt 2ND COLUMN) 1 X "0.408 0.859 2 SQUAR 1.374 0.86T 3 X CUBE -0,790 0.783

SQUARED BETA WEIGHTS 1.255 14.276 3.779

B WEIGHTS -7.246 26.715 -18.221 INTERCEPT CONSTANT « 5.40*

ANOVA TABLE FOR POLYNOMIALS

REDUCTION OUE TO LINEAR FIT» WITH 1 OF B 0.133 RESIDUAL S.S, a 0.867 OF • 95 RESIDUAL M.S. a

F FOR LINEAR FIT a 14.560 0,009

REDUCTION OUE TO GENERAL OUAORATIC FIT WITH OF 2. a 0.136 REDUCTION M.S. a 0.068 RESIDUAL S.S. a 0.864 OF a 94 RESIOUAL M.S. a 0*009

F FOR QUADRATIC FIT a 7«420 REDUCTION DUF TO CUADHATK TERM ALONE. WITH 1 6?» " 0.003 F FoR QUADRATIC TERM ALONE a 0.375

REDUCTION DUE TO GENERAL CUBIC FIT WITH OF 3» a 0.176 REDUCTION M.S. a 0.059 RESIOUAL S.S. a 0.824 OF a 93 RESIOUAL M.S. ■ 0.009

F FOR GENERAL CUBIC FIT a 6.615 REDUCTION DUF TO CUBIC TE«H ALONE WITH 1 OF F FOR CUBIC TFflM ALONE a 4.460

0.040

1-3

0.370 0.364 0.359 1.953

0.979 1.00Ö 0.990 0.297

0.942 0.990 1.000 0.322

0.2*7 0.297 ft. 322 l.POO

POLYNOMIAL FITTINO FOR 2 VAHJ A<JL£Ji-PR£ AHO T? SCOFIE- 97 OBSERVATIONS,

EM RATING

THE PREDICTOR VARIARi.EJX) IS PRE. AND THE CRITERION VARIA8U<Y) IS TP SCORE.

TEST MEANS ANO STANDARD DEVIATIONS I t 0.681 ?. SQUAR 0.599 3 X CUBE 0.539 4 Y 6.000

CORRELATION MATRIX 1 X 1.000 2 SQUAR 0.979 3 X CUBE 0.9*2 * Y 0.2*7 cBisBaaaaasaaaaaii.saaisssa.sasaasaassszsasaassaaaaoiaas.ascKiaaaiasaafla« FIRST. SECOND, AND THIRD njrGREE POLYNOMIALS. caosaasaaiaezsaasaassaaaaaaBaaaaaaaasaaeaaasaasaaaflBaBaaaaaaBaaaaaaaeaaaaa MULTIPLE R SoiiARE ■ 0.061 MULTIPLE R » 0.2*7 N.D.F.I • 1 N.D.F.? a 95 F FOR ANALYSIS OF VARIANCE ON R ■ 6.163 BET» WEIOHTS

0.2*7 CONTRIBUTIONS TO MULTIPLE CORRELATION

ÄNO REGRESSION FACTOR LOAOINOS» 2ND COLUMN) T X 0.061 1.000

SQUARED SETA WEIOHTS 0.061

B WEIGHTS 1.303

INTERCEPT CONSTANT ■ 3,u2 EaanaasBaaaBaaaasaaaaaasaaaBsaaBsiaasaaaaaaaaaaaaaaBBaaBaaaaaÄawaaaraaatajaaa MULTIPLE R SQUARE ■ 0.133 MULTIPLE R ■ 0.363 N.O.F.I ■ 2 N.0.F.2 ■ 94 F FOR ANALYSIS OF VARIANCE ON « ■ 7.238 BET« WEIOHTS -1,033 1,307

CONTRIBUTIONS TO MULTIPLE CORRELATION AND REGRESSION FACTOR LOAOINOS» 2N0 COLUMN) T X -0.255 ' 0.6?6 ? SOUAR 0.388 0.813

SQUARED BETA WEIOHTS 1.066 1.7Ä9

8 WEIOHTS -5.453 7.66* INTERCEPT CONSTANT ■ 5.516 BaSB33a8naa83S3SBilaaa8=8BiS3as33aaaiaaiai8aasBaB8BaaiiBBBsiaaaas3aBaa3aaa

MULTIPLE R SQUARE ■ 0.133 MULTIPLE R ■ 0.365 N.O.F.I ■ 3 N.0.F.2 ■ 93 F FOR ANALYSIS OF VARIANCE ON R ■ *.774 BETA WEIOHTS -1.006 1.2*1 0.6*1

CONTRIBUTIONS TO MULTIPLE CORRELATION AND REGRESSION FACTOR LOADINGS» 2NO COLUMN) 1 X -0,2*8 0.676 ? SQUAR 0.368 0.813 3 X CUBE 0.013 0.882


B WEIGHTS -5.313 6.647 0,22* INTERCEPT CONSTANT ■ 5.515


REDUCTION DUE TO LINEAR FIT, WITH 1 OF ■ 0.061 RESIDUAL S.S. ■ 0.939 DF ■ 95 RESlOUAL M.S. ■ O.OlO

F FOR LINEAR FIT ■ 6.163 Baaa8a8as88a33B3aaa3iaaiaB8B3Baaai8B88Bsaaa8B338aa3saiaaaaaBai83B3a8aaiaaa

REDUCTION DUFT TO GENERAL QUADRATIC FIT WITH OF 2. ■ 0.133 REDUCTION M.S. ■ 0.067 RESIDUAL S.S. ■ 0.B67 DF » 9* RESlOUAL M.S. • 0.009

F FOR QUADRATIC FIT a 7.238 REDUCTION DUE TO OUADRATIC TERM ALONE. WITH I CFt " 0.073 F FOR QU4DRATTC TERM ALONE • 7*867

REDUCTION DUE TO GENERAL CUBIC FIT WITH OF 3. a 0.133 REDUCTION M.S. a 0,0** RESIDUAL S.S. ■ 0.867 DF ■ 93 RESlOUAL M,S. a 0.009 J_4

F FOR OFNERAt. CUBIC FIT a 4.77* REDUCTION DUF TO CUBIC TERM ALONE WITH 1 OF ■ , 0.000 F FOR CURIC TERM ALONE a 0.000

Bi:B3aaiiflj3a^n.T:iDa«i.:ai>d'janfBs'iaii3:!J3ijn--.*(i;a3?naB«itij3B»a33i*iBiw*HBqaii3i POLYNOMIAL ftrTINO FOR 2 VARI A>iLES-<5R£ ANO IP SCOHg. 97 OBSERVATIONS,

THE PREDICTOR VARIARLE(X) IS GRE» ANO THE CRITERION VAn'lABLE'(Y) IS. Tp SCORE.

EM RATING

TEST MEANS AWl STANDARD DEVIATIONS 1 X 2 SQUAR 3 X CUBE * Y

0,960

0.A91 6.000

CORRELATION MATRIX 1 X 1,000 2 SOUAR 0,999 3 X CUBE 0.997 * Y 0.301

0.051 0.093 0.129 1.953

0.999 1.000 0.999 0.311

0.997 0.999 1.000 0.321

0.301 0.311 0.321 1.000

AStl3a3Bt383.aasa3183E33S33S83S33l33aaS3e3|3alS|sa«B3laaSIBiaal8lla||l3agsal FIRST, SECOND, ANO THIRD DEGREE POLYNOMIALS. etnaasasosaaaaaaaaajaaBaatsasaflasssaaacaBsasaaasaaaaaassaaasaaaaaaaBBasaaaassa MULTIPLE R SQUARE ■ O.Ö'O MULTIPLE R ■ 0.301 N.O.F.I a \ W.O.F.? a 95 F FOR ANALYSIS OF VARIANCE ON R ■ 9.OS BETA WEIGHTS

0.301 CONTRIBUTIONS TO MULTIPLE CORRELATION

ÄNO REGRESSION FACTCR LOADINGS» 2ND COLUMN) T X 0.090 1.000


B WFIGHTS 11.507

INTERCEPT CONSTANT a -5.0*3 BaBaassaaaaaassaeaaaaassaaaaaBaasaaaaaaEBaaaaaaaaaaaaaaaasaaaaaiataaaasaal MULTIPLE R SQUARE ■ 0.15* MULTIPLE R ■ 0.393 N.D.F.I a j N.D.F.3 ■ 9* F FOR ANALYSTS OF VARIANCE ON R " 8*560 9ETa WEIGHTS -5.5B5 5.«91

CONTRIBUTIONS TO MULTIPLE CORRELATION AND »EGRESSION FACTOR LC-AOINGS» 2ND COLUMN) 7 X -1,679 0.7*6 ? SQUAR 1,833 0.793

SOUÄRE0 BETA «EIGHTS 31,196 3*.7n7

8 WEIGHTS 213.819 123.123 INTERCEPT CONSTANT ■ 97.*86 aaaaBBsaaBaaasusaaaaaaasaaaaaaaBaaaaaaaaaBaciaiaaaasaaaasaaaaaaaBiaiBaaaaB MULTIPLE R SQUARE ■ 0.221 MULTIPLE R ■ 0,*70 N.O.F.I a 3 N.O.F.2 a 93 F FOR ANALYSIS OF VARIANCE ON R ■ 8.T9* BETA WEIGHTS 127,159.266.6Ö9 139,99* CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTOR L°A0INGS» 2NO COLUMN) 1 X 3«,221 0.639 ? SQUAR >82,9*7 0,662 3 X CUBE **.9*7 0.683

SOUÄREO BETA WEIGHTS

8 WFIGHTS 867.907««»««i«#2117,081 INTERCEPT CONSTANT a «1*06,053

ANQVA TABLE FOR POLYNOMIALS BBBaaaaaaaeBa:BBaaaB9aaa3i>BaaBaa<laa»8BBBiBaiiiiBaaiBBaaaiailBB8BaBaaaai REDUCTION OIIF TO LINEAR FIT, WITH 1 OF a 0.090 RESIDUAL S.S. ■ 0.910 OF a 95 RESIDUAL M.S. ■ O.OlO

F FOR LINEAR FIT ■ 9,*35

REDUCTION OIIF TO GENERAL OUAORATIC FIT WITH OF 2. a 0.15* REDUCTION M.S. • 0.077 RESIDUAL S.S. ■ O.B*6 DF • 9* RESIOUAL M,J, ■ 0.009

F FOR OUAORATIC FIT a 8.560 REDUCTION OUF TO CUAORATIc TERM ALONE» WITH 1 OF» a 0.06* F FOR QUADRATIC TERM ALONE a 7.081

REDUCTION DUE TO GENERAL CUBIC FIT WITH OF 3» a 0,221 REDUCTION M.S. • 0.07* RESIDUAL S.S. • 0.77« DF ■ 93 RESIOUAL M.S. ■

F FOR GENERAL CUBIC FIT a 8,79* REOUCTION DUE TO CUBIC TERM ALONE WITH 1 OF " , F FOR CUBIC TFRM ALONE a 7.9B7

0,067

0,008 1-5

• cBj»a«s»M«iuaa«a«aa»i3a,»»»»«t«««!'«U'1«ä«««xsus»»»«a«jiisa»iU8ii3<sissai

POIYNOM'AI. FTrTIW'3 FOR 2 VAH t A'^tS-rtüE AND Ti> SCO«fc"- 97 OBSEM7A r IOWS,

EM RATING

THE. PflEÖICTon VAflt'ÄÄLE'cyV IS }tot't »«0. THE CHITRRION VARIABLE (V.'IS TP SCORE.-.' '-

TEST MEANS AND STANDARD DEVIATIONS .* . . '• "'" ■ 1 ■' X . 0,ft47 0.1B5 ? SQUAR 0.453 0.225 T. X CUBE 0.133 n.236 4 :OR«

Y »ELATION

6, 000 MATRIX

1.953

J X 1.000 0,963 0, .918 0.492 2 SQUAR 0.963 1.000 0 .9ß9 o.so* ? X CUBE 0.911? 0.989 1 .000 o.*«i 4 Y 0.492 0.50* 0 .*«) 1.000

•oaaaaaarasaaainaauaaaaaaaaasio^caaaaaaaaaaiacaaaaaanjaaaaaaaaasBiaaaaaaaa FIRST, SECriNn. AND THIRO DEGREE POLYNOMIALS, BBanaasaaasaaaaaaaBBaaiBaaaaaaBnBaaaaaaaaaBaBaBaaaasBaaaaaaaaaaBBaBBaaaBaaa

MULTIPLE R SQUARE » 0«f;*2 MULTIPLE R » 0.*9? N.O.F.j - \ N.O.F.? a 95 F FOR ANALYSIS OF VARIANCE ON B ■ 30.345 BFTÄ WF.IfiHTS

0,492 CONTRIHUTIONS TO MULTIPLE CORRELATION

ANO »FGRFSSION F&cTCR LOADINGS» 2ND COLUMN) 1 X 0,242 1.000

SOUARED RETA «EIGHTS 0.242

B WFIGHTS S.194

INTERCEPT CONSTANT a 2.639 ■aaaaaaaaaaaaaaaaaaaasaasaaxaaaaaaaasaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa MULTIPLE R SQUARE ■ 0.254 MULTIPLE R ■ 0.504 N.O.F.I ■ 2 N.O.F.? at 94 F FOR ANALYSIS OF VARIANCE ON R ■ 16.024 BET» WEIGHTS

0.09ft 0.4Ü CONTRIBUTIONS TO MULTIPLE CORRELATION

ANO REGRESSION FACTCR LOADINGS« 2N0 COLUMN) 1 X 0.047 0.9/6 2 SOUAR 0.207 0.999

SQUARED BETA WEIGHTS 0.0O9 6.1*9

B WFIGHTS 1.012 3.«573

INTFRCEPT CONSTANT » 3«7?8 aaaasaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaiiaa MULTIPLE B SQUARE ■ 0.3J0 MULTIPLE R ai 0.548 N.O.F.I ■ 3 N.0.F.2 ■ 93 F FOR ANALYSIS OF VARIANCE ON R a 13.273 BET« WEIGHTS , . - - -1.475 5,TÖ6 «3,210 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR LOAOINGSI 2ND COLUMN) T X -0,726 0.8?9 2 SOUAR 2,568 0.920 ■«. X CURE -1.543 0.878

SQUARED BETA WEIGHTS 2,176 26,012 10,307

R WFIGHTS -15.571 44.3f9 -26.«561 INTERCEPT CONSTANT ai *.B*9

ANOVA TABLE FOR POLYNOMIALS «BBBBBaBBaaBBBasBBBaBaBaaaBBBBBBBBaasaBaBaBBBaaaBaaBBBsBaaBBaBBBaaaaBaBBaa OEnUCTION OUF TO LINEAR FIT, WITH 1 OF a 0.242 °ESlOUAL S.S. ■ 0,758 OF a 95 RESIDUAL M.S. • 0.008

F FOR LINEAR FIT a 30.3*5 ■laiaasaaaiiaaaaaiieiaiaaaitaaaaaaaacBtaaaaaasiaaaaaaaaiaaaaaaaaaaaaaaaaia

REDUCTION OIIF TO GENERAL OUAORATIC FIT WITH OF 2. ■ 0.254 RFOUCTION M.S. a 0.12? RESIDUAL S.S. a 0.746 OF a 94 RESlOUAL M.S. a 0.008

F FnR OUAORATIC FIT a 16*024 RFOUCTION OUF TO CUAORAT'C TERM ALONE» WITH 1 fiF» a 0.012 F FOR QUADRATIC TERM ALONE a 1.532 aaaaaaaaaaaaniaaaaaa aaaaaaaaaaaaaiaaaaaaa anaiaaaaaa aana

RFOtlCTION OUF TO GENERAL CUBIC FIT WITH OF 3, a 0.300 REDUCTION M.S. a 0.100 RESlOUAL S.S. a 0.700 OF a 93 REStOUAL M.S. a O.OOfl I-ß

F FDR (3ENERAL CUBIC FIT a 13.273 RFOUCTION OUF TO CUBIC TERM ALONE WITH 1 OF a , 0.0**. F FOR CUBIC TFRM ALONE " 6.051

anatnJaaaatuas^aiuaauaaaaaaanaaaaavsBaaasasQansaasaaxaaiftaaaiiaauBaanaLaa'aaiBaj-ja POLYNOMIAL PITTING FOH 2 VAfUA^LES-SRE ANÜ TP SCORE- 173 OBSERVATIONS,

THE PREDICTOR VAfilAHLe<X)' I5:SPE, AND THE CRITERION VARIAöLE(Y) IS TP ScoftC,

TEST MEANS ANn STANDARD DEVIATIONS 1 X 0.168 0.208 ? SQUAR 0.Ö7I 0.1*9 3 X CUBE 0.0*0 0.127 4 Y 6.514 1.784

CORRELATION MATRIX \ X 1.ÖO0 0.909 0.778 0.318 2 SQUAR 0.909 1.000 0.963 0.703 3 X CUBE 0.778 n.9?3 1.000 0.12* * . Y 0.318 0.205 0.12« 1.000

FIRST, SECOND. AND THIRD ofc'QREE POLYNOMIALS. SB33333aSB3fl333i:Ba983llXSSi'S3D:Ba3aB33338l3SaSB3333aill&IIISIIItB3alia33ag MULTIPLE R SQUARE « O.lJl MULTIPLE R ■ n,3lfl N.O.F.I aa j N.O.F.2 a 1?1 F FOR ANALYSIS OF VARIANCE ON R ■ 19.191 BFTÄ WEIGHTS

0.318 CONTRIRUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTOR LOAOINGS» 2ND COLUMN) T X 0.101 1.000

SQUARED BETA WEIGHTS 0,101

a WEIGHTS 2.720

INTERCEPT CONSTANT aa 6.057 BEsaaBsaaaaisassassaaaasssaBigBtsasaaBiassaaiailasaaasDaatiitataiiaaaasaiifliaiga MULTIPLE R SOIIARE ■ 0.1*2 MULTIPLE R ■ 0,376 N.O.F.I ■ 2 N.O.F.2 a 17Ö F FOR ANALYSIS OF VARIANCE ON R • 14.021 BET» WEIGHTS

0.759 -6.4S5 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR LOAOINGS» 2ND COLUMN) 1 X 0.241 0.8** 2 SOUAR • -0,099 0.5*5

SQUARED BETA WEIGHTS 0,576 0,21.5

B WFTOHTS 6.498 -5.793

INTERCEPT CONSTANT • 5,835 BcBniaaisasaasssiaasaiaaaa'asisBsaoK'aiaaassaagaaaiaiaaaaiiiaaaaaaagiaia MULTIPLE R SOIIARE * 0,1*2 MULTIPLE R a 0.377 N.O.F.I ■ 3 N.O.F.2 a 169 F FOR ANALYSIS OF VARIANCE ON K ■ 9.323 BETA WEIOHTS

0,845 -0,710 0.152 CONTRIRUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTOR LOAOINGS» 2N0 COLUMN) T X 0.268 0.8*3 7> SQUAR -0,145 0.3** 3 X CUBE 0.019 0.33*

SQUARED BETA WEIOHTS 0.714 0.504 0,023

B WEIGHTS 7.235 -8.477 2,129

INTERCEPT CONSTANT ■ 5!Öl7

ANQVA TABLE.FOR POLYNOMIALS aoaaaaaasaaaBBBaaBaaaBaBaaaaaaBaaaaisaaaaasaaBgsaBaaaaaaaaaaaaiaaaaiiaaaaa HEOUCTION OUF TO LINEAR FfT. WITH 1 OF ■ 0.101 RESIDUAL S.S. ■ 0,899 OF » 171 RESIDUAL M.J. a 0,005

F F^R LINEAR FIT ■ 19.191 BaBaacaaaaaaas^saBaasaasaaaBaaasaaaaaaaBaaBaaaaaaaBaaaaaaaaBaaaaaaaaaBaaaa

REDUCTION OUF TO GENERAL QUAORATIC FIT WITH OF 2. a 0.1*2 REDUCTION M.S. a 0.071 HFsTOUAL S.S. a 0.858 OF a 170 RESIOUAL M.J. a 0.003

F FOR QUADRATIC FIT a 14.021 REOUCTION OUE TO CUAORATIC TERM ALONE, WITH 1 fiF, a 0.0*1 F FOR QUADRATIC TERM ALONE a fl.OSfl

ET RATING

REDUCTION DUE TO GENERAL CUBIC FIT WITH OF 3t a 0.1*2 REDUCTION M.S. a 0.0*7 RESIDUAL S.S. ■ 0.858 OF • 169 RESIDUAL M.S. ■

F FoR GENERAL CUBIC FIT ■ 9.323 RFOIICTION DUE TO CURIC TERM ALONE WITH 1 OF ■ , F FOR CURIC TFHM ALONE a 0.078

0,000

0,003 1-7

POLYNOMIAL rtTTINO FrtH 2 VAWIAÜIES-PRE ANO TP SCOH£- 173 QBSEnV&TlOwS«

ET RATING

THE PREDICTOR VARIABt.E(X> IS PRE, AND THE CRITERION VAP1.1BLE(Y> IS Tp SCORE..

TEST MEANS AND STANDARD DEVIATIONS \ X 0.40* 0.352 ? SOUAR 0.?87 0.288 3 X CUBE 0.?13 0.2*8 A Y 6,514 1.784

CORRELATION MATRIX 1 X 1.000 0.967 0.904 0.366 ? SQUAR 0,967 1.000 0.982 0.361 3 X CUBE 0.904 0.982 1.000 0.3*3 * . Y 0.366 0.361 0.34J 1.000

3S3a3aza3aagxsis9ilasBi=3ElB3SS33aaa:flaasB39SS3:B3saaaBsiaia9&l3iaaiaa93sl FIRST. SECONn, «\N0 THIRD ÖEGREE POLYNOMIALS. aasssasasaaaasanasao^nszsnaassaaaaaas^assaaaaaaasgaaaaaaaaaaBaxsaaaaBasaaa MULTIPLE R SQUARE ■ 0.134 MULTIPLE R ■ 0.366 N.D.E.l ■ T M.D.E.2 ■ 171 F FOR ANALYSIS OF VARIANCE ON R ■ 26.433 BET» WEIGHTS


AND REGRESSION FACTOR LOAOINGS» 2N0 COLUMN) T X 0,134 1.000


B WFIGHTS 1.BS4

INTERCEPT CONSTANT ■ 5.765 aasassaBaaaaasaaaaaaaaasssaaiBsastasaaaaaassaaasaaaaaaasaBBaaiaaBaaiaaaasI MULTIPLE R SOIIARE » 0.135 MULTIPLE R » 0,367 N.O.F.I a J N.0.F.2 a 176 F FOR AN4LYSIS OF VARIANCE ON R ■ 13.222 BETA WEIGHTS

0.260 0.1Ä9 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR LOADINGS» 2N0 COLUMN) T X 0.095 0.997 ? SOUAR 0.039 0.984

SOUÄREO BETA «EIGHTS 0.0 68 0.Ö12

H WEIGHTS 1.319 Ö.677

INTERCEPT CONSTANT » 5.787 asasassasaaaasssssssaaassaaaaaaaaaassssasaaassasaaaasaasaaaaasaaaaaaaasaaa MULTIPLE R -SQUARE " 0.135 MULTIPLE R ■ 0,367 N,O.F.I a 3 N.0.F.2 ■ 169 F FOR ANALYSIS OF VARIANCE ON R ■ 8.769 8ET» WEIGHTS _ . _

0.364 -0.137" OVt'48 '"' *"*" "" "■ CONTRIBUTIONS TO MULTIPLE CORRELATION

ÄNO REGRESSION FAcTCR LOADINGS» 2N0 COLUMN) T X 0.133 0.997 7, SOUAR -0.049 0.983 3 X CUBE 0,051 0.93S

SQUARED BETA WEIGHTS 0.133 0.Ö19 0,022

fl WEIGHTS 1.845 -6,«47 1,068

INTERCEPT CONSTANT ■ 3.783

ANQVA TABLE FOR POLYNOMIALS aiaaaaaaaaaiaaaaaiiaaagaailaaiaaaBiaBaiaaiBBaaiaaaaaaitaiiatiiiaiiiiaiiBBI REOUCTION DUF TO LINEAR FIT, WITH 1 OF » 0.134 RESIDUAL S.S. ■ 0.A66 OF a 171 RESlOUAL M.S. ■ 0.005

F FOR LINEAR FIT a 26.423 BB8a8aOBaflBaaaBaaB3BflB8BB>aBBtaaaiaasBaBaBBaBataaaaaaaaBaaBBiiiBiaaiflaiBBB

REnUCTlON OUF TO GENERAL QUADRATIC FIT WITH OF 2» ■ 0.135 REDUCTION M.S. • 0.067 RESIDUAL S.S. a n.B6S OF a 170 RESlOUAL M,S, ■ 0*005

F FOR QUADRATIC FIT a 13.222 REOUCTION DUE TO OUADRATIC TERM ALONE» WITH 1 pF» a 0.001 F FOR QUADRATIC TERM ALONE a 0,152

REDUCTION DUE TO GENERAL CuBIC FIT WITH OF 3, . 0,135 RFOUCTION M.S. a (1.0*5 - Q RESIOU»L S.S. • 0.B65 OF a 169 RESlOUAL M,S, a o.OOS A ~°

F FOR GENERAt CUBIC FIT » 8.769 REOUCTION OUF. TO CUBJC TERN ALONE WITH 1 OF • , 0,000 F FOR CUBIC TFHM ALOKJE B 0.017

POLYNOMIAL FITTINO FOI< 2 VARIABLES-ORE ANO TP SCORE- 173 OBSEBVATIOKS,

ET RATING THE PREDICTOR VARIABLES» IS ORE, ANO THE CRITERION' VARIABLE(YJ•IS TP SCCRe!,

TEST MEANS ANO STANOARD DEVIATIONS 1 X . 0.P.64 2 SOUAR 0.796 3 X CUBE 0,7*0 * Y 6.51*

CORRELATION MATRIX 1 X 1.000 2 SQUAR 0.<J*0 3 X CUBE 0.A93 * Y 0.508

0.232 0.2*3 0.272 1.78*

0.960 1.000 0.9Ö* 0.512

0.893 0.98* 1.000 0.49S

0.508 0.512 0,*95 t.000

iianasini39i39Ba3ssi33i3aai.3sss3afl93s3iai.8i.3X3flsisiii3itis.as3axiaifisai FIRST, SFCOND, ANO THIRD nEGREE POLYNOMIALS. l30 = saa03asas = n = 3aa33sa = :in33B«3BaSBa33a«**33a*33a3a33»aa3M«S3asi3:i«33«aaa»B3i MULTIPLE R SOIIARE ■ O.f^B MULTIPLE R ■ 0.508 N.O.F.I a T N.O.F.2 ■ 171 F FOR ANALYSIS OF VARIANCE ON R ■ 59.470 BFT» WEIGHTS


ÄNO REGRESSION FACTOR LOADINGS. 2ND COLUMN» 1 X 0.258 1.000

SOUAREO BETA WEIGHTS 0.258

B WEIGHTS *.0B0

INTERCEPT CONSTANT ■ 2«988 t3B33338838a38833Ba33Ba33383B333BaB33aa33aa33333a83>333Sai383a3aa33atiaaaa

MULTIPLE R SOIIARE ■ 0.2*6 MULTIPLE R ■ 0.515 N.O.F.I ■ 2 N.O.F.2 a 170 F FOR ANALYSIS OF VARIANCE ON R • 30.763 BET» WEIGHTS

0.209 0,312 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR LOAOINGSI 2N0 COLUMN) 1 X 0,106 0.9OS 7. SOUAR 0.160 0.993

SOUAREO BETA WEIGHTS O.O** Ö.Ö97

B WEIGHTS 1.676 2.27*

INTERCEPT CONSTANT a 3.2S5 383338383838883=8333383333333838338833833833383833838833888888383388038333

MULTIPLE R SOIIARE ■ 0.27t MULTIPLE R ■ 0.52i N.O.F.I ■ 3 N.O.F.2 ■ 169 F FOR ANALYSIS OF VARIANCE ON R ■• 20.966 BET» WEIGHTS

2.199 -4,675 3.129 CONTRIBUTIONS TO MULTIPLE CORRELATION

ÄNO REGRESSION FACTOR LOAOINGS. 2ND COLUMN» T X 1.117 0.9J5 7 SOUAR -2,39* 0.983 i * CUBE 1.5*9 0.950

SOUAREO BETA WEIGHTS ♦,837 2l.fl«9 9,788

B WEIGHTS 17.66* -34.ÖA5 30,481 INTERCEPT CONSTANT ■ 3.227

ANOVA TABLE FOR POLYNOMIALS B38>a3383a3B33S:388833833833aI8I38333383B3BS33 3331 REOUCTION DUF TO LINEAR FIT, WITH 1 OF ■ 0.258 RESIDUAL S.S. a 0.7*2 DF a 171 RESIDUAL M.S. a

F FOR LINEAR FIT a 59.470 0.004

833338383808338838833838888888833838

0.266 REOUCTION ODF TO GENERAL OUADRATlC FIT WITH DF 2, a REDUCTION M.S. s 0.133 RESIDUAL S.S. a 0,73* DF a 170 RESlOUAL M.S. ■

F FOR OUAORATIC FIT a 30.763 REOUCTION DUE TO CUApRATTC TERM ALONE. WITH 1 CF. a 0.008 F FOR OUAORATTC TERM ALONE 8 1,783

0.004

REDUCTION HUE TO 0ENER&L CUBIC FIT WITH OF 3i ■ 0,271 REOUCTION M.S. a 0.090 RESIDUAL S.S. ■ 0.729 DF a 169 RESlOUAL M.S. •

F FnR GENERAL CURtC FIT a 20.966 RFOUCTION ODF TO CU^TC TERH ALONE WITH I OF F FOR CUBIC TFPM ALONE a 1,27*

0.005

0,004 1-9

0.210 0.22S 0.207 1.78*

0.977 0.93? 0.445 1.000 0.987 0.413 0.987 1.000 0.380 0.413 0.380 l.nOO

■ BEaBsaaiiiaansaaiiBaüsiiBjflaajiaaosaaöasaiÄjasaioisaiaainaaBiKiaaKiaaijaji,^^;,!

POLYNOMIAL PtrTlNO FOH 2 VAHIAHL£S-WRt£ ANO TP SC0«K- 173 OBSERVATIONS«

ET RATING THE PREDICTOR VAHIARLE(X) IS WRE.'AND THE-CRITERION VARIABLE<Y> tS TP'SCORC,'

TEST MEANS ANn STANOARn DEVIATIONS 1 X 0.573 ? SQUAR 0.372 "» X CUBE n.?59 4 Y 6.514

CORRELATION MATRIX 1 X 1.000 2 SOUAR 0.977 3 X CUBE 0.9.32 A V 0.445

8:S3ES33=3=B3ss393333a::3iafi3ts89Baisa33:a3333as389sa»:ansx*xaxiasOigdxag FIRST, SECOND, ANO THIRD nfGREE PULYNOMIAUS. BBB=3S3B33S33ss=83333a3 3 33aa38s33aaa8sis3333a=aaa3sa>8aasaaaaa3aii8iiaas8i MULTIPLE R SOIIARE ■ 0.198 MULTIPLE 0 " 0.445 N.O.F.I ■ T N.0.F.3 « 171 F FOR ANALYSIS OF VARIANCE ON R ■ 42.260 BETA WEIGHTS


AND REGRESSION FACTCR LOAOINGS. 2N0 COLUMN) T X 0,198 1.000


R WRIGHTS 3.789

INTERCEPT CONSTANT « 4.344 «B33as53BSB33a==aBaa5»3t3Ba38B3aB«a3saaea»3ssas3a5aiaiaeiaaai«aBi»alai3sas MULTIPLE R SOIIARE ■ 0.?Ä9 MULTIPLE R ■ 0.457 N.O.F.I ■ 2 N.D.F.2 ■ l7o F FOR ANALYSIS OF VARIANCE ON fl • 22.416 BET« WEIGHTS

0,913 -0,479 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR LOAOlNGSf 2N0 COLUMN) T X 0.406 0.9/4 ? SOUAR »0.193 0.9Ö4

SOUAREO BETA WEIGHTS 0.O33 (1.2?9

B WFIGHTS 7.769 -3.8*2

INTERCEPT CONSTANT a 3,478 B38333:3 3s8a333=a3a33 3333aa33as3aa33r33=833aas3aa3 8BaBe33a3saBaaaa83^33aai MULTIPLE R SOIIARE ■ O.^IO MULTIPLE R ■ 0.458 N.O.F.I a 3 N.0.F.2 ■ 169 F FOR ANALYSTS OF VARIANCE ON fl ■ 14.942 BETA WEIGHTS

1.285 -1.349 0,534 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR LOADINGS» 2NO COLUMN) T X 0,572 0.972 2 SOUAR -0,565 0.902 1 x CU8F 0.203 0.830

SOUÄREO BETA WEIGHTS 1.651 1.875 0,283

B WEIGHTS 10.937 -10,877 4,602 INTFRCEPT CONSTANT « 3.103

ANOVA TABLE FOR POLYNOMIALS aBBaBBBSsaBasBaaasassaQ'saBaaaaaBBaaaaaasBaaaaBaBBBBBaasBaaBBBaaaaaaaaaaaaaB REDUCTION DIIF TO LINEAR FIT, WITH 1 DF ■ 0.198 RESIOUAL S.S. ■ 0.802 OF ■ I7I RESIDUAL M.S. ■ 0.009

F FOR LINEAR FIT • 42,260

PEnUCTlON DUE TO GENERAL QUADRATIC FIT WITH OF 2, ■ 0.209 REOllCTION M.S. ■ 0.104 RESIOUAL S.S. B 0.791 OF » 170 REStOUAL M,S. ■ 0»003

F FOR OtIAORATIC FIT a 22.41* REDUCTION OUF TO GUAORATIC TERM ALONE. WITH 1 CF» a 0.011 F FQR QUADRATIC TERM ALONE ■ 2,261

REOilCTJON DUE TO OENERAL CUBIC FIT WITH DF 3i a 0.210 REDUCTION M.S. ■ 0.070 » - -> RESIOUAL S.S. a 0.790 OF » 169 RESIDUAL M,«, a 0.00S *"IU

F FOR GENERAL CUBIC FIT a 14.942 REDUCTION OUF TO CU»IC TERM ALONE WITH 1 OF • , 0.001 F Fo» CUBIC TfRM ALONE a 0,204 niiiaaiaiiajiiauaj.ma •aniiiaiiaaa»iiiil3,aaa«aiiiilaiiiaai

POIYNOMIAL FITTING FOR 2 VARIA^LES-SHE ANO fi> SCOHF. 154 OBSERVATIONS,

THE PREDICTOR VARIAHLE(X) IS SRE, ANO THE CRITERION VARIABLES) IS TP SCORE, FT RATING

0.293 0.?5S 0.2^1 2.0*2

0.9*4 0.871 0.346 1.000 0.981 0.122 0.9dl 1.000 0.294 0.322 0.294 1.000

TEST MEANS ANO STANOARC DEVIATIONS 1 X 0,103 2 SQUAR 0.177 3 X CUBE 0.12* 4 Y 6,*4ft

CORRELATION MATRIX 1 X 1.000 2 SQUAR 0.944 3 X CUBE 0.«71 A V 0.346

flaaaaaaaaaaaasasBaaaaBaaaaaaaaDaaBaaaiaaaaaaasaaaaaaaaaaaiaaaiaanaaaaaaaai FIRST, SECOND, ANO THIBO nFlREE POLYNOMIALS, BcasaaaaaaaBaaaBBBBasaarBBaBBasaaaoaaaaaaBaaaaaoaaaaaaaBBBBBBBBaBBaBaaBaaia MULTIPLE R SOIIARE a 0,119 MULTIPLE R ■ 0,346 N,O.F.I a x N.O.F.2 a 152 F FOR ANALYSIS OF VARIANCE ON R a 20«*23 BFTii WEIGHTS


ANO REGRESSION FACTCR LOAOINOS» 2N0 COLUMN) T X 0,119 1.000

SOUÄREO BETA «EIGHTS 0.119

R «FIGHTS 2.410

INTERCEPT CONSTANT a 5.5ft9 SBBaaaaaaaBBBsaasaaaBaasBaaaaaBasaaasaaaaaaaaaasaaaaaasaaaaaBaBasaaaaaaaaa HULTTPLE R SQUARE « 0.120 MULTIPLE R a 0.34ft N,O.F.I a ? N.O.F.2 a 151 F FOR ANALYSIS OF V4RIANCE ON R a 10.259 BETA WEIGHTS

0.382 -Ö.018 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR LOAOINGS, 2N0 COLUMN) \ X 0.132 0.999 ? SOUAR -0.012 0.932

SOUÄREO BETA «EIGHTS 0.146 0.ÖÖI

R «FIGHTS 2.663 -Ö.3Ö7

INTERCEPT CONSTANT a 5.4B7 scaaaaaaaaBaaaaassaaaaaaaaaaasassaaaaaasaasaaasaaaBaaaaaaaasBaaaaaaaBaaaaa MULTIPLE R SOIIARE ■ 0.120 MULTIPLE R " 0.346 N,O.F.I ■ 3 N.O.F.2 a 15ft F FOR ANALYSIS OF VARIANCE ON R a 6.798 BETA WEIGHTS

0,339 0.0*3 -0.6B3 CONTRIBUTIONS TO MULTIPLE CORRELATION

ÄNO REGRESSION FACTCR LOAOINGS» 2NO COLUMN) 1 X 0.117 0.999 2 SOUAR 0.027 0.932 3 X CUBE -0.024 0.850

SQUARED BETA «EIGHTS 0.115 0.Ö07 0.Ö07

B WEIGHTS 2.364 Ö.665 -0,730

INTERCEPT CONSTANT a 5«495

ANOVA TABLE FOR POLYNOMIALS Bcaa8aaa3Bi8a3E8iia3aB3.asaaBaaBcagasaBia8aaa3a23aaaaasaa3aiaiiBaaiaiaaiia REDUCTION OUF TO LINEAR FIT, WITH 1 OF a 0.119 RESIDUAL S.S. a 0.881 OF ■ 152 RESIDUAL M.S. a 0.006

F FOR LINEAR FIT « 20.623 acaaaaaaaaaaaasaBiaaaaaaaaBaaflaasaaaaaacaaaaaaialaaaaaiaaaaaaaBeaaiBiaxaaa

REDUCTION OUF TO GENERAL QUADRATIC FIT WITH OF 2, a 0.120 REDUCTION M.S. a 0.060 RESIDUAL S.S. a O.BflO OF a 151 RESIDUAL M,«J, a

F FOR QUADRATIC FIT a l0,2e,9 REOUCTION OUF TO CUAnRATIC TERM ALONE. WITH 1 rFi a 0.000 F FOR QUADRATIC TERM ALONE a 0.027

RFOUCTION OUF TO GENERAL PUfllC FIT WITH DP 3t a 0,120 REDUCTION M.S. a 0,040 RESIDUAL S.S. a 0.88<> OF a 150 RESIDUAL M.S. a

F FOR GENERAL CUBIC FIT a 6.798 REDUCTION OUF TO CUoiC TERM ALONE WITH 1 OF a , F FOR CUBIC TFPM ALONE a 0,010

0.000

0,006

0,006 1-11

3a3 3iiasaoas3'j.-i!i33aflai]9i3.T^3iDnsa«!ixiSi}iänai33ai39aaasa«ac'jafl3ava!i'iai3aiaaaa POLYNOMIAL FITTING FOH 2 VARI AÖL£S-FRt AND TP SCO«*;- 15* OHSf-BVA r IONS«

FT RATING THE PREDICTOR VARIABLES) IS PREf AND THE CRITERION VARlAflLElY) IS TP SCORE, ■

TEST MEANS AND STANDARD DEVIATIONS 1 X 0.568 0.393 ? SQUAR 0.476 0.365 3 X CURE 0.410 0.3*7 4 Y 6,;>*o 2.0*2 QRRELATION MATRIX 1 X 1.000 0.976 0.931 0.364 2 SQUAR 0.976 1.000 0.987 0.296 3 X CURE 0.931 0.987 1.000 0.312 4 Y 0,?6* 0.296 0.312 1.000

i:::s:33:i3oz:3:39>3!S3:s:3:i3 saisaicisiiaiaoiassiiatiuiisiiiajasi FIRST. SECOND, AND THIRD riEGREE POLYNOMIALS. ssBssssssssasssxsiassaaraBaaassaaiaBsasasaasasiaflissasiBatsaiVaisaRaiiaaaa MULTIPLE R SQUARE ■ O.O'O MULTIPLE R ■ 0.264 N.D.F.l a 1 N.D.F.2 * IS? F FDR ANALYSIS OF VARIANCE ON R ■ 11.394 BETA WEIGHTS


AND REGRESSION FACTCR LOAOINGSI 2N0 COLUMN) T x 0.070 l.ooo


B WEIGHTS 1.372

INTERCEPT CONSTANT ■ 5.460 tea^aaaaaaiaaaaasaasaasisaasgaaaaaaasaaaaaaaaaaaaaaaaaaaaaaaiaaaaaaaaaaaaa MULTIPLE R SOIIARE a 0.1Ö0 MULTIPLE R ■ 0.317 N.O.F.I ■ 2 N.0.F.2 a 151 F FDR ANALYSTS OF VARIANCE ON R ■ 8.408 BETA WEIGHTS -0.524 Ö.9Ö7

CONTRIBUTIONS TO MULTIPLE CORRELATION AND REGRESSION FACTCR LOAOlNGSi 2ND COLUMN) T X -0.138 0.834 ? SQUAR 0.239 0.934

SOUÄREO BETA wpjGHTS 0.275 Ö.AS2

B WFIGHTS -2.724 4.5?fl INTERCEPT CONSTANT a 5.635 aasssssssaaaaaasasaaasaassaaasaaaaaaaaaaaaaaaaaEaaaaaaaaaaaaaaaaaaaaaaaaaaaa MULTIPLE R SOIIARE a 0.1Ö4 MULTIPLE R ■ 6.322 N.O.F.I a 3 N.0.F.2 a 150 F FDR ANALYSIS OF VARIANCE ON R a 5.784 BETA WEIGHTS

0,394 -1.375 1.303 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR LOADINGS. 2NO COLUMN) T X 0.104 0.820 2 SQUAR -0.407 0.918 3 X CURE 0.406 0.968

SQUARED BETA WEIGHTS 0.155 1.R92 1,697

fl WEIGHTS * 2,048 -7,699 7,657

INTERCEPT CONSTANT ■ 5,60°

ANoVA TABLE FOR POLYNOMIALS aaaaeisaaaaiaiaaaaaaaaaaa'aaaaaaaiaaaasaaaaaaaa aaiiaiaaaiaaaaiaaaaaal REDUCTION OUF TO LINEAR FjT, WITH 1 OF a 0,070 RESIDUAL S.S. a 0.930 OF a 152 RESIOUAL M.S. ■ 0.006

F FDR LINEAR FIT a 11.394 aaasassasaaataaaaaaaaaasaaanaBaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaataaaaaaaaa

REDUCTION OUF TO GENERAL QUADRATIC FIT WITH OF 2» a 0,100 REDUCTION M.S. a 0.050 RESIDUAL S.S. a 0.900 OF ■ 151 RESIOUAL M.J, a 0.006

F FOR QUADRATIC FIT a 8.408 REDUCTION DUE TO CUAORATIC TERM ALONE» WITH 1 CFi a 0.030 F FOR OUAORATTC TERM ALONE a S.114

REDUCTION OUF TO GENERAL CuBlC FIT WITH OF 3« a 0,104 REDUCTION M.S. a 0.035 T 19 RESIDUAL S.S. a 0.896 OF ■ 150 RESIDUAL M,J, ■ 0a006 1 -1 £

F FDR GFNERAi. CUBIC FIT a 5.784 REDUCTION OUF TO CU«IC TE«M ALONE WITH 1 OF a , 0,003 F FDR CUBIC TFRM ALONE a 0.5H2 ■aaaaaBaaaaiaaaBaaaigBaanaaaaaaaaiaaardaaaaaaaaaaaaaaaaaBi

POLYNOMIAL CTTTTNO FOR 2 VARIAOLES-GRE AND f? SCORE- 15* CHSEdVATIONS.

THE PREDICTOR VARIAHLE(X) IS ORE,.ANO THE'.CRITERION VAPI*BLE(Y) IS Tp"s'cQRE,

FT RATING

TEST MEANS AND STANOARC OEV/IATIONS 1 X 2 SOllAR 3 X CUflE 4 Y

0.932 0,873 0,822 6.340

CORRELATION MATRIX 1 X 1.000 2 SOUAR 0.997 3 X CUBE 0.991 4 Y 0.374

O.071 0.12S 0.170 2.0*2

0.997 1.000 0.998 0.360

0.99i 0.998 1.000 0.382

0.37* 0.360 0.362 1.000

C3B3=2Caas=ac3aaia3aaaa3aa.azaas:B33s38SBa=^93333i3aiiaaaiaQao«iaatttliafa3i

aaaaaaaaa FIRST, SECOND. ANO THIRO DEGREE POLYNOMIALS, CBBzasBaaosaaaaasaaaasaaaasBEiBaaacssasassBaasassaaasBasBasaaaasaaas MULTIPLE R SQUARE » 0.1*0 MULTIPLE R ■ 0.374 N.O.F.l ■ T N.0.F.2 ■ 152 F FOR ANALYSIS OF VARIANCE ON R " 24.72J BETA WEIGHTS


AND REGRESSION FACTCR LOADINGS. 2ND COLUMN) T x 0.140 l.ooo

SOUARED BETA WEIGHTS 0.140

B WFIGHTS 10.818

INTERCEPT CONSTANT a -3.840 osnssaaacasaasaaasaaaarcsaaaaaaaeaaaanaaaanaaraaataaaaaasBaaaaaasaaaaajaaBaa MULTIPLE R SOIIARE ■ 0.U8 MULTIPLE R ■ 0.385 N.O.F.l ■ 2 N.0.F.2 o 151 F FOR ANALYSIS OF VARIANCE ON R ■ 13.1S6 BETA WEIGHTS -0.926 i.ün4

CONTRIBUTIONS TO MULTIPLE CORRELATION ÄNO REGRESSION FACTCR LOAOINGSI 2ND COLUMN) 1 X -0.346 0.971 2 SOUAR 0,495 0.9§S

SOUAREO BETA WEIGHTS 0.858 1.699

B WFIGHTS -26.793 21.215 INTERCEPT CONSTANT a 12.681 3^aa3 3a3aflnia:sKaaa33at3=3aa3ß5B3aa3saaaB8C3l3l3daa63aaa3Bla3ai3flaaBB38BBB MULTIPLE R SQUARE ■ 0.1«>5 MULTIPLE R ■ 0.406 N.O.F.l a 3 N.O.F.2 a 150 F FOR ANALYSIS OF VARIANCE ON R • 9.871 BETA WEIGHTS -24.807 52.818 -27,746 CONTRIBUTIONS TO MULTIPLE CORRELATION

ANO REGRESSION FACTCR LOAOlNGS, 2N0 COLUMN) ! X -9.278 0.921 2 SOUAR 20.050 0.93S 3 X CUBE -10.607 0.942

SOUÄREO BETA WEIGHTS 615,3702789,744 769,824 B WFIGHTS 717.524 859.598-334,?75 INTERCEPT CONSTANT ■ 199,144

ANQVA TABLE.FOR POLYNOMIALS ia33iai38i>iB3iaii333ii3i><a3>aBa>iga>ii8i33i33iiB3BiB3BBiaaiBaa8aaBai »EDUCTION miF TO LINEAR FIT, WITH 1 OF a 0.140 RESIOUAL S.S. a 0.860 OF a 152 RESlOUAL M,j, ■ 0.006

F F"R LINEAR FIT ■ 24.721

0.148 REDUCTION DUE TO GENERAL OUAORATIC FIT WITH OF 2i ■ REDUCTION M.S. a 0.074 RFSI0U4L S.S. a 0.852 OF • 151 RESIDUAL M,S, ■

F FOR OUAORATIC FIT a 13,156 REDUCTION OUF TO CUAORATIC TERM ALCNEt WITH I CF» a 0,009 F FOR QUADRATIC TERM ALONE a 1.S08

0.006

REDUCTION OUF TO GENERAL CUBIC FIT WITH OF 3» . 0,165 REDUCTION M.S. a 0.059 RESIDUAL S.S. a 0.638 DF a igo RESIDUAL M.S. a

F FOR GENERAL CUBIC FIT a 9.B71 REDUCTION DUF TO CUBIC TERM ALONE WITH 1 OF a f 0,016 F FDR CUBIC TFRM ALONE a ?,959

0.006 1-13

B33aS9cif!ia«3.t0»S8ls8iS3oza«saac9ax9at3as9Ss»isiii9usauaiiatnat)aaMMi3aartHitg4

POLYNOMIAL FJTTtNO *"OH 2 VAHIAÖLES-WRE ANO TP SCOn«.., 154 OBSERVATIONS«

FT RATING

THE PREDICTOR VARIABLE(X) IS WPE., «NO THE CRITERION VAPIARLE<Y) '13 Tf> SCORE!,

TEST MEANS ANO STANDART DEVIATIONS 1 X 0.620 0.190 ? SOUAR 0.420 0.213 3 X CUBE 0.300 0.211 4 ORf

Y »ELATION

6.740 MATRIX

2.0*2

\ X 1.000 0.970 0.919 0.430 ? SOUAR 0.970 1.000 0.98(i 0.388 3 X CUBE 0.419 0.9Ö6 1.000 0.344 4 Y 0.430 0t338 0.344 1.000

BB8al3aiS>flD53SSIII85aS:3SaSsS823a«B3.aiB«SSBS|11flMSI88aiSafDB.II3|lgl|3fK

FIRST, SECONn, ANO THlRO hEGREE POLYNOMIALS. 33a = 3»aB = »BMn = 33308PB333i = = = "03«i8iaoa»»3anKna3iia = 3BaaaarBiaiasa8ia = «8i3«8i«ai8i«8iaaaai

MULTIPLE R SOIIARE ■ 0.1ti5 MULTIPLE R a 0.*30 N.O.F.I « i N.0.F.2 * 152 F FOR ANALYSIS OF VARIANCE ON R * 34.532 BETA WEIGHTS


AND REGRESSION FACTOR LOADINGSI 2N0 COLUMN) T X 0.185 1.000

SOUÄREO BETA WEIGHTS 0.18S

8 WEIGHTS 4,635

INTERCEPT CONSTANT ■ 3.366 ■ S8 = BSCB883a8SSsaS8ls38SS888g8Basaa«sa38Sa3StS3888S8ll8fl8t8iaa8>K8B888338)

MULTIPLE R SQUARE ■ 0.?fl0 MULTIPLE R ■ 0.448 N.O.F.I • i N.O.F.2 ■ 151 F FOR ANALYSTS OF VARIANCE ON R » 18.907 BETA WEIGHTS

0.923 -Ö.5Ö8 CONTRIBUTIONS TO MULTIPLE CORRELATION

»NO REGRESSION FACTOR LPAOINGSI 2ND COLUMN! I X 0,397 0.9&1 ? SOUAR -n.197 0.866

SOU4REO BETA UEI6HTS O.S52 Ö.?SS

B WFIGHTS 9.943 -4.83?

INTERCEPT CONSTANT a 2>lo5 BS833SB3aaBaa:E:38Bas33:3>8aa3BBBaBB3aB3Bflaaa33BBaa38B83a388BB3Ba33333a3aa

MULTIPLE R SOIIARE ■ Ö.2Ä4 MULTIPLE R ■ 0.*5i N,O.F.I ■ 3 N.O.F.2 ■ ISO F FOR ANALYSIS OF VARIANCE ON R * 12.786 BETA WEIGHTS .... ....... .. .

0.328 Ö.9B4 .0,928 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR LOADINGS» 2N0 COLUMN) 1 X 0.141 0.953 \ SOUAR 0.382 0.859 3 X CU8E -0,319 0.762

SOUAREO BETA WEIGHTS 0.107 0.969 0.861

B WEIGHTS 3.531 9.3*5 -8.994

INTERCEPT CONSTANT ■ 2.8,1

ANQVA TABLE FOR POLYNOMIALS

REnUCTtON OUF TO LINEAR FIT, WITH 1 OF ■ 0.185 RESIDUAL S.S. ■ 0.815 DF • 1S2 RESlOUAL M.S. ■ 0.008

F FOR LINEAR FIT a 3*.532

REDUCTION DUE TO GENERAL QUADRATIC FIT WITH DF 2. ■ 0.200 REDUCTION M.S. ■ 0.100 RESIDUAL S.S, » 0.800 OF ■ 151 RESlOUAL M.S. ■ 0.005

F FOR QUADRATIC FIT a 18;907 REDUCTION DUE TO CUAnRATIe TERM ALONE, WITH 1 of, a 0.019 F FOR QUADRATIC TERM ALONE ■ 2.860

REDUCTION DUE TO GENERAL CUBIC FIT WITH OF 3« a 0.20* REDUCTION M.S. ■ 0.068 RESIDUAL S.S. • 0.796 OF a 15o RESIDUAL M.S. a 0.005 1-14

F FnR GENERAL CUBIC FIT a 12.786 REOIICTION DUE TO CUBIC TE«M ALONE WITH 1 OF a 0,003 F FOR CUBIC TFRM ALONE a 0,635

atiiiioCitafliiaait?aafl«a3 3sa3 = asia)iea«*BiaK3avsiaf)iaiiiaii34ai«a4*s9il3iti:«3aaal4

POLYNOMIAL FITTINO FQH 2 VARIAöLES-SHE AND TP SCO»E- SH OBSERVATIONS.

THE PREDICTOR VARIABLE(X) IS SRE> AND THE CRtTERION'VARIABLE<Y) jS,Tp. SCORE,

IC RATING

TEST MEANS ANn STANDARC DEVIATIONS 1 x 0.?66 0.280 ? SQUAR o.i *e 0.2*0 3 X CUBE 0.096 o.ia« 4 Y 5.55? 2.371

CORRELATION MATRIX 1 X 1.000 0.9*1 2 SQUAR 0.9*1 1.000 3 X CUBE 0.B46 0.973 4 Y 0.161 0.3*5

0.846 0.361 0.971 0.34S I.000 0.309 0.3o9 l.noo

oBBBaaEBaiaaaoatsaBajataBssasjoaaaBafaeaaaaEamstaaasaaaaaaaaaaBaaaBaaaBaBaaaaBaaaa FIRST, SECOND, ANO THIRO nEGREE POLYNOMIALS. aessaaaaaBaarfiScaiisssiaa'KiiaaaaisaeBasaiasaasaaaasaaaaaaaiaaiaaaRiiaaaai MULTIPLE R SQUARE ■ 0.131 MULTIPLE R ■ 0,361 N.D.F.l a T N.O.F.? o 54 F FOR ANALYSIS OF VARIANCE ON R ■ 8.4fl7 BETA WEIGHTS


UNO REGRESSION FACTOR LOAOINGSI 2ND COLUMN) i x o.i3i i.ooo


B «FIGHTS 3.062

INTERCEPT CONSTANT ■ 4,738 B;D:3::a3Bia3ai:ais33a3ssaiiE3:aias33aa3aas:aaiasaaaai:3aa>aiaaaa3ii]aaaa> MULTIPLE R SQUARE ■ 0.131 MULTIPLE R » 0,362 N.D.F.l a 2 N.0.F.2 ■ 55 F FOR ANALYSIS OF VARIANCE ON R • 4.136 BETA WEIGHTS

0,320 0.Ö44 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACKR LOäOINGS» 2ND COLUMN) f X 0.116 0.999 ? SOUAR 0.015 0.954

SQUARED BETA WEIGHTS 0.102 0.ÖÖ2

B WFIGHTS 2.711 Ö.474

INTERCEPT CONSTANT «i 4,761 BDaaaaasBaaaasBxiBSBaassBsasaBaaaBaaaaaastaalaaataaaaaaaaaaaaiiaiailagaaaa MULTIPLE R SQUARE • 0.133 MULTIPLE R ■ 6,364 N.O.F.I » j N.0.F.3 ■ 54 F FOR ANALYSIS OF VARIANCE ON R - 2*753 BET» WEIGHTS

0,063 Ö.69P -0,423 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR LOADINGS» 2N0 COLUMN) 1 X 0.023 0.992 2 SOUAR 0.241 0*947 3 X CUBE «0,131 0.847

SQUARED BETA WEIGHTS 0.004 Ö.4R7 0,179

6 WFIGHTS 0.534 T.SOO «5,306

INTERCEPT CONSTANT ■ 4.814

ANQVA TABLE FOR POLYNOMIALS Boaj3»333B3Ba;»il«aua«i:»n»<saa3iB3a3Bisaa.3ainB.n>iBiBi

OEnUCTION OIIF TO LINEAR FIT, WITH 1 OF » 0.131 nESIDUAL S.S. ■ 0.869 DF • 56 RESIDUAL M.S.

F FOR LIN£»9 FIT • P.407 0.016

REDUCTION DUE TO GENERAL QUADRATIC FIT WITH OF 2» ■ 0.131 REDUCTION M.S. « 0.065 «ESIOUAL S.S. ■ 0,869 OF ■ 55 RESlOUAL M.S. ■ 0*616

F FOR OUAORATIC FIT a 4,136 REDUCTION DUE TO QUADRATIC TERM ALONEi WITH 1 C'» ■ 0,000 F FnR QUADRATIC TERM ALONE a 0.014

REDUCTION DUE TO GENERAL CUBIC FIT WITH OF 3a ,. 0.133 REDUCTION M.S. • 0.044 PESIOUAL S.S. ■ 0.867 OF ■ 54 RESlOUAL M.S. • O.Ol«

F FnR GENERAL CUBIC FIT m 2.755 REDUCTION DUF TO CUBIC TERM ALONE WITH 1 OF F FOR CUBIC TFPM ALONE a 0,125

0.00«

1-15

POLYNOMIAL FITTINQ FOR ü VARI AoLES-PRE AM) JP SCOMf- 53 OBSERVATIONS,

THE PREDICTOR VARIABUIXI 13'-PRE, ANO THE CRITeHION VAfl'AfiLECÖ IS'TP SCORE.

IC RATING

TFST MEANS AND STANOARC DEVIATIONS 1 X 0.S08 2 SOUAR 0.414 3 X CUBE 0,350 * Y 5,552

CORRELATION MATRIX 1 X 1,600 2 SOUAR 0,974 3 X CUBE 0.936 ♦ Y 0,322

0.399 0.369 0.3*1 2.371

0.322 0.330 0.362 1,100

iiasaisiisiicsiiiiiisaasssasassssaiiassi

0.976 0.93« 1.000 0.989 0.989 1.000 O.3S0 0.362

eaansssaiaoailiaaiBBsKsaisasiaaiti FIRST, SECOND, AND THlSO (iEGREE POLYNOMIALS, ssaasaaaaaaaasasBaaaaaasaaaasflaaaiiaaaacaaiaaasiiiiBiBaasiisaaiiiaaaifiBaB MULTIPLE R SQUARE ■ 0.1*4 MULTIPLE R ■ 0.322 N.O.F.I ■ f N.0.F.2 » 56 F FOR ANALYSIS OF VARIANCE ON R » 6,493 BETA WEIGHTS


AND REGRESSION FACTCR LOADINGS, 2NO COLUMN) T X 0,104 1.000


B «FIGHTS 1.914

INTERCEPT CONSTANT « 4.S80 caa^aaaaaaaaraasaaaaaaaaaaaaaaaaaiiaaiaaaaasaaaaaaaaaaiaaaaaaaaafaaliiaaaa MULTIPLE R SQUARE • 0.l3l MULTIPLE R » Ö.36? N.D.F.I a 2 N.0.F.2 ■ 55 F FOR ANALYSIS OF VARIANCE ON R ■ 4.142 BETA WEIGHTS -0.417 6.7*7

CONTRIBUTIONS TO MULTIPLE CORRELATION AND REGRESSION FACTOR LQADINGSI 2NO COLUMNI T X -0.134 0.891 ? SOUAR 0,265 0.968

SOUÄREO BETA WEtGHTS 0.174 Ö.S73

B WEIGHTS -2.475 4.9?2 INTERCEPT CONSTANT ■ 4,773 asasaasaaaaaaaaaaaaaaBassaaaaBaBaasasaaaaaaaBaaaaaaasBaaaaaaaaaaaaBBaaaaaB MULTIPLE R SQUARE ■ 0.1-34 MULTIPLE R ■ 0,367 N.O.F.I ■ 3 N.0.F.2 a 54 F FOR ANALYSis OF VARIANCE ON R « 2.796 BETA WEIGHTS

0,378 -i,2«.i 1.246 CONTRIBUTIONS TO MULTIPLE CORRELATION

JND REGRESSION FAcTCR LOAOINGS» 2ND COLUMN) 1 X 0.122 0.879 ? SOUAR -0,438 0.95S 3 X CUBE 0.451 0.987

SOUAREO BETA WEIGHTS 0,143 1,5*5 1.552

B WEIGHTS 2.24S -8.TT6 8,666


ANOVA TABLE.FOR POLYNOMIALS

REDUCTION OUF TO LINEAR FIT, WITH 1 OF ■ 0.104 RESIDUAL S.S. « 0.896 OF ai 56 RESIDUAL M.S. ■ 0.016

F FOR LINE4R riT a 6,493 aaagaaaaaaaaaaaaaaaaaaaaaaiaaaai

0.131 REDUCTION DUE TO GENERAL QUADRATIC FIT WITH OF 2, « REDUCTION M.S. • 0.065 RESIDUAL S.S. « 0.A69 OF ■ 55 RESlOUAL M,S, ■.

F FOR QUADRATIC FIT ai 4,142 REOIICTION OUF TO QUADRATIC TERM ALONE, WITH 1 C?t ■ 0.027 F F^R OUAORoTIC TERM ALONE • 1.709

0.018

REDUCTION DUE TO GENERAL CUBIC FIT WITH OF 3« • 0,134 REDUCTION M.S. - 0.045 RESIDUAL S.S. ■ 0.B66 DF ■ 54 RESIDUAL M.S. •

F FOR OENEPAl. CUBIC FIT ■ 2.796 REDUCTION OUF TO CUBIC TE«M ALONe WITH J OF « , F FOR CUBIC TFRM ALONE « 0.222

0,004

0.016 1-16

POLYNOMIAL, FITTINO FON Ü VARI Antes-QBE AND TP SCORE- 58 OBSEUVATIOKS,

IC RATING THE PREDICTOR VARIARLE(X) IS ORE, AND THE CRITERION VARIABLE<V) IS Tp SCOPE.

TEST MEANS AND 5TANOAH0 DEVIATIONS 1 X ft,886 0.16I 2 SOUAR O.flll 0.2M 3 X CUBE 0.751 0.261 * Y 5,552 2.3/1

CORRELATION MATRIX 1 X l .000 0.954 0.901 0.387 2 SOUAR 0,954 1.000 0.9B9 0.388 3 X CUBE 0.90I 0.989 l.OOO 0.381

,4 V . 0.387 0.388 0.3«l I.000 iaiaiasiaaiiciasssiiaiisaiaBssstsaisstiiiiiaiiiiisisigiHMiiiDcj,),,,]

FIRST, SECONO, ANO THIRD ÖEGREE POLYNOMIALS. caBsaasssasassssssiaasxssaJaaBaBsaaBaaaaaaaaataaaasssasaaaBaaasaaaaaaaaaaaasa MULTIPLE R SOIIARE • 0.J49 MULTIPLE R a 0.387 N.O.F.I ■ T N.O.F.2 a 56 F Fo« ANALYSTS OF VARIANCE ON R ■ 9.838 8FT» WEIGHTS


AND REGRESSION FACTCR LOAOINOSI 2ND COLUMN) T X 0,149 l.OOO

SOUÄRED BETA WEIOMTS 0.J49

B WEIGHTS 5.706

INTERCEPT CONSTANT » 0.494 aaaaaaaaagaiaaaaaaaaaaaaaaataaaaaiicaaiiaaiiaaaaiaaaaaiaiaaaiiaaiaaaiaiaM MULTIPLE B SOIIARE ■ 0.*s4 MULTIPLE P ■ 0.392 N.O.F.I ■ 2 N.0.F.2 a 5S F FOR ANALYSIS OF VARIANCE ON R • 4.995 BET» WEIGHTS

0.178 6.219 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND «EGRESSION F*cTCR LOAOINGSI 2N0 COLUMN) T X 0.069 0.9Ü6 2 SOUAR 0.085 0.991

SOUÄREO BETA WEIGHTS 0.032 0,048

B WFIGHTS 2.626 2.4T9

INTERCEPT CONSTANT o 1.263 aaaasasasaiaaaasaBaaaasaaaisaaaaasaBasaaiaasiaaasoaaiasiiliaiMiiaaiiaaiai MULTIPLE R SOIIARE » 0.169 MULTIPLE P ■ 0.411 N.O.F.) "3 N.O.F,2 a 54 F FOR ANALYSIS OF VARIANCE ON P ■ 3.668 BETA WEIGHTS

1.813 -4.772 3.467 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR LOAOINGS» 2ND COLUMN) j t 0,701 0.939 ? SOUAR -1.853 0.944 3 X CUBE 1,322 0.927

SOUARED BETA WEIGHTS 3.287 22.7A7 12.023

B WFIGHTS ?6.763 -52.768 31,519 INTERCEPT CONSTANT ■ 0*953

ANOVA TABLE FOR POLYNOMIALS aaaaaBaaaaaaaaasaaaaaaaaaaaaaaaaaaaaaaaaaaaaiaaaaaaaaBaBataaaaaaaaaaaliiBa REOUCTION OIJE TO LINEAR FfT, WITH 1 OF ■ 0.149 RESIDUAL S.S. ■ 0.851 OF ■ 56 RESlOUAL M.S. n 0.015

F FOR LINEAR FIT » 9.838 BaaiaaaaaaaaaaaaBaaaaBaaaaaaaaaaaaaaaaaBsaBBaeaBaaaasaaBBBBaaBBaBBilaflaaBa

REOUCTION ODE TO GENERAL QUADRATIC FIT WITH OF 2» ■ 0.134 REOUCTION M.S. ■ 0.O77 RESIDUAL S.S. ■ 0.846 DF ■ 5S RESlOUAL M.S. ■ 0.015

F FOR QUADRATIC FIT a 4«995 REDUCTION DUE TO CUADRATIC TERM ALONEi WITH 1 OF» » 0.004 F FOR OUAORATTC TERM ALONE • 0.280 BBaaaaaaaaaaaaaiaaaasaaBBBiBaaaaaBaBaBaBaaaBaiaaBaaaaaaiBBBBBaaaaaaaaaaflBa'

REDUCTION OUF TO GENERAL CUBIC FIT WITH OF 3i . 0,169 _ '_ REDUCTION M.S. ■ 0,056 1-17 RESlOUAL S.S. ■ 0.B31 DF ■ 54 RESlOUAL M.S. • 0.013

F FoR OENERAl. CUBIC FIT " 3.66« REOUCTION DUE TO CUHIC TERH ALONE WITH 1 OF ■ , 0,01« F FOR CUHIC TFRM ALO^E ■ 1.012

POLYNOMIAL KITTING FOH 2 VAHIA^LES-WRE ANO TP SCORE- 5A OBSERVATIONS,

TH€ PREDICTOR VARIARLE(X) IS WRE, ANO THE CRITERION VARIABLE(Y) IS TP SCORE.

IC RATING

TEST MEANS ANO STANOARC DEVIATIONS 1 X ? SOUAR 3 X CUBE 4 Y

0,5*2 0.129 0.?16 S.S52

CORRELATION MATRIX 1 X 1.000 2 SQUAP 0.982 3 X CUBE 0.942 * Y 0.434

0.190 0.207 0.189 2.371

0.982 1.000 0.988 0.382

0.942 0.98A 1 .000 0.343

■aaasstaaaaaagaszaaxisaassasaztiBssasaaaasasrasg FIRST, SECOND, ANO THIRD nEOREE POLYNOMIALS. ■oBaasBaaacasaasssBsaaasaasaasaaaaasssaiiKaasaaa MULTIPLE R SOIIARE « MULTIPLE H N.O.F.I ■ N.O.F.?

aaaaaa

aaaaaa o.iea

0.43* 0.382 0.1*3 1.000

aaaaaa

aaaaaa

aaaaaa

aaaaaa

aaaaaaaaa

latadaaai

1 0.43*

56 12.997 F FOR ANALYSIS OF VARIANCE ON R ■

BET» WEIGHTS 0,43*

CONTRIBUTIONS TO MULTIPLE CORRELATION AND REGRESSION F»cTCR LOäOINGSI 2N0 COLUMN» T X 0.1e8 1,000


0 WEIGHTS S.*26

INTFRCEPT CONSTANT a 2,Ml aiBaaaaaaaaaaaesaaaiaaaaaai MULTIPLE R SOIIARE ■ 0.2*1 MULTIPLE « ■ 6.491 N.O.F.l a 2 N.O.F.2 a 55 F FOR ANALYSIS OF VARIANCE ON R ■ BET» WEIGHTS

1,616 -1.2Ä4 CONTRIBUTIONS TO MULT

AND REGRESSION FAC I X 0. ? SOUAR -0.


B WFIGHTS 20.199 -13.A*9 INTERCEPT CONSTANT a aaaaaaaaaaaassaaaaaaa

8t746

IPLE CORRELATION TCP LOADINGS» 2ND COLUMN) 701 0.8Ö* 460 0.778

-0.852 aaaaaaaaoaaaaaaaaa

o.f:?2 aaaaaaaaaaaaaaaaaasaaaaaasBaSaaasaa

MULTIPLE R SOIIARE ■ MULTIPLE « • 0a5*I N.D.F.I a 3 N.O.F.2 a 54 F FOR ANALYSIS OF VARIANCE ON R BET» WEIGHTS

5,460 -9,93* 5.Ö CONTRIBUTIONS TO MULT

»NO REGRESSION FAc i x 2. r> SOUAR -3. 3 X CUBF 1.

SOUÄREO BETA WEIGHTS 29.815 9A.AR7 25.138

B WEIGHTS 6A.262-113.94fl 63,629 INTERCEPT CONSTANT ■ -7.572

7.4*1

)1« IPLE CORRELATION TCR LOADINGS! 2ND COLUMN) 370 0.803 796 0.706 718 0.633


REOUCTtON DUE TO LINEAR FIT, WITH 1 OF a 0.188 RESIDUAL S.S. a 0.812 OF ■ 56 RESIDUAL M,$, a 0.01*

F FOR LINEAR FIT a 12,997

REDUCTION DUF TO GENERAL OUAORATIC FIT WITH OF 2, • 0.2*1 REOUCTION M.S. • 0.121 RESIDUAL S.S. a 0.75« OF ■ 55 RESiOUAL M.S. ■

F FOR QUADRATIC FIT a fl;746 REDUCTION 0UF TO QUADRATIC TERM ALONE» WITH 1 OF» « 0.053 F FOR QUADRATIC TERM ALONE a 3.836

0-014

REOUCTION DUE TO GENERAL CUBIC FIT WITH OF 3» a 0.2<>2 REDUCTION M.S. a 0.097 RESIDUAL S.S. a 0.708 OF ■ 5* RESIDUAL M.S. a

F FOR GENERAL CUBIC FIT a 7,**1 REDUCTION OUF TO CURtC TERM ALONE WITH \ OF « F FOR CUBIC TERM ALONE a 3.9()A

0,051

0.013 1-18

POLYNOMIAL FITTINO FOR 2 VARIABLES-SHE ANÜ TP SCOW«. 139 OBSERVATIONS.

THE PREDICTOR VARIAHLE(X) IS SPE. AND THE CRITERION, VAflTABLEjY) tS TP SCORE,

RD RATING

TEST MEANS AND STANOAHO DEVIATIONS 1 X 0.3*2 ? SOUAP 0«?*5 3 x CUBE o.ios * Y Z.flO«

CORRELATION MATRIX 1 X 1.600 ? SQUAR 0.977

'.3 X CUBE 0.930 4 Y -ft,j68

0.358 0.283 0.232 1.564

0.977 1.000 0.966 O.llO

0.930 0.98* 1.000

-0.053

•0.168 -0.110 -0.093

1.000 ■ txsxesctaai FTRST. SECOND, ANO TWIRO ÖEGREE POLYNOMIALS,

MULTIPLE R SOtlARE ■ 0.028 MULTIPLE R ■ 0,168 N.O.F.I a I N.0.F.3 ■ 137 F FOR ANALYSTS OF VAQIANCE ON R ■ 3,979 8FT» WEIGHTS -0.1A8

CONTRIBUTIONS TO MULTIPLE CORRELATION AND REGRESSION FACTOR LOAOlNGSt 2NO COLUMN) 1 V 0,028 -1.000


B WEIGHTS -0.733

INTERCEPT CONSTANT » 3.057 • Mi:s3i3i»93i:3ai»iiiii»im3als»3i,iaii,I>iiiisgagaa||||iilla,aui|ia

MULTIPLE R SQUARE » 0,093 MULTIPLE R * 0.305 N.O.F.I ■ 2 N.0.F.2 a 136 F FOR ANALYSIS OF VARIANCE ON R ■ 6,961 BETS WEIGHTS -1.326 l.T«5

CONTRIBUTIONS TO MULTIPLE CORRELATION AND REGRESSION FACTOR L°«OlNGSt 2ND COLUMN) T X 0.223 -0.551 ? SOUAR -0.130 -0.360

SOUÄREO BETA WEIGHTS 1.758 i.iftS

B WEIGHTS -5.788 6.559 INTERCEPT CONSTANT • 3«182 ■a,D.*a,oisaai«3 9ll9iia33aisiloaa*»3saiaaiiagaaaaiaiaaaaaaaK>laaaaaaaaaaa

MULTIPLE R SQUARE ■ 0.131 MULTIPLE R ■ 0.362 N.O.F.I "3 N.0.F.2 a 135 F FQR ANALYSIS OF VARIANCE ON R * 6.801 8FTÄ WEIGHTS

1,065 -4,4.16 3.332 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTOR LOAOlNGSt 2N0 COLUMN) 1 X -0.179 -0.4«4 3 SQUAR 0,486 -0,302 3 X CUBE -0.176 -0.1*6

SOUÄREO BETA WEIGHTS 1,133 19.677 11.102

B WEIGHTS 4.647 -24,544 22,430

INTERCEPT CONSTANT ■ 3*138

ANQVA TABLE FOR POLYNOMIALS

REDUCTION DUE TO LINEAR ElT, WITH 1 OF » 0.028 PESIOUAL S.S. ■ 0.972 OF a 137 RESlOUAL M.S.

F FOR LINEAR FIT • 3.97» 0.007

PEOUCTION OUF TO GENERAL OUAORATIC FIT WITH OF 2, • 0.093 REOtlCTlON M.S. ■ 0.0*' PFStDUAL S.S. - 0.907 OF ■ 136 RESlOUAL M,j, ■

F FOR OUAORATIC FIT ■ 6.961 REDUCTION DUE TO OUAORATIC TERM ALONE» WITH 1 tf, a 0.065 F FQR QUADRATIC TERM ALONE ■ 9,690 ■■■■■■■■■■■■■■■"'•■■""'•■■»••"•i"«»"""»"««'»»»"»»»»»»««»««««»»««««»

REDUCTION DUE TO OENERAL CUBIC FIT WITH DF 3, a 0,131 PEOUCTION M.S. ■ 0,0** RESIDUAL S.S. ■ 0.86« DF ■ 135 RESlOUAL M,J, a

F FOP GFNERAL CUBIC FIT a 6.A01 RFOIICTION OUF TO CUBIC TER* ALONE WITH 1 OF • , F FOP CUBIC TFRM ALONE • 5,972

0,038

0,007

0.006 1-19

■ o»s»aaii»»ic»»3B«ii»»iiiiO««3|iiaoa»«aB«aa»a»oa»»»Bm»B»o««»ll9«»«««»»iJBaai>»»«a«»» POLYNOMIAL FTTTING FDR 2 VAHIA^LES-PRE AND TP SCORE- 139 OBSEPVATIONS.

THE PREDICTOR VARIABLE (X) 'IS PRE, AND THE CR.J TP.RION VÄPI ABLE (Y> . IS TB SCORE,

RD RATING

TEST MEANS AND STANDARD DEVIATIONS 1 X ? SOUAR 1 X CUBE A V

0.475 0.4*7 0.429 2.A06

CORRELATION MATRIX 1 X 1.000 ? SOUAR 0.994 3 X CUBE 0.982 A Y - o. ? 15

0.473 0.456 0.444 1.564

0.99* 1.000 0.997

■0.213

0.98J 0.997 1 .000 0.20S

aaaaaasaaaa:

IALS.

6.612

Q933aisia8alses33iS3aaii!B3isca3»-fl3iS3

FIRST, SECONO, ANO TWI90 SFGREE POLYNOM oaaaaiaasaaaasasaaiaaaassaBasaaaaaaasta

MULTIPLE R SOIIARE ■ 0.046 MULTIPLE R ■ 0.215 N.O.F.I • f N.0.F.2 ■ 137 F FOR ANALYSIS OF VARIANCE ON R ■ 8FT» WEIGHTS -0,215

CONTRIBUTIONS TO MULTIPLE CORRELATION ÄNO REGRESSION FACTOR LOAOINGS. 2N0 COLUMN) T X 0,046 -1,000


B WEIGHTS -0.710 INTERCEPT CONSTANT » 3.143

•0.219 -0.213 ■fl.208 l.flOO

aaaaaia

uasasoaaoaanaaBoaaaa

allnaaa

: aa anaaaa

3.282

asiai asssaiaiaaaaaaa'aaa ana

MULTIPLE R SOIIARE ■ 0.0+6 MULTIPLE f> » 0.215 N.O.F.I ■ j» N.O.F.2 ■ 136 F FOP ANALYSIS OF VARIANCE ON R * BETA WEIGHTS -0,204 -0.Ö10

CONTRIBUTIONS TO MULTIPLE CORRELATION ÄNO REGRESSION FACTOR LOADINGS« 2ND COLUMN) T X 0,044 -1,000 ? SOUAR 0,002 -0,994

SOUÄREO BETA "EIGHTS 0.042 0.ÖÖ0

B WFIGHTS -0.676 -0.0^6 INTERCEPT CONSTANT " 3,1*3

aaaaaaBaaaaaaaaaaatiaKiaaaiaaaasai

iBaaaaaBsaaaaaaaaaaaaaaiaaaiaaaaaaiataaaaaaaiaaaaaa 0,093

4.392

BsaaBaaaaaaaasaaBaaaaa

MULTIPLE R SOIIARE « MULTIPLE R ■ 0,304 N.O.F.I ■ 3 N.0.F.2 ■ 135 F FOR ANALYSIS OF VARIANCE ON R BET« WEIGHTS 10.549 -25.903 13.252

CONTRIBUTIONS TO MULTIPLE CORRELATION AND REGRESSION FAcTCR LOADINGS» 2N0 COLUMN) T X -2.263 -0.705 ? SOUAR 5.527 -0.701 3 X CUBE -3,171 -0.683

SOUAREO BETA WEIGHTS 111.2B3 67Ö.OS1 232,638 B WEIGHTS 34.915 -8fl.B«fl 53,730 INTERCEPT CONSTANT - 3*132

ANOVA TABLE FOR POLYNOMIALS ■ aaa3B3io»iaaisiiaassasaBsaas8Si REDUCTION OUF TO LINEAR FlT, WITH 1 OF ■ 0.046 RESIOUAL S.S. ■ 0.954 DF ■ 137 RESIDUAL M.S.

F FOR LINEAR FIT • 6,612 0,007

REDUCTION DUF TO GENERAL QUAORATIC FIT WITH OF 2. » 0.0+6 REDUCTION M.S. - 0,023 RESIDUAL S.S. ■ 0.9S4 OF ■ 136 RESIOUAL M,S, ■ 0,n07

F FDR QUADRATIC FtT ■ 3«2B2 REOllCTION DUF TO QUADRATIC TERM ALONE. WITH 1 I>F, ■ OaOOO F FDR OUADRATTC TERM ALONE ■ 0.000

REDUCTION OUF TO GENERAL CUBIC FIT WITH OF 3t a 0,093 REOllCTION M.S. ■ 0.031 RESIDUAL S.S. ■ 0.907 OF • 135 RESIOUAL M.S. ■ 0.007 1-20

F FDR GFNERAi. CUBIC FIT » 4.592 REDUCTION OUF TO CUBIC TE«M ALONE WITH I OF ■ F FOR CUBIC TFPM ALONE » 6,9?6

0.0*7

■ B3!aaaaBai>atia/itiaaia3uaaaa3aaeaaaaaz<«aiaaaaa88aa3»a.iaaaaaB

POLYNOMIAL FITTING FOR 2 VARIAbLES-GRE AND TP SCORE. 139 OBSERVATIONS,

RD RATING THE PREDICTOR VARIABLE(X) IS ORE, AND THE CRITeRION VARlABLE(Y) IS TP SCORE,

TEST MEANS ANn STANDAfiC DEVIATIONS I X ? SQIIAR 3 X CURE * V

fl.069 0.9*3 0,919 2,«06

0.054 0.091 0.119 1.5<>*

CORRELATION MATRIX 1 X 1.000 2 SOUAR 0.992 3 X CUBE 0.972 4 Y 1.Ö21

■IBIlillll

■»■•Illllt

0.992 0.972 0.021 1.000 0.994 o.no* 0,994 1,000 -*o.no3 0.006 ■0,00'! 1.O00

l3H33S9Bflll3VS3eSS&.33BI=8ll98ia33338 = aaS»SMa3B93aBiaB&lia,lla

FTRST, SECOND, AND THIRD nEGREE POLYNOMIALS. BessBSBssaaaassessassasEssaBaaaaaaaasaaBaaaaasaaBasBaaaBsaaaaiaa

MULTIPLE R SOIIARE » O.OnO MULTIPLE R a 0.021 N.O.F.I a T N.O.F.2 a 137 F FOR ANALYST«! OF VARIANCE ON R ■ 0*060 BETA WEIGHTS


AND REGRESSION FACTCR LOADlNGSa 2ND COLUMN) T X 0.000 1.000


B WEIGHTS 0.603

INTERCEPT CONSTANT • 2.222 BBa3t3BD3BS,3=333BlS3BS3Ba38BB8333333B3383333saB33338B3B33333333BSaaBaaaaB MULTIPLE R SOIIARE " 6.013 MULTIPLE R ■ 0.113 N.O.F.i ■ 2 N.0.F.2 si 136 F FOR ANALYSIS OF VARIANCE ON R ■ 0.873 BET« WEIGHTS

0.871 -0.857 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR IOAOINGS» 2N0 COLUMN) 1 x o.oia o.ias ? SOUAR -0.006 0.057

SOUÄRED BETA "EIGHTS 0.759 Ö.735

B WEIGHTS 2S.145 -14.818 INTERCEPT CONSTANT a -7.6o2 83B3333aa3:B338a88B333833aa33aa83833aaaSB83Saa3

MULTIPLE R SOIIARE ■ 0.047 MULTIPLE R N.O.F.I a N.0.F.2

333338883383388380:88383338

0a217

135 2.218 F FOR ANALYSIS OF VARIANCE ON R ■

BETA WEIGHTS 39.917 -85.574 46.272

CONTRIBUTIONS TO MULTIPLE CORRELATION ÄNO REGRESSION FACTCR LOADINGSI 2ND COLUMN) } X 0.833 0.096 2 SOUAR -0.550 O.o3o 1 X CUBE -0.236 -0.02*

SOUÄRED BETA WEIGHTS 593,4047322.9562141.134 8 WEIGHTS 152.251«»«»«.)«» 610,24* INTERCEPT CONSTANT • -280.67*


REDUCTION DUE TO LINEAR ElT, WITH 1 OF • 0.000 RESIDUAL S.S. ■ 1,000 DF a 137 RESIOUAL M.S. • 0.00?

F FOR LINEAR FIT • 0,060

0.013 REDUCTION DUE TO GENERAL QUADRATIC FIT WITH OF 2. ■ RFOllCTION M.S. a 0,006 RESIDUAL S.S. a 0.987 OF a 136 RESIOUAL M,S, a

F FOR QUADRATIC FIT a 0.873 REDUCTION OUF TO CUAORATIC TERM ALONE. WITH 1 rjF» ■ 0.012 F FOR QUADRATIC TERM ALONE a 1,686

0.007

REDUCTION DUE TO GENERAL ?UBlC FIT WITH OF 3i a 0.0*7 REDUCTION M.S. a 0.O16 RESIOUAL S.S. a 0.933 OF a 139 RESIOUAL M.S. ■

F FOR GENERAL CUfltC FIT • 2.218 REOHCTION DUE TO CUBIC TERM ALONE WITH 1 OF a F FOP CUBIC TERM ALONE a 4,858

0,03*

COOT 1-21

POLYNOMIAL FITTING FOR Z VARIAöL£S-WRE ANO TP SCO«£« 13« OBSERVATIONS.

THE PREDICTOR VARIARLE(X) IS WRE, ANO THE CRITERION V/RIABLEIY) IS TP SCORE,

RD RATING

TEST MEANS ANO STANOARC OEVIATIONS 1 X 0.A17 0.183 ? SOUAR 0.41* 0.227 3 X CUBE 0.29« 0.332 A ORf

Y (ELATION

2,«Oft MATRIX

1.564

1 X 1.000 0.983 0.949 •O.nSl 2 SOUAR 0,983 1.000 0,990 -0.073 i X CUBE 0,444 0.990 1.000 •0,095 A Y -O.oSl -0.0/5 -0.083 1.000

llS3aaSl81fl33ISI3Sl3ll£XBI,|iaStllSSZ2s»15,x|||SISI,l3altB,|inR|a|||g FIRST, SECOND, ANO THIRD ÖEGREE POLYNOMIALS.

MULTIPLE R SOIIARE " O.0Ä3 MULTIPLE R » O.OST N.O.F.l ■ 1 N.0.F.2 ■ 137 F FOB ANALYSTS OF VARIANCE ON R • 0.362 BET» WEIGHTS •0.051

CONTRIBUTIONS TO MULTIPLE CORRELATION AND REGRESSION FACTCR LOADINGS. 2N0 COLUMN) T X 0.003 -1.000


B WFIGHTS -0.438 INTERCEPT CONSTANT » 3?076 »» » '•"."S>3:>"."<l">e»<M3Hai3siagnlli||gl|ai,|ia3|

MULTIPLE R SOIIARE ■ 0.021 MULTIPLE R ■ 0.145 N,O.F.I ■ 2 N.0.F.2 ■ 136 F FOR ANALYSIS OF VARIANCE ON R ■ 1.457 8FT« WEIGHTS

0,669 -6.7:13 CONTRIBUTIONS TO MULTIPLE CORRELATION

ÄNO REGRESSION FspTCR LOAOINGS» 2N0 COLUMN) T X -0,034 -0.3->* 2 SOUAR 0.05S -0.521

SQUARED BETA '-'EIGHTS 0.448 0,538

H WEIGHTS 5.715 -5.Ä44

INTERCEPT CONSTANT ■ 1.368 ■3333&33S3aaaasBnB8iis333a.n,«aaa33Ba3«233333gx3a33Bgg33oai3igaB«3f.aa3- MULTIPLE R SQUARE ■ 0.041 MULTIPLE R ■ 0.204 N,O.F.I »3 N.0.F.2 . 135 F FOR ANALYSIS OF VARIANCE ON R " 1.946 BET* WEIGHTS

3,072 -6,399' 3,337 " " ' CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR L°AOlNGS» 2ND COLUMN) T X -0.1S8 -0.252 2 SOUAR 0,483 -0.371 3 X CUBE -0.284 -0.418

SOUÄRE0 BETA WEIGHTS 9.439 40,947 11.134

B WFIGHTS 26.233 -44,ni7 22,482 INTERCEPT CONSTANT ■ -1.814


REOUCTION OUF TO LINEAR FIT» WITH 1 OF ■ 0.003 RESIDUAL S.S. ■ 0.997 OF ■ 137 RESlOUAL M.S.

F FOR LINEAR FIT ■ 0.362 0.007

REDUCTION OUF TO GENERAL QUADRATIC FIT WITH OF 2. • 0.021 REOUCTION M.S. ■ 0,010 RESI0U4L S.S. a 0,97? OF ■ 136 RESlOUAL M.j, a 0f007

F FOR OIIAORATIC FIT ■ 1.457 REOUCTION OUF TO CUAORATIC TERM ALONE» WITH 1 CF« ■ 0.018 F FOR OUOORATIC TERM ALONE ■ 2.549

REOUCTION DUE TO GENERAL CUBIC FIT WITH OF 3» <■ 0.041

REOUCTION M.S. ■ 0,01* RESlOUAL S.S. • 0.959 OF ■ 135 RESlOUAL M.j, • 0.007

F FnR GENERAL CUBIC FIT ■ 1.9*6 REOUCTION OUF TO CUmC TE°M «LONE WITH I OF ■ , 0,020 F FOR CUBIC TfPM ALONE ■ 2.B83

1-22

on«MÄa**»»aa»aa«a»:iwa«**aa»rt.iBJi«»aJa«»Ka33äaaO'3jj3!»aiat:'it»aaia«a«»(aa«i*-3Biia)»?ii»ai

POLYNOMIAL fltriNO FOR 2 VAHIAOLES-SRE AND TP SCOUR- 137 OBSEOVATIONS.

THE PREDICTOR VARIARLE(X) IS SRE, AND THE CRITERION VARIABLEM) IS TP SCCHE,'

RM RATING

NOARC DEVIATIONS 0.628 0.331 0.S03 0.348 0.42« 0.3S6 4.4S3 1.92S

1.000 0.960 0.960 1.000 0.907 0.9Ö8 0.?U 0.23* laetiisBSOsiBstaassaai

THlRO OEGREE POLYNOM

• 0.044 2ii

35 VARIANCE ON R B

0.907 0.988 1.000 0.244

eassatsiasss

IALS. BsisBiaiiiaa

0.21 0.23 0.24 1.00

saaaaaaaaaaaaa

aaaaaaaaa

aaaaaaaaa

6.285

ULTIPLE CORRELATION FACTCR LOAOINOSI 2N0 COLUMN) 0.044 1.000

TS

a 3.681 saaassaaBcataasssaaaaa asaaaasaaaaaaaaaaaaaaaaaaaaaaaaaaaa

TEST MEANS ANn STA 1 X 2 SOUAR ■* X CURE 4 Y

CORRELATION MATRIX i x 2 SOUAR 3 X CURE * Y Bassaaaaaaaaasaaaa

FIRST. SECOND, AND aaaaaaaaaaaaaaaaaa MULTIPLE R SOUARE MULTIPLE R ■ 0. N.O.F.l a T N.D.F.? a i F FOR ANALYST«? OF BFT» WEIGHTS

0,211 CONTRIRUTIONS TO M

ÄND REGRESSION T x

SOUÄREO BETA WEIGH 0.044

B WFtGHTS 1.228

INTERCEPT CONSTANT asasaaaaaaaasassaa MULTIPLE R SQUARE ■ 0.0S7 MULTIPLE R ■ 0.238 N.D.F.l a j N.O.F.2 a 13A F FOR ANALYStS OF VARIANCE ON R » 4.034 BFTo WEIGHTS -0.168 Ö.1R4

CONTRIBUTIONS TO MULTIPLE CORRELATION UNO REGRESSION FACTCR L°AOlNGS» 2N0 COLUMN) T X -0,035 0.8*15 ?. SOUAR 0.092 0.980

SQUARED BETA WEIGHTS 0.028 Ö.1S6

B WEIGHTS -0.976 2.1S1 INTE"CEPT CONSTANT a 3,968 asaaaasBaasaaasaaaBaaaaaaBaaaaaaaaaaaaaaasaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa MULTIPLE R SOUARE ■ 0.068 MULTIPLE R a 0.260 N.O.F.l a 3 N.D.F.2 a 133 F FOR ANALYSIS OF VARIANCE ON R B 3.218 BETA WEIGHTS

0,746 «2.231 1.770 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTOR LOAOINGS» 2N0 COLUMN) T X 0,157 0.811 3 SOUAR -0,521 0.898 i X CUBE 0.431 0.937


R WFIGHTS 4.346 -12,317 9.567

INTERCEPT CONSTANT a 3.857

ANOVA TABLE FOR POLYNOMIALS aaaaaaaaaBsaaaaaaaaaaaaasaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaBBBaBBaaaaaaaaaaaa REDUCTION DUF TO LINEAR FlT, WITH 1 OF a 0.044 RESIDUAL S.S. B 0,956 OF a 135 RESlOUAL M.S. a 0.007

F FnR LINFAR FIT a 6,285 aaaaaBaasaBaaaasaaaaaaaaBaaaaaaaaaaasaaaaaaaBaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

0.037 REDUCTION DUF TO GENERAL QUADRATIC FIT WITH OF 2. ■ REDUCTION M.S. a 0.028 uFSlDUAL S.S. a 0.943 OF a 134 RESIDUAL M.S. •

F FOR QUADRATIC FIT a 4*034 REOIICTION DUE TO QUADRATIC TERM ALONEi WITH 1 T)F» ■ 0.012 F FoR QUADRATIC TERM ALONE a 1.748

O.flOT

RFOllCTION DUF TO GENERAL CUBIC FIT WITH OF 3» a 0,0*8 REDUCTION M.S. a 0.023 RESIDUAL S.S. B 0.932 OF ■ 133 RESIDUAL M,«, a

F FOR GENERAL CUBIC FIT a 3.218 RFOllCTION DUE TO CUHTC TERM ALONE WITH 1 OF • , 0,011 F FOR CUBIC TFPM ALONE a 1.554

0.007 1-23

POLYNOMIAL FTTTINO FOR 2 VARIA^LES-PHE ANO TP SCO«E* 137 OBSERVATIONS,

RM RATING THE PREDICTOR VARIABLE(X) IS PRE, AND THE CRITERION VAOIABLE(Y) IS TP SCORE.

TFST MEANS AND STANDARO DEVIATIONS 1 X O.flOO 0.341 ? SOUAR 0.756 0.342 3 X CURE 0.719 0.348 4 OR»

Y »ELATION

4.453 MATRIX

1.925

1 X 1.600 0.985 0.953 0.156 ? SOUAR 0.983 1.000 0.991 0.16* 1 X CUBE 0.953 0.99J 1.000 0.16T 4 Y 0 .15« 0.104 0.167 1.000

toiaiicD3ai9::3:i:>ai3::»>ciS39i"3;33ii:iiii-39ii-3ia93ailiiai3iiaiiI,i,I,a

FIRST, SECONn, AND THIRO ÖE6REE POLYNOMIALS. ■sa9a93B39aa3 9SCSflBBaBasflBa>BBB3SSaasxaaaadaagaiaaaa]i33eiBi9iai4sasiiaaa«i MULTIPLE R SOIIARE « 0.024 MULTIPLE R ■ 0.156 N,O.F.I ■ T N.D.F.2 ■ 13<5 F FOR ANALYSIS OF VARIANCE ON R ■ 3.387 8FT» WEIGHTS


AND REGRESSION FACTCR LOADINGS, 2N0 COLUMN) T X 0.024 1.000


B WFIGHTS O.fl«?

INTERCEPT CONSTANT ■ 3«747 33=:a33a33aa3::3a3aB3a3333aB3aa9B>3393a8aa:3a3aa3aa3aaa:aaa33agaa3B9aa339t MULTIPLE R SOIIARE ■ 0.028 MULTIPLE R ■ 0.166 N.O.F.I ■ ? N.O.F.2 »134 F FOR ANALYSIS OF VARIANCE ON R ■ 1.902 BET» WEIGHTS -0.159 0,3?6

CONTRIBUTIONS TO MULTIPLE CORRELATION AND REGRESSION FacTCR LOADINGS. 2ND COLUMN) T X -0.025 0.9*2 j? S*UAP 0,052 0.986

SQUARED BETA «EIGHTS 0.025 0.i«2

B WEIGHTS -0.894 l.BM INTERCEPT CONSTANT » 3,8o7 ■Bag99Ba939C9aa=aaa9Ba3ss9a93e89a3ag93a93a9a3383a3B9aaas9«B3a83aa33S9aa333 MULTIPLE R SOUARE ■ 0.631 MULTIPLE R ■ Ö.177 N.O.F.l ■ i N.O.F.2 ■ 133 F FOR ANALYSIS OF V4RIANCE ON R ■ 1.428 BETA WEIGHTS

1.852 -4.3*6 "2.71Ö *""•" CONTRIBUTIONS TO MULTIPLE CORRELATION

ÄNO REGRESSION FAcTCR LOAOINGS» 2N0 COLUMN) 1 X 0.290 0.fl«6 ? SOUAP -0.712 0.927 3 X CURF 0.453 0.947

SQUÄREO BETA WEIGHTS 3.430 lfl.890 7,345

B WEIGHTS 10.440 -24.458 15,010

INTFRCEPT CONSTANT ■ 3.795

ANoVA TABLE FOR POLYNOMIALS B«at: = a5^3«»*»»3 = a»«a» = sao3aa3»BaBa«3o:3:in3«:j3^3aa»a3s^»»a:a«»3»amaaaaasa«a»» REDUCTION OtIF TO LINEAR fjl, WITH 1 DF » 0.024 RESIDUAL S.S. ■ 0.976 OF ■ 13b RESIOUAL *.S. ■ 0.007

F FOR LINEAR FIT ■ 3.387 aaasBsaasBBtaisraasaaoissaaBaaaBiaaasaBasaaaBiaalsataaiaaaBaaaaaai.aiaasas

REhUCTlON OUF TO GENERAL QUADRATIC FIT WITH OF 2, ai 0,028 REDUCTION M.S. • 6.014 RESIDUAL S.S. - 0.972 OF ■ 134 RESIDUAL M.S. • 0.007

F FnR QUADRATIC FIT a 1*902 REOllCTION OUF TO CUAORATIC TERM ALONE, WITH 1 oF» " 0.003 F FnR OUAORATTC TERM ALONE a 0.432

REOllCTION DUE TO OENFRAL CUBIC FIT WITH DF 3, . 0,011 REOllCTION M.S. ■ 0.010 RESIDUAL S.S. a, 0.969 OF ■ 133 AESlOUAL M.S. • 0.007 1 — 24

F FOR OFNERAl. CUBIC FIT ■ 1.428 REDUCTION DUF TO CUBjC TERM ALONE WITH 1 DF ■ , 0.004 r FOR CimlC TFRM ALOK'E ■ 0.493

■ ■ ■ u ts * a ■ a u ■ ■ 3 s s 2 u .1 ■ n * a « & a a ° a a a a s u ■ " * a a n n a a s « c ? 3 a a it a 3 • a x 3 s * » s a ■ ta t* a 3 a ■. s a s a a a

POLYNOMIAL FITTING FOR 2 VAWIAHLES-GPE ANO TP SCORE- 137 OBSERVATIONS.

THE PREDICTOR VARIABLER) IS GRE, «NO THE CRITERION VARIABLE(Y) IS TP SCORE.

RM RATING

TEST MEANS ANO STANDARD OEvIATIONS 1 X 7 SOIIAR 3 X CUBE 4 Y

CORRELATION MATRIX X

SQUAR X CUBE

Y

0.997 0.99« 0.992 4.453

( 1.000 1.000 1.000 0.6*1

0.007 0.013 0.019 1.925

1.000 1.000 1.000 0.0**

1.000 1.000 1.000 0.0**

0.0*1 0.1** 0.0*6 1.000

Beas8sss3BBtsB:3asai3933a3>sa:3SBS9ass8333taa3:s>iB8lifsait3iiifsflai«4si3a

FIRST, SECOND, ANO THIRO nEGREE POLYNOMIALS. easaesaaBaBBeaaaaBBBBBaaaaBaaaaaBaaaaaas:

MULTIPLE R SQUARE a MULTIPLE R N,O.F.I ■ N.O.F.2

0.0*2

1 0.041

135 0.227

10.869

F FOR ANALYSIS OF VARIANCE ON R ■ BETA WEIGHTS

0.0*1 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR LOAOINGSI 2N0 COLUMN» 1 X 0.002 1.000


B WFIGHTS 12.039 INTERCEPT CONSTANT ■ -7.551 BBBaBSBaBaaaacBaaaaiaaasaaaBBaaasaiisaiaaataaaaaaaaaaaaiaaaaaaaiaaiiiaBiaa

MULTIPLE R SQUARE » 0.1*0 MULTIPLE R ■ 0.37* N.D.F.I ■ j» N.O.F.2 ■ 134 F FOR ANALYSTS OF VARIANCE ON R ■ BETA WEIGHTS -51,053 51.095 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR L<>AOlNGS» 2N0 COLUMN) 1 X -2.092 0.U0 ? SQUAR 2.231 0.117

SQUARED BETA (.«EIGHTS 606.3AB26i6.A91 B WFIGHTS • •••«••76*4,e,70 INTFRCEPT CONSTANT ■ 7356.34* BBBsBa:3e3iai::i:aBaaaBB3Bi3:BBS3iaaaaaBaBB aaaaaBaaaaBaaaaaaa

MULTIPLE R SQUARE ■ 0.139 MULTIPLE R ■ 0.372 N.O.F.I ■ 3 N.O.F.2 ■ 133 F FOR ANALYSTS OF VARIANCE ON R ■■ 7.131 BET» WEIGHTS -35,0*7 19.<?i 15.S63 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR LOADINGS. 2ND COLUMN) 1 X -l.*36 O.llO ? SOUAR 0.853 0.117 3 X CUBE 0.722 0.125

SOUÄREO BETA WEIGHTS 2?8.327 381,346 2*2.1*93 B WFIGHTS •••••••2921.6O71580.930 INTERCEPT CONSTANT ■ 5796.982

BiaiBBaa

ANpVA TABLE FOR POLYNOMIALS BaaBaaaaaaBaaBsaaBBaaaBaaBaBBaBaaaaaaaaBaaBBaaBaaaaaaaaai REDUCTION OUF TO LINEAR FIT, WITH 1 OF a 0.002 RESIDUAL S.S. ■ 0.998 OF a I35 RESIOUAL M.S.

F FOR LINEAR FIT a 0.227 O.OO7

REDUCTION OUF TO GENERAL QUADRATIC FIT WITH OF 2. ■ 0.1*0 REDUCTION M.S. a 0.070 HFSIDUAL S.S. ■ 0.860 OF ■ 13* REStOUAL M.S, a

F FOR QUADRATIC FIT a 10.869 REDUCTION DUE TO QUADRATIC TERM ALONE» WITH 1 nF, a 0.138 F FDR QUADRATIC TERM ALONE ■ 2l.*76

0.006

REDUCTION OUF TO GENERAL CUBIC FIT WITH DF 3» a 0.119 REDUCTION M.S. ■ 0,0*6 RESIDUAL S.S. ■ 0.861 OF ■ 133 RESIDUAL M.S. ■

F FQR GENERAL CUBIC FIT • 7,131 REDUCTION oijF TO CUHjC TERM ALONE WITH 1 OF ■ , -0,001 F FOR CUHIC TFRM ALONE a -0.156

0.006 1-25

POLYNOMIAL KITTING FOR i VARJA"LES-wHt ANU TP SCOftE- 137 ORSEQVATIONS,

RM RATING THE PREDICTOR VARIABLE(X) IS WRE, AND THE CRITERION VAPIARLE(Y)' IS TP SCOHE,- .

TFST MEANS AMn STANOAPC OEVIATIONS 1 X 0.632 0.232 ? SOUAR 0.453 0.252 3 X CURE 0.3*2 0.253 A ORC

Y (ELATION

4.AS3 MATRIX

1.925

1 X l.iioo 0.956 . 0.899 0.366 ? SQUAR 0.956 1.000 0.983 0.390 3 X CUBE o.«9e 0.9ÜS 1.000 0.387 A Y 0.3*« 0.390 0.387 l.ftOO

scG3SXi::s38«33=3al3:B2s:i3a5Bers3a3sia:s2r::::33sa.at3:BSisaB33a3aSait.sa FIRST, SECOND. AND THIRD OFGREE POLYNOMIALS. asassasasassssssasassssssaSsassssBassaasaaasaasaassaaasiaBaaaBBssaaaaaaassa MULTIPLE R SQUARE ■ 0.134 MULTIPLE R ■ 0.366 N.O.F.I a T N.O.F.2 a 139 F FOR ANALYSIS OF VARIANCE ON R ■ 20A845 BFTA WEIGHTS

0,3*6 CONTRIBUTIONS TO MULTIPLE CORRELATION

ÄNO REGRESSION FACTCR LOADINGSI 2ND COLUMN) T X 0.134 1.000


B WFIGHTS 3.037

INTFRCEPT CONSTANT » 2.533 aassasssssssssasssBassssssBsaisssssaassssBSBasssaaaassaaaaaasaassaaaaaaaaas MULTIPLE R SQUARE =■ 0.1*3 MULTIPLE R a 0.391 N.D.F.I a it N.D.F.2 a 134 F FOR ANALYSIS OF VARIANCE ON R a 12.103 BET» WEIGHTS -0.0R9 Ö.47S

CONTRIBUTIONS TO MULTIPLE CORRELATION AND REGRESSION FACTCR LOAOlNGSt 2N0 COLUMN) T X -0.033 0.935 ? SOUAR 0,ie6 0.998

SOUÄREO BETA «EIGHTS 0.008 Ö.226

B WEIGHTS -0.739 3.6"»0 INTERCEPT CONSTANT a 3.276 SBaassssaaa&saaasCBaalassaaasOBEalaasaicaaaaiaaalllailaaaiaalilatBalaaaaaa MULTIPLE R SOUAPE 3 0,154 MULTIPLE H ■ 0.392 N.n.F.l a i N.P.F.2 3 133 F FOR ANALYSIS OF VARIANCE ON R a 8.042 BFTÄ WEIGHTS -0.260 0.9*9 -0.324

CONTRIBUTIONS TO MULTIPLE CORRELATION ANO REGRESSION FAcTCR LOAOlNGS. 2ND COLUMN» } X -0.095 0.933 2 SOUAR 0.374 0.996 3 X CUBE -0.126 0.9Ö7

SQUARED BETA WEIGHTS 0.068 0.9(9 0.105

B WFIGHTS -2.159 7,320 -2.472 INTERCEPT CONSTANT a 3*347

ANOVA TABLE FOR POLYNCMIALS laaaaaiaaaiiaaiaaaaiaaiaa'iagaaaaaigsaBaciiaiaiiiBliliaaittilNiiialaaiaia «EDUCTION OUF TO LINEAR FIT, WITH 1 OF a 0.134 RESIDUAL S.S, a 0.866 OF a 135 RESIDUAL M.S. a 0.006

F FOR LINEAR FIT a 20.845 taaaaaaaaaaaaaaagggaaaBaaaBaaaaaaBgaaaaBsiagaaagliBaaiaataiiaiaaiiiaalBaai

REDUCTION OUF TO GENERAL QUADRATIC FIT WITH OF 2i'■ 0.133 REDUCTION M.S. a 0.077 RESIDUAL S.S, » 0.8*7 OF a J3* RESIDUAL M,S, a 0.006

F FoR QUADRATIC FIT ■ 12.103 REDUCTION OUF TO QUADRATIC TE&M ALONE» WITH 1 cP» ■ 0.019 F FOR QUADRATIC TERM ALONE a 3,046

REDUCTION DUE TO GENERAL CUBIC FIT WITH OF 3. a 0,114 REDUCTION M.S. a 0.051 RESIDUAL S.S. a 0,846 OF a 133 RESIOUAL M,S, • 0.006 1-26

F FOR OFNERAL CUBIC FIT a 8.042 REDUCTION OUF TO CUBIC TE«M ALONE WITH 1 OF ■ , O.0O1 f FOR CURIC TFRM ALONE a 0.086

ö3nnnB».-8aa«BaTaaD3iiir,Jnsjo:,3333ta3fl3cmijiiaju33'j3:i3an3sajiiia3n*»Biiaa»333a« POLYNOMIAL FITTING FOR 2 VAP.IAöLtS-SHE ANO 7P SCORE« 152 CBSERVATIONS.

ST RATING THE PREDICTOR VAHIABLEIX» IS. SHE, AND THE .CRITERION VARIABLE (Y>-"is TP SCORE.

TEST MEANS ANn STANDARC DEVIATIONS 1 X 0,280 0.283 ? SQUAB O.jSP 0.250 3 X CVHF. O.|l0 0.23* 4 Y S.783 2.425

COBPELATION MATRIX I X l.flOO 0.938 0.83S 0.199 ? SQUAR 0.938 1.000 0.980 0.2** ■) X CURE 0.P58 O.9Ö0 i .000 0.259 * Y 0.199 0.2*6 0.259 1.000 Bcs33B3as>aaizaaflsBBB~ss3C33assssaa3c3ia3B3S3=3 838«8iitaxao3BSaaia3«aS9aai FIRST, SECOND, ANO THIRD DEGREE PULYNOHIALS. cr333 333aa3B==3=Bi3Baacc=3 3 33=7=3aa=axfl3=a3a3a3aa3B3Baaaaa33aa3aia3laaaaaa MULTIPLE R SOilARE ■ 0.0*0 MULTIPLE R ■ 0.199 N.n.F.l a T N..0.F.2 « 150 F FOP ANALYSTS OF VARIANCE ON R ■ 6.217 BETA WEIGHTS


AND REGRESSION FAeTCR LOADINGS» 2NO COLUMN) 1 X 0.040 1.000

SnUAREO RETA WEIGHTS 0.0*0

R WPIGHTS 1.711

INTERCEPT CONSTANT ■ 5«3o* a:nB:::a3a:B33::Eaaa3B33:aB33:s:asa33a8:B33:aa:3B3BBBBaa3Ba3aBaaB3aBBS3a3a MULTIPLE R SQUARE » 0.0*9 MULTIPLE R « 0.262 N.O.F.l a ? N.0.F.2 a . 1*9 F FOR ANALYSIS OF VARIANCE ON R » 5.*83 BETA WEIGHTS -0,259 Ö.4BB

CONTRIBUTIONS TO MULTIPLE CORRELATION ANO REGRESSION FACTCR LOADINGS» 2ND COLUMN) T X -0.051 0.762 2 SOUAR 0,120 0.9*0

SDUÄREO BETA WEIGHTS 0.067 0.23°

B WFlGHTS -2.213 4.7?4

INTERCEPT CONSTANT ■ 5.657 Bi:3 3=33i:SB3BB3 3 33aa33°3a3 3aaaB:aBa3=8a3 33B3SB33 3B333SS3BBa33a3a3Bfl8SaSa3Qa MULTIPLE R SOUARE ■ Ö.069 MULTIPLE R » Ö.263 N.O.F.l a 3 N.0.F.2 ■ 1*8 F FOR ANALYSIS OF VARIANCE ON R a 3.656 BET« WEIGHTS_ -0.142 Ö.1S3 0.231

CONTRIBUTIONS TO MULTtPLE CORRELATION . ANO REGRESSION FAcTCR LOADlNGSf 2ND COLUMN)

T X »0.028 0.7->9 ? SOUAR 0.038 0.936 3 X CUBE 0.060 0.965

SQUARED BETA WEIGHTS 0.020 O.Ö?3 0.Ö53

P WEIGHTS -1.218 1.4B3 2.390 INTERCEPT CONSTANT ■ 5,627

ANOVA TARLE FOR POLYNOMIALS BaBaaBaasaaaaaeBSsaaaBBBaBaaaBaaaaBasaasasgassBaaaaBaaaaaaaaaaBaaaaaaaaaaa REDUCTION OIIF TO LINEAR FIT, WITH 1 OF a 0.040 RESIDUAL S.S. a 0.960 DF a 150 RESIDUAL M.S. ■ 0.006

F FOR LINEAR FIT • 6.21? asaaasBBBaBaaratBaaaaBaBBasaaaBBaaaaaaaaaaBassssBsaaaaaaBBaaBaaaaaaBaaBassaa

REDUCTION DDF TO GENERAL QUADRATIC FIT WITH OF 2. a 0.069 REDUCTION M.S. a 0.03* RESIDUAL S.S. a 0.931 DF a 149 RESIDUAL M.S. a 0>006

F FDR QUADRATIC FIT a 5«483 REDUCTION nilF TO CUADRATIC TERM ALONE. WITH 1 CF» a 0.029 F FnR QUADRATIC TERM ALONE a 4,600

RFDIICTION DUF TO GENERAL CUBIC FIT WITH DF 3, . 0.069 REDUCTION M.S. a 0,023 T 07 RESIDUAL S.S. a 0.931 DF a US RESIDUAL M,S. " 0.006 l~C'

F FDR GENERAL CUBIC FIT a 3.656 RFDIICTION DUF TO CUBIC TER* ALONE WITH | DM , 0.000 F FOR CUBIC TFRM ALONE a 0.071

«ijE]|aiiisaiail:i3i!:siiiSii-jl3anfl;iBa2;ia-jn",'j-ja3BilJl:3aanla3:iia(ii.iM3a«<aauaiiaaaBiisa POLYNOMIAL PITTING FOH 2 VARlAtJLtS-PRfc AMO JP SCO^- 15? OBSERVATIONS,

ST RATING

THE PREDICTOR VARIABLE(X) IS PREI AMO THE CRITERION VAPIABLE(Y) IS TP SCORE.

TFST MEANS ANn STANOARC DEVIATIONS l x 0.S49 0.349 ? SOUAR 0,452 0.3<>3 3 X CURE 0,38s 0.347 4 Y 5,783 2.425 ORRELATION MATRIX l X 1,000 0.972 0.923 0.098 2 SOUAR 0.972 1.000 0.987 0.055 3 X CUBE 0,923 0.987 1.001) 0.O63 4 Y 0.058 0.055 0.063 1.000

FIRST, SECOND. «MO THIRD nEGRtE POLYNOMIALS. BSSaaaaaaaaaeaaaaasanaBBaanOaaaaaEaaasaaaxaaaaaaaaBaaaasBaEoaaaaBaaaaaaaaaasal MULTIPLE R SOi.iAPE o 0.0Ä3 MULTIPLE R » 0.05B N.O.F.l » T N.D.F.2 a« 150 f FoR ANALYSIS.OF VARIANCE ON R a 0.909 BFTÄ WEIGHTS

0,058 CONTRIBUTIONS TO VULTIPLE CORRELATION

«NO REGRESSION FACTCR LOADINGS» 2NO COLUMN) T X 0.003 1.000

SOUARED BETA WEIGHTS 0.003

8 WpIGHTS 0.362

INTERCEPT CONSTANT ai 5»584 Baa^BaaaBaaBasBBSSBaaaiBassBBassssBasaaaBaBassXasaaBsssasassaBsssaasaBsasaaBBasa MULTIPLE R SOIIAPE ■ 0.0fi3 MULTIPLE R aa 0.058 N.O.F.I aa p N.0.F.2 aa 149 F FOR ANALYSIS OF VARIANCE ON R ■ 0.255 BFT» WEIGHTS

0,083 «0.0?S CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR LOADINGS» 2NO COLUMN) 1 X 0.005 0.995 7 SOUAR -0,001 0.9*3

SOUÄRED BETA «EIGHTS 0,007 Ö.ÖÖ1

R WFIGHTS 0,516 -0.170

INTERCEPT CONSTANT aa S»577 srjarrBssasasBBaacsBaBsaaaBaa^saaBSsaBB-aasswaoasBaasafBBBaraaaanaBaaBBaaaaaaaB MULTIPLE R SOHARE ■ 0.1 j>2 MULTIPLE R aa 0#32Ö N.D.F.I " 3 N.P.F.2 aa 148 F FOR ANALYSIS OF VARIANCE ON R ■ 5.634 BFTÄ WEIGHTS

4.864 -11.7S6 7.177 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION FACTCR LOADINGS» 2ND COLUMN) T X 0.283 0.1U2 ? SOUAR «0.648 0.1*2 3 X CUBE 0,468 0.20*

SOUÄRED BETA WEIGHTS P3.659 138.2n8 51.513

8 WFIGHTS 30.272 -78.604 50.161 INTFRCEPT CONSTANT ■ 5.410

ANOVA TABLE FOR POLYNOMIALS ■laialBeaiBaszsaaaaaaBasBaaBiaaBBBaiBBaBaaaaaBaaaaBBaaaBaaaiBaaaaaaaiaaiia PEnUCTlON DIIF TO LINEAR FJT, WITH 1 OF « 0.003 RESIDUAL S.S. ■ 0.997 OF ■ 150 RESIDUAL M.S. " 0.007

F FOR LINEAR FIT aa 0,509 Ra»a*BaBa3BBx,ettaaaRB3aB>33t3aiiai>i=iiaaiis«aiaiiiiiaiiiaaaiiisaiiiBtiiBaa

OEOUCTTON Ollf TO GENERAL QUADRATIC FtT WITH OF 2t ■ 0.003 REDUCTION M.S. • 0.002 »ESinUAL S.S. ■ 0.997 OF ■ 149 RESIDUAL M.S. a 0.007

F FOR OUAORATIC FIT ■ 0.255 REDUCTION OUF TO CUADRATIC TERM ALCNE» WITH 1 CF» • 0.000 F FOR QUADRATIC TERM ALONE ■ 0.005

REDUCTION DUE TO GENERAL CUBIC FIT WITH DF 3. a 0.10? RFDUCTION M.S. ■ 0.034 RESIOUAL S.S. ■ 0.898 OF ■ 148 RESlOUAL M.S. * 0.006 1-28

F FnR GENERAL CUBIC FIT aa 5.634 RFOIICTION DUF TO CUBtC TERM ALONE WITH J OF ■ , 0.099 F FOR CUBIC TE-PM ALONE ■ 16.339

POLYNOMIAL FITTING FO« 2 VAHlAÜLES-tiRE «NU IP SCORE- 152 OBSERVATIONS,

ST RATING

THE PREDICTOR VARIARL£<X> IS GRE, AND THE CRITERION VARIABLEIY) IS TP SCORE,

TFST MEANS AMD STANDARC OEVlATIONS 1 X 0.«97 0.187 ? SOUAR O.B*O 0.203 3 X CURE 0.790 0.22Ö A ORt

y »ELATION

5,783 MATRIX

2.425

i X 1.000 0.969 0.9U 0.099 ? SOUAR 0,969 1.000 0.985 0,183 3 X CUBE 0.914 0.9(13 1.000 0.243 A Y 0.099 0.1Ö3 0.2*3 1.000

BsDasaaasaaacaaaaaaaaasaasaaaaaaaaBasaaaaaaaaaaaaazBaaaaaaaaaaaaaaoaisaas« FIRST, SECONO. ANO THIRD nEGREE POLYNOMIALS. aaeaaaaaaaaaaaaaaaaasaaaaaaaaacaaaaaaaaaaaaiaaaa aaaaaaaaaaiagnaaaa-a MULTIPLE R SOKAPE a O.OlO MULTIPLE R ■ 0.099 N.O.F.I a I N.D.F.2 a ISO F FnR ANALYSIS OF VARIANCE ON R " 1.490 BET« WEIGHTS


»NO REGRESSION FACTCR LOADINGS» 2N0 COLUMN) 1 X 0,010 1.000

SOUaREO BETA WEIGHTS 0.010

8 WFIGHTS 1.285

INTERCEPT CONSTANT " 4.630 BaaoaaaaaaaaaaaaaaaaaaaaaaaaaaaiaaaaaaaaaaasaaaaaaaaaaaiHaaatiaataaaaaaaa MULTIPLE R SOUARE » 0.13* MULTIPLE R a 0.366 N.O.F.I a 2 N.0.F.2 ■ 149 F FOR ANALYSIS OF VARIANCE ON R ■ 11.350 BETA WEIGHTS -1,294 1.417

CONTRIBUTIONS TO XULTIPLE CORRELATION »ND REGRESSION FACTOR LEADINGS« 2N0 COLUMN) I X -0.128 0.271 ? SOUAR 6.263 0,*99

SOU«REO BETA «EIGHTS 1.67* 2.«A*

B WrtGHTS -16.765 17.133 INTERCEPT CONSTANT * 6,*37 »aaasaaBaaaiaaaaeaaaaaaassaaaaaaiaaassaseaasiaaatiaaaaaaiHaaatisaalaaaaai MULTIPLE R SQUARE ■ 6.178 MULTIPLE R ■ 0.*22 N.O.F.I • 3 N.0.F.2 ■ 1*8 F FOR ANALYSIS OF VARIANCE ON R a 10.692 BET» WEIGHTS

1,686 -5.777 *.192 ""■ CONTRIBUTIONS TO MULTIPLE CORRELATION

»NO REGRESSION FACTOR LOADINGS» 2ND COLUHN) T X 0.167 0.235 ? SOUAR -1,056 0.*33 1 X CURE 1,067 0.575

SOUÄRE0 BETA WEIGHTS 2.9*3 33.37* 19,?88

B WFIGHTS 21.8*9 -68.PR6 48,351 INTERCEPT CONSTANT a 5,825

ANOVA TABLE FOR POLYNOMIALS aaBaaaaaaaBaaaBaBaaBaaaBB3BaaaaaaaaBaaa«aaaaflBaaaaaBaaaaaaaaiaaiaaaaBaaaaa REDUCTION OUF TO LINEAR FlT, WITH 1 OF ■ 0.010 RESIDUAL S.S. » 0.990 DF ■ 150 RESIDUAL M.S. • 0.007

F FnR LINEAR PIT ■ 1.490

RFnUCTTON OUF TO GENERAL QUAORATIC FIT WITH OF 2» ■ 0.13* REDUCTION M.S. ■ 0.067 RFStOUAL S.S. ■ 0.866 OF a U9 RESIOUAL M.S. a 0»006

F FnR QUADRATIC FIT a 11,550 REDUCTION OUF TO QUADRATIC TERM ALONE, WITH 1 CE» ■ 0.124 F FnR QUADRATIC TERM ALONE ■ 21.408

REDUCTION DUE TO GENERAL CUBIC FIT WITH OF 3« -i 0,178 REDUCTION M.S. a 0.059 RESIDUAL S.S. a 0,fl22 OF • 148 RESIDUAL H,S. a 0.006 1-29

F FOR GENERAL CUBIC FIT a 10.692 REDUCTION OUF TO CUBIC TERN ALONE ttlTH 1 OF ■ , 0,0*4 F FnR CUBIC TFRM ALONE a 7.9o*

POLYNOMIAL FITTING Fow 2 VARIAOLfcS-WRE «NO IP SCO«E. 152 CRSEOVATIONS.

ST RATING THE PREDICTOR VARIAHLEIX) IS WRC, AND THE CRITERION VARIAHLEIY) IS TP SCOME.

TEST MEANS ANO STANOäR0 DEVIATIONS

i X 0.A03 0.207 ? SQUAR 0.406 0.237 3 X CURE 0,?92 0.23« * V 5.783 2.425

CORPELATION MATRIX 1 X 1.000 0.973 0.92» 0.234 2 SQUAR 0.973 1.000 0.983 0.2*3 3 X CUBE 0,921 0.983 1.000 0.263 4 Y 0.234 0.2«>5 0.26? l.flOO adB;:aaa=B3a==3raiaasi2s=33 3=a=ac3t=zai=33==a;3S3aa3fa3Salsaiaa3aaiaiii«gga FIRST, SECOND, ANO THIPO ÖE6PEE POLYNOMIALS. nÄE = o = = oa«aa33c = "aait3:aiiannai3 = i:K = n3*a5caja:jiBi»a»aioaB«3(3ai»Bat*aiai*at»3aairlaiaiar3laiaiaitt MULTIPLE R SQUARE ■ 0.055 MULTIPLE R a 0.234 N.n.F.l « 1 N.P.F.2 ■ 150 F FOR ANALYSIS OF VARIANCE ON R ■ 8.707 BFTj WEIGHTS


»NO REGRESSION FACTOR LADINGS. 2ND COLUMN) T X 0.055 1.000


B WEIGHTS 2.746

INTERCEPT CONSTANT ■ 4;128 tsssaaaasssaaaaaaaisaaasEOaasgstiaiaaaaaiiaaaaaaaaaaaaBiiaaaaaaaiaaaaaaiii MULTIPLE R SOIIARE ■ 0.0?0 MULTIPLE R ■ Ö.283 N.D.F.I a j N.0.F.2 a 149 F FOR ANALYSIS OF VARIANCE ON R • 6.467 9ET» WEIGHTS -0.426 Ö.679

CONTRIBUTIONS TO MULTIPLE CORRELATION ÄNO REGRESSION FACTOR LOADINGSI 2ND COLUMN) 1 X -O.lOO 0.829 7. SOUAR 0.18O 0*936

SQUARED BETA WEIGHTS 0.183 6.4M

B WEIGHTS -4,999 6.917 INTERCEPT CONSTANT ■ 5;982 aaaassaaaaaassissaaaaaaaaaaasaftaaBBsaaaB&aaaeaaaaaaBaiaaaaaaasaaaaaaaaBBaaaa MULTIPLE R SOIIARE ■ 0.123 MULTIPLE R ■ 0.351 N.O.F.l ■ 3 N.0.F.2 ■ 148 F FOR ANALYSTS OF VARIANCE ON fl ■ 6.925 BET« WEIGHTS -2.679 6.115 -3.314

CONTRIBUTIONS TO MULTIPLE CORRELATION ANO REGRESSION FACTCR LADINGS» 2ND COLUMN) 1 X -0.62s 0.668 ? SOUAR 1.624 0.754 1 X CUBE -0.873 0.75»

SQUARED BETA WEIGHTS 7.179 37,644 10.980

B WEtGHTS -31.413 62.653 -34,060 ' INTERCEPT CONSTANT ■ 9.241

ANOVA TABLE FOR POLYNOMIALS BaBBBBaBaaBBaBaBaBaaaaaaaaaaaaaBBaaaaaaaaaaBBaaBaaaaaaBaaaaaaaaaaaaaaa REDUCTION DUE TO LINEAR FIT, WITH 1 OF a 0.055 RESIDUAL S.S. a 0.945 DF a 150 RESIDUAL M,j, a 0.006

F F*R LINEAR FIT a P.707 aaaaaaaaaaaasaaaaaaaaaaaaaaaaaaaaaaaaaaaaBaaBaaaaaaaaaaaaaaaaaaaaaaaaa

REOUCTION DUE TO GENERAL CUADRATIC FIT WITH OF 2, a 0.080 REPUCTION M.S. a 0.040 oEsIOUAL 5.«, a 0.920 OF a 149 RESIDUAL M,J, a 0.Ö06

F FOR QUADRATIC FIT a 6.467 REDUCTION OUF TO CUAfjRATIC TERM ALONE» WITH 1 tf, a 0.023 F FOR QUADRATIC TERM ALONE a 4.050 .t.i..aataflBaxaiiilBallDialiBtlll3.aaaBaM.33aaaaa3alafii8i>*tlaaa.8ii

REDUCTION DUE TO GENERAL ?uaIC FIT w'™ 0* 3» a 0.123 REDUCTION M.S. a 0.041 RESIDUAL S.S. ■ 0.177 DF ■ 148 RESlOUAL M,J, a o.006

F FOR GENERAL CUBIC FIT a 6.925 REDUCTION OUF TO CUBIC TERM ALONe WITH 1 OF ■ , 0,041 F FOR CUBIC TFRM ALONE a 7.296

1-30

aaaaaaaaaatzoaaa rca« a n nn a a a ua «o jaaasislflasatiBsasn JS'

POLYNOMIAL FITTING FOR 2 VARIAöLES-SRE AND T*» SCO«E- laaaaaaaaaaaeti-da»»«

39 OBSERVATIONS,

TM RATING THE PREDICTOR VARIABi.ElX) IS SRE, AND THE CRITERION VArtiaBLE(Y) IS TP SCORE.

0.19«) 0.135 0.109 lo986

0o905 1.000 0o9?2 0.05Z

0.79J 0.972 1.000

-0.057

0.1*>8 0.092

-0.057 1,000

TEST MEANS AND STANOARC DEVIATIONS 1 X 0.173 ? SOUAR 0.667 3 X CUBE 0.035 * V 5,?82

CORRELATION MATRIX 1 X 1.000 2 SOUAR 0.Q0S 3 X CUBE 0,791 A V 0,)9P

lBs = ai»SB:BD3lsa3,:a3taai>«aa3Ei!ci3S3:aaci»iB3i>3i333Saa,lli3itiBisnlia33iB FIRST, SECONO, AN0 THIRO nEGREE POLYNOMIALS. 833£3333a3SB32BsBaS33aaaa3aaaBsa3333aa3a8sa3a338S3:3SB3aaaa3aB3aa3BaalBaaa

MULTIPLE R SOIIARg a 0.039 MULTIPLE R • Ö.19R N.O.F.l ■ T N.0.E.2 ■ 37 F FnR ANALYSIS OF VARIANCE ON R ■ 1.303 BET» WEIGHTS


»NO REGRESSION FAeTCR LOAOINGS, 2NO COLUMN) T X 0.039 1.000

SOUÄAEO BETA WEIGHTS 0.039

B WEIGHTS 2.024

INTERCEPT CONSTANT ■ 4.932 ■sBacsssaaaassaasaaaaasasaaaaQsaaaaasaaasaaaaaaaaBasaiaaBlBBBBaaaesaalsaaa MULTIPLE R SQUARE ■ 0.12^ MULTIPLE R N.O.F.l i N.O.F.2

0.359

3« F FOR ANALYSIS OF VARIANCE ON R ■ g.6S6 BFT» WEIGHTS

0.835 -Ö.7Ä4 CONTRIBUTIONS TO MULTIPLE CORRELATION

»NO REGRESSION FACTCR LOAOINGS» 2ND COLUMN) T X 0.165 0.5*1 ? SOUAR -0.036 0.144

SOUÄREO BETA WEIGHTS 0.698 Ö.496

8 WFlGHTS 8.558 -10,1*8

INTERCEPT CONSTANT ■ 4.490 ■ afltaaiaaasasnsiicaasaaSBsisiafliiaBataQasa.ltBflifaiaaasasaaaiiBSliai'laa MULTIPLE R SQUARE » 0.?17 MULTIPLE R ■ 0.466 N.O.F.l - 3 N.O.F.2 ■ 3S F FOR ANALYSIS OF VARIANCE ON R a 3.230 BET« WEIGHTS •0,463 3,0*7 «2,651

CONTRIBUTIONS TO MULTIPLE CORRELATION ÄNO »EGRESSION FACTCR LOADINGS» 2N0 COLUMN) T X -0.092 2 SOUAR 0.158 3 X CU8F 0,151

SOUAREO BETA WEIGHTS 0.215 9.285 7.Ö88

B WFlGHTS -4.749 44.T64 -48.424 INTERCEPT CONSTANT ■

0.424 O.IU

■0.122

4.802

ANQVA TABLE FOR POLYNOMIALS aBcaBaiiaiaBcasBaBaaaaaaaiaiaBBaBSiaiaostiiiiasBaiaaaai REDUCTION DUE TO LINEAR FJT, WITH 1 OF » 0,039 "ESIOUAL S.S. » 0.961 OF ■ 37 RESIDUAL M.5. ■

F FoR LINEAR FIT ■ 1.503 0.Ö26

REnUCTlON OWE TO GENERAL CUADRATIC FIT WITH OF 2. ■ 0.129 RFOUCTION M.S. ■ 0,064 RESIDUAL S.S. ■ 0.871 DF » 36 RESIDUAL M.S. ■

F FOR QUADRATIC FIT a 2*656 REOUCTION OUE TO CUAORATIC TERM ALCNE. WITH 1 RFi ■ 0.090 F FOR QUADRATIC TERM ALONE « 3.700

REDUCTION DUE TO GENERAL CUBIC FIT WITH OP 3, . 0.217 REDUCTION M.S. ■ 0.072 RESIDUAL S.S. a 0.783 OF ■ 35 RESIDUAL M.S. a

0.024

F FoR OFNERAi. CUBIC FIT ■ 3 = 230 REOUCTION DUF TO CUPIC TERM ALONE «ITH 1 OF • F FO» CUBIC TFRM ALONE ■ 3.944

0,08»)

0.022 1-31

aasaBaaoaaaaaaaaasattaaaasaaaaaaaiaattaaBisasaaBaaaasavsBauaacaaaaeaaBaaBaaaaasa . POLYNOMIAL FITTING FOR 2 VARlA«LE<i-PR£ ANO TP SCO»e- 39 OBSERVATIONS,

TM RATING THE PREDICTOR VARIARLE(X) IS PREa AND THE CRITERION VARIABLE(Y) IS TP SCORE,

TEST MEANS ANn STANOARfi OEVIATIONS 1 X 0,450 0.354 ? SOUAR 0.325 0.2?« 3 X CUBE 0.2*5 0.256 4 OR(

V »ELATION

5.282 MATRIX

l ,9§<a

1 X 1.000 0.971 0.911 0.243 2 SOUAR 0.971 1.000 0.982 0.281 y X CUBE 0.911 0.982 1.000 0,280 4 Y 0.2*3 0.2Ö1 0.280 1.000 iiiiaassaiitissisaiasBieBSSsiissisfsesszBiixisiaciiiiiasiiiiiiiiiialiifaii FIRST, SECOND, ANO THIRD oEOREE POLYNOMIALS, aaaBaasiaaaaaasaaBaaaaatasaiiiicaaaasaaaEiBaaaaaaBBBaaaaaiaiaaaaadaiaaiiii MULTIPLE R SoiiARE ■ 0,059 MULTIPLE R ■ 0.243 N.O.F.l ■ T N.O.F.? ■ 37 F FOR ANALYSIS OF VARIANCE ON R ■ 2»329 BETA WEIGHTS


ÄNO REGRESSION FACTCR LOADINGSI 2ND COLUMN» T X 0.059 1.000


B WjrlGHTS 1.366

INTERCEPT CONSTANT « 4,667 BsBaaBaaaassasiasaBaagiassBaaasBBBaaiiiiiaaaaaaiaaiaaBaaaiBBBBIBSBfliiBaaaafl MULTIPLE R SOUARE ■ 0.0'4 MULTIPLE R ■ 6.306 N.O.F.I ■ 2 N.O.F.z ■ 36 F FOR ANALYSIS OF VARIANCE ON R » 1.857 8FT» WEIGHTS -0,502 6,T«.8 CONTRIBUTIONS TO MULTIPLE CORRELATION

AND REGRESSION F4CTCR L°AOlNGS, 2NO COLUMN! \ X -0.122 0.796 ? SOUAR 0.216 0.918

SOUÄREO BETA «EIGHTS 0.252 6.591

B WEIGHTS -2.821 5.154 INTERCEPT CONSTANT • 4,879 BaBraEaasaaBaaiiBaaaaaaasaaaBaaaaaaBacaaaaassBsaaaaaaBaaaiBaBBaaaBaaBBaaaaB MULTIPLE R SOiiARE ■ 0,153 MULTIPLE R Bi 0.39i N.D.F.I ■ 3 N.0.F.2 a 35 F FOR ANALYSIS OF VARIANCE ON R ■ 2.112 BETA WEIGHTS -3.0B3 6,T»6 -3.575

CONTRIBUTIONS TO MULTIPLE CORRELATION AND REGRESSION FACTCR LOAOINGSI 2N0 COLUMN! 1 X -0.750 0.622 2 SOUAR 1.906 0.717 3 X CURE -1.002 0,716

SOUAREO BETA WEIGHTS 9.506 46.044 12.784

B WFIGHTS -17.309 45.SÖ8 -27.737 INTERCEPT CONSTANT » 3,lo3

ANoVA TABLE FOR POLYNOMIALS

REDUCTION DUE TO LINEAR FJT, WITH 1 OF a 0.059 RESIDUAL S.S. ■ 0.9*1 OF ■ 37 RESIOUAL M.J, • 0,025

F F*R LINEAR FIT » 2.329

BEOUCTION DUE TO GENERAL OUADRATIC FIT WITH OF 2, a 0.094 REDUCTION M.S. a 0.0*7 RESIDUAL S.S. a 0.906 OF ■ 36 REStOUAL M,5, a 0,025

F FnR OUAORATIC FIT a 1.857 REDUCTION OUF TO CUAORATIC TERM ALONE» WITH 1 CFi a 0.034 F FOR QUADRATIC TERM ALONE a 1.363

\ REDUCTION OtIE TO. OENfRAL CUBIC FIT WITH OF 3» a 0.153 REDUCTION M.S. a 0.051 RESIDUAL S.S. a 0.B47 DF a 35 RESIDUAL M,$, • 0.024 l"OC

F FOR GENERAL CUBIC FIT ■ 2.112 REDUCTION OUF TO CUBIC TERH ALONE WITH 1 OF ■ ', 0,060 F FOR CUBIC TFRM ALONE a 2,470

POLYNOMIAL FITTING FOR 2 VARIAöLES-GRE AND TP SCORE- 39 OBSERVATIONS.

TM RATING THE PREDICTOR VARIABLEM) IS ORE. ANO THE CRITERION VARIABLES) IS TP SCORE.

TEST MEANS AND STANOARC DEVIATIONS 1 X 0.B71 0.W3 ? SOUAR 0.788 0.2l0 3 X CUBE 0,720 0.2*S * Y 5,?8? 1.996

CORRELATION M«TRIX I X 1.O00 0.952 0.888 0.186 ? SQUAR 0.952 1.000 0.98« O.lTO 3 X CUBE 0.88? 0.986 1.000 0.1*3 4 Y 0.18« 0.170 0.163 1.000

FIRST, SECOND. AND THlBO DEGREE POLYNOMIALS. ■ tl3i9ili3ig:3l3lili3)inilEiiciiliili<9iil»3ciilliilliiilli»liliillili),t MULTIPLE R SOiiAflE ■ 0.035 MULTIPLE « • 0.186 N.O.F.I » T N.0.F.2 ■ 37 F FOR ANALYSTS OF VARIANCE ON R • 1.327 BET» WEIGHTS


ÄNO REGRESSION FACTCR LYINGS» 2N0 COLUMN) T X 0.03S 1.000

SoUÄREO BETA WEIGHTS 0.035

B WpJGHTS 2.131

INTERCEPT CONSTANT ■ 3,*?6 ■ ilKisifisicsiii asiiaEiiiaiitiliiitiiUKiti.illiiiiiiiiiiinn

MULTIPLE R SOIIARE ■ 0.035 MULTIPLE R • 0.187 N.O.F.I ■ 2 N.O.F.? ■ 36 F FOR ANALYSIS OF VARIANCE ON R ■ 0.656 BET« WEIGHTS

0.2S7 -Ö.Ö75 CONTRIBUTIONS TO MULTIPLE CORRELATION

jiNO REGRESSION FACTCR LOAOINGS. 2ND COLUMN) 1 t 0.048 0.9*3 ? SOUAR -0.013 0.907

SOUAREO BETA.«EIGHTS 0.066 o.nn6

B WFIGHTS 2.9*4 -6.7«*

INTERCEPT CONSTANT ■ 3.272 a3SBS3al3»IS33l3IIS8M:ia*e.llBSf3lll»SISS88Sllltlllllätlltllllll«ltliait

MULTIPLE R SOIIARE ■ 0.085 MULTIPLE R ■ 0.291 N.O.F.I • 3 N.0.F.2 ■ 35 F FOR ANALYSTS OF VARIANCE ON R « 1.082 BFTÄ WEIGHTS

3,436 -8.9B7 5.971 CONTRIBUTIONS TO MULTIPLE CORRELATION

flNO REGRESSION FAßTCR LOADINGS» 2N0 COLUMN) ? X 0.639 0.639 ? SQUAR -1.529 0.584 •% X CU8F 0.97* 0.560

SOUÄRE0 BETA WEIGHTS 11.807 80.7*8 35.65S

B WFTGMTS 39.356 -84.A77 48,463 INTpRCEPT CONSTANT ■ 2*981

ANQVA TABLE FOR POLYNOMIALS ■ BBH3iasiB»»3ai«tl»B»»»iBB«*33<»»»onain»»«»"«««'««»aa3iia«na«»»»«»«amm«B««««««««B»« REnUCTION OIJF TO LINEAR FiT, WITH 1 DF ■ 0.035 RESIOUAL S.S. ■ 0.965 DF ■ 37 RESIOUAL M.S. a 0.026

F FOR LINEAR FIT « 1.327

REDUCTION ODE TO GENERAL QUADRATIC FIT WITH OF 2| ■ 0.035 REDUCTION M.S. ■ 0.018 RESIOUAL S.S. ■ 0.965 OF ■ 36 RESIDUAL M.S. a 0.027

F FOR QUADRATIC FIT ■ 0,656 REDUCTION OUF TO CUADR«TIC TERM ALCNE» WITH 1 CE» ■ 0,001 F FOR QUADRATIC TERM ALONE ■ 0.020

RFOllCTION DUE TO GENERAL 9^8IC FIT WITH DF 3, . 0.0R5 REDUCTION M.S. ■ 0,028 T-?^

RESIOUAL S.S. ■ 0.919 OF ■ 35 RESIDUAL M.S. ■ 0,026 x *JvV

F FOR OFNERAI CUBIC FIT ■ 1,082 REDUCTION ntlF TO CURIC TERM ALONE WITH 1 DF ■ , 0.050 F FOR CUBIC Tf-BM ALONE ■ 1.903

POLYNOMIAL FITTING FOR 2 VAHIAÖLES-WRE ANO TP SCORE- 39 OBSEPVAMOSS,

TM RATING T*E PREOICTrtR VARIABLE(X) IS WHE. ANO THE CRITERION VARlARLE(Y) IS TP SCORE,

TFST MEANS ANn STANOAflC DEVIATIONS I X 0.496 0.194 ? SOUAR 0.?B3 0»193 3 X CUBE O.ir« 0*165 * Y 5,282 1,986

CORRELATION MATRIX 1 X 1.000 0#978 0.930 O.*03 2 SOUAR 0.978 1.000 Q.98« n.*l9 3 X CU8E 0.930 0,9«« 1,000 0.409 * V. 0.403 0.419 0.*0<5 1.000

tsssiiBiai=S3sBa9iai3Bf29tiiec:saitss88:,i,se58zia3aigasaii8i»istiiiiiiii

FIRST, SECONO, ANO THIRD OEGREE POLYNOMIALS, • :cs»:i»iBc;:BS3aa8i8:s88a]8:8a88ai8a889«E>a38i»iaiaa<8»»3itHitiiafi

MULTIPLE R SOMARE ■ 0.1*2 MULTIPLE R ■ 0.403 N.O.F.I at T N.0.F.2 ai 37 F FOR ANALYSIS OF VARIANCE ON R ■ 7.172 BET» WEIGHTS


AND REGRESSION FACTCR LOAOINGS. 2N0 COLUMN) T X 0.162 1.000

SOUÄREO BETA WEIGHTS 0,1ft?

R WEIGHTS ».127

IK-TERCEPT CONSTANT » 3;234 BBOssssBBasBsssssBaaBBssaaBBeBaBBBBBSBaBsaBBBaaBBaaaBaaBBBBaaaaBBBBBaaasaa MULTIPLE R SOUARE « 0.177 MULTIPLE R ■ 0.421 N.n.F.l ■ J» N.0.F.2 a 36 F FOR ANALYSIS OF VARIANCE ON R ■ 3.867 BET» WEIGHTS -0,161 Ö.S76

CONTRIBUTIONS TO MULTIPLE CORRELATION ÄNO REGRESSION FACTOR L^AOINGSS 2N0 COLUMN) T X "0,065 0.958 ? SOUAR 0,242 0.997

SOUÄREO BETA '.'EIGHTS 0.026 Ö.312

B WEIGHTS •1.644 3.918 INTERCEPT CONSTANT a 4,424 sssaBSEaaaaBBssassaaaassaa'ssssaaaassaassaaaassasaaaaaaaaaaaaBaaaBaaaaaaaa MULTIPLE R SOHARE » 0.224 MULTIPLE R ■ 0.473 N.o.F.i "3 N.0.F.2 a 35 F FOR ANALYSIS OF VARIANCE ON R • 3«362 BFTÄ WEIGHTS -3,771 8,708 -4,667

CONTRIBUTIONS TO MULTIPLE CORRELATION AND REGRESSION FACTCR LOAOINGS» 2N0 COLUMN» 1 X -1.S20 0.8*2 ? SOUAR 1.650 0,886 ? X CU8F -1.907 0.864

SOUÄRED BETA WEIGHTS 14.222 75, «13 21,785

B WFIGHTS -38.622 89,446 -56,664 INTERCEPT CONSTANT ■ 9.017

»NOVA TABLE FOR POLYNOMIALS BsssBSBBsaaassasasaaaasBsaBaaBBBBSaaaBaaaaasBssaaBBaaaaaaaaaaBaaaaaaaaaBaai «FnUCTION ptip T(l LINEAR FIT» WITH 1 OF a 0.162 RESIDUAL S.S. a 0.838 OF a 37 RESIOUAL M.S. • 0,023

F F"R LINEAR FIT ■ 7.172

REDUCTION OUF TO GENERAL GUAORATIC FIT WITH OF 2i • 0.177 . REOUCTION M.S. ■ 0.088 »ESIDUAL S.S. » C.P23 OF • 36 RESIDUAL M,S. • 0.623

F FOR OUAOBATTC FIT a 1.867 REDUCTION OUF TO CUAORATIr TERM ALONEI WITH 1 OF» ■ 0.014 F FOR OIIAORATTC TERM ALONE a 0.634

REOUCTION OUF TO GENERAL CUBIC FIT WITH OF 3» a 0,224 REOUCTION M.S. a 0.075 - ». RESIOUAL S.S. ■ 0.776 OF ■ 35 RESIOUAL M.S. a 0.022 1-J4

F FOR GENERAL CUBIC FIT ■ 3.362 REOUCTION OUF TO CUBIC M*» ALONE WITH 1 OF ■ , 0,047 F FOR CUBIC TERM ALONE a 2,113

APPENDIX J

JOB TASK CONDITIONAL AND JOINT FREQUENCIES BY RATING

J-1

THIS PAGE IS BLANK

J-2

JOB TASK CONDITIONAL AND JOINT FREQUENCIES BY RATING

This Appendix will be devoted to presenting the Job Task Conditional and Joint Frequency Matrices by rating for the data collected on the Technical Proficiency Checkout Form (TPCF). The matrices are symetric about their main diagonal and therefore only the upper triangular portion

will be presented. Denote an arbitrary entry in the i row and j column (i = l, ...,8) as a. •, read left to right and top to bottom for

i »J rows and columns respectively.

For the Job Task Conditional Frequency Matrices, the entries on the main diagonal, a. . (i=l, ...,8), are the number of technicians CHECKED

OUT on the (9-i)th j0b task, but NOT CHECKED OUT on an any more difficult task. The entry off the main diagonal, a. • (i=l, ..., 8; j>i) is the

1 +■ h number of technicians CHECKED OUT on the (9-j)Ln task given they are

CHECKED OUT at most on the (9-i) task. For example, from page J-4, there were 17 EM's who were CHECKED OUT at most on Job Task Mo. 7, i.e., they were NOT CHECKED OUT on Job Task No. 8. Of those 17 men 4, 13, 14, 17, 17, and 17 were CHECKED OUT on Job Task No.'s 6, 5, 4, 3, 2, and 1 res, pectively.

In the Job Task Joint Frequency Matrices, the entries on the main diagonal, a- • (i=l, •.., 8), are the number of individuals CHECKED OUT

th on the i task, regardless of their performance on any of the other tasks. The entries off the main diagonal, a- • (i = 1, ..., 8; j>i), are the number

'»J th th of technicians CHECKED OUT on both the i and j job task. For example, from page J-4, 75 EM's were CHECKED OUT on Job- Task No. 5, 42 on Job Tasks No.'s 5 and 6, etc.

J~3

ELECTRICIAN'S MATE (EM) RATING (M = 97)

Conditional Frequencies

55

Job Task 7 6 5 4 3 2 1

4ft 33 50 50 50 52 55

17 4 13 14 17 17 17

6 6 5 5 4 6

6 5

3

6

3

6

6

3

5

0

6

3

6

0

2

The number of technicians not CHECKED OUT on any job task is 2

Joint Frequencies

9S

Job Task

2 3 4 5 6 7 8

87 87 77 75 43 63 55

87 84 . 73 70 40 63 52

H7 71 69 40 62 50

77 6<5 40 56 50

75 4?

4 3

56

35

63

50

33

46

55

J-4

ELECTRONICS TECHNICIAN (ET) RATING (N'= 173)


132

Job Task 7 6 5 4 3 2 l

114 PI 113 131 127 123 131

IS 5 6 15 15 li 15

10 S 6 10 7 9

2 2

9

2

H

2

1

4

1

0

2

8

1

0

?

The number of tech nicians not CHECKED OUT on any job task is 1

Joint Frequencies

16*

Job Task

2 3 4 5 6 7 8

14«. 160 163 126 95 129 131

147 146 143 117 89 118 123

164 159 124 95 124 127

165 125 94 129 131

126 88

96

108

83

129

113

81

114

132

J-5

FIRE CONTROL TECHNICIAN (FT) RATING (N = 154)


102

Job Task 7 6 5 4 3 2 1

9? 74 86 101 94 90 102

8 s 6 8 8 8 P

14 q 12 14 12 14

15 14 12 10 14

8 4

2

4

1

0

7

2

0

1


Joint Frequencies

14*

Job Task ? 3 4 5 6 7 8

125 134 141 115 93 100 102

1?5 124 121 101 86 94 90

134 128 108 91 96 94

143 114 90 99 101

116 76

9.1

86

76

100

86

74

92

102

J-6

INTERIOR COMMUNICATIONS ELECTRICIAN (IC) RATING (N = 58)


PI

Job Tas k 7 6 5 4 3 2 1

27 20 23 27 26 26 27

6 1 6 S 6 6 6

5 4 2 4 4 5

8 7 6 4 8

3 3

4

2

3

0

3

4

0

4


Joint Frequencies

57

Job Task

2 3 4 5 6 7 8

45 49 44 41 26 33 27

45 45 38 36 25 32 26

49 41 38 25 32 26

44 37 23 32 27

41 25

?6

29

21

33

23

20

27

27

J-7

RADARMAN (RD) RATING (N = 139)


2*

Job Task 7 6 s 4 3 2 l

7 6 25 14 4 5 26

1 1 l l 1 l 1

1 l l 0 0 1

70 31

3

30

2

1

16

2

1

2

70

3

1

2

32


Joint Frequencies

Job Task

13h 27 38 50 97 R 8 26

27 23 ie 21 3 1 5

3e 23 34 2 2 4

50 47 7 6 14

97 8

8

8

3

8

25

6

7

26

J-8

RADIOMAN (RH) RATING (N = 137)


13

Job Task

7 6 5 4 3 2 1

10 9 13 13 13 12 13

17 8 17 17 11 11 17

29 28 22 28 24 29

55 32 40 34 55

4 3

5

3

4

0

4

5

0

12


Joint Frequencies

1 2 3

135 PB 100

88 fa8

100

Job Task

4 5

88 113

69 81

76 91

88 83

113

8 7 8

46 27 13

36 20 12

40 21 13

39 27 13

45 27 13

46 17 9

27 10

13

J-9

SONAR TECHNICIAN (ST) RATING (N = 152)


80

Job Task

7 6 5 4 3 2 1

64 74 69 78 77 77 HO

13 11 9 12 13 13 13

18 16 10 IS 14 IP

21 15 10 10 24

2 1

3

1

3

0

2

3

0

4


Joint Frequencies

144

Job Task 2 3 4 5 6 7 8

118 119 115 118 103 77 80

118 118 102 99 98 76 77

119 102 100 99 76 77

117 101 94 76 78

121 92

103

69

75

77

69

74

64

80

J-10

TORPEDOMAN'S MATE (TM) RATING (N = 39)


Job Task

7 6 s 4 3 2 1

ft 6- 8 e 8 8 8

12 5 12 li 12 12 12

3 3 3 3 3 3

11 4 6 6 10

5 3

n

2

0

0

5

0

0

0


Joint Frequencies

3«

Job Task

2 3 4 5 6 7 8

30 31 31 33 14 lfl 8

31 31 27 29 14 18 8

3? 28 29 14 1« 8

31 26 14 17 8

34 14

14

18

10

18

8

6

ft

B

J-ll

THIS PAGE IS BLANK

J-12

APPENDIX K

THE UTILITY OF THE WRE

K-l

THIS PAGE IS BLANK

K-2

THE UTILITY OF THE WRE

The purpose of this section is to demonstrate the utility of the WRE as an estimator of individual performance. Essentially it is a useful type of estimator in that it is not dependent on a convention to be adopted for the case wherein a man did not work at particular job activity. As such the convention need only provide reliability ratios for those job activities for which the man being evaluated received ZUE = 0 and ZUI = 0 from his supervisor.

Each square in Table 1 represents a breakdov/n of Table E-l into the number (and proportion) of men who did not work at a particular job activity and those men who received ZUE = 0 and ZUI = 0. For example, on job activity Number 1, 19 (19.6% of the EM's) received ZUE = 0 and ZUI = 0 and 6 (6.2% of the EM's) did not work at that job activity. The composite reliability values need then only be employed on 19.6% of the men in that rating and job activity rather then on 25% of the men as required by the SRE, PRE, and GRE. More significantly, in the case of RD's and RM's, for example, at most 59% (as compared to 99% for the SRE, PRE, and GRE) of the men in those ratings derive reliability ratios for some job activities from the composite reliability table. Clearly this is a significant improvement which should improve individual performance estimates. The statistical analyses reported in the main text of this paper verified this conjecture.

Derivation of the Weights Employed by the WRE

On the JPQ ANSWER SHEET is Appendix A, page A-4, in column (c) for each job activity the following question was answered by the supervisor on the man he was evaluating:

QUESTION (c) Considering this man's overall performance, it is your opinion that the importance of this job activity, as a factor in determining the overall performance of this man, is best described as being:

3. of central and primary importance 2. a significant factor, but of secondary importance 1. of only moderate importance in estimating overall

performance 0. of little or no importance

The weights (w^) for the itn job activity are determined by the formula

If the supervisor recorded the ith job activity as:

of central and primary importance, the weight w^ = 1.0

of secondary importance, the weight w^ = .75

of moderate importance, the weight w^ = .5

of little or no importance, the weight w.j = .25.

K-3

TABLE K-l

NUMBER AND PROPORTION OF TECHNICIANS IN EACH PROBLEM AREA

Job Act.

EM ET FT NZ NW NZ NW NZ NW

Rating

IC RD RM ST TM NZ NW NZ NW NZ NW NZ NW NZ NW

19 6 36 0 51 5 9 0 46 4 34 19 39 6 6 0

0,19« 0,062 0,220 0,000 0,331 0,032 0,155 0.000 0.331 0.029 Ö.24B 0,139 0,257 0,039 0,154 0.000

16 7 47 0 48 1 7 0 52 1 27 15 36 9 13 0

0,165 0,072 0,272 0,000 0,312 0,006 0,121 0,000 0.374 0.007 Ö.197 0,109 0,237 0,059 0,333 0.000

28 1« 42 3 70 5 17 1 71 62 47 57 53 16 20 6

0,269 0,144 0.243 0,017 0,455 0,032 0;29S 0,017 0,511 0,446 0.343 0,416 0,349 0,105 0,513 0.154

29 1< 53 30 67 26 22 5 50 25 39 38 57 19 8 2

0,299 0,144 0,306 0,173 0,435 0,169 0,379 0,066 0,360 0,180 0,285 0,277 0,375 0,125 0,205 0.051

16 5 82 1 79 1 7 0 7i 5 39 11 58 4 10 1

0,166 0,052 0.474 0,006 0,513 0,006 0,121 0,000 0,511 0.036 0.285 0,080 0,382 0,026 0,256 0.026

38 12 63 33 79 36 24 11 59 41 42 46 58 32 15 5

0,392 0,124 0,480 0,191 0,513 0,234 0,414 0.190 . 0.424 0,295 0,307 0,336 0,382 0,211 0,385 0.128

13 5 47 1 «9 14 5 0 50 87 »4 74 49 27 23 2

0,134 0,052 0,272 0,006 0,448 0,091 o;o86 0,000 n.367 0,626 0,394 0,540 0,322 0,178 0,590 0.051

22 13 48 0 60 4 12 0 51 81 50 64 45 16 18 4

0,227 0,134 0,277 0,000 0,390 0,026 0,207 0,000 0.367 0.583 0.365 0,467 0,296 0,105' 0,462 0.103


97 173 154 58 139 137 152 39

NZ = Number and Proportion of Technicians Who Received ZUE=0 and £1)1=0

NW = Number and Proportion of Technicians Who Did Not Work at that Job Activity

KM

REFERENCES

[1] Anderson, T. W. An Introduction to Multivariate Statistical Analysis. John Wiley and Sons, Inc., New York, 1958.

[2] Bartlett, M. S. The use of transformations. Biometrics, 1947, 3_, p. 39.

[3] Cooley, W. W. and Lohnes, P. R. Multivariate Data Analysis. John Wiley and Sons, Inc., New York, 1971.

[4] D'Agostina, R. B. Simple compact portable test of normality: Geary's test re- visited. Psychological Bulletin, 1970, 74, pp. 138-140.

[5] D'Agostina, R. B. and Cureton, E. E. Test of normality against skewed alternatives. Psychological Bulletin, 1972, 78, pp. 262-265.

[6] Dixon, W. J. and Massey, F. J. Introduction to Statistical Analysis. McGraw-Hill Book Company, Inc., New York, 1957.

[7] Draper, N. R. and Smith, H. Applied Regression Analysis. John Wiley and Sons, Inc., New York, 1966.

[8] Ferguson, T. S. Mathematical Statistics - A Decision Theoretic Approach. Academic Press, New York, 1967.

[9] Flanagan, J. C. Critical requirements: a new approach to employee evaluation. Personnel Psychology, 1949, 2, pp. 419-425.

[10] Guilford, J. P. Fundamental Statistics in Psychology and Education. McGraw-Hill Book Company, Inc., New York, 1965.

[11] Guttman, L. The basis for scalogram analysis. In S. A. Stouffer, et al., Measure- ment and Prediction. Princeton: Princeton University Press, 1950, pp. 60-90.

[12] Hogg, R. V. and Craig, A. T. Introduction to Mathematical Statistics. The Macmillian Company, 3rd edition, 1970.

[13] Jaspen, N. Serial correlation. Psychometrika, 1946, Vl_s pp. 23-30.

[14] Layard, M. W. J. Robust large-sample tests for homogeneity of variances. Journal of the American Statistical Association, 1973, 68, p. 195.

[15] Scheffe, H. The Analysis of Variance. John Wiley and Sons, Inc., London, 1959.

[16] Siegel, A. I. and Federman, P. J. Development of Performance Evaluative Measures: Investigation into and Application of aFleet Posttraining Performance Evaluation System. Wayne, Pa.: Applied Psychological Services, 1970.

[17] Siegel, A. I. and Fischl, M. A. Absolute scales of electronic job performance: Empirical validity of an absolute scaling technique. Wayne, Pa": Applied Psychological Services, 1965.

[18] Siegel, A. I. and Pfeiffer, M. G. Posttraining performance criterion development and application. Personnel Psychophysics: Estimating personnel subsystem reliability through magnitude estimation methods'. Wayne, Pa.: Applied Psychological Services, 1966.

[19] Siegel, A. I. and Schultz, D. G. Posttraining performance criterion development and application: A comparative multidimensional scaling analysis of the tasks )erformed by Naval aviation electronics technicians at two job levels. Wayne, Pa. ipplied Psychological Services, 1963. B

[20] Siegel, A. I., Schultz, D. G., and Lanterman, R. S. The development and application of absolute scales of electronic job performance. Wayne, Pa.: Applied Psychological Services, 1964.

[21] Whitlock, G. H. Application of the psychophysical law to performance evaluation. Personnel Psychology, 1949, 2, pp. 419-425.

DISTRIBUTION LIST

NAVY

4 Director, Personnel and Training 1 Research Programs (Code 458)

Office of Naval P^esearch Arlington, VA 22217

1 Director 1 ONR Branch Office 495 Summer Street Boston, MA 02210

1 Director 1 ONR Branch Office 1030 East Green Street Pasadena, CA 91101

1 Director 1 ONR Branch Office 536 South Clark Street Chicago, IL 60605

1 Commander (Code 20P) Operational Test and Evaluation Force 1 U.S. Naval Base Norfolk, VA 23511 ATTN: Mr. James Long

6 6 Director

Naval Research Laboratory Code 2627 Washington, DC 20390 1

12 Defense Documentation Center Cameron Station, Building 5 5010 Duke Street Alexandria, VA 22314 1

1 Chairman Behavioral Science Department Naval Command and Management Division U.S. Naval Academy 1 Luce Hall Annapolis, MD 21402

1 Commanding Officer Service School Command 3 U.S. Naval Training Center San Diego, CA 92133 ATTN: Code 303

Chief of Naval Training (Code 017) Naval Air Station Pensacola, FL 32508 ATTN: CAPT Allen E. McMichael

Chief of Naval Technical Training Naval Air Station Memphis (75) Millington, TN 38054 ATTN: Dr. G. D. Mayo

Chief Bureau of Medicine and Surgery Code 513 Washington, DC 20390

Chief Bureau of Medicine and Surgery Research Division (Code 713) Department of the Navy Washington, DC 20390

Commandant of the Marine Corps (Code A01M) Washington, DC 20380

Chief of Naval Personnel (Pers-A3) Navy Department Washington, DC 20370

Commander Naval Air Systems Command Navy Department, AIR-413C Washington, DC 20360

Commander Submarine Development Group Two Fleet Post Office New York, NY 09501

Commanding Officer Naval Medical Neuropsychiatric

Research Unit San Diego, CA 92152

Commanding Officer Naval Personnel and Training

Research Laboratory San Diego, CA 92152

Head, Personnel Measurement Staff 1 Capital Area Personnel Service Officer Ballston Tower No. 2, Room 1204 801 N. Randolph Street Arlington, VA 22203

Program Coordinator 1 Bureau of Medicine and Surgery (Code 71G) Department of the Navy Washington, DC 20390

Research Director, Code 06 Research and Evaluation Department 1 U.S. Naval Examining Center Building 2711 - Green Bay Area Great Lakes, IL 60088 ATTN: C.S. Winiewicz

Superintendent 1 Naval Postgraduate School Monterey, CA 93940 ATTN: Library (Code 2124)

1 Chief of Naval Personnel (Pers-llb) Navy Department Washington, DC 20370

COL George Caridakis Director, Office of Manpower Utilizatioi Headquarters, Marine Corps (A01H) MCB Quantico, VA 22134

Special Assistant for Research and Studies

OASN (M-RA) The Pentagon, Room 4E794 Washington, DC 20390

Mr. George N. Graine Naval Ship Systems Command (SHIPS 03H) Department of the Navy Washington, DC 20360

CDR Richard L. Martin, USN CONFAIRMIRAMAR F - 14 MAS Miramar, CA 92145

Mr. Lee Miller (AIR 413E) Naval Air Systems Command 5600 Columbia Pike Falls Church, VA 22042

Naval Submarine Medical Center Naval Submarine Base New London Groton, Connecticut 06340

Technical Director Personnel Research Division Bureau of Naval Personnel Washington, DC 20370

Technical Library (Pers-llB) Bureau of Naval Personnel Department of the Navy Washington, DC 20360

Technical Library Naval Ship Systems Command National Center Building 3 Room 3 S-08 Washington, DC 20360

Technical Reference Library Naval Medical Research Institute National Naval Medical Center Bethesda, MD 20014

1 Dr. Jamers J. Regan Code 55 Naval Training Device Center Orlando, FL 32813

1 Dr. A. L. Slafkosky Scientific Advisor (Code Ax) Commandant of the Marine Corps Washington, DC 20380

1 LCDR Charles J. Theisen, Jr., MSC CSOT Naval Air Development Center Warminster, PA 18974

1 Center for Naval Analyses 1401 Wilson Blvd Arlingtion, VA

ARMY

1 Behavioral Sciences Division Office of Chief of Research and

Development Department of the Army Washington, DC 20310

U.S. Army Behavior and Systems Research Laboratory

Rosslyn Commonwealth Building, Room 239

1300 Wilson Boulevard Arlington, VA 22209

Director of Research U.S. Army Armor Human Research Unit ATTN: Library Building 2422 Morade Street Fort Knox, KY 40121

COMMANDANT U.S. Army Adjutant General School Fort Benjamin Harrison, IN 46216 ATTN: ATSAG-EA

Commanding Officer ATTN: LTC Montogomery USACDC - PASA Ft. Benjamin Harrison, IN 46249

Director Behavioral Sciences Laboratory U.S. Army Research Institute of

Environmental Medicine Natick, MA 01760

AIR FORCE

Dr. Robert A. Bottenberg AFHRL/PHS Lackland AFB Texas 78236

ARHRL (TR/Dr. G.A, Eckstrand) Wright-Patterson Air Force Base Ohio 45433

Headquarters, U.S. Air Force Chief, Personnel Research and Analysis Division (AFPDPL-R)

Washington, DC 20330

AFHRL/MD 701 Prince Street Room 200 Alexandria, VA 22314

AF0SR (NL) 1400 Wilson Boulevard Arlington, VA 22209

COMMANDANT USAF School of Aerospace Medicine ATTN: Aeromedical Library (SCL-4) Brooks AFB, TX 78235

Commandant United States Army Infantry School ATTN: ATSIN-H Fort Benning, GA 31905

U.S. Army Research Institute Room 239 Commonwealth Building 1300 Wilson Boulevard Arlington, VA 22209 ATTN: Dr. R. Dusek

Armed Forces Staff College Norfolk, VA 23511 ATTN: Library

Mr.'Edmund Fuchs BESRL Commonwealth Building, Room 239 1320 Wilson Boulevard Arlington, VA 22209

1 Personnel Research Division AFHRL Lackland Air Force Base San Antonio, TX 78236

1 Headquarters, U.S. Air Force Chief, Personnel Research and Analysis Division (AF/SPXY)


1 Research and Analysis Division AF/DPXYR Washington, DC 20330

1 CAPT Jack Thorpe USAF Dept. of Psychology Bowling Green State University Bowling Green, OH 43403

DOD

Mr Joseph J. Cowan, Chief Psychological Research Branch (P-l) U.S. Coast Guard Headquarters 400 Seventh Street, SW


1 Dr. Ralph R. Canter 1 Director for Manpower Research Office of Secretary of Defense The Pentagon, Room 3C980

OTHER GOVERNMENT 1

1 CNO (0P-964) Room 4A 538 Pentagon Washington, DC 1

1 CNO (OP-987) Room AB 525 Pentagon Washington, DC

1 1 CNO (OP-987F)

Room 4B 489 Pentagon Washington, DC

1 Dr. Lorraine D. Eyde 1 Bureau of Intergovernmental Personnel

Programs Room 2519 U. S. Civil Service Commission 1900 E. Street, NW Washington, DC 20415 1

MISCELLANEOUS

CAPT John F. Riley, USN Commanding Officer, U.S. Naval

Amphibious School Coronado, CA 92155

Dr. Scarvia Anderson Executive Director for Special

Development Educational Testing Service Princeton, NJ 08540

Professor John Annett The Open University Waltonteale, BLETCHLEY Bucks, ENGLAND

Dr. Bernard M. Bass University of Rochester Management Research Center Rochester, NY 14627

Dr. David G. Bowers Institute for Social Research University of Michigan Ann Arbor, MI 48106

Dr. Kenneth E. Clark University of Rochester College of Arts and Sciences River Campus Station Rochester, NY 14627

Dr. Rene' V. Dawis Department of Psychology 324 Elliott Hall Univeristy of Minnesota Minneapolis, MN 55455

Dr. Marvin Dunnette University of Minnesota Department of Psychology Elliott Hall Minneapolis, MN 55455

ERIC Processing and Reference Facility 4833 Rugby Avenue Bethesda, MD 20014

Dr. Victor Fields Department of Psychology Mbntogomery College Rockville, MD 20850

Professor Gerald L. Thompson Carnegie-Mellon University Graduate School of Industrial Admin. Pittsburgh, PA 15213

Dr. David Weiss University of Minnesota Department of Psychology Elliott Hall Minneapolis, MN 55455

Dr. Albert S. Glickman American Institutes for Research 8555 Sixteenth Street Silver Spring, MD 20919

Dr. Richard C. Atkinson Department of Psychology Stanford University Stanford, CA 94305

LCOL Austin W, Kibler, Director Human Resources Research Office ARPA 1400 Wilson Boulevard Arlington, VA 22209

Century Research Corporation 4113 Lee Highway Arlington, VA 22207

Dr. Joseph W. Rigney Behavioral Technology Laboratories University of Southern California 3717 South Grand Los Angeles, CA 90007

Dr. Lawrence B. Johnson Lawrence Johnson and Associates, Inc. 2001 "S" Street, NW Suite 502 Washington, DC 20009

Dr. Stanley M. Nealy Department of Psychology Colorado State University Fort Collins, CO 80521

Dr. Robert D. Pritchard Assistant Professor of Psychology Purdue University Lafayette, IN 47907

Dr. Diane M. Ramsey-Klee R-K Research & System Design 3942 Fidgemont Drive Malibu, CA 90265

Dr. Leonard L. Rosenbaum, Chairman Department of Psychology Montgomery College Rockville, MD 20850

Dr. George E. Rowland Rowland and Company, Inc. Post Office Box 61 Haddonfield, NJ 08033

Dr. Frederick M. Lord Educational Testing Service Princeton, NJ 08540

Dr. Richard S. Hatch Decision Systems Associates, Inc. 11428 Rockville Pike Rockville, MD 20852

Dr. Robert R. Mackie Human Factors Research, Inc. Santa Barbara Research Park 6780 Cortona Drive Goleta, CA 93017

Mr. Luigi Petrullo 2431 North Edgewood Street Arlington, VA 22207

Psychological Abstracts American Psychological Association 1200 Seventeenth Street, NW Washington, DC 20036

Dr. Arthur I. Siegel Applied Psychological Services Science Center 404 East Lancaster Avenue Wayne, PA 19087

Dr. Henry Solomon George Washington University Department of Economics Washington, DC 20006

Dr. Benjamin Schneider Department of Psychology University of Maryland College Park, MD 20742

U15A574

CM AND DEVELOPMENT LABORATORY - DTIC · 2018. 11. 8. · linear regression analysis. An emphasis on...

Documents

Transcript of CM AND DEVELOPMENT LABORATORY - DTIC · 2018. 11. 8. · linear regression analysis. An emphasis on...