Validity generalization and hypothetical reliability distributions: a test of the Schmidt-Hunter procedure

Article Abstract:

A Monte Carlo study was conducted to determine how variations in true reliability distributions affect validity generation estimates that are based on hypothetical reliability distributions proposed by Schmidt and Hunter. Both interactive and noninteractive validity generalization equations were examined. True reliability distributions and sample sizes were systematically varied. Depending on the sample size and nature of the true distributions, artifactual variance was either overestimated or underestimated by both equations. In some cases, extremely large errors were observed. Even when the data were generated with true reliability distributions identical to the hypothetical distributions, both equations systematically overestimated artifactual variance. The severity of estimation errors increased proportionally as sample size per validity coefficient increased. Implications of using assumed distributions and recommendations for validity generalization researchers are discussed. (Reprinted by permission of the publisher.)

Monte Carlo method, Monte Carlo methods, Psychology, Applied, Applied psychology

---

A note on the sampling variance of the mean uncorrected correlation in meta-analysis and validity generalization

Article Abstract:

This article compares the accuracy of several formulas for the standard error of the mean uncorrected correlation in meta-analytic and validity generalization studies. The effect of computing the mean correlation by weighting the correlation in each study by its sample size is also studied. On the basis of formal analysis and simulation studies, it is concluded that the common formula for the sampling variance of the mean correlation, Vt = Vr/K where K is the number of studies in the meta-analysis, gives reasonably accurate results. This formula gives accurate results even when sample sizes and ps are unequal and regardless of whether or not the statistical artifacts vary from study to study. It is also shown that using sample-size weighting may result in underestimation of the standard error of the mean uncorrected correlation when there are outlier sample sizes. (Reprinted by permission of the publisher.)

Meta-analysis

---

Bootstrap estimates of standard errors in validity generalization

Article Abstract:

Bootstrapping is introduced as a method for approximating the standard errors of validity generalization (VG) estimates. A Monte Carlo study was conducted to evaluate the accuracy of bootstrap validity-distribution parameter estimates, and nonparametric bootstrap confidence intervals. In the simulation study we manipulated the sample sizes per correlation coefficient, the number of coefficients per VG analysis, and the variance of the distribution of true correlation coefficients. The results indicate that the standard error estimates produced by the bootstrapping procedure were very accurate. It is recommended that the bootstrap standard-error estimates and confidence intervals be used in the interpretation of the results of VG analyses. (Reprinted by permission of the publisher.)

Similar abstracts:

• Abstracts: Validity generalization in the context of situational models. A note on validity generalization procedures
• Abstracts: Operationalization of goal difficulty as a moderator of the goal difficulty-performance relationship. Test of the mediating role of goals in the incentive-performance relationship
• Abstracts: The effects of television consumption on social perceptions: the use of priming procedures to investigate psychological processes
• Abstracts: Antecedents and outcomes of work-family conflict: testing a model of the work-family interface. Chronic occupational stressors, self-focused attention, and well-being: testing a cybernetic model of stress

This website is not affiliated with document authors or copyright owners. This page is provided for informational purposes only. Unintentional errors are possible.