A simulation of the impacts of machine learning to combine psychometric employee selection system predictors on performance prediction, adverse impact, and number of dropped predictors
通过模拟12亿参与者的数据，比较现代机器学习与传统回归在员工选拔中预测工作绩效的效果，发现机器学习主要在丢弃预测因子而非提升预测精度上有优势，并建议未来研究关注非传统设计场景。
Abstract We compare modern machine learning (MML) techniques to ordinary least squares (OLS) regression on out‐of‐sample (OOS) operational validity, adverse impact, and dropped predictor counts within a common selection scenario: the prediction of job performance from a battery of diverse psychometrically‐validated tests. In total, scores from 1.2 billion validation study participants were simulated to describe outcomes across 31,752 combinations selection system design and scoring decisions. The most consistently valuable improvement from adopting MML over traditional regression was from dropping predictors rather than by improving prediction. On average, MML improved prediction of performance from psychometric scale composites only when the ratio of sample size to scale count was less than approximately 3, although algorithm choice, predictor count, and selection ratio affected outcomes as well. We also simulated the effects of design choices when combining item scores, which showed consistent, superior predictive accuracy for several MML algorithms, especially elastic net and random forest, over traditional regression. Given these results, we suggest the potential of machine learning for employee selection is unlikely to be realized in selection systems focusing on the combination of scale composites from previously validated psychometric tests. Instead, it will be realized in unconventional design scenarios, such as the use of individual items to make multiple trait inferences, or with novel data formats like text, image, audio, video, and behavioral traces. We therefore recommend researchers focus on the potential value of MML in future selection contexts rather than continuing to focus on the current value of MML in current selection contexts.