talent-initiative-evaluation
talent-initiative-evaluation
talent-initiative-evaluation

Evaluate Solution Validity & Decision Quality

Classification of applicants to staff positions, promotion of top performers, downsizing a workforce, acquiring one of several competitors and team leader emergence are all selection decision problems. Transparent applicant selection decisions require evaluatory evidence of the reliability, content validity, construct validity, criterion validity and fairness of selection solutions and component assessment tools. Moreover, the tests, measures, rating scales and simulations comprising predictive selection models are only useful to the extent they inform HR decision making better than alternative selection tool options. Therefore, it is essential the theoretic-based rationales and accumulated empirical evidence support the appropriateness of the interpretations, inferences and actions taken from selection solution scores.

Generating quantitative evidence to evaluate a selection solution necessarily involves research design. Alternative selection models such as multiple cutoff, multiple regression, and multiple hurdle sequential models mandate different designs and analyses to collect situationally specific supporting evidence. There are also applicant, employee, primary study and/or position element sampling issues to consider. When generating evidence for selection solutions it is important to understand the local tradeoffs of sampling current employees when calibrating selection solutions during concurrent validation studies, sampling position candidates when piloting selection solutions during predictive validation designs and sampling studies when leveraging meta-analytic evidence to underpin validity generalization arguments. When sufficient samples do not exist synthetic validity studies provide a useful and defensible approach.

Typically the emphasis when evidencing selection solutions is on minimizing selection errors by maximizing metric reliability, measurement accuracy and predictor performance criteria relationships. Reliability, validity and predictive efficiency are unarguably essential technical components of selection solutions but there are deeper considerations that are often even more critical when selecting talent. For example, alternative predictive validities, selection ratios, base rates and predictor set cutoff scores collectively shape the overall value of an HR selection solution. Understanding the relative importance of correlated selection tools is also critical to yielding a clearer picture of which predictors comprising a multicomponent solution matter most for predicting performance and warrant continued investment.

It is also essential to document the incremental validity of each test or measure of a predictor battery because this evidence suggests it adds something unique to the prediction of a performance criterion. Each core test or measure comprising a sequenced multicomponent selection solution must have incremental validity over the other tools already administered in order justify the investment in its use. The incremental validity of a test or measure predictor for selection should never be taken for granted. Employers often depend upon subjective telephone-based interviews to initially screen new candidates. If sales applicants are subject to an unstructured interview which measures their interpersonal acumen then the later use of a personality scale measuring extraversion may only yield redundant information. In addition to informally measuring interpersonal skills, cognitive ability and knowledge, other attributes are also inadvertently assessed by applications and interviews such as conscientiousness (completing all sections of an application, being on time), extraversion (influencing an interviewer to approve one for the next hurdle of the selection process), and emotional stability (maintaining composure under stress).

The above example also illuminates the importance of precisely understanding how HR representatives collect, combine, and apply the information generated from the use of applicant selection protocols. Interviewers and other HR officers detect and combine applicant cues such a years of prior experience, grade point average, interview scores and knowledge scores to make fit judgments and hiring decisions. This wide-spread practice is based on the belief superior candidates can be easily identified by those in the know. Unfortunately, different people use different decision rules when synthesizing applicant data and even experts are often ineffective at subjectively combining predictor data to make hiring decisions. Talent Threshold can help you understand the decision policy employed to evaluate your applicants relative to the optimal policy as well as how to strike the ideal balance of human and mechanical touch.

In addition to the reliability, validity and predictive efficiency technical components of a selection solution, and the selection ratio, base rate and cutoff situational aspects shaping the value of selection, the outcomes of selection decisions must also be evaluated to tune protocols and justify HR resources. Although the prior considerations minimize erroneous rejections and acceptances, as well as maximize correct rejections and acceptances, they are still deficient for failing to fully reflect the different inherent utilities placed on these four selection decision outcomes. A singular hits and misses focus ignores the degree errors are reduced, selection costs, opportunity loss and the tight interconnections between applicant selection and other HR processes (recruiting, training). A decision theory approach to selection also estimates the costs and benefits of alternative options in light of capital budgeting considerations such as taxation and discounting so investments are competed.

Talent Threshold implements selection solutions for positions from the shop floor to the c-suite and we believe in directly addressing the subtle but critical HR selection issues some others far too often ignore. We can help you document required selection evidence and apply it to field an optimized HR solution. Our advisors identify and execute the most appropriate validation approach given localized contextual affordances and constraints yielding the predictor dominance and incremental validity evidence to gauge if HR investments explain anything unique in performance or if limited HR resources are wasted. We also help you understand the current and optimal decision policies of your selection practices. Our advisors also help you gauge the return on investment and net present value of selection solutions. To speak to a talent advisor about selection solutions please email us at advisor@talentthreshold.com.