Assess AI-based assessments in personnel selection

Talent management tools and technologies using artificial intelligence (AI) have proliferated in recent years, being sold to organizations with the promise of improving recruitment, selection and development strategies.

There are, however, concerns about the shortcomings and potential biases of AI-based assessments. The Society for Industrial and Organizational Psychology (SIOP) offers guidance in a brief video, “IA and Selection,” to help employers make realistic assessments of these tools.

In the presentation, Fred Oswald, Professor of Industrial Organizational Psychology (IO) at Rice University, outlines four key considerations surrounding the use of AI-based assessment tools.

  1. AI-based assessments should meet professional testing standards that reflect the consensus of professional opinion on fair and accurate assessments. Standards for the Validation and Use of Personnel Selection Procedures and Educational and Psychological Testing can provide employers with guidance on the validity, reliability and fairness of AI-based tools and are freely available online.
  2. AI-based assessments must be relevant to the job, measuring the knowledge, skills, abilities or other characteristics (KSAO) needed to perform the job. While not all work-relevant KSAOs need to be measured, those that are not work-relevant need not be. Good data is more important than “big data”.
  3. AI-based assessments should minimize bias in score differences between different demographic groups. Yet employers should demand more than freedom from bias – after all, a coin toss is an unbiased method of selection. For a tool to be both fair and effective, it must also be relevant to the job (see #2).
  4. AI-based assessments are multifaceted. To minimize bias and error, employers should consider and evaluate many of the components and aspects of these tools, including:
    • Interactive technologies that candidates will use with the tool, such as games, video interviews, etc.
    • How the data will be extracted and transformed before being subjected to machine learning.
    • Real machine learning algorithms and data mining techniques that will be applied to the data.
    • Scores that will emerge from machine learning analyses.

This video is part of the “Spotlight on Science: Applying Research Outcomes for Better Workplaces” series, a repository of SIOP evidence-based HR resources produced in partnership with SHRM. These materials present data that infuses the science of IO psychology and other HR disciplines into management practices for everyday use.

Comments are closed.