In 2022, AI recruitment and assessment tools are still in their infancy and, in many cases,
are “less than perfect”. Ai recruiting systems are plagued by problems that limit AI
development in many fields. The most important of these is that machines cannot use
abstract reasoning to perform complex reasoning tasks and extract deep meaning from
data. Even the most advanced AIs are far from replicating the human brain’s capabilities.
One of the essential functions of the human brain is the ability to understand other people
and to attribute meaning to their behaviour. This requires a higher-level analysis of
cognition that can see the whole from the sum of its parts. We are primarily talking about
psychological science here. Psychology is based on the idea that many factors (or
“individual differences”) make each of us unique. Psychology attempts to study and
measure these individual differences to understand human behaviour’s causes and
predict how individuals will behave in certain future situations.
When hiring, measuring and interpreting individual differences is key to determining a
candidate’s success. Psychologists conduct behavioural studies specifically for this
purpose. For an “AI Psychologist” to do an equal or better job than humans, they need to
understand and interpret individual differences the same way as humans. In other words,
AI researchers need to think like psychologists to make a real difference.
This begs the question; how do psychologists measure individual differences?
Psychologists measure individual differences using psychometric methods that
emphasise the interaction of cognition and expert judgment.
Psychometrics is: “A field of study dealing with the theoretical and methodological
assessments of psychology, including the measurement of knowledge, skills, attitudes,
and personal characteristics. This field is primarily concerned with the study of differences
between people. It consists of two main components.
Research Project Areas,
1.Equipment design and measurement methods
2.The development and improvement of theoretical measurement methods.”
Psychology provides us with the tools to ensure accurate and reliable measurements of
the individual characteristics that make us who we are (e.g., our personality, attitudes,
intelligence etc.). In psychometric science, we refer to these properties as “constructs”.
Constructs are defined as:
“a proposed characteristic of a person, which usually cannot be measured directly, but can
be measured by several measurable indicators or variables.”
Measuring constructs requires a set of expertly constructed tools that can “see below the
surface” of a person to understand their individual differences.
Psychometrically robust measures of structure-based individual differences are
fundamental in establishing quality recruitment assessments. By creating predictive
recruiting tools, trained professionals (usually Industrial or Organisational psychologists)
use their judgment to identify the specific constructs needed to succeed in a particular
job, then psychometric tools to assess the candidate’s status in determining the
relationship to those constructs.
Artificial intelligence does not rely on individual differences because it cannot understand
them. Instead of making expert predictions based on psychometric measures of
constructs, AI essentially “learns” patterns in the data and applies that learning to make
predictions on new data sets. For instance, AI can examine datasets such as CVs to
identify word and phrase structures that can predict job success. This AI can be “trained”
on a dataset based on the CVs of high-performing incumbents.
The AI compares patterns in the data in the applicants’ CVs to patterns in the information
in the CVs of successful employees. While this approach is viable, it represents a
theoretical, data-driven approach to predictive modelling known as Dustbowl empiricism.
Thus, Although AI can be trained to recognise “signatures” that can predict human
behaviour, it cannot generalise this information to match psychometrically measured
constructs. Without this understanding, AI cannot identify individual differences. An AI’s
inability to understand people through the lens of individual differences means that AI
systems make bad psychologists!
What are the Key Issues surrounding the use of recruitment AI?
∙We are still in the early days of AI-based recruitment technologies, and many do
not meet muster. Fully automated recruitment bots have yet to outperform reliable
models for measuring individual differences using cognitive tools.
∙AI behaviours are not generic. AI struggles to generalise in many situations. AI tools
are very specific to the dataset they were trained on. Thus, results may be difficult
to replicate when applying AI to new data. This issue is called model “overfitting”
and can be highly problematic when applying AI to other contexts. Empirically
keyed models, such as those used in AI, tend to be unstable over time and need to
be constantly retrained.
∙AI comes with risks. Although it is not illegal to use AI to conduct personnel
research, it is required to adhere to the same standards as all recruitment research.
Ai tools must demonstrate that their performance assessments are attributable to
critical business success factors. Compliance is at risk if an AI cannot be shown to
measure these parameters directly. This is especially true for complex neural
networks that make up deep learning AIs.
∙Empirical methods of pattern recognition and AI-assisted decision-making can
introduce biases. Although machines can be trained to avoid these biases, the risk
remains. This is the case with very sophisticated black boxes that learn in ways that
cannot be fully explained. Fortunately, many research professionals using AI are
doing the right thing, evaluating processes and eliminating bias.
Significant resources have been invested in developing artificial intelligence that can use
large datasets to predict career success; individual psychometrics remain essential for
accurate employee evaluation.
Although AI technology continues to develop and forge forward through solid investment,
it is far from reaching a human-like understanding. This will take some time and may well
require human-machine collaboration. Chess is an excellent example of what can happen
when humans and machines work together. While a computer can beat any human 100%
of the time, a computer-human playing together can beat even the most advanced chess
computer. AIs don’t need higher intelligence to help us hire good people. Today there are
many scenarios where AI works with psychologists to help them do their jobs better. In
terms of recruitment, combining AI and synergistic assessments can assist organisations
in reliably improving their ability to predict business success by understanding individual
differences.
So, what does the future hold?
Beware of “Silver bullet” solutions claiming an immediate effect on results. The
implementation of “New AI” requires a long-term approach. Think of your data as
constructs, not just patterns. Don’t discount psychometric-based measurements as part of
the assessment mix. Don’t rely solely on data patterns to predict career success.
An intelligent approach may be to assemble multidisciplinary teams consisting:
Psychologists, Psychometricians, Data Scientists, Developers and Business / HR SMEs.
Firstly, conduct a thorough job analysis and then collaborate on using AI to build the
psychometric models needed to understand and measure individual differences.
Success will require psychologists and computer scientists to understand and accept
each other’s perspectives. Although challenging, there is a lot to gain by using AI to help
psychologists better understand individual differences.
While AI may be able to automate some aspects of the recruitment process, it is unlikely
to fully replace human recruiters. Instead, AI can be seen as a tool to help recruiters be
more efficient and effective in their roles. In addition, by automating some of the more
mundane tasks associated with recruitment, AI can free up recruiters to focus on more
strategic tasks, such as identifying top talent.
