Speech Synthesis – Listen using AWS Polly
Voice is pitched lower and speaks more slowly. This creates the normal pauses a listener would expect when a speaker breathes or pauses during speech. Synthesizer is unable to identify irony or sarcasm. A normal human speaker has either rehearsed the transcript or reads ahead while speaking to catch the change in tone.
The single faulty assumption of every employment test for professional and technical jobs is that it predicts job success. There is no evidence to support this claim.
Consider the one test most of us are familiar with, the SAT. The SAT does have enough quantitative data to be predictive. The purpose of the SAT is to predict success in college. But, what do they mean by success? Their own study used data from 223,000 students and 171 four year colleges and only predicts the likelihood of a higher GPA in the first year of college and a student’s return from their first year in college to a second year. Beyond that it has no predictive power. It doesn’t predict the likelihood of graduating, furthering your education or even comprehension of the material. With that much data and it’s limited predictive power, how could an employment test possibly have any meaning at all? Well, a few testing business are working hard to convince us their products can.
A study touted by the Criteria Corp, a commercial provider of pre-employment tests, predicts that scoring well on a pre-employment test can increase an employee’s tenure (note, not performance) from 11-18%. Criteria Corp asserts, “The impact of this study, however, may be greater than many of its predecessors because of the pedigree and credibility of its source.” True enough, the study was conducted by a respected, private, non-profit organization, National Bureau of Economic Research (NBER)
The abiliity to predict that increase in percent is significant, especially when you consider companies with tens if not hundreds of thousands of employees. But like every study, you need to look closely at what they studied. The NBER studied 15 firms, 300,000 hires and 555 hiring managers for entry level service sector employees in industries where the median worker stays 99 days. Yes, that’s right, 99 days. You cannot take the logical leap that this study is applicable to any industry whose metrics don’t meet the study’s criteria. So, it’s utility is limited to only those entry level service sector occupations with short tenures. Not much use for any hiring I’ve done recently and certainly of no value for any professional job
Also, consider this article and graph published in the Harvard Business Review (HBR) This article in HBR was based on a seminal work published by Frank L. Schmdit, which was a meta analysis of the previous 100 years of studies done on assessing candidates.
HBR published this graph, with this introductory text, “The table below shows the predictive validity of some commonly used selection practices, sorted from most effective to least effective, according to Frank L. Schmidt’s latest analysis that was shared at the Personnel Testing Counsel Metropolitan Washington chapter meeting this past November…”
Wait a minute? Read the text at the top of the graph, Schmidt states, “The higher the test score, the higher correlation between the test scores and predicted job performance.” Note the word predicted, not actual. The graph simply communicates which tests are better at predicting a prediction – not the job itself.
I thought this might be a mistake in HBR’s graphic, so I read through Schmidt’s paper and some accompanying papers that conducted a similiar meta-analysis of job performance testing and studies. In general, the consensus is that General Cognititive Ability (i.e., IQ Tests) are predictive in some cases, but have only the same predictive ability as 5 years of experience in a similar job. The consensus, but not proof, is if candidates interviewing for jobs with no experience or training are smarter, the faster they learn, the faster they learn, the more productive they become.
While that may be true for a small minority of jobs whose success can be comprehensively and quantitatively measured, the major flaw of testing is for every other profession is there is no uniform and repeatable way to measure success in a job. Without that measurement, it is meaningless to attempt to match a quantitative analysis of a pre-employment test against the abstraction of job success.
So, with all that in mind, I believe a more effective way to vet a potential employee for a position that relies on judgement, insight and experience is to put a test in front of them, if they give you a puzzled look and ask why? Stop the test, and hire them.
Sources:
Validity of the SAT® for Predicting First-Year Grades and Retention to the Second Year May 2019
CriteriaCorp – https://tinyurl.com/y8sh3hnw
NBER – Testing Versus Manager Discretion in Hiring
Copyright Honner Consulting LLC 2020