Eddy’s HR Mavericks Podcast
Created by Eddy and our HR Mavericks community.
Using Behavioral Assessments in Talent Acquisition w/ Fred Rafilson
In episode 72, Fred Rafilson draws on his knowledge of I/O psychology to explain how behavioral assessments work and why they’re beneficial.
Episode 72
It takes two things to make a good employee: the right skills and the right behavior. Fred Rafilson, chief I/O psychologist at Talview, doesn’t underestimate the importance of skills—but he also knows that neglecting to measure candidates’ behavioral attributes could be a big hiring mistake. The best candidates have the knowledge and abilities to perform a job, and the behavioral tendencies (motivation, attitude, and personality) to become standout employees. In this episode, Fred draws on his knowledge of I/O psychology to explain how behavioral assessments work and why they’re beneficial. We discussed:
- Why it’s important to screen candidates for both skills and behavioral traits
- The qualities that behavioral assessments measure
- How providers ensure that the tests are fair
- What a behavioral assessment might look like—and what goes on behind the scenes
- Three things that the best assessments have in common
Fred Rafilson
Full Transcript
[00:00:00] Garrett Jestice: Welcome to the next episode of the HR Mavericks podcast. I'm Garrett Jestice, and today I'm joined by Fred Rafilson, who's the chief I/O psychologist at Talview. Fred, how you doing today?
[00:00:13] Fred Rafilson: I'm doing really well, Garrett. Thank you.
[00:00:15] Garrett Jestice: Thanks for being on the show today. How's the weather in, beautiful, Sunny Bend, Oregon today?
[00:00:20] Fred Rafilson: Well, that's a great question because it's been snowing for the last two days and, I'm pretty excited. I've got my season pass coming from the mountain, so it's perfect.
[00:00:28] Garrett Jestice: That's awesome. I've, I've been thinking the same thing. You know, we're based in Utah here and winter just decided to come the last few days, but looking forward to hitting the ski slope soon too. So, Fred, we're, we're excited to have you on the show today. and, to dive into this really interesting topic.
Before we do that though, give our listeners a little bit more context on and about you and your background as well as what your company Talview does.
[00:00:52] Fred Rafilson: Sure, thanks. I'm an industrial and organizational psychologist, an I/O psychologist, and what we do is we focus on the psychology of people and work. and specifically, I'm a testing guy. I'm an assessment guy. I'm all about measuring people and determining what are their skills, abilities, knowledge, you know, what are their competencies as, in, in terms of are they gonna be predictive of success on a job?
So I've been doing this for, longer than I care to say, you know, I'm just gonna throw out, a few decades is good enough. I had my own, business years ago. We developed, exams for police and fire departments around the country. Became the nation's largest provider of those, I since sold that to my employees.
moved up to Bend and I've worked for several large, testing companies as their chief I/O psychologist, until I found, Talview which I really love. It's an amazing company. We do some really cool things. That I'll tell you about. And, I came to tell you, and I'm the chief I/O psychologist here.
[00:01:46] Fred Rafilson: So I work with our customers to ensure that their assessment programs are number one valid, they're helping them select the best people. number two, fair. They're doing it in a way that works the same way for everyone. And number three, that our candidates have a great experience going through the process. so in a nutshell, that's what I do. Talview is, I, I struggle with the best way...
it's, it's an end to end talent measurement platform and it's AI driven, really state of the science. So we start with everything from reading resumes to administering assessments that are, you know, job related based on the requirements of your job to, asynchronous interviews prerecorded with psycho linguistic capabilities, which we'll talk about later to help not only look at the answers to the interview questions, but look at the behavioral traits of of the candidates that are answering them.
To live interviews and scheduling, full suites of assessments so we can really help an organization, acquire the, the right talent for their positions.
[00:02:46] Garrett Jestice: I love it. That's so great. It's such a great elevator speech. I think a, a great precursor to what we're gonna talk about today. Tell me a little bit more about, who your typical customers are. Who do you typically work with at Talview? Is it all across the board or are there, are there certain types of companies that have really found value with working with Talview?
[00:03:02] Fred Rafilson: So I guess two answers. One would be industry, and the second would be, you know, size of organization. we work in all industries, right? Everybody needs to hire. And, especially these days, it's really tricky because, you know, good candidates, they're, they're not sticky. If we can't get them right away, we're gonna lose them to somebody else.
So, you know, hiring is really, in an interesting place right now.
[00:03:24] Garrett Jestice: Mm-hmm.
[00:03:25] Fred Rafilson: All industries. we tend to focus on enterprise organizations because they hire thousands of people and have tens of thousands of organizations of, of, candidates for these jobs. But we work with medium and small businesses as well.
I've always said, especially being a business owner myself, a small business owner, you know, clearly when you have to hire thousands of people, Making sure you make valid decisions is critical, right?
[00:03:51] Garrett Jestice: Hmm.
[00:03:52] Fred Rafilson: Hire one person, you make a bad decision there, it's really not a good thing. So, what we applicable to any size of organization, in any industry,
[00:04:01] Garrett Jestice: Yeah. I love that. And I think our topic today is too, so I think what we're gonna talk about today is really this idea of behavioral assessments, right? And how do they, how should they, and how can they help in your talent acquisition efforts? And that's a topic that I think can be applicable for any size business.
Right. And so to really just start us on this path, tell us a little bit more about what behavioral assessments are and the different types of insights they can provide when you are interviewing job candidates.
[00:04:31] Fred Rafilson: Yeah, that's a great question. I think people tend to think of assessments as skills assessments, right? Does a person, can they code? Do they have the medical knowledge they need to be, an ER tech? you know, do they have the finance knowledge they need to, to work in a bank? And of course, those skills are critical, right?
If you don't know, if you don't have coding skills, you're not gonna be much of a software development, right? You don't have the medical skills, I don't want you looking at me when I come to the ER. So skills are critical. But skills are only a part of the puzzle, right? It takes two things to make a good employee, and we know this through years of research, and I think common sense.
It takes two things. One is what you can do, and that's skills. And the other is what you will do. And that's your behavioral tendencies: your motivation, your attitude, your personality. A person can have all the skills in the world, but if they're not motivated to come to work and they don't wanna be a team player and they can't get along and what have you, they're gonna be a terrible employee, right?
On the other hand, someone might have, you know, great motivation, great attitude, just an all-around great person, but they don't have the skills. That's not gonna work either. So you need to assess both. And what we're here to talk about today is really looking at those behavioral things.
So, what kinds of things do we measure? And I, I touched on it a little bit. We really try to predict how a person is going to behave in a particular situation. And we do that by looking at their personality traits. we look at their attitude, we look at their communication skills, and I use the word skills as sort of a blurry line sometimes.
we look at their integrity and we think about all of the jobs where integrity is key, where people have access to money and merchandise and sensitive records, et cetera. You wanna make sure that you know, We'll have a good moral compass and are gonna do the right thing and aren't gonna steal, aren't gonna use drugs on the job, aren't gonna engage in counterproductive behaviors.
[00:06:32] Fred Rafilson: All of those kinds of things. You know, teamwork orientation -- with remote working now and, and, and teams and, the importance of working in teams. We wanna make sure that people are suited for that. some people are really suited for working independently, and that's great for certain jobs. but it's difficult to take a person like that, make them part of a team and expect them to function if they're not wired that way.
Certainly there's training, but we wanna select the right people. And vice versa. We don't wanna take someone who needs that social contact and needs that teamwork and put them in a position where they're never gonna talk to anyone. They're not gonna be happy, they're not gonna be engaged.
[00:07:06] Garrett Jestice: Yeah.
[00:07:07] Fred Rafilson: So those are the types of things that we look at with behavioral assessments.
I always kind of liken it to when people say, well, you know, we just need to measure their skills. And I'm like, that's awesome. But like when I go to a restaurant and I get a hamburger, I don't just ask for the patty. The patty's great, but I really like the bun and all the condiments and everything else that's in there because that's a hamburger.
Right? I know. It's kinda a crazy analogy. I'm hungry. It's, it's almost lunchtime here, but, you need to measure the whole thing, not just, not just what this goes, not just the patty. We need the entire package and that's where behavioral assessment comes in.
[00:07:44] Garrett Jestice: Yeah, that makes total sense. And I, I'm gonna jump in just to, we're just gonna dive into the deep end here with this next question because I'm curious, you know, your thoughts on this. because I'm sure there's people out there who, who, who get it. They're from a business perspective, like you need to be able to, to test skills and these behavioral traits and tendencies.
But I, I think there's also people who might be wondering, are these behavioral assessments fair to do? Like when someone asks you that question, what's, what's your response to that?
[00:08:15] Fred Rafilson: So my response is, yes, they're fair. And I'll tell you why I believe that. One, we have decades of research. I/O psychology is not a new field, and we can talk about that later. But, We have decades of research to show that, different types of personality characteristics do indeed predict success at work.
So we know empirically that there's a correlation between certain types of personalities and success on certain types of job. So they're fair in terms of being job related. They're also fair in terms of well developed tests, properly constructive tests being provided by reputable providers. I mean, God only knows there's a million things out on the internet, so be careful.
You know, what you buy or what, what you read. We all know that now. Right. But assuming you're dealing with, you know, respectable test vendors, we also know that the tests are fair and the reason is that we go to great lengths to ensure that they're fair. So that starts with the, very beginning with item construction.
And when I say items, I mean test questions. So when we're developing questions to measure a particular construct, whatever that may be, teamwork orientation, you know, we as psychologists know that, well, this question should be predictive of that. This question should be predictive of that. We write a big pool of questions, more than we think we're gonna need. We then review them very carefully, for any signs of bias at all. And we even bring people in who are experts at looking for cultural bias, gender bias, and they review the questions. And so we all as a group say that question should be good. but the thing is, we're all human and we all have unconscious biases.
[00:09:50] Garrett Jestice: mm.
[00:09:51] Fred Rafilson: Anyone tells you they don't, they just don't know themselves. We can't help it. We grow up in a particular society. We know what we know. You know what you know, I know what I know. We all have our biases. Okay? and it doesn't mean we're not well intentioned. They're just there. It's, it's human nature. So what we then need to do is take all of those questions and administer them to large groups of people, diverse groups of people, ethnically diverse, gender diversity, geographic diversity, socioeconomic diversity.
People with disabilities. So we administer these items and we collect all this demographic information, and now we look at how do these different groups respond to these items? If we see a systematic difference between the way one group responds to an item, we say there's something going on there. sometimes we see it and we're like, oh yeah, that makes sense, and we can fix it.
Quite often we're like, no idea why, but the numbers are showing us that that item is not fair to that group. So that item gets thrown out.
[00:10:47] Garrett Jestice: Hmm.
[00:10:48] Fred Rafilson: Now we can develop our tests based on items that have been field tested. We know that they're fair, but now we put the whole test together. We administer the test.
Your company, goes ahead and administers this test and we monitor it. And, you know, there are federal guidelines we work with, you know, the EEOC, the Equal Employment Opportunity Commission, who rightly so says tests need to be fair. They can't discriminate against people. and what we, we then do is we routinely, whatever cadence based on how many people you hire, we'll look at per thousand hires or whatever.
or we'll look quarterly, but we'll then take all of those test scores and we'll take all of the demographics and we'll see, are there any differences? Are we seeing any patterns? Hopefully not. Everything's great. If we see something, we dive in and we say, What's going on here?
[00:11:31] Fred Rafilson: Fix it if we can, remove it if we can't. So the answer is yes, they're fair.
[00:11:37] Garrett Jestice: I love it. I love hearing the context behind all of the thought and you know, work that goes into making sure that they're fair and proving, you know, again, this is a science that is predictive of actual behavior in the future, and that's been proven. So, I know there's probably people out there who are thinking, Okay, this sounds great, but we really, especially in today's crazy hiring market, we really need to be competitive.
And it sounds like, you know, assessments like this will take, will slow down the hiring process. They will, they'll take a long time and won't be great from a candidate experience perspective. I mean, who, who wants to go take a big, huge, long test just to be....
You know, have an opportunity to get into a job. So how, how do you, how do you balance that? How do you, have effective,assessments without hurting really that candidate experience and slowing down the hiring process?
[00:12:31] Fred Rafilson: That, that's a great question and I can relate to that. Having been in this field for a number of years, when I started, you would have to take a psychometric test that, you know, could have taken you an hour and a half to complete and you would do it because you had no choice. And that's what everyone was doing.
But the world is different, and candidate experience is everything. so how do we balance that? How do we ensure that the tests are just as predictive, but in a way that only takes six minutes? Right. And also doesn't turn off the candidate. So, amazing question. And just like every other technology, the science of, of psychometrics has increased and, and improved dramatically over the years.
And I'll give you an example of something that we do. That is one of the reasons I love Talview and we, we really lead the industry with this. Let's look at a particular role. I'm thinking like a QSR, a quick service restaurant role, an entry level role. So need to get people in the door, need to know if they're gonna stay on the job.
We wanna screen them, but we really don't have a lot of time and we have a big funnel, so what can we do? I don't have a million recruiters that can be on the phone and interviewing these people. I don't have that kinda staff. I don't have that kinda time. and all of these, you know, and also I don't wanna ruin the candidate experience.
So all of those things feed into it. Well, here's, you know, kinda a real use case. What we can do is, of course, everything's online and everyone's online, so they're gonna find a link to this job app, whether it be a QR code that they see in a particular restaurant or they're on a job site, whatever.
So they, they click on it, you know, are you interested in applying? Yeah, I need a job. A chat bot comes up and says, you know, are you available these times? That's a screener. Yes I am. Can you legally work in the US, whatever screening question. So that 15, 20 seconds. then, hey, are you available now for a quick interview?
candidate says, no. Okay. Well, here's a calendar. When are you available? So they put that in or yeah, sure, I can talk now. I, I've got my phone. So then they immediately go into what we would call an asynchronous interview, meaning that it's a lot, it's a recorded interview and they're asked a series of questions, and they respond to them on, on their phone.
[00:14:47] Fred Rafilson: It doesn't need to be done at a desktop, which is one of the cool things because we really wanna level the playing field for everyone. And you, you don't, you shouldn't have to have, you know, an internet connection and nice equipment. You can do this on your phone, and just about everybody has a phone or has an, has the phone. So they give their responses, they, they, answer these questions.
we work with the companies to determine what those questions should be, and maybe that interview takes three or four minutes. now here's the magic as far as I see it. So that interview is saved and the hiring manager can go back and look at the interview and they can, they can look at the candidate or listen to the candidate and, and hear the responses to those questions.
[00:15:26] Fred Rafilson: But in the background, what we're doing is we're taking that candidate's responses and we're running it through what we call a psycho linguistics engine. And psycho linguistics is a science that's been around for a long time. And has relatively recently, we've been applying it to talent acquisition. So we're actually looking at the words you use, the way you form your sentences, the way you speak, the way you think, and we can actually derive these behavioral scores and competency scores that are unrelated to the questions you asked that will give you the same kind of output that you're used to seeing from those psychometric tests, just from listening to the candidate speech. So now you've done this really brief process. You know, they've clicked, they've answered a few questions, they've answered a few interview questions.
You get to look at the interview, but now you're also getting a report on, you know, their team orientation, you know, on all these different behavioral characteristics. And the candidate was not even aware that, they were being assessed on these things. Which as the added benefit of you recall, if you've ever taken one of those tests, you're always like, Well, how should I answer this?
You know, I need to make myself look good. I wanna answer in a socially desirable way. Well, that's not happening. So these things aren't being gamed. You're getting a really good measure of a person's behavioral tendency. so that's one example of how we can make the candidate experience good, make the, assessment brief.
And still get that kind of predictive, information. So I, sorry. I tend to lose track. Just push me back when I, when I do. But...
[00:16:56] Garrett Jestice: no, that's great. Yeah, it's fa it's a fascinating example and I can definitely see the use case there. And it's, it's cool to learn about that too, that it doesn't take an hour long assessment now it takes, you know, a couple minutes of questions and sometimes people even know, don't even know they're taking the test, right.
But, and so you can get those really honest results and, and really predictive of behavior. So it's fascinating. It's great to hear that. So if someone out there is looking, they, they understand the value of behavioral assessments. They're looking for, you know, the right behavioral assessments, whether they go with Talview or they're looking somewhere else.
What are characteristics of the best behavioral assessments out there? Like, what should, what should you be looking for?
[00:17:36] Fred Rafilson: That's awesome. I'm gonna, we're gonna call this I/O Psychology 101, and we're gonna an extra hour to this podcast just this, but I'm gonna say to narrow it down, there's three things they should be looking for. There's validity, reliability, and fairness. And your test vendors are gonna have information on these three things. If they don't, you know, this goes back to my earlier point, find someone who does. Right?
So validity, does the test do what it's supposed to do? Test scores, should. Predict success on the job, right? Or they should indicate a master of a particular skill. There needs to be documentation that those test scores do what they're supposed to do.
That's validity. Ok. And what that really comes down to is are they job related? That's validity. Reliability is an interesting term that even a lot of I/O psychologists get wrong, but it's a really cool term. and what it has to do is with, is the accuracy of the test. And you can sort of picture it this way.
Say, looking at your ability to do math, or I'm looking at your extroversion, right? Cause you seem very extroverted to me. So in your head you have this level of math ability or extroversion, this is you. Okay? but I can never get in your head and pull that out of there. No matter how hard I try, that's gonna be a bad candidate experience, right?
So all I can do is give you some sort of an assessment. And what we call this, what's in here, that's your true score on that trait or that skill. When we give you an assessment, what we end up is what we call an observed score. So, so we're getting a measurement. We're observing a score. An observe score is always an estimate of the true score.
The closer the estimate is, the closer the observe score is to the true score, that's reliability. So, and there's ways of, of providing evidence to show that tests are reliable. It, it's a mathematical concept, requires lots of people. but that's a trait or a psychometric property of tests that you wanna look for.
So we've got validity, we've got reliability, and then we've got fairness. And we talked a lot about fairness already. We wanna make sure that your test works the same way for everyone, regardless of their demographic, that it predicts success and in the same way that the passing rates are the same, that there's no, what we would call differential test or question functioning within that.
So those are the three things that I think people need to look for. I hope that made some sense.
[00:20:02] Garrett Jestice: Totally did. Yeah, I think that's, that's excellent. Excellent tips. So, Fred, I, I, I think I could go for another hour here too, and just, I, this is such fascinating stuff. thank you for being on the podcast today. I know we just barely scratched the surface of this, but this has been such an excellent conversation and I definitely see the value, of interjecting, especially these behavioral assessments in the hiring process. Right.
as we wrap up here, if there are listeners who want to learn more about this topic or about Talview, or if they have questions for you, what's the best way for them to, to reach out and connect?
Sure. And boy, that went by fast and then I enjoyed it and, I would love to come back and talk more.
[00:20:45] Garrett Jestice: Yeah.
[00:20:46] Fred Rafilson: I would love for people to come to Talview. Go to talview.com. That's the best way to find this. if you wanna reach me, I think the best way is through my LinkedIn profile. Hopefully, you can share that with the audience.
Please reach out. I, I look forward to continuing the conversation.
[00:21:02] Garrett Jestice: Thank you so much Fred. We'll drop links for all those listeners out there. We'll drop links to both the Talview site and Fred's LinkedIn profile where you can find and connect with him in the show notes so you can find them there. So Fred, thank you again so much for being with us today, sharing your knowledge with us, and we hope you have a great rest of the day.
[00:21:18] Fred Rafilson: My pleasure. You too. Great to meet you.
[00:00:13] Fred Rafilson: I'm doing really well, Garrett. Thank you.
[00:00:15] Garrett Jestice: Thanks for being on the show today. How's the weather in, beautiful, Sunny Bend, Oregon today?
[00:00:20] Fred Rafilson: Well, that's a great question because it's been snowing for the last two days and, I'm pretty excited. I've got my season pass coming from the mountain, so it's perfect.
[00:00:28] Garrett Jestice: That's awesome. I've, I've been thinking the same thing. You know, we're based in Utah here and winter just decided to come the last few days, but looking forward to hitting the ski slope soon too. So, Fred, we're, we're excited to have you on the show today. and, to dive into this really interesting topic.
Before we do that though, give our listeners a little bit more context on and about you and your background as well as what your company Talview does.
[00:00:52] Fred Rafilson: Sure, thanks. I'm an industrial and organizational psychologist, an I/O psychologist, and what we do is we focus on the psychology of people and work. and specifically, I'm a testing guy. I'm an assessment guy. I'm all about measuring people and determining what are their skills, abilities, knowledge, you know, what are their competencies as, in, in terms of are they gonna be predictive of success on a job?
So I've been doing this for, longer than I care to say, you know, I'm just gonna throw out, a few decades is good enough. I had my own, business years ago. We developed, exams for police and fire departments around the country. Became the nation's largest provider of those, I since sold that to my employees.
moved up to Bend and I've worked for several large, testing companies as their chief I/O psychologist, until I found, Talview which I really love. It's an amazing company. We do some really cool things. That I'll tell you about. And, I came to tell you, and I'm the chief I/O psychologist here.
[00:01:46] Fred Rafilson: So I work with our customers to ensure that their assessment programs are number one valid, they're helping them select the best people. number two, fair. They're doing it in a way that works the same way for everyone. And number three, that our candidates have a great experience going through the process. so in a nutshell, that's what I do. Talview is, I, I struggle with the best way...
it's, it's an end to end talent measurement platform and it's AI driven, really state of the science. So we start with everything from reading resumes to administering assessments that are, you know, job related based on the requirements of your job to, asynchronous interviews prerecorded with psycho linguistic capabilities, which we'll talk about later to help not only look at the answers to the interview questions, but look at the behavioral traits of of the candidates that are answering them.
To live interviews and scheduling, full suites of assessments so we can really help an organization, acquire the, the right talent for their positions.
[00:02:46] Garrett Jestice: I love it. That's so great. It's such a great elevator speech. I think a, a great precursor to what we're gonna talk about today. Tell me a little bit more about, who your typical customers are. Who do you typically work with at Talview? Is it all across the board or are there, are there certain types of companies that have really found value with working with Talview?
[00:03:02] Fred Rafilson: So I guess two answers. One would be industry, and the second would be, you know, size of organization. we work in all industries, right? Everybody needs to hire. And, especially these days, it's really tricky because, you know, good candidates, they're, they're not sticky. If we can't get them right away, we're gonna lose them to somebody else.
So, you know, hiring is really, in an interesting place right now.
[00:03:24] Garrett Jestice: Mm-hmm.
[00:03:25] Fred Rafilson: All industries. we tend to focus on enterprise organizations because they hire thousands of people and have tens of thousands of organizations of, of, candidates for these jobs. But we work with medium and small businesses as well.
I've always said, especially being a business owner myself, a small business owner, you know, clearly when you have to hire thousands of people, Making sure you make valid decisions is critical, right?
[00:03:51] Garrett Jestice: Hmm.
[00:03:52] Fred Rafilson: Hire one person, you make a bad decision there, it's really not a good thing. So, what we applicable to any size of organization, in any industry,
[00:04:01] Garrett Jestice: Yeah. I love that. And I think our topic today is too, so I think what we're gonna talk about today is really this idea of behavioral assessments, right? And how do they, how should they, and how can they help in your talent acquisition efforts? And that's a topic that I think can be applicable for any size business.
Right. And so to really just start us on this path, tell us a little bit more about what behavioral assessments are and the different types of insights they can provide when you are interviewing job candidates.
[00:04:31] Fred Rafilson: Yeah, that's a great question. I think people tend to think of assessments as skills assessments, right? Does a person, can they code? Do they have the medical knowledge they need to be, an ER tech? you know, do they have the finance knowledge they need to, to work in a bank? And of course, those skills are critical, right?
If you don't know, if you don't have coding skills, you're not gonna be much of a software development, right? You don't have the medical skills, I don't want you looking at me when I come to the ER. So skills are critical. But skills are only a part of the puzzle, right? It takes two things to make a good employee, and we know this through years of research, and I think common sense.
It takes two things. One is what you can do, and that's skills. And the other is what you will do. And that's your behavioral tendencies: your motivation, your attitude, your personality. A person can have all the skills in the world, but if they're not motivated to come to work and they don't wanna be a team player and they can't get along and what have you, they're gonna be a terrible employee, right?
On the other hand, someone might have, you know, great motivation, great attitude, just an all-around great person, but they don't have the skills. That's not gonna work either. So you need to assess both. And what we're here to talk about today is really looking at those behavioral things.
So, what kinds of things do we measure? And I, I touched on it a little bit. We really try to predict how a person is going to behave in a particular situation. And we do that by looking at their personality traits. we look at their attitude, we look at their communication skills, and I use the word skills as sort of a blurry line sometimes.
we look at their integrity and we think about all of the jobs where integrity is key, where people have access to money and merchandise and sensitive records, et cetera. You wanna make sure that you know, We'll have a good moral compass and are gonna do the right thing and aren't gonna steal, aren't gonna use drugs on the job, aren't gonna engage in counterproductive behaviors.
[00:06:32] Fred Rafilson: All of those kinds of things. You know, teamwork orientation -- with remote working now and, and, and teams and, the importance of working in teams. We wanna make sure that people are suited for that. some people are really suited for working independently, and that's great for certain jobs. but it's difficult to take a person like that, make them part of a team and expect them to function if they're not wired that way.
Certainly there's training, but we wanna select the right people. And vice versa. We don't wanna take someone who needs that social contact and needs that teamwork and put them in a position where they're never gonna talk to anyone. They're not gonna be happy, they're not gonna be engaged.
[00:07:06] Garrett Jestice: Yeah.
[00:07:07] Fred Rafilson: So those are the types of things that we look at with behavioral assessments.
I always kind of liken it to when people say, well, you know, we just need to measure their skills. And I'm like, that's awesome. But like when I go to a restaurant and I get a hamburger, I don't just ask for the patty. The patty's great, but I really like the bun and all the condiments and everything else that's in there because that's a hamburger.
Right? I know. It's kinda a crazy analogy. I'm hungry. It's, it's almost lunchtime here, but, you need to measure the whole thing, not just, not just what this goes, not just the patty. We need the entire package and that's where behavioral assessment comes in.
[00:07:44] Garrett Jestice: Yeah, that makes total sense. And I, I'm gonna jump in just to, we're just gonna dive into the deep end here with this next question because I'm curious, you know, your thoughts on this. because I'm sure there's people out there who, who, who get it. They're from a business perspective, like you need to be able to, to test skills and these behavioral traits and tendencies.
But I, I think there's also people who might be wondering, are these behavioral assessments fair to do? Like when someone asks you that question, what's, what's your response to that?
[00:08:15] Fred Rafilson: So my response is, yes, they're fair. And I'll tell you why I believe that. One, we have decades of research. I/O psychology is not a new field, and we can talk about that later. But, We have decades of research to show that, different types of personality characteristics do indeed predict success at work.
So we know empirically that there's a correlation between certain types of personalities and success on certain types of job. So they're fair in terms of being job related. They're also fair in terms of well developed tests, properly constructive tests being provided by reputable providers. I mean, God only knows there's a million things out on the internet, so be careful.
You know, what you buy or what, what you read. We all know that now. Right. But assuming you're dealing with, you know, respectable test vendors, we also know that the tests are fair and the reason is that we go to great lengths to ensure that they're fair. So that starts with the, very beginning with item construction.
And when I say items, I mean test questions. So when we're developing questions to measure a particular construct, whatever that may be, teamwork orientation, you know, we as psychologists know that, well, this question should be predictive of that. This question should be predictive of that. We write a big pool of questions, more than we think we're gonna need. We then review them very carefully, for any signs of bias at all. And we even bring people in who are experts at looking for cultural bias, gender bias, and they review the questions. And so we all as a group say that question should be good. but the thing is, we're all human and we all have unconscious biases.
[00:09:50] Garrett Jestice: mm.
[00:09:51] Fred Rafilson: Anyone tells you they don't, they just don't know themselves. We can't help it. We grow up in a particular society. We know what we know. You know what you know, I know what I know. We all have our biases. Okay? and it doesn't mean we're not well intentioned. They're just there. It's, it's human nature. So what we then need to do is take all of those questions and administer them to large groups of people, diverse groups of people, ethnically diverse, gender diversity, geographic diversity, socioeconomic diversity.
People with disabilities. So we administer these items and we collect all this demographic information, and now we look at how do these different groups respond to these items? If we see a systematic difference between the way one group responds to an item, we say there's something going on there. sometimes we see it and we're like, oh yeah, that makes sense, and we can fix it.
Quite often we're like, no idea why, but the numbers are showing us that that item is not fair to that group. So that item gets thrown out.
[00:10:47] Garrett Jestice: Hmm.
[00:10:48] Fred Rafilson: Now we can develop our tests based on items that have been field tested. We know that they're fair, but now we put the whole test together. We administer the test.
Your company, goes ahead and administers this test and we monitor it. And, you know, there are federal guidelines we work with, you know, the EEOC, the Equal Employment Opportunity Commission, who rightly so says tests need to be fair. They can't discriminate against people. and what we, we then do is we routinely, whatever cadence based on how many people you hire, we'll look at per thousand hires or whatever.
or we'll look quarterly, but we'll then take all of those test scores and we'll take all of the demographics and we'll see, are there any differences? Are we seeing any patterns? Hopefully not. Everything's great. If we see something, we dive in and we say, What's going on here?
[00:11:31] Fred Rafilson: Fix it if we can, remove it if we can't. So the answer is yes, they're fair.
[00:11:37] Garrett Jestice: I love it. I love hearing the context behind all of the thought and you know, work that goes into making sure that they're fair and proving, you know, again, this is a science that is predictive of actual behavior in the future, and that's been proven. So, I know there's probably people out there who are thinking, Okay, this sounds great, but we really, especially in today's crazy hiring market, we really need to be competitive.
And it sounds like, you know, assessments like this will take, will slow down the hiring process. They will, they'll take a long time and won't be great from a candidate experience perspective. I mean, who, who wants to go take a big, huge, long test just to be....
You know, have an opportunity to get into a job. So how, how do you, how do you balance that? How do you, have effective,assessments without hurting really that candidate experience and slowing down the hiring process?
[00:12:31] Fred Rafilson: That, that's a great question and I can relate to that. Having been in this field for a number of years, when I started, you would have to take a psychometric test that, you know, could have taken you an hour and a half to complete and you would do it because you had no choice. And that's what everyone was doing.
But the world is different, and candidate experience is everything. so how do we balance that? How do we ensure that the tests are just as predictive, but in a way that only takes six minutes? Right. And also doesn't turn off the candidate. So, amazing question. And just like every other technology, the science of, of psychometrics has increased and, and improved dramatically over the years.
And I'll give you an example of something that we do. That is one of the reasons I love Talview and we, we really lead the industry with this. Let's look at a particular role. I'm thinking like a QSR, a quick service restaurant role, an entry level role. So need to get people in the door, need to know if they're gonna stay on the job.
We wanna screen them, but we really don't have a lot of time and we have a big funnel, so what can we do? I don't have a million recruiters that can be on the phone and interviewing these people. I don't have that kinda staff. I don't have that kinda time. and all of these, you know, and also I don't wanna ruin the candidate experience.
So all of those things feed into it. Well, here's, you know, kinda a real use case. What we can do is, of course, everything's online and everyone's online, so they're gonna find a link to this job app, whether it be a QR code that they see in a particular restaurant or they're on a job site, whatever.
So they, they click on it, you know, are you interested in applying? Yeah, I need a job. A chat bot comes up and says, you know, are you available these times? That's a screener. Yes I am. Can you legally work in the US, whatever screening question. So that 15, 20 seconds. then, hey, are you available now for a quick interview?
candidate says, no. Okay. Well, here's a calendar. When are you available? So they put that in or yeah, sure, I can talk now. I, I've got my phone. So then they immediately go into what we would call an asynchronous interview, meaning that it's a lot, it's a recorded interview and they're asked a series of questions, and they respond to them on, on their phone.
[00:14:47] Fred Rafilson: It doesn't need to be done at a desktop, which is one of the cool things because we really wanna level the playing field for everyone. And you, you don't, you shouldn't have to have, you know, an internet connection and nice equipment. You can do this on your phone, and just about everybody has a phone or has an, has the phone. So they give their responses, they, they, answer these questions.
we work with the companies to determine what those questions should be, and maybe that interview takes three or four minutes. now here's the magic as far as I see it. So that interview is saved and the hiring manager can go back and look at the interview and they can, they can look at the candidate or listen to the candidate and, and hear the responses to those questions.
[00:15:26] Fred Rafilson: But in the background, what we're doing is we're taking that candidate's responses and we're running it through what we call a psycho linguistics engine. And psycho linguistics is a science that's been around for a long time. And has relatively recently, we've been applying it to talent acquisition. So we're actually looking at the words you use, the way you form your sentences, the way you speak, the way you think, and we can actually derive these behavioral scores and competency scores that are unrelated to the questions you asked that will give you the same kind of output that you're used to seeing from those psychometric tests, just from listening to the candidate speech. So now you've done this really brief process. You know, they've clicked, they've answered a few questions, they've answered a few interview questions.
You get to look at the interview, but now you're also getting a report on, you know, their team orientation, you know, on all these different behavioral characteristics. And the candidate was not even aware that, they were being assessed on these things. Which as the added benefit of you recall, if you've ever taken one of those tests, you're always like, Well, how should I answer this?
You know, I need to make myself look good. I wanna answer in a socially desirable way. Well, that's not happening. So these things aren't being gamed. You're getting a really good measure of a person's behavioral tendency. so that's one example of how we can make the candidate experience good, make the, assessment brief.
And still get that kind of predictive, information. So I, sorry. I tend to lose track. Just push me back when I, when I do. But...
[00:16:56] Garrett Jestice: no, that's great. Yeah, it's fa it's a fascinating example and I can definitely see the use case there. And it's, it's cool to learn about that too, that it doesn't take an hour long assessment now it takes, you know, a couple minutes of questions and sometimes people even know, don't even know they're taking the test, right.
But, and so you can get those really honest results and, and really predictive of behavior. So it's fascinating. It's great to hear that. So if someone out there is looking, they, they understand the value of behavioral assessments. They're looking for, you know, the right behavioral assessments, whether they go with Talview or they're looking somewhere else.
What are characteristics of the best behavioral assessments out there? Like, what should, what should you be looking for?
[00:17:36] Fred Rafilson: That's awesome. I'm gonna, we're gonna call this I/O Psychology 101, and we're gonna an extra hour to this podcast just this, but I'm gonna say to narrow it down, there's three things they should be looking for. There's validity, reliability, and fairness. And your test vendors are gonna have information on these three things. If they don't, you know, this goes back to my earlier point, find someone who does. Right?
So validity, does the test do what it's supposed to do? Test scores, should. Predict success on the job, right? Or they should indicate a master of a particular skill. There needs to be documentation that those test scores do what they're supposed to do.
That's validity. Ok. And what that really comes down to is are they job related? That's validity. Reliability is an interesting term that even a lot of I/O psychologists get wrong, but it's a really cool term. and what it has to do is with, is the accuracy of the test. And you can sort of picture it this way.
Say, looking at your ability to do math, or I'm looking at your extroversion, right? Cause you seem very extroverted to me. So in your head you have this level of math ability or extroversion, this is you. Okay? but I can never get in your head and pull that out of there. No matter how hard I try, that's gonna be a bad candidate experience, right?
So all I can do is give you some sort of an assessment. And what we call this, what's in here, that's your true score on that trait or that skill. When we give you an assessment, what we end up is what we call an observed score. So, so we're getting a measurement. We're observing a score. An observe score is always an estimate of the true score.
The closer the estimate is, the closer the observe score is to the true score, that's reliability. So, and there's ways of, of providing evidence to show that tests are reliable. It, it's a mathematical concept, requires lots of people. but that's a trait or a psychometric property of tests that you wanna look for.
So we've got validity, we've got reliability, and then we've got fairness. And we talked a lot about fairness already. We wanna make sure that your test works the same way for everyone, regardless of their demographic, that it predicts success and in the same way that the passing rates are the same, that there's no, what we would call differential test or question functioning within that.
So those are the three things that I think people need to look for. I hope that made some sense.
[00:20:02] Garrett Jestice: Totally did. Yeah, I think that's, that's excellent. Excellent tips. So, Fred, I, I, I think I could go for another hour here too, and just, I, this is such fascinating stuff. thank you for being on the podcast today. I know we just barely scratched the surface of this, but this has been such an excellent conversation and I definitely see the value, of interjecting, especially these behavioral assessments in the hiring process. Right.
as we wrap up here, if there are listeners who want to learn more about this topic or about Talview, or if they have questions for you, what's the best way for them to, to reach out and connect?
Sure. And boy, that went by fast and then I enjoyed it and, I would love to come back and talk more.
[00:20:45] Garrett Jestice: Yeah.
[00:20:46] Fred Rafilson: I would love for people to come to Talview. Go to talview.com. That's the best way to find this. if you wanna reach me, I think the best way is through my LinkedIn profile. Hopefully, you can share that with the audience.
Please reach out. I, I look forward to continuing the conversation.
[00:21:02] Garrett Jestice: Thank you so much Fred. We'll drop links for all those listeners out there. We'll drop links to both the Talview site and Fred's LinkedIn profile where you can find and connect with him in the show notes so you can find them there. So Fred, thank you again so much for being with us today, sharing your knowledge with us, and we hope you have a great rest of the day.
[00:21:18] Fred Rafilson: My pleasure. You too. Great to meet you.
Eddy’s HR Mavericks Podcast
Using Behavioral Assessments in Talent Acquisition w/ Fred Rafilson
In episode 72, Fred Rafilson draws on his knowledge of I/O psychology to explain how behavioral assessments work and why they’re beneficial.
Episode 72
It takes two things to make a good employee: the right skills and the right behavior. Fred Rafilson, chief I/O psychologist at Talview, doesn’t underestimate the importance of skills—but he also knows that neglecting to measure candidates’ behavioral attributes could be a big hiring mistake. The best candidates have the knowledge and abilities to perform a job, and the behavioral tendencies (motivation, attitude, and personality) to become standout employees. In this episode, Fred draws on his knowledge of I/O psychology to explain how behavioral assessments work and why they’re beneficial. We discussed:
- Why it’s important to screen candidates for both skills and behavioral traits
- The qualities that behavioral assessments measure
- How providers ensure that the tests are fair
- What a behavioral assessment might look like—and what goes on behind the scenes
- Three things that the best assessments have in common
Fred Rafilson
Full Transcript
[00:00:00] Garrett Jestice: Welcome to the next episode of the HR Mavericks podcast. I'm Garrett Jestice, and today I'm joined by Fred Rafilson, who's the chief I/O psychologist at Talview. Fred, how you doing today?
[00:00:13] Fred Rafilson: I'm doing really well, Garrett. Thank you.
[00:00:15] Garrett Jestice: Thanks for being on the show today. How's the weather in, beautiful, Sunny Bend, Oregon today?
[00:00:20] Fred Rafilson: Well, that's a great question because it's been snowing for the last two days and, I'm pretty excited. I've got my season pass coming from the mountain, so it's perfect.
[00:00:28] Garrett Jestice: That's awesome. I've, I've been thinking the same thing. You know, we're based in Utah here and winter just decided to come the last few days, but looking forward to hitting the ski slope soon too. So, Fred, we're, we're excited to have you on the show today. and, to dive into this really interesting topic.
Before we do that though, give our listeners a little bit more context on and about you and your background as well as what your company Talview does.
[00:00:52] Fred Rafilson: Sure, thanks. I'm an industrial and organizational psychologist, an I/O psychologist, and what we do is we focus on the psychology of people and work. and specifically, I'm a testing guy. I'm an assessment guy. I'm all about measuring people and determining what are their skills, abilities, knowledge, you know, what are their competencies as, in, in terms of are they gonna be predictive of success on a job?
So I've been doing this for, longer than I care to say, you know, I'm just gonna throw out, a few decades is good enough. I had my own, business years ago. We developed, exams for police and fire departments around the country. Became the nation's largest provider of those, I since sold that to my employees.
moved up to Bend and I've worked for several large, testing companies as their chief I/O psychologist, until I found, Talview which I really love. It's an amazing company. We do some really cool things. That I'll tell you about. And, I came to tell you, and I'm the chief I/O psychologist here.
[00:01:46] Fred Rafilson: So I work with our customers to ensure that their assessment programs are number one valid, they're helping them select the best people. number two, fair. They're doing it in a way that works the same way for everyone. And number three, that our candidates have a great experience going through the process. so in a nutshell, that's what I do. Talview is, I, I struggle with the best way...
it's, it's an end to end talent measurement platform and it's AI driven, really state of the science. So we start with everything from reading resumes to administering assessments that are, you know, job related based on the requirements of your job to, asynchronous interviews prerecorded with psycho linguistic capabilities, which we'll talk about later to help not only look at the answers to the interview questions, but look at the behavioral traits of of the candidates that are answering them.
To live interviews and scheduling, full suites of assessments so we can really help an organization, acquire the, the right talent for their positions.
[00:02:46] Garrett Jestice: I love it. That's so great. It's such a great elevator speech. I think a, a great precursor to what we're gonna talk about today. Tell me a little bit more about, who your typical customers are. Who do you typically work with at Talview? Is it all across the board or are there, are there certain types of companies that have really found value with working with Talview?
[00:03:02] Fred Rafilson: So I guess two answers. One would be industry, and the second would be, you know, size of organization. we work in all industries, right? Everybody needs to hire. And, especially these days, it's really tricky because, you know, good candidates, they're, they're not sticky. If we can't get them right away, we're gonna lose them to somebody else.
So, you know, hiring is really, in an interesting place right now.
[00:03:24] Garrett Jestice: Mm-hmm.
[00:03:25] Fred Rafilson: All industries. we tend to focus on enterprise organizations because they hire thousands of people and have tens of thousands of organizations of, of, candidates for these jobs. But we work with medium and small businesses as well.
I've always said, especially being a business owner myself, a small business owner, you know, clearly when you have to hire thousands of people, Making sure you make valid decisions is critical, right?
[00:03:51] Garrett Jestice: Hmm.
[00:03:52] Fred Rafilson: Hire one person, you make a bad decision there, it's really not a good thing. So, what we applicable to any size of organization, in any industry,
[00:04:01] Garrett Jestice: Yeah. I love that. And I think our topic today is too, so I think what we're gonna talk about today is really this idea of behavioral assessments, right? And how do they, how should they, and how can they help in your talent acquisition efforts? And that's a topic that I think can be applicable for any size business.
Right. And so to really just start us on this path, tell us a little bit more about what behavioral assessments are and the different types of insights they can provide when you are interviewing job candidates.
[00:04:31] Fred Rafilson: Yeah, that's a great question. I think people tend to think of assessments as skills assessments, right? Does a person, can they code? Do they have the medical knowledge they need to be, an ER tech? you know, do they have the finance knowledge they need to, to work in a bank? And of course, those skills are critical, right?
If you don't know, if you don't have coding skills, you're not gonna be much of a software development, right? You don't have the medical skills, I don't want you looking at me when I come to the ER. So skills are critical. But skills are only a part of the puzzle, right? It takes two things to make a good employee, and we know this through years of research, and I think common sense.
It takes two things. One is what you can do, and that's skills. And the other is what you will do. And that's your behavioral tendencies: your motivation, your attitude, your personality. A person can have all the skills in the world, but if they're not motivated to come to work and they don't wanna be a team player and they can't get along and what have you, they're gonna be a terrible employee, right?
On the other hand, someone might have, you know, great motivation, great attitude, just an all-around great person, but they don't have the skills. That's not gonna work either. So you need to assess both. And what we're here to talk about today is really looking at those behavioral things.
So, what kinds of things do we measure? And I, I touched on it a little bit. We really try to predict how a person is going to behave in a particular situation. And we do that by looking at their personality traits. we look at their attitude, we look at their communication skills, and I use the word skills as sort of a blurry line sometimes.
we look at their integrity and we think about all of the jobs where integrity is key, where people have access to money and merchandise and sensitive records, et cetera. You wanna make sure that you know, We'll have a good moral compass and are gonna do the right thing and aren't gonna steal, aren't gonna use drugs on the job, aren't gonna engage in counterproductive behaviors.
[00:06:32] Fred Rafilson: All of those kinds of things. You know, teamwork orientation -- with remote working now and, and, and teams and, the importance of working in teams. We wanna make sure that people are suited for that. some people are really suited for working independently, and that's great for certain jobs. but it's difficult to take a person like that, make them part of a team and expect them to function if they're not wired that way.
Certainly there's training, but we wanna select the right people. And vice versa. We don't wanna take someone who needs that social contact and needs that teamwork and put them in a position where they're never gonna talk to anyone. They're not gonna be happy, they're not gonna be engaged.
[00:07:06] Garrett Jestice: Yeah.
[00:07:07] Fred Rafilson: So those are the types of things that we look at with behavioral assessments.
I always kind of liken it to when people say, well, you know, we just need to measure their skills. And I'm like, that's awesome. But like when I go to a restaurant and I get a hamburger, I don't just ask for the patty. The patty's great, but I really like the bun and all the condiments and everything else that's in there because that's a hamburger.
Right? I know. It's kinda a crazy analogy. I'm hungry. It's, it's almost lunchtime here, but, you need to measure the whole thing, not just, not just what this goes, not just the patty. We need the entire package and that's where behavioral assessment comes in.
[00:07:44] Garrett Jestice: Yeah, that makes total sense. And I, I'm gonna jump in just to, we're just gonna dive into the deep end here with this next question because I'm curious, you know, your thoughts on this. because I'm sure there's people out there who, who, who get it. They're from a business perspective, like you need to be able to, to test skills and these behavioral traits and tendencies.
But I, I think there's also people who might be wondering, are these behavioral assessments fair to do? Like when someone asks you that question, what's, what's your response to that?
[00:08:15] Fred Rafilson: So my response is, yes, they're fair. And I'll tell you why I believe that. One, we have decades of research. I/O psychology is not a new field, and we can talk about that later. But, We have decades of research to show that, different types of personality characteristics do indeed predict success at work.
So we know empirically that there's a correlation between certain types of personalities and success on certain types of job. So they're fair in terms of being job related. They're also fair in terms of well developed tests, properly constructive tests being provided by reputable providers. I mean, God only knows there's a million things out on the internet, so be careful.
You know, what you buy or what, what you read. We all know that now. Right. But assuming you're dealing with, you know, respectable test vendors, we also know that the tests are fair and the reason is that we go to great lengths to ensure that they're fair. So that starts with the, very beginning with item construction.
And when I say items, I mean test questions. So when we're developing questions to measure a particular construct, whatever that may be, teamwork orientation, you know, we as psychologists know that, well, this question should be predictive of that. This question should be predictive of that. We write a big pool of questions, more than we think we're gonna need. We then review them very carefully, for any signs of bias at all. And we even bring people in who are experts at looking for cultural bias, gender bias, and they review the questions. And so we all as a group say that question should be good. but the thing is, we're all human and we all have unconscious biases.
[00:09:50] Garrett Jestice: mm.
[00:09:51] Fred Rafilson: Anyone tells you they don't, they just don't know themselves. We can't help it. We grow up in a particular society. We know what we know. You know what you know, I know what I know. We all have our biases. Okay? and it doesn't mean we're not well intentioned. They're just there. It's, it's human nature. So what we then need to do is take all of those questions and administer them to large groups of people, diverse groups of people, ethnically diverse, gender diversity, geographic diversity, socioeconomic diversity.
People with disabilities. So we administer these items and we collect all this demographic information, and now we look at how do these different groups respond to these items? If we see a systematic difference between the way one group responds to an item, we say there's something going on there. sometimes we see it and we're like, oh yeah, that makes sense, and we can fix it.
Quite often we're like, no idea why, but the numbers are showing us that that item is not fair to that group. So that item gets thrown out.
[00:10:47] Garrett Jestice: Hmm.
[00:10:48] Fred Rafilson: Now we can develop our tests based on items that have been field tested. We know that they're fair, but now we put the whole test together. We administer the test.
Your company, goes ahead and administers this test and we monitor it. And, you know, there are federal guidelines we work with, you know, the EEOC, the Equal Employment Opportunity Commission, who rightly so says tests need to be fair. They can't discriminate against people. and what we, we then do is we routinely, whatever cadence based on how many people you hire, we'll look at per thousand hires or whatever.
or we'll look quarterly, but we'll then take all of those test scores and we'll take all of the demographics and we'll see, are there any differences? Are we seeing any patterns? Hopefully not. Everything's great. If we see something, we dive in and we say, What's going on here?
[00:11:31] Fred Rafilson: Fix it if we can, remove it if we can't. So the answer is yes, they're fair.
[00:11:37] Garrett Jestice: I love it. I love hearing the context behind all of the thought and you know, work that goes into making sure that they're fair and proving, you know, again, this is a science that is predictive of actual behavior in the future, and that's been proven. So, I know there's probably people out there who are thinking, Okay, this sounds great, but we really, especially in today's crazy hiring market, we really need to be competitive.
And it sounds like, you know, assessments like this will take, will slow down the hiring process. They will, they'll take a long time and won't be great from a candidate experience perspective. I mean, who, who wants to go take a big, huge, long test just to be....
You know, have an opportunity to get into a job. So how, how do you, how do you balance that? How do you, have effective,assessments without hurting really that candidate experience and slowing down the hiring process?
[00:12:31] Fred Rafilson: That, that's a great question and I can relate to that. Having been in this field for a number of years, when I started, you would have to take a psychometric test that, you know, could have taken you an hour and a half to complete and you would do it because you had no choice. And that's what everyone was doing.
But the world is different, and candidate experience is everything. so how do we balance that? How do we ensure that the tests are just as predictive, but in a way that only takes six minutes? Right. And also doesn't turn off the candidate. So, amazing question. And just like every other technology, the science of, of psychometrics has increased and, and improved dramatically over the years.
And I'll give you an example of something that we do. That is one of the reasons I love Talview and we, we really lead the industry with this. Let's look at a particular role. I'm thinking like a QSR, a quick service restaurant role, an entry level role. So need to get people in the door, need to know if they're gonna stay on the job.
We wanna screen them, but we really don't have a lot of time and we have a big funnel, so what can we do? I don't have a million recruiters that can be on the phone and interviewing these people. I don't have that kinda staff. I don't have that kinda time. and all of these, you know, and also I don't wanna ruin the candidate experience.
So all of those things feed into it. Well, here's, you know, kinda a real use case. What we can do is, of course, everything's online and everyone's online, so they're gonna find a link to this job app, whether it be a QR code that they see in a particular restaurant or they're on a job site, whatever.
So they, they click on it, you know, are you interested in applying? Yeah, I need a job. A chat bot comes up and says, you know, are you available these times? That's a screener. Yes I am. Can you legally work in the US, whatever screening question. So that 15, 20 seconds. then, hey, are you available now for a quick interview?
candidate says, no. Okay. Well, here's a calendar. When are you available? So they put that in or yeah, sure, I can talk now. I, I've got my phone. So then they immediately go into what we would call an asynchronous interview, meaning that it's a lot, it's a recorded interview and they're asked a series of questions, and they respond to them on, on their phone.
[00:14:47] Fred Rafilson: It doesn't need to be done at a desktop, which is one of the cool things because we really wanna level the playing field for everyone. And you, you don't, you shouldn't have to have, you know, an internet connection and nice equipment. You can do this on your phone, and just about everybody has a phone or has an, has the phone. So they give their responses, they, they, answer these questions.
we work with the companies to determine what those questions should be, and maybe that interview takes three or four minutes. now here's the magic as far as I see it. So that interview is saved and the hiring manager can go back and look at the interview and they can, they can look at the candidate or listen to the candidate and, and hear the responses to those questions.
[00:15:26] Fred Rafilson: But in the background, what we're doing is we're taking that candidate's responses and we're running it through what we call a psycho linguistics engine. And psycho linguistics is a science that's been around for a long time. And has relatively recently, we've been applying it to talent acquisition. So we're actually looking at the words you use, the way you form your sentences, the way you speak, the way you think, and we can actually derive these behavioral scores and competency scores that are unrelated to the questions you asked that will give you the same kind of output that you're used to seeing from those psychometric tests, just from listening to the candidate speech. So now you've done this really brief process. You know, they've clicked, they've answered a few questions, they've answered a few interview questions.
You get to look at the interview, but now you're also getting a report on, you know, their team orientation, you know, on all these different behavioral characteristics. And the candidate was not even aware that, they were being assessed on these things. Which as the added benefit of you recall, if you've ever taken one of those tests, you're always like, Well, how should I answer this?
You know, I need to make myself look good. I wanna answer in a socially desirable way. Well, that's not happening. So these things aren't being gamed. You're getting a really good measure of a person's behavioral tendency. so that's one example of how we can make the candidate experience good, make the, assessment brief.
And still get that kind of predictive, information. So I, sorry. I tend to lose track. Just push me back when I, when I do. But...
[00:16:56] Garrett Jestice: no, that's great. Yeah, it's fa it's a fascinating example and I can definitely see the use case there. And it's, it's cool to learn about that too, that it doesn't take an hour long assessment now it takes, you know, a couple minutes of questions and sometimes people even know, don't even know they're taking the test, right.
But, and so you can get those really honest results and, and really predictive of behavior. So it's fascinating. It's great to hear that. So if someone out there is looking, they, they understand the value of behavioral assessments. They're looking for, you know, the right behavioral assessments, whether they go with Talview or they're looking somewhere else.
What are characteristics of the best behavioral assessments out there? Like, what should, what should you be looking for?
[00:17:36] Fred Rafilson: That's awesome. I'm gonna, we're gonna call this I/O Psychology 101, and we're gonna an extra hour to this podcast just this, but I'm gonna say to narrow it down, there's three things they should be looking for. There's validity, reliability, and fairness. And your test vendors are gonna have information on these three things. If they don't, you know, this goes back to my earlier point, find someone who does. Right?
So validity, does the test do what it's supposed to do? Test scores, should. Predict success on the job, right? Or they should indicate a master of a particular skill. There needs to be documentation that those test scores do what they're supposed to do.
That's validity. Ok. And what that really comes down to is are they job related? That's validity. Reliability is an interesting term that even a lot of I/O psychologists get wrong, but it's a really cool term. and what it has to do is with, is the accuracy of the test. And you can sort of picture it this way.
Say, looking at your ability to do math, or I'm looking at your extroversion, right? Cause you seem very extroverted to me. So in your head you have this level of math ability or extroversion, this is you. Okay? but I can never get in your head and pull that out of there. No matter how hard I try, that's gonna be a bad candidate experience, right?
So all I can do is give you some sort of an assessment. And what we call this, what's in here, that's your true score on that trait or that skill. When we give you an assessment, what we end up is what we call an observed score. So, so we're getting a measurement. We're observing a score. An observe score is always an estimate of the true score.
The closer the estimate is, the closer the observe score is to the true score, that's reliability. So, and there's ways of, of providing evidence to show that tests are reliable. It, it's a mathematical concept, requires lots of people. but that's a trait or a psychometric property of tests that you wanna look for.
So we've got validity, we've got reliability, and then we've got fairness. And we talked a lot about fairness already. We wanna make sure that your test works the same way for everyone, regardless of their demographic, that it predicts success and in the same way that the passing rates are the same, that there's no, what we would call differential test or question functioning within that.
So those are the three things that I think people need to look for. I hope that made some sense.
[00:20:02] Garrett Jestice: Totally did. Yeah, I think that's, that's excellent. Excellent tips. So, Fred, I, I, I think I could go for another hour here too, and just, I, this is such fascinating stuff. thank you for being on the podcast today. I know we just barely scratched the surface of this, but this has been such an excellent conversation and I definitely see the value, of interjecting, especially these behavioral assessments in the hiring process. Right.
as we wrap up here, if there are listeners who want to learn more about this topic or about Talview, or if they have questions for you, what's the best way for them to, to reach out and connect?
Sure. And boy, that went by fast and then I enjoyed it and, I would love to come back and talk more.
[00:20:45] Garrett Jestice: Yeah.
[00:20:46] Fred Rafilson: I would love for people to come to Talview. Go to talview.com. That's the best way to find this. if you wanna reach me, I think the best way is through my LinkedIn profile. Hopefully, you can share that with the audience.
Please reach out. I, I look forward to continuing the conversation.
[00:21:02] Garrett Jestice: Thank you so much Fred. We'll drop links for all those listeners out there. We'll drop links to both the Talview site and Fred's LinkedIn profile where you can find and connect with him in the show notes so you can find them there. So Fred, thank you again so much for being with us today, sharing your knowledge with us, and we hope you have a great rest of the day.
[00:21:18] Fred Rafilson: My pleasure. You too. Great to meet you.
[00:00:13] Fred Rafilson: I'm doing really well, Garrett. Thank you.
[00:00:15] Garrett Jestice: Thanks for being on the show today. How's the weather in, beautiful, Sunny Bend, Oregon today?
[00:00:20] Fred Rafilson: Well, that's a great question because it's been snowing for the last two days and, I'm pretty excited. I've got my season pass coming from the mountain, so it's perfect.
[00:00:28] Garrett Jestice: That's awesome. I've, I've been thinking the same thing. You know, we're based in Utah here and winter just decided to come the last few days, but looking forward to hitting the ski slope soon too. So, Fred, we're, we're excited to have you on the show today. and, to dive into this really interesting topic.
Before we do that though, give our listeners a little bit more context on and about you and your background as well as what your company Talview does.
[00:00:52] Fred Rafilson: Sure, thanks. I'm an industrial and organizational psychologist, an I/O psychologist, and what we do is we focus on the psychology of people and work. and specifically, I'm a testing guy. I'm an assessment guy. I'm all about measuring people and determining what are their skills, abilities, knowledge, you know, what are their competencies as, in, in terms of are they gonna be predictive of success on a job?
So I've been doing this for, longer than I care to say, you know, I'm just gonna throw out, a few decades is good enough. I had my own, business years ago. We developed, exams for police and fire departments around the country. Became the nation's largest provider of those, I since sold that to my employees.
moved up to Bend and I've worked for several large, testing companies as their chief I/O psychologist, until I found, Talview which I really love. It's an amazing company. We do some really cool things. That I'll tell you about. And, I came to tell you, and I'm the chief I/O psychologist here.
[00:01:46] Fred Rafilson: So I work with our customers to ensure that their assessment programs are number one valid, they're helping them select the best people. number two, fair. They're doing it in a way that works the same way for everyone. And number three, that our candidates have a great experience going through the process. so in a nutshell, that's what I do. Talview is, I, I struggle with the best way...
it's, it's an end to end talent measurement platform and it's AI driven, really state of the science. So we start with everything from reading resumes to administering assessments that are, you know, job related based on the requirements of your job to, asynchronous interviews prerecorded with psycho linguistic capabilities, which we'll talk about later to help not only look at the answers to the interview questions, but look at the behavioral traits of of the candidates that are answering them.
To live interviews and scheduling, full suites of assessments so we can really help an organization, acquire the, the right talent for their positions.
[00:02:46] Garrett Jestice: I love it. That's so great. It's such a great elevator speech. I think a, a great precursor to what we're gonna talk about today. Tell me a little bit more about, who your typical customers are. Who do you typically work with at Talview? Is it all across the board or are there, are there certain types of companies that have really found value with working with Talview?
[00:03:02] Fred Rafilson: So I guess two answers. One would be industry, and the second would be, you know, size of organization. we work in all industries, right? Everybody needs to hire. And, especially these days, it's really tricky because, you know, good candidates, they're, they're not sticky. If we can't get them right away, we're gonna lose them to somebody else.
So, you know, hiring is really, in an interesting place right now.
[00:03:24] Garrett Jestice: Mm-hmm.
[00:03:25] Fred Rafilson: All industries. we tend to focus on enterprise organizations because they hire thousands of people and have tens of thousands of organizations of, of, candidates for these jobs. But we work with medium and small businesses as well.
I've always said, especially being a business owner myself, a small business owner, you know, clearly when you have to hire thousands of people, Making sure you make valid decisions is critical, right?
[00:03:51] Garrett Jestice: Hmm.
[00:03:52] Fred Rafilson: Hire one person, you make a bad decision there, it's really not a good thing. So, what we applicable to any size of organization, in any industry,
[00:04:01] Garrett Jestice: Yeah. I love that. And I think our topic today is too, so I think what we're gonna talk about today is really this idea of behavioral assessments, right? And how do they, how should they, and how can they help in your talent acquisition efforts? And that's a topic that I think can be applicable for any size business.
Right. And so to really just start us on this path, tell us a little bit more about what behavioral assessments are and the different types of insights they can provide when you are interviewing job candidates.
[00:04:31] Fred Rafilson: Yeah, that's a great question. I think people tend to think of assessments as skills assessments, right? Does a person, can they code? Do they have the medical knowledge they need to be, an ER tech? you know, do they have the finance knowledge they need to, to work in a bank? And of course, those skills are critical, right?
If you don't know, if you don't have coding skills, you're not gonna be much of a software development, right? You don't have the medical skills, I don't want you looking at me when I come to the ER. So skills are critical. But skills are only a part of the puzzle, right? It takes two things to make a good employee, and we know this through years of research, and I think common sense.
It takes two things. One is what you can do, and that's skills. And the other is what you will do. And that's your behavioral tendencies: your motivation, your attitude, your personality. A person can have all the skills in the world, but if they're not motivated to come to work and they don't wanna be a team player and they can't get along and what have you, they're gonna be a terrible employee, right?
On the other hand, someone might have, you know, great motivation, great attitude, just an all-around great person, but they don't have the skills. That's not gonna work either. So you need to assess both. And what we're here to talk about today is really looking at those behavioral things.
So, what kinds of things do we measure? And I, I touched on it a little bit. We really try to predict how a person is going to behave in a particular situation. And we do that by looking at their personality traits. we look at their attitude, we look at their communication skills, and I use the word skills as sort of a blurry line sometimes.
we look at their integrity and we think about all of the jobs where integrity is key, where people have access to money and merchandise and sensitive records, et cetera. You wanna make sure that you know, We'll have a good moral compass and are gonna do the right thing and aren't gonna steal, aren't gonna use drugs on the job, aren't gonna engage in counterproductive behaviors.
[00:06:32] Fred Rafilson: All of those kinds of things. You know, teamwork orientation -- with remote working now and, and, and teams and, the importance of working in teams. We wanna make sure that people are suited for that. some people are really suited for working independently, and that's great for certain jobs. but it's difficult to take a person like that, make them part of a team and expect them to function if they're not wired that way.
Certainly there's training, but we wanna select the right people. And vice versa. We don't wanna take someone who needs that social contact and needs that teamwork and put them in a position where they're never gonna talk to anyone. They're not gonna be happy, they're not gonna be engaged.
[00:07:06] Garrett Jestice: Yeah.
[00:07:07] Fred Rafilson: So those are the types of things that we look at with behavioral assessments.
I always kind of liken it to when people say, well, you know, we just need to measure their skills. And I'm like, that's awesome. But like when I go to a restaurant and I get a hamburger, I don't just ask for the patty. The patty's great, but I really like the bun and all the condiments and everything else that's in there because that's a hamburger.
Right? I know. It's kinda a crazy analogy. I'm hungry. It's, it's almost lunchtime here, but, you need to measure the whole thing, not just, not just what this goes, not just the patty. We need the entire package and that's where behavioral assessment comes in.
[00:07:44] Garrett Jestice: Yeah, that makes total sense. And I, I'm gonna jump in just to, we're just gonna dive into the deep end here with this next question because I'm curious, you know, your thoughts on this. because I'm sure there's people out there who, who, who get it. They're from a business perspective, like you need to be able to, to test skills and these behavioral traits and tendencies.
But I, I think there's also people who might be wondering, are these behavioral assessments fair to do? Like when someone asks you that question, what's, what's your response to that?
[00:08:15] Fred Rafilson: So my response is, yes, they're fair. And I'll tell you why I believe that. One, we have decades of research. I/O psychology is not a new field, and we can talk about that later. But, We have decades of research to show that, different types of personality characteristics do indeed predict success at work.
So we know empirically that there's a correlation between certain types of personalities and success on certain types of job. So they're fair in terms of being job related. They're also fair in terms of well developed tests, properly constructive tests being provided by reputable providers. I mean, God only knows there's a million things out on the internet, so be careful.
You know, what you buy or what, what you read. We all know that now. Right. But assuming you're dealing with, you know, respectable test vendors, we also know that the tests are fair and the reason is that we go to great lengths to ensure that they're fair. So that starts with the, very beginning with item construction.
And when I say items, I mean test questions. So when we're developing questions to measure a particular construct, whatever that may be, teamwork orientation, you know, we as psychologists know that, well, this question should be predictive of that. This question should be predictive of that. We write a big pool of questions, more than we think we're gonna need. We then review them very carefully, for any signs of bias at all. And we even bring people in who are experts at looking for cultural bias, gender bias, and they review the questions. And so we all as a group say that question should be good. but the thing is, we're all human and we all have unconscious biases.
[00:09:50] Garrett Jestice: mm.
[00:09:51] Fred Rafilson: Anyone tells you they don't, they just don't know themselves. We can't help it. We grow up in a particular society. We know what we know. You know what you know, I know what I know. We all have our biases. Okay? and it doesn't mean we're not well intentioned. They're just there. It's, it's human nature. So what we then need to do is take all of those questions and administer them to large groups of people, diverse groups of people, ethnically diverse, gender diversity, geographic diversity, socioeconomic diversity.
People with disabilities. So we administer these items and we collect all this demographic information, and now we look at how do these different groups respond to these items? If we see a systematic difference between the way one group responds to an item, we say there's something going on there. sometimes we see it and we're like, oh yeah, that makes sense, and we can fix it.
Quite often we're like, no idea why, but the numbers are showing us that that item is not fair to that group. So that item gets thrown out.
[00:10:47] Garrett Jestice: Hmm.
[00:10:48] Fred Rafilson: Now we can develop our tests based on items that have been field tested. We know that they're fair, but now we put the whole test together. We administer the test.
Your company, goes ahead and administers this test and we monitor it. And, you know, there are federal guidelines we work with, you know, the EEOC, the Equal Employment Opportunity Commission, who rightly so says tests need to be fair. They can't discriminate against people. and what we, we then do is we routinely, whatever cadence based on how many people you hire, we'll look at per thousand hires or whatever.
or we'll look quarterly, but we'll then take all of those test scores and we'll take all of the demographics and we'll see, are there any differences? Are we seeing any patterns? Hopefully not. Everything's great. If we see something, we dive in and we say, What's going on here?
[00:11:31] Fred Rafilson: Fix it if we can, remove it if we can't. So the answer is yes, they're fair.
[00:11:37] Garrett Jestice: I love it. I love hearing the context behind all of the thought and you know, work that goes into making sure that they're fair and proving, you know, again, this is a science that is predictive of actual behavior in the future, and that's been proven. So, I know there's probably people out there who are thinking, Okay, this sounds great, but we really, especially in today's crazy hiring market, we really need to be competitive.
And it sounds like, you know, assessments like this will take, will slow down the hiring process. They will, they'll take a long time and won't be great from a candidate experience perspective. I mean, who, who wants to go take a big, huge, long test just to be....
You know, have an opportunity to get into a job. So how, how do you, how do you balance that? How do you, have effective,assessments without hurting really that candidate experience and slowing down the hiring process?
[00:12:31] Fred Rafilson: That, that's a great question and I can relate to that. Having been in this field for a number of years, when I started, you would have to take a psychometric test that, you know, could have taken you an hour and a half to complete and you would do it because you had no choice. And that's what everyone was doing.
But the world is different, and candidate experience is everything. so how do we balance that? How do we ensure that the tests are just as predictive, but in a way that only takes six minutes? Right. And also doesn't turn off the candidate. So, amazing question. And just like every other technology, the science of, of psychometrics has increased and, and improved dramatically over the years.
And I'll give you an example of something that we do. That is one of the reasons I love Talview and we, we really lead the industry with this. Let's look at a particular role. I'm thinking like a QSR, a quick service restaurant role, an entry level role. So need to get people in the door, need to know if they're gonna stay on the job.
We wanna screen them, but we really don't have a lot of time and we have a big funnel, so what can we do? I don't have a million recruiters that can be on the phone and interviewing these people. I don't have that kinda staff. I don't have that kinda time. and all of these, you know, and also I don't wanna ruin the candidate experience.
So all of those things feed into it. Well, here's, you know, kinda a real use case. What we can do is, of course, everything's online and everyone's online, so they're gonna find a link to this job app, whether it be a QR code that they see in a particular restaurant or they're on a job site, whatever.
So they, they click on it, you know, are you interested in applying? Yeah, I need a job. A chat bot comes up and says, you know, are you available these times? That's a screener. Yes I am. Can you legally work in the US, whatever screening question. So that 15, 20 seconds. then, hey, are you available now for a quick interview?
candidate says, no. Okay. Well, here's a calendar. When are you available? So they put that in or yeah, sure, I can talk now. I, I've got my phone. So then they immediately go into what we would call an asynchronous interview, meaning that it's a lot, it's a recorded interview and they're asked a series of questions, and they respond to them on, on their phone.
[00:14:47] Fred Rafilson: It doesn't need to be done at a desktop, which is one of the cool things because we really wanna level the playing field for everyone. And you, you don't, you shouldn't have to have, you know, an internet connection and nice equipment. You can do this on your phone, and just about everybody has a phone or has an, has the phone. So they give their responses, they, they, answer these questions.
we work with the companies to determine what those questions should be, and maybe that interview takes three or four minutes. now here's the magic as far as I see it. So that interview is saved and the hiring manager can go back and look at the interview and they can, they can look at the candidate or listen to the candidate and, and hear the responses to those questions.
[00:15:26] Fred Rafilson: But in the background, what we're doing is we're taking that candidate's responses and we're running it through what we call a psycho linguistics engine. And psycho linguistics is a science that's been around for a long time. And has relatively recently, we've been applying it to talent acquisition. So we're actually looking at the words you use, the way you form your sentences, the way you speak, the way you think, and we can actually derive these behavioral scores and competency scores that are unrelated to the questions you asked that will give you the same kind of output that you're used to seeing from those psychometric tests, just from listening to the candidate speech. So now you've done this really brief process. You know, they've clicked, they've answered a few questions, they've answered a few interview questions.
You get to look at the interview, but now you're also getting a report on, you know, their team orientation, you know, on all these different behavioral characteristics. And the candidate was not even aware that, they were being assessed on these things. Which as the added benefit of you recall, if you've ever taken one of those tests, you're always like, Well, how should I answer this?
You know, I need to make myself look good. I wanna answer in a socially desirable way. Well, that's not happening. So these things aren't being gamed. You're getting a really good measure of a person's behavioral tendency. so that's one example of how we can make the candidate experience good, make the, assessment brief.
And still get that kind of predictive, information. So I, sorry. I tend to lose track. Just push me back when I, when I do. But...
[00:16:56] Garrett Jestice: no, that's great. Yeah, it's fa it's a fascinating example and I can definitely see the use case there. And it's, it's cool to learn about that too, that it doesn't take an hour long assessment now it takes, you know, a couple minutes of questions and sometimes people even know, don't even know they're taking the test, right.
But, and so you can get those really honest results and, and really predictive of behavior. So it's fascinating. It's great to hear that. So if someone out there is looking, they, they understand the value of behavioral assessments. They're looking for, you know, the right behavioral assessments, whether they go with Talview or they're looking somewhere else.
What are characteristics of the best behavioral assessments out there? Like, what should, what should you be looking for?
[00:17:36] Fred Rafilson: That's awesome. I'm gonna, we're gonna call this I/O Psychology 101, and we're gonna an extra hour to this podcast just this, but I'm gonna say to narrow it down, there's three things they should be looking for. There's validity, reliability, and fairness. And your test vendors are gonna have information on these three things. If they don't, you know, this goes back to my earlier point, find someone who does. Right?
So validity, does the test do what it's supposed to do? Test scores, should. Predict success on the job, right? Or they should indicate a master of a particular skill. There needs to be documentation that those test scores do what they're supposed to do.
That's validity. Ok. And what that really comes down to is are they job related? That's validity. Reliability is an interesting term that even a lot of I/O psychologists get wrong, but it's a really cool term. and what it has to do is with, is the accuracy of the test. And you can sort of picture it this way.
Say, looking at your ability to do math, or I'm looking at your extroversion, right? Cause you seem very extroverted to me. So in your head you have this level of math ability or extroversion, this is you. Okay? but I can never get in your head and pull that out of there. No matter how hard I try, that's gonna be a bad candidate experience, right?
So all I can do is give you some sort of an assessment. And what we call this, what's in here, that's your true score on that trait or that skill. When we give you an assessment, what we end up is what we call an observed score. So, so we're getting a measurement. We're observing a score. An observe score is always an estimate of the true score.
The closer the estimate is, the closer the observe score is to the true score, that's reliability. So, and there's ways of, of providing evidence to show that tests are reliable. It, it's a mathematical concept, requires lots of people. but that's a trait or a psychometric property of tests that you wanna look for.
So we've got validity, we've got reliability, and then we've got fairness. And we talked a lot about fairness already. We wanna make sure that your test works the same way for everyone, regardless of their demographic, that it predicts success and in the same way that the passing rates are the same, that there's no, what we would call differential test or question functioning within that.
So those are the three things that I think people need to look for. I hope that made some sense.
[00:20:02] Garrett Jestice: Totally did. Yeah, I think that's, that's excellent. Excellent tips. So, Fred, I, I, I think I could go for another hour here too, and just, I, this is such fascinating stuff. thank you for being on the podcast today. I know we just barely scratched the surface of this, but this has been such an excellent conversation and I definitely see the value, of interjecting, especially these behavioral assessments in the hiring process. Right.
as we wrap up here, if there are listeners who want to learn more about this topic or about Talview, or if they have questions for you, what's the best way for them to, to reach out and connect?
Sure. And boy, that went by fast and then I enjoyed it and, I would love to come back and talk more.
[00:20:45] Garrett Jestice: Yeah.
[00:20:46] Fred Rafilson: I would love for people to come to Talview. Go to talview.com. That's the best way to find this. if you wanna reach me, I think the best way is through my LinkedIn profile. Hopefully, you can share that with the audience.
Please reach out. I, I look forward to continuing the conversation.
[00:21:02] Garrett Jestice: Thank you so much Fred. We'll drop links for all those listeners out there. We'll drop links to both the Talview site and Fred's LinkedIn profile where you can find and connect with him in the show notes so you can find them there. So Fred, thank you again so much for being with us today, sharing your knowledge with us, and we hope you have a great rest of the day.
[00:21:18] Fred Rafilson: My pleasure. You too. Great to meet you.
Eddy's HR Newsletter
Sign up for our email newsletter for helpful HR advice and ideas.
