Hello again, everyone. Welcome to another episode of The Testing Psychologist podcast. Hope you’re all doing well today. I am just in a great mood this morning. It’s a Monday morning. And the reason is because I think we have finally had our last snowstorm here.
Colorado is notoriously crazy for having late spring snowstorms. I thought we had escaped it this year, it’s obviously mid-May, but last week we had two straight days of this heavy, wet snow and ended up with probably 6 to 8 [00:01:00] inches of snow in the middle of May, which is not ideal as far as I’m concerned.
Growing up in the south, I’m definitely a summer person. I like the heat. I like the humidity. So, that was a bummer to have that thrown upon us all of a sudden, but as is typical with Colorado, the next day it was first about 55 degrees and then the next day it was 60 or 65 degrees and all the snow had melted and now it seems like we are actually, I think done with the snow and we’re really cruising into the summertime, which I’m really excited about.
Both my kids have learned to ride their bicycles. They love it. So we have been taking some family bike rides and it has set the stage I think for a really cool summer. So hope all is going well wherever you are and maybe you’re getting into the summertime as well.
I’m going to be doing a solo episode today talking all [00:02:00] about how to hire and train psychometricians. I get a lot of questions about how psychometricians operate in our practice and why you might want to hire a psychometrician. I’m going to talk all about that today and try to give you some info if you’re considering that or maybe even have some psychometricians in your practice, maybe some ideas to streamline that process and get everybody on the same page and make it work as effectively as possible. So I’m going to dive right into it.
One of the first things when you’re thinking about hiring a psychometrician, and just to back up a little bit, I should say the term psychometrician is a mouthful. Some people will call these individuals techs or testing assistants or technicians, but either way, the [00:03:00] psychometrician is someone who administers the tests and scores the tests for you.
A good analogy would be, for example, radiologists are not the ones who actually put the vest on you and get you into the radiography machine, whatever that might be. They are not the ones that do that and they don’t print out the x-ray or anything like that, but they are the ones who interpret the x-ray.
So for us, the psychologists are the radiologist and the psychometrician is more like the x-ray tech; the person who plays this really important role to get everything set up appropriately and make sure the x-ray is taken precisely and then relaying those results accurately and effectively back to the radiologist. That gives you some idea.
When you’re thinking about whether a psychometrician [00:04:00] may or may not be a good fit for your practice, I can say that the main reason that I ended up taking on psychometricians way back, in the beginning, is because I needed more time.
I think I’ve told the story here and maybe on two other podcasts about how back when I launched the testing part of the practice, it was due to getting probably 5-6 times my typical amount of referrals all at once. I went from getting maybe 2 or 3 referrals a month for testing up to, I think I had 15 or 20 all at once at one point due to a change in policy here at our local university. So I had to figure out how to find the time.
Now, I had served as a psychometrician when I was in grad school. I worked under a clinical neuropsychologist for two years. So I was familiar with that model and I was pretty open to it right off the [00:05:00] bat just because of my experience there. So when I knew that I was getting extra busy and physically could not see all these testing cases, then I started to think it’s time to look for some psychometricians.
I went to our local graduate school- the department that I did my doctorate and just put out the word that I was looking for two students to come and get trained and be psychometricians in the practice. And that ended up working out really well.
A big reason that I think a lot of psychologists hire psychometricians is to have more time. If you think about doing the testing, actually administering the testing, depending on the battery you’re doing, that could be 4 hours, 6 hours, maybe 8 hours. So that’s a lot of time that the psychologist could otherwise be using for other things. And that’s why I think a lot of folks do hire a psychometrician to help with that.
So, if you’re deciding, [00:06:00] I’d really sit down and think what could you be doing with that time: Will that allow you to be free and maybe not work as much but still be able to make some amount of passive income? Would you be able to see more cases? That’s a big motivation. So you could certainly spend your time more on intakes and feedback and interpretation rather than just administering the tests. So there are two factors that you may consider.
Now, two things that are a little bit tougher about having a psychometrician though is that it does certainly take some time to invest in training depending on the level of experience of that person.
Now, I typically bring on advanced doctoral students, so these are individuals who are in the latter stages of their PhDs. They typically have at least a year’s experience administering and scoring tests already, and they all have master’s degrees already. So in my [00:07:00] training, there is quite a bit that happens and I’ll talk about that here in just a bit, but they really are ready to hit the ground running for the most part. So that’s something that you would want to take into account.
We will talk a little bit more as well about the board of certified psychometrists. This is an organization that’s moving to create standards for certifying psychometricians. I’ve met 2 folks who have had some experience with that board, and they just have an incredible experience with testing and are ready to almost just like plug and play in practice because they are so skilled. So the level of training is going to vary, but that’s going to be an investment in your time. I also do ongoing supervision with my psychometrician. We meet for an hour every week. So that’s an investment as well, just on an ongoing basis.
The other piece I think that’s really crucial if you’re thinking about hiring a psychometrician is that you [00:08:00] have to let go of a certain amount of control. Now, as a practice owner and as someone who likes to do testing in general, I think many of us really enjoy being precise, looking at data, facts, and standardization. So this can be challenging. It was certainly challenging for me as well.
Now, you can address that with some clear training processes, which I will talk about here in just a little bit as well. But I think that’s just something to think about for yourself, like, would you really be able to let go of that amount of control? And if so, how do you do that?
Let’s start to talk about how you actually do that.
For me, the interview and selection process is huge in selecting the psychometrician. Now, of course, I think this makes intuitive sense, but I start from the very beginning when I’m considering who might come on as a psychometrician [00:09:00] here in the practice. When I put the advertisement out, obviously you get applications and emails and that sort of thing. And I am looking right off the bat at the quality of those applications.
One of the skills that I think is just huge for a psychometrician is attention to detail. They have to be accurate in their scoring, in their writing, in their test administration. Attention to detail is huge. So, I am actually really strict and very attentive to those initial emails that I get. If there are any errors in those emails, I mean, capitalization, grammar, the wrong agency name on the cover letter, any errors at all in the application or on the resume, for better for [00:10:00] worse, I take that as a sign that this person may not have the best attention to detail. And so, I rule out those applications pretty quickly.
Once you get past that, then you can really start to focus on the applicants and how they might be a good fit. So in my interviews, I am really upfront with the applicants about what the job entails: what it means to be a psychometrician and what qualities are important for that. Of course, I’m kind. I don’t drop the hammer right away and scare them off, but I’m very clear that two of the biggest things that you have to consider are:
1) Attention to detail like we’ve talked about, but
2) Efficiency is a big one as well.
The way that our practice is set up, we do typically all of our testing in one day. So that tends to be a lengthy day. It’s probably 6 [00:11:00] hours face to face with the client when all is said and done with a lunch break and everything like that. And then it takes at least 2 hours to score everything and do the writing. That’s all the psychometrician’s job.
I’m very upfront that if the individual is not good at using time efficiently and working fairly quickly and accurately, then it’s going to be really easy to get behind, the cases stack up and then that can be a bad situation. So I’m pretty clear with that. I talk about timeliness very clearly. So making sure that the individual is on time for appointments, on time for supervision, and on time with the reports.
So, again, I’m very clear about the expectations here in our clinic where my ideal expectation is that the scoring and writing is done at the end of the day after testing. There are some cases when [00:12:00] that might extend out a little bit just working primarily with graduate students- they have schedule issues that they have to work with and sometimes they have exams and that kind of thing. So I’ll give them a little bit of leeway sometimes to make sure that they’re on time, but generally, the expectation is that the writing and scoring is done on the day of testing.
Now, other things that I really talk with them about include their willingness to ask for help. My worst nightmare is a psychometrician who might think that they are doing things okay but they just cruise along doing it the wrong way rather than ask me a question about it. So I really try to emphasize that I have an open-door policy and I’m very willing to talk with them about any issues that might come up.
And then another piece that I talk with them about is their ability to take feedback. I’ll touch on this a little bit more as we go along, but a big part of [00:13:00] the report writing process is feedback for me. I use Track Changes to correct and edit their reports, especially initially. And I’m also giving ongoing feedback as we go along and as they are learning and even later stages, sometimes things will come up. And so, I always make sure that the folks are open to taking feedback and hopefully not going to be super defensive. And of course, I try to be kind as well. And that helps, of course.
So the interview process I think is really important. Generally speaking, I feel like administration skills can be taught fairly easily. Again, I’ll talk about that here and how we do that, but my philosophy is that personality and these other skills that I’ve discussed: timeliness, efficiency, attention to detail, that kind of thing. I think those are way more important than administration skills when someone comes in.
[00:14:00] Now, of course, they need to be familiar with the different measures, but I’m not drilling someone during the interview about what do you do if the person does not give an answer within so many seconds on the WISC. I’m not talking about that kind of stuff. I just feel like that stuff can be taught and can be taught well. So I’m really focused more on personality and how this person is going to connect with others in the agency, how they’re going to connect with my clients during testing, and how they’re going to get the job done.So just to emphasize that, I think the personality is huge. And again, connection with the client. I should have maybe said more about that as I went along, but that is actually really, really important for me and I think for the client too. Testing is just such an intimidating process that you need to have someone who can connect with your client and help people feel comfortable during that process.
Now, one other aspect of the, I guess you’d say interview processes that I [00:15:00] do ask for sample reports. As I said, everybody who comes in has already done a fair amount of testing. So they always have sample reports ready. I look at those not so much for the content of the interpretation, but really more just to see what their writing styles look like, do they have good grammar and good punctuation? Again, do they have attention to detail? So I’m looking for any mistakes in those reports. That really helps to give me a decent idea of what they’re capable of and how they pay attention to those important pieces.
Once people get here, then we are talking about training. So during the interview, I always ask them, are they familiar with these tests? I show them our typical battery and try to get some sense of what they are going to be bringing in. Most people have some familiarity with the battery, but not complete familiarity.
So I always ask: Is there any way you can do some [00:16:00] research on the internet? Can you go to your local clinic? Does your grad school program have these measures? And I ask them to look over any measures that they can just to be somewhat familiar. So it’s not the very first time they’re seeing that particular measure when they come into our practice. That just happens right off the bat during the interview.
Now on my side, I think the training documentation is really important. And so for you all, if you’re thinking about hiring a psychometrician, I would start right now writing down literally everything you do during test administration. Now I’m not talking about, you don’t have to rewrite the Testing manual or anything like that, but all of those little things like what tests you give, the order that you give the tests, what you do in between, why you’re not giving the tests, [00:17:00] what you say when you greet the client, how you explain the testing day, when you take a lunch break, what order you score things in, how you write, where all the scoring software is, do you do it by hand or do you do it on the computer? You get the idea.
I would go through your typical testing day and operationalize virtually everything that you can. So just write everything down and try to start. This is going to be the basis of your training manual. Even the little things. That’s one of the big things that I noticed is that I was doing a lot of little things that were second nature to me that other people might not think to do. For example, asking for the parent’s cell phone number so that we can text them when the testing is over, something like that. And that’s just a little piece that’s important that you might skip over.
Anyway, I [00:18:00] would start and just write down everything that you do, take two testing days. It’ll take you a little extra time, but I think this is really important.
Once you do that, you can start to build your training manual. For us, our training manual at this point is pretty lengthy. We have what I would call a macro document that has I’d say a big-picture checklist for the testing day. It’s about 1-2, it might be 2 pages now- a 2-page little document that basically has a 50-item checklist for the actual testing day.
The idea with this macro document is that someone could come in, they could open this document and as long as they know how to administer the particular tests, they could follow this checklist and get through the testing day and [00:19:00] administer a battery effectively. So it has big picture stuff like greet the client, give them the schedule for the day, ask for their phone number, tell them what the testing day is going to look like, so on and so forth.
And then it gives the order for the battery that we typically administer. It tells what time to do lunch, how to determine when to do lunch, and then it describes what order to write things in most effectively, where all the scoring software is, and how to do the note in our EHR system, which is TherapyNotes. So, big picture checklist. 50 items sounds like a lot but that includes every test that we administer. So there’s a lot that I think is pretty simple and straightforward on there and just makes up for a lot of items. So that’s our macro document.
But then we also have micro documents for each test. These are housed in a [00:20:00] binder though we also have them electronically on Google Docs as well, which I would definitely recommend because then you can edit them at will. So, micro documents for each test.
And these go into a lot more of the nuances of test administration. I don’t really duplicate anything that’s already in the manual, but we do talk about little things to keep in mind that might be buried in the manual or things that you might look over or just little eccentricities that happen when you’re administering that particular test and things to look for and just be aware of. And then, of course, we talk about how to score it and typical questions that might come up and things like that. So we have little micro documents, like I said, for each test that we administer.
That really makes up our training manual. It’s, like I said, pretty lengthy at this point. The individuals get access to this from day one. I will actually provide access to the [00:21:00] training manual before they come on board. And I really encourage them to look over it even though it’s all theoretical, but just to get some familiarity with our process and with what we do.
In terms of the training schedule, when someone first starts, we have a fairly structured schedule for how we “onboard them”. The first two days, they observe either myself or another advanced grad student who’s been here for a while. They observe one of those individuals administering all of the tests. So they just sit in the room back in the corner. We, of course, tell the client and everything like that but all they do is observe, and then afterward they stick around and we score everything together. We talk about it, answer any questions, let them look around on the computer, and familiarize themselves with the [00:22:00] scoring software and things like that. So that’s the first two days where they observe someone else administering.
The 3rd and 4th day, I observe them, or again, another advanced psychometrician would observe them doing the administration. They would do the scoring themselves with supervision, of course. So we’re sitting side by side this whole time. If they don’t need anything, then I can work on other work. If they do, then of course I’m there to help them or the other grad students there to help them. And then walk them through the writing process, and answer any questions. At the end of that fourth day, they’ve either observed or conducted their own assessment for about four days in a row. If they’re good to go, then that’s great. Then I turn them loose and if I feel confident, then they’re ready to go.
One thing that I [00:23:00] know some other practices do, and this is what I’ve done when I worked for a neuropsychologist was, they do test administrations. So the new psychometrician would have to practice administering on another employee, for example, before they get turned loose with a client. So that is an aspect that I’ve certainly considered adding. I haven’t done it yet, but I know that other practices will do that.
Now another aspect of this training is that I always have them just shadow me out to the waiting area. So they get a sense of how I interact with the client, talking with the parents if it’s a kid, also modeling interaction with the kids. I think generally speaking, here in our practice, we tend to err on the side of being informal whenever possible. So joking around with the kid, say something like, all right, let’s go back to the torture chamber, something like that, just [00:24:00] to lighten the mood and break the ice a little bit. You get that idea.
I’ll be pretty informal, of course, as long as it doesn’t interfere with actual test protocol or test administration or anything like that. So joking around, being casual and friendly with the parents and the kiddos. I like to model that really clearly for the grad students just to let them know. I think sometimes they’ll come in and, of course, it’s a new job, they want to do a good job and be pretty precise and professional, but I’ll let them know right away it’s okay to joke around, keep people in a lighter mood. So I think that’s really important.
Another aspect of training is that we are really big on behavioral observations here. I think this is one of the biggest things when hiring a psychometrician that it’s a big hurdle to get over is, people say, Oh, I’m going to lose all these observations. [00:25:00] Interacting with the client for all those hours gives you so much data. And I totally agree. So that’s a big one. That’s hard to get over. I’ll just put that out there.
The way that I’ve addressed that, there are two ways. There’s the documentation standpoint. We have actually a pretty lengthy behavioral observations document that I put together that addresses all the different aspects of test-taking behavior, all the way from gate and motor skills to eye contact, to mood and affect, to their effort, response to failure, how much caffeine they drank, were they excited, happy, sad, nervous, all that kind of stuff.
We have that larger scale document that has a bunch of check [00:26:00] boxes. I try to make it easy where folks can just go through and check off certain characteristics or observations. So that’s pretty important.
And then I also emphasize taking notes during the actual test administration. So lots of notes in the margins of the scoring booklets trying to document as much as possible what the actual test-taking behavior was like.
For example, if you’re familiar with the WISC or the WAIS, on the block design, I really want to know, are people missing these items because they are running out of time or because they worked quickly but got the wrong answer. Just a small example like that. With like reading tests, I want to know what kind of errors they’re making and how they responded to that, were they upset, were they not, different things like that. So a lot of notes on the margins.
Now, one thing that I am working on is an [00:27:00] even more detailed behavioral observation sheet where we have a separate sheet for each test. So let’s say there will be a WISC observation sheet with spaces or blocks for taking notes on each particular subtest so that you don’t have to cram them into the margins. But that’s an ongoing project. So, that wraps it up in terms of training for administering the tests.
The next part of our process is writing reports. Again, I do a lot of quality control so to speak with scoring to make sure that scoring is being done effectively and accurately. And that leads of course, to writing an accurate report to make sure that the scores are correct. So, there is that piece.
With the actual reports, it’s pretty simple. We have cloud-based electronic records, so everyone can access the reports that are [00:28:00] written. I just use Track Changes or the editing feature in Google Docs, which is great, for shared document editing. And I using one of those methods make it really clear what changes I’m making to the reports. This tends to be more relevant at the beginning while folks are learning how to write the report.
And I will say, actually, I should back up just a little bit, that I have honed our report template, particularly the results section where you’re just talking about scores and what range the score fell in and that kind of thing. I’ve really nailed that down to where it’s very structured. There’s not a whole lot of room for creative writing so to speak. And it’s, like I said, pretty structured and straightforward. So that was some work just on the front end that was put in place to make sure that there is standard language in the reports.
[00:29:00] Either way, I do go through, I edit the reports. I use Track Changes just to let people know little tweaks and nuances here and there that I am making and things that I like to address in the reports.And then again, on an ongoing basis, we do supervision each week. So we meet for an hour talking about specific tests, things to watch for, and answering any questions. So we use supervision a lot and of course, talk about interpretation of the tests and things to watch out for there. So those pieces tend to… We tend to focus on that over the first month or maybe 6 weeks of supervision, and then after that, we transition to the work and case consultation and that kind of thing.
That’s our training process in a nutshell. I know that there are many other models out there. Like I alluded to at the beginning, there is, I think an increasingly popular [00:30:00] push to get your psychometricians certified. So there’s now board of certified psychometrists. I would really love to get a professional from that board to come on the podcast. So I’m going to be working on that. But there is a push to certify your psychometrists. And this is, I think like any other certification, you have to get a certain number of practice hours or supervised hours, there is an exam and then there is a certification that you can obtain. I think the opinion coming out of this so far is that it’s really nice.
If you do a lot of forensic work, you are probably aware that they will question the qualifications of your psychometrician in court. And so, having a certified psychometrician can really help. And then it’s also, I think just nice for standardization period. You know that you have someone who’s quality and is [00:31:00] administering the tests effectively.
A step short of that is that I know in children’s hospitals, for example, where they might have a team of psychometricians, they will often have a lead psychometrician who is in charge of supervising and training all of the psychometricians just to ensure standardization as well. So there are a few steps that are possible to make sure that you have a standardized administration process. But like I said, hopefully, I can get someone on to talk about that board of certified psychometrists and really give us more detail on that.
Thank you as always for listening. This is really cool. I love doing these podcasts and sharing some of this information with you. We have some really cool interviews coming up over the next few weeks. I’m going to be talking with Dr. Bryn Harris down at CU Denver about culturally competent assessment. [00:32:00] I am also going to be talking with Dr. Karen Postal, who wrote the book, Feedback That Sticks, an amazing book about giving feedback and difficult feedback. So I’m really excited for those interviews.
In the meantime, if you want more information or want to learn more about The Testing Psychologist or testing in general, you can go to the website, which is thetestingpsychologist.com. You can also check us out on Facebook. We have a nice Facebook group going with some discussion talking about all sorts of things testing- that is The Testing Psychologist Community on Facebook.
And if you are interested in growing your own testing services or need to chat with somebody about how to do that, I would love to talk with you about that. You can schedule a call on The Testing Psychologist website. We can chat and see where you’re at and if it would be helpful to do some consulting around testing in your practice.
Take care, enjoy the oncoming summer, and I will [00:33:00] talk to you next time. Bye. Bye.