This episode is brought to you by PAR.
PAR offers the RIAS-2 and RIST-2 Remote to remotely assess or screen clients for intelligence. They also offer in-person e-stimulus books for these two tests for in-person administration. Learn more at parinc.com.
Hello everyone, and welcome back to The Testing Psychologist podcast.
Today is a clinical episode on a topic that is near and dear to many of our hearts, and if not yet, maybe will be in the future. We are talking all about Psychometrists.
My guest today, Dr. Christine Ghilain is an [00:01:00] ABPP board-certified lifespan clinical neuropsychologist who maintains a private practice in Northern New Jersey. She’s on faculty at the NYU Grossman School of Medicine and sees patients on a consultative basis for the NYU Child Study Center.
A recent graduate of the Goldman Sach’s 10,000 Small Business Program, Christine is passionate about increasing access to neuropsychological services and spearheading multiple expansion initiatives. Her research and clinical interests include refining assessment practices to enhance diagnostic accuracy, that is important, and achieving optimal outcomes for individuals with diverse neurological and neurodevelopmental conditions. She has a passion for training and promoting competence in the field of neuropsychology.
Christine is fantastic. She wrote the book on Training Psychometrists. As you all know, we have talked about psychometrists many times on the podcast, and comes up often in The Testing Psychologist Community [00:02:00] on Facebook.
I am just so honored to have her on the podcast to go through the research, the history, the necessary qualifications for a psychometrist, compensation for psychometrists, how to onboard and train them, and many other things. Christine has a lot going on. She was very knowledgeable. This is a fabulous conversation.
Now, if you are a practice owner and you would love some support or you need some support in running your practice or growing your practice, I would love to have you join one of The Testing Psychologist Mastermind Groups. There are levels for all stages of practice development, beginner, intermediate, and advanced, and it’s a great environment. It’s group coaching. It’s supportive. There’s accountability built in. There’s a private Facebook community just for cohort members. It’s lovely. If this sounds interesting to you, you can go to [00:03:00] thetestingpsychologist.com/consulting and book a pre-group call to see if it’s a good fit.
All right y’all, let’s get to my conversation with Dr. Christine Ghilain.
Hey, Christine, welcome to the podcast.
Dr. Christine: Hey there.
Dr. Sharp: Glad to have you
Dr. Christine: Glad to be here. I am very excited.
Dr. Sharp: Me too. This is a topic that comes up so much in this community and in my Facebook group and consulting. There’s so many questions about psychometrists. It is a surprisingly undefined area of our practice or maybe it was until now. There are so many questions I want to ask you. I’m really [00:04:00] grateful that you are here. Thank you.
Dr. Christine: My pleasure. Looking forward to it.
Dr. Sharp: I’ll start with a question that I ask everybody when we get on here, which is, of all the things that you could spend your time and life on, why psychometrists and writing this book?
Dr. Christine: Fair question. As I think with many folks who go into psychology or neuropsychology, my very first time working with a psychometrist was when I was an intern. I had never even heard of psychometry or using psychometrists or technicians to do testing until I was an intern and somewhat even learning some of the tests from them.
So, I realized that this was an area that even though I didn’t know much about it, it was a huge component of testing practices. A lot of major medical centers were using these [00:05:00] folks for seeing additional patients and getting additional patient care out there.
Having never heard of it before, and then getting my first job straight out of fellowship, I was tasked with hiring and training a psychometrist for a large medical practice or a medical center, and all of a sudden I was like, Oh, okay, sure. I’m sure there’s resources. I’m sure there’s things I can look at. I’ll just do my little research and figure it out. But having come upon nothing or at least things that weren’t published since 1991, I felt like, okay, we need some updates here. I somewhat stumbled into it and have been grateful for the opportunity to delve deeper and write the book ever since.
Dr. Sharp: That’s great. I think it’s the same as many other things born of our own experience and necessity, right?
Dr. Christine: Yes.
Dr. Sharp: It’s funny. My journey was, I worked as a psychometrist when I was in grad school in a neuropsych practice, [00:06:00] I thought that’s how everyone did it, and so, I went that direction pretty soon after starting my practice. And then it was only later that I figured out that it was maybe not totally typical for everyone. It was also ill-defined as well as far as what psychometrists are or who they should be, what we do with them, and what they can do. It’s a vague path.
Dr. Christine: A lot of people have said to me, hey, do they have to have a master’s degree? Do they have to have a bachelor’s? If they know they want to leave in two years and go to graduate school, should I not bother hiring them? Are they the best candidate to hire because they have such an interest or vested interest in the practice and the programs?
I love hearing your story. I think it’s super awesome that you trained as a psychometrist first. That to me makes logical sense, but again, falling into it somewhat later, it was funny to me to think that that [00:07:00] wasn’t how my journey was.
Dr. Sharp: It’s pretty wild. There’s just so much variability. To call it trained as a psychometrist, maybe that’s a glorification. I don’t know that it was rigorous by any means, but we’ll see. We’re going to talk about this. Well, let’s dive into it.
Dr. Christine: Hence the book.
Dr. Sharp: Yeah, exactly. Hence the book. One of many, I’m sure we’re going to mention this book many times throughout our interview, but if folks are interested in picking that up, which I would recommend, we’ll make sure to have a link in the show notes for people to do that. It’s pretty comprehensive and we’re just going to touch on a small part of it today.
Let’s start though. I do want to do a brief history of psychometrists and where this concept came from. Can you speak to that a little bit?
Dr. Christine: Absolutely. I recently gave a talk on this at the AACN conference in DC back in June. So some of this info is fresh in my [00:08:00] mind. Something, when I first started researching the history that I was surprised by was that psychometrists and what they used to call psychological technicians, and I think, as I’m sure you’ve experienced, they’ve been called all kinds of different things. I have a long list of techs or Psychometrist or otherwise, so I’ll try to stick with psychometrists just to maintain our own language consistency, but psychological technicians as they were described were used by William Hunt in the 1930s.
He was evaluating World War II naval soldiers. That was happening already by the 1930s. And then, Halstead-Reitan. The more I dove into it, the more I realized people were using technicians in labs, were using them in practice settings, and hospital settings was a big one. But again, defining who these individuals were and what kind of training they [00:09:00] needed wasn’t as easily found or readily available.
Personally, I was surprised. I didn’t think in the 1930s that technicians were being used or psychometrists were being hired, but it seemed like over time, certainly, APA was talking a little bit more about their training and the very first division 40 task force of the APA put out some training guidelines in 1991. So that is our first benchmark for what they described as non-doctoral personnel. And that’s who gave us our first understanding of what kind of training and supervision these individuals would be expected to have. There hasn’t been an update since. Time stops a little bit.
Certainly, there have been salary surveys. In 2019, there was a salary survey put out for a psychometrist. [00:10:00] Interestingly in there, authors did mention, there isn’t a consistent guideline or a consistent framework for training. So I’m hoping that my book and ongoing conversations, I’m hoping folks are picking up, I am not considering myself the absolute expert in psychometry. I don’t think I have some special training that no one else does other than just taking the time to delve deeply into the literature to better understand it.
In 2006, insurance created the CPT codes that allowed us to bill insurance for technicians. Similarly, that’s also when AACN, NAN, and Veterans Affairs, all put out support documents that said we support you using psychiatrists in practice. But again, one of the biggest surprises there was there was no additional information provided [00:11:00] around training, state qualifications, et cetera.
Dr. Sharp: It seems counterintuitive that in a field where we spent so much energy defining what a neuropsychologist is, we haven’t spent a whole lot of time defining the role of a psychometrist who presumably plays a pretty big role in neuropsychologists’ practice. Do you have any sense, even guesses as to why psychometrists have flown under the radar?
Dr. Christine: I think because all of these folks are working under the psychologist or neuropsychologist license. Certainly, I feel like I have a vested interest in how a psychometrist working for me is going to be trained because it’s my license on the line. My diagnostic impressions are very much based on those test scores or behavioral observations that that individual provides.
So, I think there is a moral and ethical [00:12:00] motivation, shall I say, for the neuropsychologists themselves. It’s interesting, I’m sure you’re on Listservs, and I think we’re all on Listservs, and it’s often a question, hey, I’m going to hire my first psychometrist. What do I do? Has anybody done this? I feel like I’m reinventing the wheel a little bit.
My goal is to centralize resources for folks who are doing this so that not everyone has to reinvent the wheel. And I think as insurance programs change the ways that we can bill and we want to increase our efficiency and all of these other things play a role, I think we’re going to be wanting to do this more often, but I think beyond our own license, I don’t have a good rationale as to why we haven’t done that deep dive into training.
Dr. Sharp: Yeah, it’s interesting. Do we have any statistics on how many folks are using [00:13:00] psychometrists or how common it is? Any way to gauge that?
Dr. Christine: It’s a great question. I feel like there’s potentially some information in the most recent AACN salary survey. It definitely doesn’t capture. It’d be interesting to do in your Testing Psychologists Facebook group, a poll, because I don’t know how many folks who aren’t necessarily board certified or aren’t part of AACN.
I don’t know how many folks are using psychometrists, but particularly in Academic medical settings, and I would say folks who take insurance, I feel like it’s becoming more and more of a need just to keep up with demands and the reimbursements, which I won’t get on that soap box during this chat just to steer clear of the political side of things.
Dr. Sharp: Fair enough. I’ll agree not to talk about it as well. I think it’s true though. I’m going [00:14:00] to immediately contradict my own statement, but running a practice that’s heavily insurance-based and running a practice at scale using psychometrists is a big part of the financial picture to have it be sustainable. So I get it. I can imagine hospitals are probably in that same boat, but there are other reasons too, which we all talk.
Let’s do a little bit of terminology. I would love to… We’ve been using the term psychometrist, but as you said, there are other terms, technician or assistant and maybe some others, and of course psychometrician which gets mixed in there a lot. So, give me the terminology. What should we be saying?
Dr. Christine: I think that it’s still up for debate. Certainly, a psychometrist is an individual who administers tests. In the book I [00:15:00] talk differences between what is testing and assessment. Even another set of terms that I think are used interchangeably.
Testing refers to the actual administration and scoring of test measures, and then assessment adds in those surrounding factors. So I always think the neuropsychologist is doing the assessment, the psychometrist is doing the testing. And so the assessment adds in the cultural factors and the behavioral observations, the diagnostic considerations.
So when I think about psychometrists, that is the term that I use. I sometimes will say techs or testing technicians, just because when I was on fellowship, that’s what they were referred to. Some old habits die hard.
Psychometricians, my understanding with some of that is also folks who are creating the tests themselves, right? A different group of folks entirely, but yes, I agree with you. I’ve heard all kinds of usages of all different forms of psychometric, [00:16:00] psychometricians, psychometrists, all of those.
Dr. Sharp: Yeah. I know that I said psychometrician for a long time, and then I don’t know, 4 or 5 years ago, I was like, wait a second. I don’t think that’s right.
Dr. Christine: Yeah, not uncommon though.
Dr. Sharp: Sure. Well, thank you. Thanks for the validation. So we’ll go with psychometrist. I like that. That was a very simple and clear distinction between testing and assessment. I think that’s important to keep in mind as we move forward as well.
Let’s dig into the nuts and bolts here. I think the biggest question on people’s minds is who can be a psychometrist? What training is required or is a degree required? If so, what degree? Any information in that realm would be so helpful to start with?
Dr. Christine: I will unfortunately start with a caveat that there are some pretty significant differences across [00:17:00] state lines. So like New York versus California versus my state of New Jersey differences in expectations. There are some states and forgive me, I’m forgetting which specific ones, where a psychometrist is expected to pass the EPPP.
Dr. Sharp: Wow, I have not heard that.
Dr. Christine: Interesting differences. What I most frequently see in the general broad strokes is a lot of folks who are looking for at minimum a bachelor’s degree in psychology or related field. I think many people will say masters’ degree preferred. Personally, and this isn’t really necessarily even in my book, I think if you find somebody who is organized, detail oriented, and really passionate about learning the ins and outs of psychological testing, sometimes to me, that means more than can you define who Freud was or or otherwise…
[00:18:00] Dr. Sharp: I agree.Dr. Christine: I have some interesting questions when I am interviewing for psychometrists, like which one energizes you more: scoring versus the test administration itself? I think you can get a lot of interesting answers from questions like that. The true nuts and bolts in terms of hard and fast qualifications can really differ across state lines. So it’s hard. The reason people, I think, don’t have a clear cut understanding is it does differ depending on what state you live in.
Dr. Sharp: As do many things, right? So that’s a totally legit answer. How do we find out though? Who will be able to tell us the requirements for a psychometrist in our state? Is that the license of the psychology licensing board?
Dr. Christine: Yeah. I would say your board of psychological examiners. I know for New Jersey, and this is [00:19:00] interesting. A little fun factoid about New Jersey, the psychometrist is not allowed to administer psychological intelligence testing. So even if I hire a psychometrist, they are not allowed to give the WISC, the WAIS, the WASI, the RIAS. That is forever going to be a job that I administer; which personally I like.
I don’t currently use psychometrists in my own practice. I do really enjoy the testing piece, but if ever I am to grow and expand, that is something where every client that comes in, the intelligence testing has to be given by the licensed professional, which is interesting. I think it’s New Jersey only. I don’t think there is anyone else who does that, but the licensing board that’s in our bylaws in terms of New Jersey, it’s like… So check your state information.
Dr. Sharp: This is good to know. Well, no consistent guidelines, I [00:20:00] suppose. That jives with what I’ve seen and shared with folks a lot is that it seems like a bachelor’s is at the minimum and like you said, a masters’ is preferred. And then some states, it sounds like have more strict requirements around psychometrists qualification.
Dr. Christine: I think even California is recently creating or looking to create some level of certification with expectations around hours spent in supervision. Obviously, that’s a goal that I would have for all 50 states nationally and internationally, but it does seem like some states are more keenly interested in training and qualifications than others at the moment.
Dr. Sharp: That makes sense. I know that the picture even gets muddied even more in the States that allow master’s [00:21:00] level psychologist licensure. Texas has a licensed psychological associate. They’re not calling them psychometrists, but they’re in that ballpark. I think it goes to your point of checking with your state to see exactly who can do this work.
Dr. Christine: Absolutely.
Dr. Sharp: I know that there’s been a, well, I don’t know if it’s a push because that would imply like some amount of inertia and I don’t know if that’s in place or not, but there’s a board of psychometrists, or a certified psychometrist entity out there somewhere. I haven’t seen a lot from that entity so that’s why I hesitated to use the word push, but I’m curious how that fits into this whole picture in terms of defining standards and certification and those kinds of things.
Dr. Christine: Great point to raise. The National [00:22:00] Association of Psychometrists or the NAP Group was started back in 1995. I’ve been in contact with a variety of folks that were part of that. Initially, I think, it was a beautiful story. There were a lot of really motivated psychometrists who felt like they were somewhat isolated in their individual groups, but maybe had questions about scoring, or maybe had questions about responding to certain behaviors or what have you. It sounded to me very much like, hey, we’re going to all get together so that we have a common space for sharing and asking questions and bouncing things off.
Certainly, I know when I was in fellowship and I had other folks around, I was always asking about scoring questions. Well, what do you think about this word versus that word? Or how do you score this 1 point, 2 points, no points?
It sounds like that was maybe part of their initial rationale for gathering as a group, but then I forget the exact timeline, but they did, a group of [00:23:00] them got together and worked towards creating a test to certify folks as certified psychometrists that is still absolutely available.
When I was writing the book, I sent out links to a whole bunch of different groups and had psychologists as well as psychometrists filling out surveys around their thoughts and it was interestingly polarizing to hear folks who said, Oh, yeah, it was great. I have all of my technicians or my psychometrists go through the program. I had other folks who said they don’t need any kind of certification if they’re working under my license. Why waste the time? I do not claim this as my own statement, but some feel like psychometry should not be a full-time job. It should be something that’s a stepping stone to graduate school.
So it was rather eye-opening to see the variations and folks’ perspectives on psychometry. But I also feel [00:24:00] like, anecdotally, the folks who I’ve spoken to who have gone through that process have found it of benefit. So I’ve looked into it a little bit and I’ve read a little bit. There’s certainly information on napnet.org, which is their website, but I think at this point, every time someone passes the test, they submit a new question. Questions are a little bit different these days. So I’m not exactly sure how many folks are pushing for it, but it does seem like a good spot for folks to gather additional information, at least to this point.
Dr. Sharp: Yeah, that makes sense. I know this is maybe just a rumor, maybe it’s rooted in some facts, but I’ve heard here and there that there’s more energy around the certification of psychometrists in the neuropsychological world. I don’t know if that is true or not.[00:25:00] Maybe you just answered it with this survey or the polarization that you talked about. But I feel like that’s come up over the last maybe 2, 3, 4 years here and there that certified psychometry might become more of a thing.
Dr. Christine: Yeah, I know they have an annual conference every year that happens in the same city as the American Academy of Clinical Neuropsychology and around the same weekend or overlapping with that weekend. So one of the years, I don’t think it was this past year, but the year before, I popped into the conference and there was a good number of folks who were there. So there’s certainly a force. I wouldn’t say that they’re getting smaller, but it did sound like maybe things are maintaining status. To me, it didn’t seem like there was a big difference from what folks were sharing while I was there in terms of enrollment or otherwise.
Dr. Sharp: Got you. [00:26:00] That leads to questions that I think you talk about in the book around competence in general and governance of psychometrists; these questions of should there be a separate board for psychometrists and should there be a separate ethics code for psychometrists and things like that? Can we touch on those questions?
Dr. Christine: Yeah. I’ll share my opinions and my thoughts but I think many feel various things about these. For me, I felt like if someone is working under my license, I want them to follow the APA ethics code. And so, even in the book, I break down the particular sections of the APA ethics code that I feel are most pertinent to a psychometrist. I remember thinking about, well, the research sections of the APA ethics code probably don’t pertain, but if you’re using a psychometrist to [00:27:00] administer research protocols, then they probably do.
I think there is a psychometry-specific ethics code that was created by the NAP group. I didn’t specifically review that in the book. I kept it a little bit more toward the APA guidelines. But it seems like there’s certainly variability in folks seeking out that information.
Dr. Sharp: Yeah. Got you. It sounds good. This is, if nothing else like I said, it’s validating that things are not as well defined and there’s a lot of variability in opinions in the field about what psychometry should look like.
Dr. Christine: Yeah, I think some folks feel like, well, if there was a board, then that’s one less thing that the neuropsychologist themselves has to have on their plate. I respect and understand. [00:28:00] Folks are busy and overwhelmed. And as you say, you weren’t even sure if you could call it a robust training that you received as a psychometrist. So I get it. I’ve seen it in private practice. I’ve seen it in group practice. I’ve seen it in medical settings. The variability is there. So, I think you’re going to get a wide variety of opinions there.
Dr. Sharp: Well, I would love to transition to the more practical aspects of psychometry and how we hire and train them. I think that’s what a lot of folks are interested in. I’m going to pick back up on a question or a point that you raised a little bit ago, though. You mentioned these questions that you ask psychometrist candidates. I’m really curious about that. What are your interview questions for your psychometrist?
Dr. Christine: I’m more along the lines of, if you are detail-oriented, you [00:29:00] are thoughtful in your process, you are organized, and verbally articulate the best we all can be on certain days, these are factors for me that I feel like are very helpful. These individuals are representing the practice or the medical center. They’re working directly with the patients.
So most of my questions are around understanding a little bit more like, what does your desk tell you about yourself? Is it messy but you know exactly where everything is? Is your desk completely clear before you leave for the day if you have one at home?
I think there are fun and funny ways to get at the ways folks approach problems, for example; flexibility around, well, the kid took the laptop and threw it off the desk. How do you respond to that? Certainly, things that don’t necessarily happen every day, but you hear the [00:30:00] stories and maybe have seen them firsthand yourself.
I think a lot of my questions stem from the flexibility around those behaviors, the unpredictability of even if you’ve seen 100 kids with ADHD, how different each one of those kids is as they walk into your office and how you respond and maintain your cool while still taking a whole bunch of behavioral observational notes in your mind about what’s going to help them succeed in a classroom or an adult to succeed in a workplace, et cetera.
Dr. Sharp: I love that. I love interview questions in general and finding creative ways to get at people’s behaviors and thoughts.
Dr. Christine: Do you have a favorite one that you ask of your psychometrists?
Dr. Sharp: Well, I don’t do the interviewing for our psychometrists anymore. So I’m out of the game. This is a whole other tangent we could get on, but I [00:31:00] have moved more toward a working interview versus a question-and-answer interview because I think that all that tells us is how well people respond and novel social situations. It doesn’t give us any sense of the work that they do. And so, it’s more of a working interview situation.
Dr. Christine: I love that. That’s awesome.
Dr. Sharp: I like all the personality questions. I tend to talk about things not work-related, honestly, in interviews. I’m more interested in how people think, what they’re reading, what they like to do instead of work, and things like that.
Dr. Christine: I think you get a lot of good information from that for sure.
Dr. Sharp: Yeah. I like your question, especially the scoring versus administration question. I am going to add that.
Dr. Christine: Pull that out for your action.
Dr. Sharp: Yeah. I definitely will. I jumped into the interview [00:32:00] portion of finding a psychometrist, but where should we start in this whole process when we think we want to hire a psychometrist?
Let’s take a break to hear from our featured partner.
The RIAS-2 and RIST-2 are trusted gold standard tests of intelligence. For clinicians using teleassessment, PAR offers the RIAS-2 Remote, which allows you to remotely assess clients, and the RIST-2 Remote, which lets you screen clients remotely for general intelligence. For those practicing in the office, PAR has in-person e-stimulus books for both the RIAS-2 and the RIST-2. These are electronic versions of the original paper stimulus books that are an equivalent, convenient, and more hygienic alternative when administering these tests in person. Learn more at parinc.com/rias2_remote.
All right, let’s [00:33:00] get back to the podcast.
Dr. Christine: I think most folks you’ll find a post on things like Indeed or those general job search entities. I appreciated one of the folks I was consulting for, Sharon. She also reached out to local graduate students. That has been extremely beneficial. Folks are interested in learning more and learning the assessments. Sometimes they get credits for it. For example, if you’re doing research within your practice or your medical center, it’s a great opportunity for those early experiences, particularly if they’re going to go on to graduate school. Interestingly enough, I have had a few folks tell me that retired teachers are the best way to go, and I can totally see that.
Dr. Sharp: I’ve heard that as well.
Dr. Christine: Having a mom who was a kindergarten teacher for [00:34:00] 30-something years. I can see my mom doing a great job with all of that; managing behavior, gathering data, all of that. It seems like there’s a good qualification set there. So if you have a retired teachers group in your area, that might be a fruitful place to seek folks out.
Dr. Sharp: That’s great.
Dr. Christine: But those are some options.
Dr. Sharp: I resonate with all of those. We primarily use grad students as our psychometrists. So just pulling from the local Ph.D. and PsyD programs. We try to grab advanced students who already have their master’s and have done at least a year of assessment experience in their programs. And that has worked well in our practice.
Dr. Christine: Do you find a lot of turnover though at that point? That’s the biggest feedback I get as well. If we do that, then every two years I’m getting a new person, right?
Dr. Sharp: Yeah, we do. It was difficult at first, but I think once we [00:35:00] set the expectation, know that’s going to be the cycle, it forced us to get our training dialed in the process and the onboarding and everything; set clear expectations for them. I think now that we’re in the groove, it’s gotten a lot easier, but it’s tough when they turn over every year or two. A little more labor intensive.
Dr. Christine: Yeah, that’s great if you ask. I agree with you. Setting those clear expectations and having a training plan going into it. That is going to make a world of difference.
Dr. Sharp: Sure. Yeah, we’ll see. We’re going to talk about your approach and thoughts on training. We’ll see if we’re in the right direction or not. I’ve also heard either part-time or retired school psychologists in a lot of states have served as really great psychometrists for folks. So there’s a lot of options.
Dr. Christine: Yeah, absolutely. If only they know. I keep thinking to [00:36:00] myself all these folks are great potential candidates. But again, every time I talk about psychometry with folks who maybe aren’t in our field or aren’t as deeply engrossed in assessment practices, they’re like, psychoma what? We’ve never heard of that. What is that?
I wonder about it, I have dreams of campaigning and helping folks to understand what psychometry is. Given how many graduates from even undergraduate programs with psychology majors come out into the workforce each year and wonder, well, we can’t be clinical psychologists yet, we got to go through a whole lot of years more of training. What can we do in the interim? Part of me just thinks there is a lack of knowledge, understandably for folks who aren’t aware that this even exists. So how to tap into those unknowing folks or unknowing groups?
Dr. Sharp: That’s a great question. Do you have any thoughts on that?
Dr. Christine: My hope is that at some point I am doing that [00:37:00] and getting that information out to people. I think there could be great opportunities to create college-level programs that help people understand that these are options that they can get into. I’ll talk a little bit. I’m sure we’re going to talk about some of the ways that I do training.
I do think early on it’s a lot of nuts and bolts basic information, basic statistics around score conversions: If you have a z score, how do you get it to a standard score or whatever it might be? Not necessarily a full statistics course. I think that’s important for the Neuropsychologist, but maybe less important to the person unless they’re interested, right? I’m never saying don’t learn about these cool things, but when I think about, practically speaking, what a psychometrist need to know walking into the room to test the patient, basic things in the statistical world are helpful, but maybe not your full stats, curricula, right?
Dr. Sharp: Yeah, absolutely. [00:38:00] I’m excited to talk about the training approach.
Let me see. I wanted to follow up on some aspects of psychometry. Oh, the salary component. You mentioned the salary survey a little bit earlier. Can you give us just a ballpark of what that looks like these days? I get a lot of questions like how much should I pay my psychometrists?
Dr. Christine: I’m going to defer to the salary survey. The salary survey does a nice job of breaking it down. In my area, in New Jersey or the Northeast, I think it’s somewhere between $30 and $35 an hour. But it does very much differ across geographic regions and the published salary survey, if folks need it, they can send me an email. I’m happy to provide that, but I’m pretty sure it’s open access. It’s a nice place to start. It’s a good place to get a sense. And [00:39:00] obviously, if someone has a master’s degree, it’s a little bit different. There are some pushes. Also, if they have that board certification in psychometry, some additional pay bump for that as well. So some incentives to continue with the formal education around it.
Dr. Sharp: Great. Yes. And we’ll try to link that in the show notes as well so folks can find it. I’m going to go look at it after we’re done.
Dr. Christine: Yeah. It’s a good one.
Dr. Sharp: So let’s dive into the training and get some details around that. Let’s say you’ve found your psychometrist, where do we start with training?
Dr. Christine: Personally, I start with a lot of reading. I don’t know about you, but I have a few different articles, and a few different chapters that I have pulled from different textbooks, but some of it is on basic welcome to the world of neuropsychology. These are the kinds of disorders that we [00:40:00] see. These are the kinds of patients you may work with. But then it delves a little bit deeper into, this is how you will experience certain types of behaviors, and here’s some behavioral management strategies around that.
So I actually have a required reading list and then a recommended reading list. I think my manual is about 90 pages now. It’s not part of my book but I’m hoping to get that out there for folks to be able to use. I do think it’s practice-specific. For example, if you happen to be in a major medical center and you are always on an epilepsy monitoring unit and you’re always doing shorter batteries, you’re going to want a lot more training for that psychometrist around seizure protocols if someone’s actively seizing in front of you, what kinds of questions to ask once they’re medically safe, obviously? Are they responsive? Those kinds of things. Versus if you’re in a private practice where your majority of clientele is ADHD, learning disabilities, and things like that?
[00:41:00] I always start with readings and a breakdown of the manual. One of my favorite handouts, and I think it’s up on my website at this point for free, is a walk through a manual, what sections of a manual are most important for the psychometrist to read, the specific information that a psychometrist is going to want from a manual, the focus around standardization is pretty mission critical for a psychometrist. So those are the early days of training for me is just really getting folks oriented to what’s going to be happening before I put them in front of a practice patient or before I become the practice patient that they’re starting to administer tests to.Dr. Sharp: Do you have any idea how long that period is before they get in front of a practice patient?
Dr. Christine: It depends is the worst answer, but I’m going to keep saying it. If somebody has experience right there in [00:42:00] advanced assessment student, then a lot of this, they’re going to fast forward through relatively quickly and they may be ready to be sitting in and observing a patient and then ready to potentially even administer some of the tests that they have administered to the clients in their graduate program quickly within a week, even. Other folks who are completely green, who think this sounds like a cool area to work in, but have never been interacting with a patient, I would say will take certainly a few weeks of the initial and then how long it takes.
I think many folks are excited. Once they hire that psychometrist, they’re like, great. They’re going to take all these hours from me. This is going to be so great. And eventually, it is great, but to begin with, it is a double job. As you likely know, you are doing the testing, you are doing the training, you are doing the scoring, you are doing the double scoring, you are taking the behavioral observations, you are talking about the behavioral [00:43:00] observations, so certainly keeping in mind how much time that training process is going to take. And I think that’s a lot of why the interview questions can be helpful. If they have no idea what scoring or testing means or is, then you’re going to expect a little longer of a process, I think.
Dr. Sharp: Absolutely. Gosh, that just made my stomach drop thinking that I might interview someone who didn’t know what scoring or administration was. Yes, that’s fair.
Dr. Christine: It happens. I have had folks who have successfully gotten those. They’re interested. They think it sounds cool and it is cool. We’re all biased or at least I will admit my own bias. I think this is the coolest field and everyone should be in it. But it’s exciting to get somebody with that thirst for knowledge and their willingness to jump in.
Dr. Sharp: Sure. So we start with reading and orientation and so forth. And then where does it go from there?
Dr. Christine: Typically, I’ll be the practice client. [00:44:00] I’ll first administer some of the testing to the person, or if they would prefer, they can administer the testing directly to me. Some folks see and then do, and they want to observe it. Others will go through the manual and say, well, let me give it a try and you can give me feedback on how it goes.
I know some folks have the benefit of being able to observe, let’s say, through a mirror or something to that effect where they can observe testing and see what that looks like. I always the two-way mirror option, because if I can sit with the psychometrist and say, okay, see how she was doing this, or you see how she jumped in and did that, those teachable moments, I think are much more beneficial than just what you read in a book.
And then practice clients. And then I become more and more of a frustrating or annoying practice client or someone who gives a curveball of some kind just to prepare them for things that they [00:45:00] can’t necessarily plan for. And then I think when it comes to being able to move them into the next level of they’re going to do assessments with actual clinical patients, you’ll be in the room observing, assisting if they need that, giving them feedback.
So it is sequential and a step-by-step iterative process, but I do tend to find that if folks can then get that final check off, then there’s usually some level of routine maintenance check-ins, whether that’s double scoring files or once a year sitting in and observing the testing, because we all tend to drift, as they say, that examiner drift, if you haven’t done it in a while, or, oh, gosh, I haven’t given this test in six months, we don’t necessarily always have the time to jump back, but it’s always good to keep a pulse check on it. Everybody’s still doing this the same way each time.
Dr. Sharp: Absolutely. [00:46:00] Two questions around all of that. Do you have any go-to difficult client behaviors that you found helpful to deploy with these practice sessions?
Dr. Christine: Yeah. Some that come to mind are the kid who wants to go to the bathroom after every question and seeing how they respond to that. Hopefully, their thought process is, oh, let me introduce first, we do these three things, then we’ll take a break, those kinds of structurings.
Certainly, I think we’ve all had a client who just refuses to engage. I’m not doing this. This is too hard. Arms crossed. We’re just done. So sometimes I do throw in that argumentative client who doesn’t want to keep participating, and being able to teach them, okay, well, let’s align with this person. Was this a hard task? Do they feel dumb [00:47:00] or like they can’t do it because it’s a really hard task for them? Why might that be? How can we align with them to say, yeah, this is hard. Let’s just get through it. And then we can move on to something different.
I think we’ve all had those clients that put us through a loop. And so, I think about those clients for myself and try to make sure they feel as prepared as they can.
Dr. Sharp: Yeah. It’s interesting to me. Both of those examples are fantastic and they’re behavioral things. I thought maybe there would be more of like a test answer example, but the behavioral stuff is the harder stuff to manage as far as I’m concerned. We can handle answers.
Dr. Christine: I think on things lists, on the D-KEFS, I think about doing verbal fluency, for example, and spewing out a whole bunch of [00:48:00] different letter names or giving them a whole bunch of rapid-fire answers can be an overwhelming or on a memory task when all of a sudden they have a list learning and they give you 10 things in 3 seconds, right? Being able to remember what they were or having some sort of short-form if they are speaking very quickly, how to get those in order.
So you’re right. There are testing ones that are important too. But I agree with you. I think the behavior and behavioral regulation within the room can be some of the more challenging things for the psychometrists as they get through that testing day.
Dr. Sharp: I agree. Have you found any particular tests that seem more difficult for folks to master in terms of administration?
Dr. Christine: I would say, I tend to review a lot of like Tower of London and Drexel Tower, remembering how you have to [00:49:00] wait until a certain time and when they move. And if they pick it up, where do you put it back? And you’re still counting how many moves they’ve made or otherwise… I don’t want to give away too much in terms of test security. So I’ll try to watch what I’m saying. But some of the more complex reasoning tasks tend to be more challenging, I think, for folks. Do you find the same in training?
Dr. Sharp: Yeah, that’s why I was nodding. Tower was the first thing that came to mind just because there is a lot, I think, for the administrator to pay attention to, and you’re tracking a lot of things at the same time.
Dr. Christine: Yes. Trying to keep up with it all.
Dr. Sharp: Yeah. Things move quickly. So we’ve done some reading, we’ve done some practicing, and at some point, you feel confident that these individuals are ready to fly solo. You’ve done some observation. I’m curious about how much training are you doing on the [00:50:00] scoring. Do you trust that they can read the manual and figure it out? How does that work? I’ll out myself a little bit. This is, I think, where I maybe assume too much. I feel like the manuals are pretty clear about how to score things as long as you give yourself the time to read them and are pretty meticulous the first several times. So I maybe pay less attention to this than I should. I’m curious how you approach this.
Dr. Christine: I’m going to be an unfavorite person in the moment…
Dr. Sharp: That’s totally fine.
Dr. Christine: …or my opinion might be unfavored but it might be unfavored. I tend to recommend hand scoring as much as possible prior to… Lists in the iPads are a beautiful invention and have increased my efficiency, but if the power went out or I don’t know [00:51:00] what would possibly happen that I couldn’t use my technology, I would want to be able to administer score and provide just the same amount of information. I’m probably just old school because that’s how I learned.
So I have folks the first couple of times that they’re doing measures hand score everything and I double score after. I think I agree with you that the manuals themselves do talk a lot about it and I tend to open myself up to like, if you have any questions, go ahead and come and ask me. But once you read through the scoring, you know, the goal is for them to be independent in their scoring.
And so I do more of a hand me your scored file and I’ll review it and double score because I think that that pulls up and as you say, doing it versus just watching me do it and then parroting off of what I’m doing versus sitting down, reading the manual, learning it, [00:52:00] internalizing it a little bit more. I think there’s certainly benefits to getting a little bit of leeway in terms of your own seeking out of that knowledge.
Dr. Sharp: That’s fair. It’s very painful thinking about hand-scoring some of these things, but I think you’re right. Just that extra layer of investment.
Dr. Christine: I remember, again, outing me probably on somewhat of how I’m not super late career, but I’ve been in the business a little while. But even hand-scoring things like the BRIEF, knowing which questions go into which categories, to me, that helps you when a parent says, okay, I’m elevated on inhibit. What does that mean? You can be using the language that they have already endorsed whether or not you have hand-scored at that time or not, I think you can again, round out your understanding of what are you [00:53:00] doing this for. Why are you making this patient do this? What could it mean for someone who gets that data or that information?
Dr. Sharp: Yeah, I think that’s valuable, especially these days where the cost of digital administration is ramping up more and more and more. We might be getting back to that tipping point where hand scoring is more cost-effective. So there’s a double advantage, I suppose, in that case.
Dr. Christine: Yes, I think some folks who didn’t shift to iPads during the COVID pandemic are now like, oh, but we hear about the pricing and we’re so glad we never switched. It’s almost like that ignorance is bliss if you don’t even know what you’re missing, you’re set in routines. I think I would miss it if I went back, but I feel confident that I could if I needed to, right?
Dr. Sharp: There you go. Yes.
Talk to me about behavioral observations. This to me is one of the most important parts of the [00:54:00] administration process, maybe the most important part. So I’m curious how you approach training that. I’ll just add another little qualification here. I think this is where a lot of people run into a hurdle in bringing on a psychometrist thinking nobody is going to be able to do behavioral observations like I do. Nobody is going to pay attention to the things that I think we need to pay attention to. And so, I’m curious how you might approach that.
Dr. Christine: I agree with you. It’s one of the most common things I hear is like, well, they won’t see it. They won’t know. And then if I miss that, that could be a huge thing that, as you know, some of these smaller behaviors may directly impact diagnostic impressions. I do think I am confident.
I used to feel very similarly having worked with them excellently trained psychometrists. It was amazing. Sometimes I felt like they were coming to me with such beautiful descriptions and knew what I was [00:55:00] looking for. I tend to find when I give similar tests, I will sit down and talk about what specifically I am looking for when administering that test.
So behavioral observations become complex and multifaceted because you’re talking about the behaviors they’re seeing in the room, but also, why am I administering this test? And what behaviors am I looking for specifically that might come from this test? So, it’s certainly another one that’s a marathon, not a sprint.
I think that sitting and being able to observe as many cases as possible can be helpful and having those conversations after the fact, comparing behavioral observation notes. I also have some behavioral observation checklists. So we start basic with checklist by domain, for example, and then move more to open-ended questions where they’re filling in additional information and maybe not necessarily relying on the [00:56:00] checklist, but they know in the attention domain, I’m going to ask about these layers. Are they even responding at all? If they are responding, is their processing speed slow? So, building from very basic skills to more advanced skills over time. So, it’s another one that it’s like, well, it depends on how long that’s going to take, but there’s lots of pieces.
Dr. Sharp: Absolutely. Well, and I see there’s two separate related constructs. There is the observations during the testing process, how they’re approaching the tests, and the tasks. And then there’s the free-floating observations like, what are they doing between tests? What are their social skills like, and what’s their appearance like, and all those kinds of things? It’s a lot to track and a lot to be aware of I think for all of us, but especially, folks just starting out. It’s like, how do you even [00:57:00] start to describe what to pay attention to?
But you mentioned that you use checklists. Can you say more about these checklists that you use?
Dr. Christine: Those should be up on my website as well. You can download them and see within as you say, some of it is basic, so general grooming and hygiene. To me, the checklist early on help to remind folks to look out for those behaviors or look out for, consider those things, and then in terms of attention, is it variable? Are there some tasks where it’s better or worse?
I’m trying to think of other specific areas. It goes domain by domain to have options for folks to say, like, oh, yeah, he was a little bit impulsive, or did he respond to early questions wrong but then get later questions right? That variability.
So sometimes I talk through what those [00:58:00] specific checklists mean, but a lot of times it’s to help people remember, like, oh, that’s right. I know what she means by that. Have I seen that at all during my work with this client?
Dr. Sharp: Yeah. That sounds good. I think checklists are super helpful. We also have a pretty comprehensive checklist. It’s maybe not as specific or domain-based, but it’s been helpful just to give a little structure and let them know what to look for.
Dr. Christine: Right. When you as the neuropsychologist or the psychologists are then writing your behavioral observations, if they know the things that you are looking for and the things that you will speak to as you go through that paragraph or section of the report, they’re almost helping to you provide you good concrete details as to why you are talking about them in certain ways or what substantiates your diagnosis based on some of the things that they say.
Dr. Sharp: Yeah, absolutely. We could have [00:59:00] touched on this earlier, but I think it fits just fine right here as well. And that’s the question of what is the psychometrist responsible for in the testing process. I’ve seen across the board, some folks are having their psychometrist only test and score. And then it goes from there. It’s like, do they put the scores into the tables in the report? Do they write the behavioral observations? Do they write the history based on your notes? Do they maybe do more than that, which I don’t think they should be doing? I’m curious from your perspective, what is the role of a psychometrist especially in terms of how much they’re writing or doing in addition to testing?
Dr. Christine: This is a huge can of worms. I’ll try to be somewhat brief, but I have sections of my book specifically talking about like, okay, we all agree, administering the test, scoring the test, [01:00:00] and giving some behavioral observations. We all agree that’s a part of the psychometrist’s role. But then as you very clearly point out, it’s really interesting, the liberties I’ll call them, with which people extend the expectations of the work of a psychometrist.
I’ll say, I got a lot of feedback from the psychometry side on those surveys that was more, I don’t know how to say to my supervisor or my attending, I don’t think I should be writing this section or I don’t feel comfortable doing this but I’m being asked of it. And there’s that supervisory distinction. And so, there are some gray lines. And there are some folks who say well, they’re just typing up the background history based on my notes as you say. So if they’re just plugging in that information, I think that’s okay versus other [01:01:00] folks who have said in those surveys, well, they sit in on my intakes and therefore they can type up all of the information just as well as I can and then I edit it after.
I don’t necessarily agree fully with that particular approach. I think some justify it as, well, if they’re going to go to grad school, that’s something they’re going to learn eventually anyway, but there does seem to be some, dare I call it, abusive power in terms of, now you can write this section and it can sometimes snowball a little bit further to where folks are feeling less comfortable.
Dr. Sharp: Is there a clear line we can draw on the sand where you can say for sure, psychometrists should not be doing this?
Dr. Christine: I’ll put myself out there as this is a personal opinion, but I don’t think psychometrists should be making diagnoses or providing feedback and [01:02:00] there are some who are. Those two to me, again, I think they are doing the testing, and the neuropsychologist or psychologist is doing the assessment. And for me, that helps to be as much of a black-and-white line as possible.
Dr. Sharp: I like that. Yeah, we can close where we started in a way with that distinction between testing and assessment. I think that is a great way to look at it. It’s important to keep in mind the power differential as well. I don’t know that I was fully conscious of that, but it’s so important that a lot of psychometrists, especially if they’re grad students, may have a harder time saying, hey, I’m not supposed to be doing this, or I’m not comfortable doing this and we need to keep that in mind.
Dr. Christine: Yeah, I think that raises just one other point that came up more frequently than I anticipated in those surveys, which was sometimes psychometrists were [01:03:00] asked to do testing that they had never seen. And it was like, Oh, it’s okay. It’s similar to like, Oh, do this NAB naming task. It’s very similar to Boston naming. You’ve given it before.
I see why a psychologist might say that or throw that in really quickly, for example, but I think we as neuropsychologists and psychologists could do a better job of saying things like, Hey, I’m going to switch this out. The patient’s already seen it, whatever it might be. Do you feel comfortable giving it? Do you have a few minutes to look at it? Would you like me to be the one administering it first so you can see it? I think opening those discussions more collaboratively versus with that expectation of, Oh, you’ll figure it out. You’ll be fine. Which came up quite a bit in those surveys that I reviewed.
Dr. Sharp: Yeah, that makes sense to me. I’m sure I’m guilty of that at some point over the years. It’s like, this is very similar, [01:04:00] take a look at the manual. Yeah, that’s important.
Dr. Christine: Right. I think in general, the biggest feedback I got from psychometrists was feeling like maybe they were a cog in a wheel or that their opinions weren’t as well received. For example, when folks say, oh, this patient is done, they don’t want to do any more testing, and you’re like, nope, we need all that data. Keep pressing through, keep going. And I think there’s pros and cons or times where both of those things are appropriate, but it seems to me like a more collaborative conversation. While understanding and respecting those professional boundaries, I think it can be a fruitful way to build more of a community within your practice rather than that strict hierarchical role that we all know that we play.
Dr. Sharp: Yeah. That makes sense. One more question before we totally wrap up. I’m just curious about the [01:05:00] interaction with the psychometrist throughout the day. Can you speak to that?
How much should we be interacting with them in terms of greeting the patient together or checking in at different points? Do we have to be on-site? Do we not have to be on-site? I get that question a lot. What if they can text me? Do I have to be able to go into the room? Thoughts on all of those questions?
Dr. Christine: They’re all very fair questions. I think it can be helpful for the psychologist or licensed psychologist to be on-site during the testing. I think there are some, and I’m pretty sure New Jersey is one of the states that mandate that. So, again, looking at your state laws and ethics is an appropriate piece.
Obviously with technology, I have had many folks say, well, I can be on a video chat in two seconds. And so even if I’m in a different… I can be down the road in 10 minutes, whatever it might be. [01:06:00] I think this is again your own ethical understanding, is this patient somebody that could potentially get aggressive, get violent? Could they potentially endorse, we can’t always know that a patient might endorse suicidal ideation, and if you need to step in and do a risk assessment, or there’s bruising that’s observed that’s consistent with potential abuse, you as the psychologist are going in to do those assessments. That’s, in my opinion, beyond the scope of a psychometrist.
And so again, that’s your license on the line. I think there are people who feel more comfortable being a little bit further out. And if your state laws and ethics don’t require you to be within a certain range, I understand. But I would say, personally for me, I know I’ve always stayed on-site because I think it is important that they can check in with me. Oh, I just noticed this behavior or I just noticed this tremor or I just noticed whatever it might be. Do you [01:07:00] have anything you want to add? Or they bombed this test. Should we add this test? With a competent psychometrist, you can get a lot more qualitative data throughout the day that could enhance your diagnostic accuracy by the end of it.
Dr. Sharp: Yeah, I like that. Well, you’ve shared a lot of knowledge with us. I appreciate it. As we wrap up, is there anything that we didn’t touch on that you feel we have to mention before concluding this discussion on psychometrists?
Dr. Christine: I would say mostly, I love to hear from folks and I love to hear what folks are doing. I am hoping to come out with some courses. Hopefully, by the time you’re hearing this podcast, those courses are out. Just to start those conversations and to get the beginnings of training under the belt of a psychometrist so that a neuropsychologist or [01:08:00] psychologist has a little bit less on their plate.
I think laws and ethics and all of those pieces can be somewhat done through virtual platforms. So, I’m hoping by the time you’re hearing this that you can go to my website and get these courses and that I can help jumpstart your training process. But I’m also available for consulting if folks have questions or want to tailor specific things to their own practice. I’m always happy to connect. So I would just welcome folks’ willingness to reach out.
Dr. Sharp: I would imagine that there are a lot of folks who would want to reach out to you. There’s so much discussion about this topic and the different channels that I’m part of. I’m sure you’ve seen the same on listservs and so forth. So, we will 100% have your website in the show notes. I also hope those courses are ready by the time people are listening to this because they are very, very needed. Thanks again.
[01:09:00] Dr. Christine: That’s motivating.Dr. Sharp: It is. Oh, yeah. I think they’re necessary. I appreciate you being here, Christine. This is a great conversation and I look forward to next time we might chat.
Dr. Christine: Yeah. Thank you so much for having me. Thank you for bringing more awareness to this topic. I appreciate it.
Dr. Sharp: All right, y’all. Thank you so much for tuning into this episode. Always grateful to have you here. I hope that you take away some information that you can implement in your practice and your life. Any resources that we mentioned during the episode will be listed in the show notes, so make sure to check those out.
If you like what you hear on the podcast, I would be so grateful if you left a review on iTunes, Spotify, or wherever you listen to your podcast.
And if you’re a practice owner or aspiring practice owner, I’d invite you to check out The Testing Psychologist mastermind groups. I have mastermind groups at every stage of practice development, beginner, intermediate, and advanced. We [01:10:00] have homework. We have accountability. We have support. We have resources. These groups are amazing. We do a lot of work and a lot of connecting. If that sounds interesting to you, you can check out the details at thetestingpsychologist.com/consulting. You can sign up for a pre-group phone call and we will chat and figure out if a group could be a good fit for you.
Thanks so much.
The information contained in this podcast and on The Testing Psychologist website is intended for informational and educational purposes only. Nothing in this podcast or on the website is intended to be a substitute for professional psychological, psychiatric, or [00:11:00] medical advice, diagnosis, or treatment.
Please note that no doctor-patient relationship is formed here, and similarly, no supervisory or consultative relationship is formed between the host or guests of this podcast and listeners of this podcast. If you need the qualified advice of any mental health practitioner or medical provider, please seek one in your area. Similarly, if you need supervision on clinical matters, please find a supervisor with an expertise that fits your needs.