Dr. Sharp: Hello, everyone. Welcome to The Testing Psychologist podcast, the podcast where we talk all about the business and practice of psychological and neuropsychological assessment. I’m your host, Dr. Jeremy Sharp, licensed psychologist, group practice owner, and private practice coach.
This episode is brought to you by PAR. PAR has recently released the Feifer Assessment of Childhood Trauma or the FACT, the first and only comprehensive instrument measuring how stress and trauma can impact children in a school-based setting. You can learn more or purchase the FACT teacher form by visiting parinc.com\fact_teacher.
Hey, welcome back everyone. Glad to be here with you. I think this episode is coming out on Thanksgiving week. So happy Thanksgiving if you celebrate. Otherwise, I hope you have a nice holiday week. This will give you something to think about while you maybe take a little bit of time off.
This is episode number 250, which is amazing. I love to reflect on these milestones at least briefly and just appreciate everything that this podcast has been. So here we are, episode 250. This is a great topic and a fantastic guest.
My guest today, Dr. David Baker is a board-certified neuropsychologist. He is an Assistant Professor through the University of Colorado School of Medicine and is a neuropsychologist at Children’s Hospital Colorado in the department of rehabilitation. He commonly evaluates patients with varying degrees of traumatic brain injury, cerebral palsy, and neuromuscular conditions. He also serves as the neuropsychology training director for the rehab track. Dr. Baker has a specific interest in concussion/mild TBI and validity testing in children and teens and has published numerous journal articles and book chapters related to this topic.
That is the topic we are talking about today. We are talking about non-credible effort in pediatric assessment or validity testing. And we get into a number of things that are super interesting. This is another one of those topics where it’s I cannot believe that we have not covered it on the podcast before, but here we are. And I think it was a great episode.
So we do some definitions. What do we mean by non-credible effort? How that contrasts with malingering or feigning. We talk about PVTs versus SVTs. We talk about the battery and how to gauge these constructs. What to do as far as administration. If kids bomb a PVT at the beginning, how do you handle that? We talk about how to share non-credible effort results with families. We also get into the fascinating area of how we’re not trying to necessarily catch kids faking or catch them doing something wrong with this testing, but we can really use these results as a springboard for intervention just like any of our other results. So, this is a good one.
If you have not subscribed to the podcast, now’s a great time to do that. Plenty of good content coming up. And I would love to have you onboard so that you don’t miss an episode.
All right. Let’s get to my conversation with Dr. David Baker.
Dr. Sharp: Hey, David. Welcome to the podcast.
Dr. David: Thank you for having me.
Dr. Sharp: I am glad to have you. Yes, this is another one of those episodes. I feel like could have been done years ago. I don’t know why it’s taken this long. I think your colleague, Amy Connery, talked just briefly about this stuff on one of the very first episodes, but we didn’t really dive into it too much, which was really my fault. So I’m glad to have you here, and I’m really excited to talk about this stuff. So, thanks.
Dr. David: Well, I’m happy to be here. I appreciate you inviting me on. I’m excited to talk about it.
Dr. Sharp: Absolutely. I’ll start with our traditional opening question which is, of all the things that you could do, I know you do many things, but why focus on this within the field? Why focus on effort or non-credible effort with kids?
Dr. David: I think the answer to that is probably complex and multi-factorial for me. First and foremost, it’s something that in my clinical work over the last decade or so I’ve really found to be extremely useful in especially working with youth with mild TBI or concussion. I’m sure we’ll dive into some of that.
I found it just extremely clinically useful to not only help me understand the validity of the test findings that I’m getting but also honestly help conceptualize a case and guide treatment in the proper places. And so, I find it extremely important in virtually all the evaluations I do now.
And then on a more personal level, I think it’s a fascinating area that in some ways blends the neuro with the psychology aspect of things. I think we all started off in this field, or at least many of us started off in clinical psychology and wanting to help others and learning about psychopathology and psychological underpinnings and motivators and all of that stuff.
And that’s never really lost me despite my more specialized training now in neuropsychology and learning about brain injury and brain-behavior relationships. So, I feel like it blends those two really nicely for me.
Dr. Sharp: Right. I know that we’re going to get into this, but I love the way that even the way that we’re framing it right off the bat that it is an important piece of the evaluation. It’s not just a black and white issue where, oh, non-credible effort we’re done here. It is a piece of the evaluation that you incorporate and can guide intervention. It’s a real process component.
Dr. David: Yeah, that’s exactly right. In our group, we’ve really learned to take invalid test results or non-credible performance as a very meaningful test finding. Even though we throw out most of the actual neuropsychological or neurocognitive test data, we find the way it is so meaningful especially once you then start to explore what’s driving it, what’s motivating it, why is it happening? And that’s really where that softer side or more clinical psychology side of things comes into play is really finding out and figuring out why they’re presenting in this way.
Dr. Sharp: Right. Well, I know we’re going to dig into all that. So I’m holding back putting the cart before the horse because I know we need to do some definitions probably and lay some groundwork here because it’s fascinating. I think this is one of those areas, at least in the folks that I’ve talked with and the audience that I have, it’s one of those things we know we need to be doing it and should be doing it, but the degree to which we are doing it and know what to do with it afterward is questionable for a lot of us, I think. So let’s back way up and just set the stage a little bit.
Can you give me some definitions or some terms that we should be using? I’ve already used the term non-credible effort. I don’t know if that’s even the right term necessarily or not. So tell us what we’re talking about here and some terms we might need to know.
Dr. David: Absolutely. And just to circle back to what you were just saying initially, I think there is historically a lot of hesitancy and apprehension in using validity tests in our everyday practice. And I think a big part of that is because when we were all in training, most of us had no training in validity testing, especially when we’re talking about working with kids or pediatrics because historically, most of the validity testing was centered around adult populations, forensic evaluations and those sorts of things.
I know in my graduate training I had very little to no exposure to this. So I think that is understandable. A lot of pediatric psychologists and neuropsychologists are a bit apprehensive for lack of a better term.
I also think it’s daunting and somewhat intimidating to imagine, okay, we do this validity testing, and then what happens if we get invalid results, right? And that is quite intimidating for a lot of practitioners. Hopefully, we can dive into some of that, what to do, and all of that to help loosen some of that. But you’re not alone.
And I think in recent surveys of pediatric neuropsychologists, I think it’s pretty clear that while most pediatric neuropsychologists are starting to incorporate more PVTs or SVTs into their practice, they’re doing so sometimes inconsistently. They might be doing it but not interpreting the results. So there’s a lot more inconsistency, I think, in the pediatric world than there is in the adult world.
I digressed but your question more was about terminology. First and foremost, I think most of what we’ll talk about today are what we call Performance Validity Tests or PVTs. And those generally are distinct from Symptom Validity Tests. And the distinction is Performance Validity Tests are really ways to measure the validity of those tests that we are giving, those performance-based tests.
Symptom Validity Tests or Testing is generally meant to be for those validity indicators that we have in the symptom reported. The questionnaires that we give parents and patients. And we have a bunch of different validity indicators there.
That’s one distinction I often like to say. Now, I will say a lot of people still use both those terms PVTs and SVTs interchangeably. But I do like to distinguish them in some ways because most of what we’ll be talking about today is really PVTs. And there’s much more I think research and literature there on PVTs in children than there are SVTs in children.
Dr. Sharp: Just to clarify a little bit. To make a distinction, a PVT would be something like the TOMM, the MSVT, or something like that, whereas SVT maybe as one of those embedded indexes on the BASC or something like that?
Dr. David: Yeah, that’s exactly right.
And then under the heading of PVTs, we can further differentiate between two types.
One is a free-standing performance validity test, such as what you just mentioned, the TOMM or the MSVT, the MVP, there are many others that we can talk about. But then there are also are embedded validity indicators or PVTs that are embedded within the test that we commonly are given. And in the realm of kids and pediatrics, I think embedded indicators or embedded PVTs are still a bit in their infancy stage and there’s not as much literature in those areas, but it’s growing. Certainly much more literature in the free-standing PVTs now in pediatrics.
Dr. Sharp: Got you. So talk me through just some differences in terminology here as far as, again, non-credible effort versus malingering. That came up a lot in my adult work in graduate school. Feigning is a separate term from those. What terms should we be using and what does that actually mean? What are we trying to capture here?
Dr. David: I think that’s a great question. I think malingering often does come up as a term, but what I will say is in the vast majority of clinical work that I do and just reviewing the pediatric literature, malingering is an uncommon term we often use, especially in kids.
Generally, malingering is thought to be by definition, feigning, some sort of condition or problem for some sort of secondary gain. And most often this is money in the form of litigation. Someone is suing somebody for a car accident because they got a head injury or concussion and they are maybe feigning their problems in order to gain some money and compensation. And that is often a pretty clear case of what some call malingering.
In kids, rarely is that truly the case, right? Rarely is a kid actively feigning or exaggerating problems because there’s obvious secondary gain. I will say it certainly can happen, but it’s pretty rare. And certainly, in the context of the clinical setting like at Children’s Hospital Colorado where I work, we would never use that term, malingering. So it’s generally something we don’t use in the pediatric world very often.
I’d say the more common term that we may use would be, I think you said non-credible effort, a non-credible presentation, non-credible performance. I think that sometimes suboptimal effort, suboptimal performance are other terms you might use.
I often prefer the term non-credible performance because when you throw effort in there, I think sometimes that can be confusing because sometimes a kid is actually putting forth quite a bit of effort in exaggerating or feigning their problems. So I think sometimes effort is somewhat misleading.
And I think just in terms of being most descriptive to folks, even folks maybe reading a report such as a pediatrician or another referring provider, saying something like non-credible performance is maybe a little more descriptive and they might understand that a little bit more.
I will also sometimes use terms such as inconsistent effort or inconsistent presentation, inconsistent performance, things that sometimes can be again, more descriptive and also maybe less pejorative or less demeaning, such as faking or even feigning. But I do think sometimes it is important to call it out and say, there was evidence of exaggeration of certain symptoms and things like that. But I do tend to avoid those terms like faking especially with the patients and families and in my documentation.
Dr. Sharp: Sure. So, is deliberateness a component of this? Is deliberate exaggeration or non-credible performance, I mean, is that always a part of it or is there a way for kids to do this somehow incidentally or without…
Dr. David: I think for the most part we conceptualize this more often than not in our concussion program, in particular, as a pretty conscious and deliberate behavior or action. I will say, I think that there are probably…
I would never be so black and white and say that it’s always that. Sometimes there are cases where it maybe is unconscious to a certain degree. Thinking about conditions such as conversion disorder, we used the term conversion disorder or functional neurological disorder, where a patient is under such extreme emotional stress for one reason or another, maybe it’s a past trauma or whatnot that they are literally unconsciously in many ways presenting in a fashion that is not consistent with what we would expect for that medical condition.
And I think other times there are kids who are just for one reason or another, that can be sort of environmental factors going on with them. They just don’t want to be there. They’re upset. They’re scared. Maybe they’re just really feeling forced to be there for one reason or another. Again, I would probably still classify that as a conscious or deliberate effort, but sometimes they’re just not engaged in the testing. And they’re just not engaging at all. And that can absolutely have effects on their performance on the validity test, but also on the performance-based tests.
Dr. Sharp: Yeah. I’m trying to be too concrete with this, but it’s it sounds like it is largely conscious for the most part, but maybe the big difference between kids and adults is that deliberate shooting for a secondary gain of some sort. Kids might have a lot more variation why they are giving non-credible effort or performance.
Dr. David: Yeah, exactly. I think with kids, it’s much more complex because rarely is there a very clear secondary gain where it’s like, oh, they’re gunning for more money or something like that. It’s much more complex than that. I will say that if you’ve given any of these PVTs before, you realize that they are extremely easy to pass. And so when someone does not pass them, it’s pretty indicative of they’re making an effort to choose maybe the wrong answer, shoot to choose the wrong answer intentionally. And so I think that that’s why most cases we do conceptualize it in that case.
Now, obviously, in cases of real significant psychiatric issues or psychosis, I think there can be reasons that one unconsciously or non-consciously fails a test like that, but most of the time it does feel like a deliberate action.
Dr. Sharp: Right. Well, let’s do a little bit of numbers. Let’s talk a little bit of numbers. And just trying to present a compelling case for why should we be doing this? I’m sure there are people listening who are like, why do I need to worry about this? So can you even just give me some basic statistics around how many kids we think are giving non-credible performance and how that might vary depending on population, of course, but anything in that realm would be helpful?
Dr. David: Absolutely. Most of the studies looking at this, the pediatric studies looking at this really differentiate between, there are clear conditions that seem to have a much higher base rate of invalid test performance or non-credible presentations. And then there are those where it’s just general clinical cases and they’re not necessarily as high base rate group.
I guess I’ll start there with sort of general clinical samples and studies of kids who are presenting for most neuropsychological evaluations like in a hospital setting, maybe at a medical center, most studies show that roughly 2 to 5% of them fail a validity test of some sort.
That’s not an insignificant number. It’s obviously a small number, but it’s not an insignificant number. And it’s honestly in keeping with what I’ve seen over my career in that cases where I would not at all expect invalid test performance or a failure on one of my PVTs, it’s happened. And often, it takes sometimes a little more digging to find out why in those particular populations, but it does happen. So roughly 2 to 5% in the general population of kids who are being evaluated.
Higher percentages we see certainly in the realm of mild TBI or concussion. So not only has our group looked at this pretty extensively over the years, but others across the country and in Canada have found a similar high base rate failure on PVTs. Roughly between 12 to 25% of those with mild TBI or concussion have been found to present non-credibly or fail a PVT.
That’s a pretty substantial number. And I think that in our group, it roughly equates to about 1 in every 5 concussion patients that we’re seeing, we will get this potentially get this presentation. I will add that also mostly these are unique concussion or mild TBI cases in that day are generally those who are having persisting symptomatology or persisting problems after a concussion.
So that said, I think this is a unique population. Usually are kids who are several weeks if not months from their injury. And we would not necessarily expect them to still be having these difficulties. And often they are then referred for a neuropsychological consultation with us.
So it is a small subset of the concussion or mild TBI population that we’re talking about. But nonetheless, it is still pretty significant and notable.
Dr. Sharp: Right. This might be getting…
Dr. David: Go ahead.
Dr. Sharp: I was just going to ask. It may be a little too granular, but do you know if it varies depending on say age or co-occurring conditions, or even any other demographic factors?
Dr. David: That’s a great question. Yes, I think we absolutely see teenagers present this way much more than younger children. And some of the co-occurring conditions that we do often see are internalizing emotional disorders such as anxiety and depression. We do see a high number of kids with learning disabilities or ADHD as well presenting in this way.
So yeah, I think it’s it is a unique subset of kids that we are seeing that often are presenting this way. But I think it’s important for those out there because many of us are seeing kids who are being referred for persisting or long-lasting and I’m using air quotes, post-concussion syndrome, or post-concussion problems. And so, I think we have to be really aware of the fact that maybe a quarter of these kids are going to present in a non-credible manner. So that’s a really important clinical population to keep in mind.
And then the other one which is starting to bleed a little bit into kids is those seeking ADHD evaluations, especially at the college and postgraduate level. So this is a group. There are several studies of college-age kids seeking ADHD diagnosis and accommodations and medications. Some studies as high as 47% of those students that are seeking those sorts of evaluations are presenting in a non-credible or inconsistent manner.
And then lastly, sorry, real quick, I think the other population is those seeking social security disability. Many of us are not doing those evaluations, but some are, and that has a very high base rate of non-credible presentations, sometimes as high as 50% to 60 of those presenting there.
Dr. Sharp: I could see that. For anybody who might be interested in more discussion around, particularly that ADHD and college-age individuals, I interviewed Allyson Harrison and Julie Suhr probably six or eight months ago. I know that they’re deep in that world. So that episode might be helpful for folks.
Dr. David: They’re great. I’ve sat on some panels with them and they’re doing great work looking at that. They really try to drive home the point of, often in most of these cases, we have to take a very common-sense approach to any of these diagnoses, right? So it’s not just the performance validity tests or symptom validity tests that can help us find those that are maybe non-credible, but it also, you just have to take your clinical judgment and what you know about these conditions like ADHD, for example, or concussion, and what is typically expected in terms of that natural course.
And more often than not, a kid should be diagnosed or show signs of ADHD well before college levels. And similar to concussions, we know that most kids with an uncomplicated concussion heal pretty darn quickly and without much complications. And so, when we’re several months to years out from an uncomplicated concussion, we do have to start to wonder about that too.
Dr. Sharp: Absolutely. Yes, I’m holding but this could so easily spin off into a concussion episode. I’m really trying to keep us on the rails here. It’s tough.
Dr. David: I know. We could go down so many roads, right?
Dr. Sharp: Right. Well, let me maybe ask a question that’ll bridge us from research to practice which is, for a lot of us who are working with, let’s just say a general clinical population, you named that 2 to 5% number. Would you suggest then that that is standard of care or standard or standard of practice to administer PVTs to all the kids that we’re seeing? Is it to that point?
Dr. David: I think it is. A lot of our national organizations I think are starting to come out with or have come out with standard practice of care statements. This should be a normal part of all evaluations.
Of course, the adult world is been saying this for a while and it’s starting to trickle down to the pediatrics world, but I would say just from my own clinical experiences that you never know. You just really never know. And why would we not have some sort of validity check on these evaluations that we’re doing that are sometimes very high stakes, right?
So sometimes we’re making determinations about school placement, IEP services. Sometimes we’re making big decisions around intellectual disability or not. We might be making decisions around medications or not medications. So, I feel like we’re making… Sometimes our test data is potentially very powerful and why not have some confidence in the validity of those findings?
And like I said, if, at the high end of that 5%, you’re going to run across that fairly frequently if you’re doing a lot of testing. And so I think it’s really important. I’ve really been shocked at how little is done in the schools for school evaluations, for IEP. And I think that’s hopefully, maybe going to be the next wave of popularity. I would hope that for making big decisions around IEP placement and classification, boy, you would hope that there would be some validity checks there.
Dr. Sharp: Right. Yeah, that’s funny. I was going to ask about that because I have not… granted I’m not a school psychologist, but I interface with a fair number of them in my audience. And I don’t hear about PVTs in schools.
Dr. David: No. I just recently had to… I did an evaluation of a kid who wanted to apply for accommodations for the SAT. I used to be in private practice and I used to be pretty familiar with what were the requirements for documentation for those SATs. And interestingly, on the college board website now for the SATs and for the AP exams, one of the requirements is that there is validity testing as part of the documentation.
So I think they’re starting to… it’s starting to trickle down to other areas. Maybe not just in the medical center or something where we’re seeing concussion or mild TBI.
Dr. Sharp: Sure. That’s great. Yeah, it’s funny. It reminds me of how in the investing world, I don’t know if you’ve heard this, that it doesn’t do any good to try and time the market and invest on specific days. You need to really be investing small amounts throughout the year or over the months, or whatever. It’s a parallel to this. We don’t know which kids are giving a credible performance or not.
Dr. David: And the end, again, it’s not black and white. Most of the time we are suggesting multiple validity checks or PVTs throughout an all-day battery just because we know that effort and motivation and engagement can wax and wane. It’s not always this stable, consistent process.
Dr. Sharp: Right. I think that’s probably a nice segue to the battery, the practice. How does this come alive in the work that we do? I’m curious, maybe we frame this. I would love to paint a picture of a typical evaluation day for you. Where these measures coming into play? At what point in the day? Which ones do you like? And just how to integrate it with the batteries that we do.
Let’s take a quick break to hear from our featured partner.
Kids are experiencing trauma like never before, but how can you figure out whether they’ve been affected and how it impacts their behavior and performance at school?
The Feifer Assessment of Childhood Trauma or the FACT is the first and only comprehensive instrument measuring how stress and trauma can impact children in a school-based setting. The FACT teacher form solicits the teacher’s perspective on the performance and behavior of children ages 4 to 18 years. It takes just 15 seconds to administer. And the available e-manual gives you detailed administration and scoring instructions. You can learn more or purchase the FACT teacher form by visiting parinc.com\fact_teacher.
Dr. Sharp: All right, let’s get back to the podcast.
Dr. David: Absolutely. I think this question comes up the most. When do you give them? What do you give?
So, probably the most commonly administered but also studied PVTs in kids would be the TOMM, The Test of Memory Malingering, and the MSVT or the WMT. Green’s Publishing has several different PVTs that are all computer-based. One is called the Medical Symptom Validity Test or MSVT. One is longer, a longer version of that called the Word Memory Test. And then another one is a non-verbal memory test as well.
The MSVT and the Word Memory Test generally require about a 2nd to3rd grade level of reading. So if you have a younger kid, it might not be as appropriate. The TOMM tends to be very well-validated in kids even as young as 5years. And it generally has an ability to be pretty sensitive, but also highly specific.
The one downfall I will say with the TOMM is it tends to be a bit long and lengthy. I think they are possibly coming out with, if not already, a digital version which might help with just not having to turn those pages a lot. And then there also is potentially an abbreviated version that they’ve been talking about.
And then another one that I commonly will use, especially in younger kids, is a relatively recently created one called the Memory Validity Profile or MVP. And that one is the first. freestanding validity test that was created specifically for children. So all the other ones were created for adults and then modified or used for children and studied with children.
The MVP, I will often use, especially in those young kids where I’m worried a little bit about reading ability, but also they’re just young and I want something quick and simple. It has a visual component to it and it also has an auditory component to it. I like that piece too.
To answer your question about when, I usually start off my evaluation, if I’m evaluating a school-aged kid, one of the first things I’m doing is usually the TOMM or the MSVT.
I think most of us probably are in the practice of doing some sort of a warm-up test, whether it’s a VMI or something that’s non-threatening to the kid, and you just kind of warm them up. And so after doing some sort of a warmup test and also talking to them about how important it is to try your best and whatnot, I’ll usually put them through the MSVT or the TOMM because I think right off the bat, you want to know. And it would change my decision-making if either of those tests were failed.
And so if they are not failed, if they pass and do fine, then I will usually progress with testing. And then usually after lunch, I might do one other freestanding validity test. And again, if I haven’t given the TOMM, then maybe I’ll give the TOMM or I might give the MVP. I might give Rey 15-Item Test which is a very old test, but it’s publicly available. Mike Kirkwood created this automatized series that we looked at which is a very quick and again, simple validity check that you can give throughout.
There are now new ones. There’s one that was just released very recently, I think through MHS called the Pediatric Performance Validity Test Suite and it’s digital. It has about, I think, five different tests that you can give throughout the day, again, to monitor steadily throughout the day.
And then lastly, I do pay attention to those embedded indicators that we are often already giving but can tell us something.
So for instance, Digit Span on the WISC is probably one of the more well-known embedded indicators. And the ChAMP which is a new memory profile- children and adolescent memory profile, has several embedded indicators throughout that evaluation. We know the CVLT-C and the CVLT-3 also have embedded indicators as well. So I might pay attention to those as well.
Dr. Sharp: Great. We have a lot to choose from. I think that’s a surprise to a lot of us. Most people have heard of the TOMM, maybe the MSVT, but there are a lot of other options out there that we can really pick and choose. And it’s a little surprising to me to hear you say that you’ll give two free-standing tests throughout the day. It makes sense, of course, but surprising. So, this is good.
Dr. David: I will say sometimes I do forensic evaluations or medical-legal evaluations where I’m called on to be an expert case and do an independent evaluation of a kid. And in those cases, I will do many more. I will sprinkle in almost all that we can do at times just to make sure. And some do argue that the more you give, the more chances you have of a false positive, but at the same time, I think it is important to just continually monitor that validity.
Dr. Sharp: Yeah. Well, that leads me to some questions around decision-making then. So you said if they do well on the first one, then you continue, right? And that seems like the easy case. So, walk me through some iterations of not passing and how that affects your clinical decision-making on that day.
Dr. David: Absolutely. This is like the million-dollar question, right? And this is, I think, why a lot of neuropsychologists are a bit squeamish about administering them. So like, what happens? What do you do that?
So, I think in our clinical setting, it’s much easier for us to, at that point, really modify and slim down the battery. I do not feel confident giving an entire WISC to a kid when they have just failed the MSVT because I’m not going to trust those results and I might not even report those results. So why put me and the patient through that? Why also expose them to the WISC when we don’t necessarily need to?
So, often, in our clinic, in particular, we really change and modify the battery quite a bit.
Now, our concussion battery is already pretty short and pretty small. I think you’ve talked with Amy Connery about this, but it’s really about an hour to an hour and a half of a battery. So slimming it down is not that hard. And we might still sprinkle in some other things as well and some performance-based tests.
I always find it interesting because more often than not in the patients that I see with persisting post-concussion problems when they fail the MSVT and or the TOMM, I’ll still give them a list-learning measure like a bonafide memory test, whether it’s the CBLT or lists on the ChAMP.
And surprisingly, they generally do well on that except maybe on the recognition trial. In the recognition trial, they often will not do well, but on the recall trials, they will often do really well. And for me, that’s nice important information to share with them and the family.
I might say something at some point. Well, it’s interesting because this patient performed worse on the really simple, basic tasks and then better on our bonafide memory tests. So that’s a really good sign. So sometimes I will use that in my dialogue about the testing.
So in the context of the concussion evaluation, it’s pretty easy and quick to shorten and modify that battery. I might spend a lot more time doing clinical interviews, giving questionnaires, really exploring the psychosocial environment of the kid and how they’re functioning. But I think in bigger evaluations, sometimes it’s challenging. And I will also say, I think often in private practice folks are really confronted with this dilemma, right?
First of all, if you completely stop testing or shorten the battery, it reduces your income, right? It reduces what you can charge. But in my mind, that’s probably the most ethical thing to do because, again, it doesn’t make sense to me to continue with a full battery in a kid where you’re really not going to trust those results and then charge the family all that time for results that you really can’t give much constructive feedback on.
So I will usually figure out why this happened in one way or the other. And I find that to be really important for the family. But really reduce that that testing data and time collecting.
Dr. Sharp: Can I press you a little bit on that and just ask, when you say, slim down the battery or reduce the battery, it sounds like you’re not saying stop testing right at that moment. So I’m curious then, what does slimming down look like, and which measures are you still giving and why? I guess that begs the question of why?
Dr. David: Well, one of the reasons I think not to just completely stop testing right at that moment is:
A) It’s a little bit of a test security issue. Then you’ve really shown your hand and made it very clear which test is a validity test, right?
B) I think you still have a job in many ways to do your due diligence and try to still piece apart what’s going on.
So, if you were really to ask me, what I would do is generally after that MSVT failure, I might still give digit span. Again, it’s an embedded. It can be an embedded indicator, but I might also give a coding subtest from the WISC as well just to see how they perform on that. Again, I’m not going to usually report these findings in my report if I have validity concerns, but sometimes it can tell me something.
I might give those tests that we think of as whole hold tests or crystallized tests such as vocabulary from the WASI or the WISC. I might give we’d like word reading from the WIAT just to see. Again, these are generally tests that we would not expect to be impacted by concussion or brain injury. And so these are often things that we do give.
And then again, like I said, I might actually give a bonafide list learning task or a verbal memory task to see how they perform. And like I said, most of the time they do perform surprisingly better than their PVT performance would predict. And then really, again, some self-report questionnaires, a lot of parent-report questionnaires, clinical interviews, really exploring things at a higher level.
Dr. Sharp: Yeah. Do you have many kids who will do fine in the morning and then fail something in the afternoon or at a different point throughout the day?
Dr. David: That’s a good question. I can’t think off the top of my head of where I’ve seen that. I think the common thing we do see is just fatigue, especially in patients that I do see with real significant neurological or neurodevelopmental conditions. They might mentally fatigue later in the afternoon.
Generally, they still do fine on the validity tests, but I will sometimes make comments about just their overall engagement and energy level later in the day seemed really down and maybe they were giving a lot of answers. And so, I will make comments on that. And I think that’s really important information and rich information to provide in a report.
Again, if you’ve given any of these PVTs, you know how easy they are. And so again, if a kid fails, it really is usually a conscious effort in many ways, many times.
Dr. Sharp: Yeah. Well, let’s talk about what to do with this information. So, we’ve talked about the testing day and what to do with it, but then where does it go from there? So there are two facets here. There’s how do you describe it in the report? And how do you describe the other test results and the report? How do you communicate this to families? And of course, I’d love to blend that into our topic from the beginning which is, how is this an actual intervention versus just sorry your kid wrecked this evaluation?
Dr. David: I’ll start with how we approach this with the patient and the family. In the concussion clinic, typically when this happens, we stop testing again, not right after the failure, but usually after we’ve done a few other things and talked. And then usually we have an independent conversation with the parent or parents and family about what happened.
And that is always very interesting because some families or some parents really get it right off the bat. And you can say, typically the spiel that I say is something to the effect of, we did a lot of different tests today and in order to make sure that we were getting the best possible results for these tests, we do give some validity tests, meaning there are some tests that measure the effort and the validity of these tests that were given. And unfortunately, on those validity checks or those validity tests, your son or daughter did not perform well. And therefore, unfortunately, all of these other test results are really called into question.
And sometimes I just pause and see, does the family get it? That’s a pretty abstract concept, I think for a lot of families. So often there’s confusion like, “Well, I’m not sure what you mean.” So then sometimes I break it down a little bit more like, okay, well, we gave a lot of tests that appear difficult, but are actually very easy and basic. And these are tests that are generally easily passable by kids with very severe brain injuries. Kids with intellectual disabilities pass these. And so they are generally very simple, but they might appear to be more difficult. And it appears that your son or daughter intentionally performed poorly on these tasks for one reason or another.
Sometimes you have to get even more specific to say, it seemed like at times they were choosing the wrong answer intentionally, they didn’t seem fully engaged. Sometimes I will use the term, it was really inconsistent and didn’t line up with what we would really expect with this type of condition.
They did really well on those harder tasks and not so well on the easier and basic tasks. And so, sometimes when you break it down a little bit, you see the light go on for the parents. And then other times you may even have to be more specific. And sometimes these tests like the MSVT has nice charts that you can print off showing where this patient’s performance was compared to really severely neurologic samples or kids with intellectual disability.
Those sorts of things can help highlight. Look, we know your kid is not intellectually disabled. We know your kid doesn’t have a severe TBI that sort of thing. Often then it’s a discussion around, I usually employ the parent to ask or to help me with figuring out why this might be the case.
And this tends to be some of the most rich information that we get. I say something to the effect of, why do you think this might’ve happened? Why do you think they might be exaggerating these memory problems, for example, or exaggerating these impairments? And usually, the parents have a pretty good idea of why and can throw out some hypotheses.
Usually, I already have some working hypotheses and so I might feed them or test them out with the family. Maybe they’re under a lot of stress right now at school. Could that be a reason? Oh yeah. But often we’ll get some really important information at that point.
The family might disclose something, a big stressful event that’s been going on in the house. Like, oh, I wonder if it’s because her dad and I are getting a divorce, or I wonder if it’s because he just broke up with his girlfriend and was really devastated by that. Or I wonder if it’s because he really doesn’t want to go back to playing football and dad’s really pressuring him to play football, different things like that. I think it really brings out those motivators and driving factors that are really important.
Dr. Sharp: Right. Yeah. It seems like this is that place. Again, circling back, it’s the psychological component. It’s the process of what’s happening versus, okay, we have this failure score. So what? I’m curious to figure out.
Dr. David: And I usually have a talk with a family at that point. Look, this is really important information. Some parents are really annoyed, like, wait, so this was just a waste. You’re telling me this was a waste. We can’t trust any of the data. And I say, no, I actually, I don’t think this was a waste at all. What we found out is your son or daughter is really maybe crying out for help right now or is really struggling in one way or the other. And this is one way that they’re communicating to us or to the school or to whoever.
So I really drive home how important it is. I also drive how important it is not to chastise or punish that patient for what they did. So then, I usually do bring the patient back in, and depending on the age of the child, again, most of them are teenagers, I usually soften that feedback a little bit, but I might say something like your test performance was a little inconsistent in here. It was on some things really good and some things not so good, but what I can say with a lot of confidence is, your test performance did not align up with what we know about concussion or mild TBI. I really don’t think that this is something that we… this is not something we see in concussion or mild TBI. I think there are probably other things going on to explain your difficulties. And then I might go into some of that.
I’ve found that if you confront a teenager and say something like, I know what you did. I know you were faking it or whatever really does not go well most of the time. And usually, you get a lot of shutdowns. Then they’re not going to listen to anything you say. They’re not going to trust you. They’re mad. And you’ve really ruined that rapport. And then, therefore, it’s going to be harder to convince them of your conceptualization and treatment plan going forward.
Dr. Sharp: Yeah. I know it’s hard to generalize, but do you get teenagers who will say, you’re right. I wasn’t trying my best. They cop to it for lack of a better term.
Dr. David: No, I have never had a teenager come clean. Usually, they might throw out some excuses. I had a really bad headache or the computer screen was hurting my eyes. There might be some excuses there that sort of thing. And to be perfectly honest, sometimes families are doing that as well because they’re pretty entrenched in maybe this narrative of my kid has brain damage or brain injury. And sometimes that is hard. That’s a hard conversation to have. I was going to say majority of the time… Yeah, go ahead.
Dr. Sharp: It was something I was thinking when you were describing the process of feedback because I thought, well, I wonder if parents are ever taking that as evidence that, well, yeah, that must mean that my kid is really, really impaired by this concussion, you know?
Dr. David: Totally. That is one of those cases where then you really have to break it down and explain to them why it’s not evidence of impairment. And so yeah, sometimes you have to be real concrete.
Dr. Sharp: Sure. And I’m sorry I cut you off.
Dr. David: No, that’s okay. I can’t remember where I was going, but I think the other piece is the documentation. You were asking, how do we document this? Again, if there is clear evidence of failure on these PVTs, then I am usually not presenting any test data. And the reason for that is I can’t really trust that test data. Even if it’s even in the average range, that still might be an underestimation of their true abilities.
And my concern always is that the family shares that with school or with somebody else. And that those test scores really just get looked at and nothing else is considered, right? I mean, how many times have you looked at a report, and sometimes go straight to the test scores at the back and you don’t read the context in which those scores may be obtained. So that is a big reason we, as a group, generally do not present those test scores for invalid testing.
I will say in the context of forensic cases usually test scores are required. The attorneys want those test scores even if you have failed PVTs across the board. I usually in those cases will put a lot of asterisks and caveats under those tables or next to those scores and just say again, this is a very likely underestimation of their true abilities. Failure on multiple PVTs.
Dr. Sharp: Sure. Thinking about the hospital setting, I know y’all write pretty short reports anyway, but in the case where you’re not reporting any test data, what does that report actually contain if anything?
Dr. David: Usually a lot of discussions around what do we feel is maybe driving this presentation and what would be important recommendations going forward? Usually a statement around, Johnny got this concussion six months ago and has continued to have these problems, but it’s our opinion that these problems are not necessarily due to concussion factors anymore. They’re likely due to several non-concussion factors including school stress, anxiety, depression, whatever might be those factors. So, a clear description of that and then a clear recommendation section that goes through what we do recommend in terms of medical follow-up or returning to their sport, and then certainly returning to school.
I do find in these cases, the school situation is a bit tricky because I sometimes am a little bit sheepish about the family bringing that report into the school and then the school going haha. I knew they were faking it or something like that. So I might sometimes write a separate letter that really does not outline the invalid test findings necessarily, but really just outlines my concerns about maybe their emotional functioning or their stress at school and how to support that. Whether it’s creating a 504 plan for anxiety or depression or just creating more of an informal plan to get the kid back to school and support them.
Dr. Sharp: Right. There’s a lot to consider here. I am going back to the, I don’t know if you call it a feedback session, but that time when you’re talking through the results with the family and the kid. It sounds like it’s really just a matter of you just have to be curious and in a way, just skip over any defensiveness or denial and just go with it and say, well, what’s going on here? What might be driving this? Tell me what’s happening.
Dr. David: I will completely agree. And I think you’re right. You have to act as a detective to figure out what is driving this. And sometimes it does require getting the parents on board to help you with what could be going on. And sometimes, to a certain degree of the patient, but usually a patient is going to be less forthcoming about some of those things. I think being very curious.
There are many cases where I say, look, I think right now there’s too much emotional noise going on for us to get a really clean picture of their neuropsychological or neurocognitive functioning. Why don’t we do some therapy? Let’s start Johnny on some individual therapy and why don’t I see him back in six months? And at that point, hopefully, we’ll have a cleaner picture. And I’ve done that multiple times in cases. And usually, when they come back, I get a valid performance at that point.
Dr. Sharp: I’m glad you brought that up. There is a question of what happens after. Do you try to retest? I guess it depends on the case, right?
Dr. David: Yeah, it does.
Dr. Sharp: Okay. Gosh, I feel like this has been a great overview. We got to dip into specific aspects here in some regards. I took a lot of notes and I’m guessing other people did too.
What about resources? I know y’all have done a lot of research in this area. It sounds like there are other folks around the country. Folks who want to learn more about this topic, where should they go?
Dr. David: I have no financial interest or conflict of interest in this book, but I did co-write a book chapter in Michael Kirkwood’s book on Validity Testing in Child and Adolescent Assessment. I think it’s published by Guilford Press. And it was published a few years ago, but in many ways, it’s kind of the end all be all of pediatric validity testing. It has great chapters on terminology, base rates as we’ve talked about, what tests are available for pediatric providers, motivations, feedback. All of that stuff is in there. So I think that’s a great first stop.
Dr. Sharp: Nice. I’ll definitely put that in the show notes for people to check it out.
Dr. David: That’d be great.
Dr. Sharp: This has been good. I really appreciate it. It’s really got me thinking about what we need to change up in our batteries and just some peace of mind too, because that question really of what do we do with this if something goes wrong or if a kid fails. That’s where a lot of people get stuck. And so, being able to answer those questions I think was super helpful. So, thank you.
Dr. David: Absolutely. Well, I feel like we just touched the tip of the iceberg and could have gone so much deeper. So if you ever need me or one of our group to come back and dive deeper into it, I think that’d be great.
Dr. Sharp: Oh, I would love that. Yeah. I appreciate that. Well, I guess it’s a goodbye for now, but thank you.
Dr. David: All right. Thank you.
Dr. Sharp: Okay, y’all, thanks so much for tuning in today. I hope you enjoyed it. This was a good one. I was taking so many notes while we were talking, and the hope is that you walk away with some things to think about and some things to implement in your practice.
Like I said at the beginning, if you have not subscribed to the podcast or followed the podcast, now’s a good time to do so. I have totally converted over to Spotify for all of my music and podcast needs. So I’m starting to use the terminology of follow more than subscribe, but whatever it looks like in whatever podcast app you’ve got going on, I hope that you have done it so that you don’t miss any episodes coming up.
All right. I hope you’re doing well. Like I said, enjoy your holiday week, whatever that looks like. I’ll catch you next time.
The information contained in this podcast and on The Testing Psychologist website is intended for informational and educational purposes only. Nothing in this podcast or on the website is intended to be a substitute for professional, psychological, psychiatric, or medical advice, diagnosis, or treatment.
Please note that no doctor-patient relationship is formed here, and similarly, no supervisory or consultative relationship is formed between the host or guests of this podcast and listeners of this podcast. If you need the qualified advice of any mental health practitioner or medical provider, please seek one in your area. Similarly, if you need supervision on clinical matters, please find a supervisor with expertise that fits your needs.