[00:00:00] Dr. Sharp: Hello, everyone. Welcome to The Testing Psychologist podcast, the podcast where we talk all about the business and practice of psychological and neuropsychological assessment. I’m your host, Dr. Jeremy Sharp, licensed psychologist, group practice owner, and private practice coach.
This episode is brought to you by PAR. The Feifer Assessment of Writing examines why students may struggle with writing. The FAW and the FAW screening form are available on PARiConnect- PAR’s online assessment platform. Learn more at parinc.com\faw.
Hello, everyone. And welcome back. Glad to be here with you. Glad to be back with another clinical episode, the first in a long time. So today I am talking with Dr. Jen Engle all about assessing motor-impaired children.
Let me tell you a little bit about Jen, and then I’ll tell you [00:01:00] a little bit about the episode. Dr. Jen Engle is a pediatric neuropsychologist who works at BC Children’s Hospital in Vancouver, Canada. She is Board Certified in pediatric neuropsychology through the American Academy of Pediatric Neuropsychology. In addition to providing direct neuropsychology services, she offers training and consultations for psychologists who wish to develop their expertise in special populations, particularly children with neuromotor conditions and children with visual impairments.
Jen has also written an ebook with several strategies and guidelines around assessing motor-impaired kids and visually-impaired kids. That’ll be linked in the show notes. I invite you to check that out. Jen was so kind and such a fantastic guest. She was super knowledgeable.
We cover a lot of ground in this interview where we’re talking about a [00:02:00] number of things related to assessing motor-impaired kids. We talk about what that even means. So some definitions and etiology of motor impairment. Different types. We talk about why it’s important to make accommodations for this population of kids. We talk about which tests Jen prefers for kids with motor impairments. We spend quite a bit of time talking through how to break standardization, when to break standardization, what to do with that data once you have it. We talk about interpreting the data. We talk about different strategies for setting up the room for kids who may have motor impairments.
So we cover a lot of ground and I think there is a lot to take [00:03:00] away from this episode. Even if you are not going to specialize in working with this population, you’re actually the perfect audience because this material is really aimed at school psychologists or clinical psychologists who may see these kids in their practices and want some handy knowledge and strategies to make sure that we’re doing the best job that we can.
So without further ado, I will give you my interview with Dr. Jen Engle.
Hey, Jen, welcome to the podcast.
Dr. Jen: Thanks. Happy to be here.
Dr. Sharp: Yes, I am happy to have you. I am [00:04:00] amazed at all of the topics that we still have not covered on the podcast even after 200 plus episodes. And this is absolutely one of those topics. I am so glad that you’re here to talk with us about assessment with motor-impaired kids.
As usual, I want to start with this question of why this work, in particular, is important to you out of everything that you could do within assessment or psychology?
Dr. Jen: Well, I work at BC Children’s. It’s a tertiary care center in Vancouver. And part of our mission is to help build capacity in the community for people who care for children. In this case is, these kids can come to the hospital. We do have special programs for assessment for kids who are deaf or hard of hearing or visual impairments or neuromotor conditions, but we’re not really able to serve [00:05:00] all of the kids in the private province, especially those with neuromotor conditions. There’s a lot of them. And actually, I think that they’re really best served in their home community by their home psychologist who can do things like classroom observations that I’m not able to do at the hospital. And also to save these kids who often have complex medical conditions from traveling. So the idea behind this project of mine was to help build capacity in our community for these kids to be seen in an accurate and meaningful way in their home community by building the capacity in the psychologist.
Dr. Sharp: I love that mission, if you want to call it that, that’s bringing services to the kids who need it the most and probably have a harder time accessing it. That’s amazing.
Dr. Jen: Yeah.
Dr. Sharp: Well, I feel like there’s a lot to dive into and I will undoubtedly [00:06:00] ask a number of naive questions as we go through this conversation, but let’s start at the beginning. We’re talking about motor-impaired kids. You used the phrase kids with neuromotor weaknesses, I think, or deficits. So what does that mean? What are we actually talking about?
Dr. Jen: Neuromotor is a broad term that means any kind of developmental or acquired disorder that affects things like movement, posture, motor ability, but it’s caused by damage to the central nervous system, the brain. And well, it’s not a spinal cord-only injury. That’s just different. Although functionally, the kind of adaptations we might make for motor impairments would be the same, but the conditions that we’re talking about, the reason why they’re coming to us in psychology is that it affects the brain.
So it’s the brain plus motor condition. Cerebral palsy being the number [00:07:00] one in children that we see the most often. That’s the one people probably know the best which is something that is basically a non-progressive abnormality of the developing brain which usually is from the prenatal or perinatal period.
Dr. Sharp: I got you. Is there anything else you could add as far as the etiology of cerebral palsy and how that comes about?
Dr. Jen: Yeah, probably the most common cause would be children who were born premature and their brain is really vulnerable to a bleed. That can happen around the time just after birth when they’re really vulnerable and there are all kinds of things going on in their brain and their body and their lungs. But there can also be, for example, a stroke that happens prenatally or postnatally. All of those, if they happened around that time and [00:08:00] cause the motor dysfunction would be called cerebral palsy.
Dr. Sharp: Sure. I know it’s maybe beyond the scope of the podcast to really get in the weeds as far as the neurology and everything that’s happening there but are there any other primary drivers for some of these motor impairments we might see or that you see?
Dr. Jen: Yeah, that is the biggest group other than what we could see any acquired injury, brain injuries may lead to motor conditions like hemiplegia with a brain injury or quadriplegia. Those are the conditions that happen from birth. And generally, the kids I’m talking about in this presentation are the ones who’ve had lifelong conditions and developmental. And [00:09:00] most of them will have not most of them, but with the kids that I’m talking about doing this assessment with where you have to do things really differently, are the kids with really severe cognitive impairments. You can have acquired conditions like that, but it’s different from intellectual disability when it happens in an acquired way.
Dr. Sharp: Of course. I know there’s a broad spectrum of impairments that might result, but is there any way to generally describe when we say motor impairment, what does that actually look like in real life?
Dr. Jen: Yeah. Some of the terminologies that people have heard probably, let’s just go over them. One that’s common is hemiplegia- so that would be one side of the body affected i.e, right arm and legs sometimes it’s the right arm worse than the right leg or vice [00:10:00] versa. The second most common probably is quadriplegia- so all four limbs are affected. Less commonly would be diplegia- so that’s usually just the legs affected and would affect our testing a lot less if they’re able to use their hands effectively.
A lot of kids with hemiplegia or hemiparesis can use their hands pretty functionally. It can be from anything from just like discoordination or shakiness or weakness in their hand to really a full completely… their hand is so spastic or tight, they can’t move it for anything in the assessment. So the range is huge- from kids who can only use their eyes to communicate to those who just have some kind of weakness in their hands.
Dr. Sharp: Sure. And you have worked with kids that run the gamut, it sounds like.
Dr. Jen: Yeah, we do see them all.
Dr. Sharp: Oh [00:11:00] my goodness. I know we’re going to really dive into the accommodations and different ways of working with these kiddos, but just right off the bat, it almost sounds like two different worlds in a way, like working with a kiddo who has maybe a little tremor or some weakness in their hands to a kiddo who can only communicate with their eyes. That’s a wide range of presentations that we have to be mindful of.
Dr. Jen: Yeah, and often, because the problem comes from the brain, it’s not just motor functioning. So that’s the thing. It goes along with cognitive impairments and social, emotional delays, epilepsy, often learning difficulties in general, sensory, often you see visual impairments particularly critical cerebral visual [00:12:00] impairment and also hearing impairments. So you can have any and all of those together.
Dr. Sharp: Right. Well, I know that the ebook that you put together has several sections and you get into each of these areas. And I think it’s too much for us to tackle just on an hour-long podcast, but just for folks to know that that is out there, we’ll link your book in the show notes.
We’ll try to focus just on adaptations for motor impairments here today, but it seems complicated right off the bat. I’m like, “This sounds very complicated.” And how in the world are we getting accurate assessment results with some of these nontraditional presentations maybe is a word for it. I’m sure there’s a better term for it, but non-standardized administrations maybe. So yeah, I would love to dive into the actual assessment [00:13:00] process. So maybe we start big picture. I know you work in a hospital setting, so what are the referral questions? Do you even have referral questions or is it just assess this kid and tell us what’s going on?
Dr. Jen: Usually in this clinic, it is pretty broad needs and assessments. Sometimes it is specific to accessing community supports, getting that diagnosis of intellectual disability. Other times it’s about the school wanting some guidance on either what program they should be in like a life skill versus academic, or just wanting some guidance of how to reach them, how to engage them, what to expect of them. Sometimes they come like a little mystery of help us understand this kid. Again that’s pretty broad. [00:14:00] Yeah, they are often very broad as the referrals.
Dr. Sharp: I’m sure. Can you walk me through the clinical decision-making process then? When you get one of these referrals, how do you tackle an assessment like this? Where do you even start with a broad question like that?
Dr. Jen: Well, at the hospital, I’m lucky that I have access to all their medical records here. I can review that at my first stop. And then next step is we collect information from the family and the school. They fill out lots of questionnaires for us. And then I do interviews with the family and sometimes with the school as well, and make my initial plan for this assessment from there. That helps me to narrow down the question of what I’m focusing on and also get to know the child.
I need to know the basics of like what their motor abilities are, [00:15:00] what is their communication like? Are they using words or gestures or a communication device? And what’s their capacity? Some kids can really only handle 5-10 minutes at a time, whereas other kids can handle a full assessment. I need to try to figure out as much as I can ahead of time about that, but I also need to be super flexible so that I’m ready for anything when they get there. And I usually will pull a whole bunch of different tests than the room just looks like stocked with staff so that I can quickly go, oh, this isn’t working, and then switch to something else because as much as I try to be prepared, I don’t always know what I’m doing until I get in the room.
Dr. Sharp: I can imagine that. So it sounds like right off the bat, flexibility is key in being ready for anything. Okay. So you’re [00:16:00] combing through records. You’re talking with the family. I think of just traditional outpatient evaluations where we are meeting with the family or the parents on a separate day for separate intake. Is that the way that you approach it as well?
Dr. Jen: Yes. Traditionally, before COVID, always been able to do that but now it’s routine certainly as prior to COVID that we do virtual meetings ahead of time and I won’t go back. That’s been fantastic to be able to do that in advance. And a lot of the families are coming from far, so having that information ahead of time for me is really important.
Another thing that I’ve done for this population specifically is when I can, I’ll get the kid on zoom so that I can see them myself a bit beforehand. And even sometimes when I’ve done these sometimes in [00:17:00] collaboration with a pediatrician when there’s an autism question, we’ll have them do a few things with their parents in their home or in their school. And that’s been amazing. This is a new thing since COVID. Everyone’s on zoom in their house or in their school. That’s been a big benefit and something I’m just learning to take advantage of.
Dr. Sharp: Sure. We found that remote assessment was a pain in the ass in a lot of ways, but it was also really nice to be able to see into people’s homes and get kids in their natural environment and hopefully a little more comfortable. You get to see the room. You get to see the state of the house and it was beneficial in that way. It sounds like y’all capitalize on that too.
Dr. Jen: Yeah, I never did go to remote assessment for most of my work. I’m a neuropsychologist in the hospitals. We were able to see kids pretty quickly here during COVID, but for this population, [00:18:00] for the specific group that I will offer a virtual in-home assessment for when the kids are not able to speak, they don’t have yes, no responses, they don’t do any choice-making. So those kids, I can walk their parents or their teachers through some things so I could observe enough to feel confident to give them that enough information and consultation and diagnosis just from interview and observation, because I know even if I had them in the room, I couldn’t do any standardized testing with them.
Dr. Sharp: That’s fair. So when you’re thinking through the assessment process, what approach do you take? Is this a pretty comprehensive battery or are you trying to get a sense of a variety of domains or are you really focusing on only intelligence or academic or social? [00:19:00] How do you approach that?
Dr. Jen: It really could vary. For the kids with severe cognitive impairment who come in maybe as a teenager and I know ahead of time that their functioning is somewhere under 3 developmentally, I know I’m not going to be doing a full battery of testing. Even if they didn’t have motor impairments, I wouldn’t be testing everything under the sun. I’d be keeping myself pretty focused. But other kids might have just more subtle motor impairments but may have counter-complex cognitive profiles that need investigating. So again, it could be a really broad range, but for the kids with severe cognitive impairment, I’m trying my best to get a full cognitive assessment in as much of a standardized way as I can. And then if I can’t, then I’m moving on to my next step of then what after that?
[00:20:00] Dr. Sharp: Yes. And I think we’ll talk about that then, but yeah, that’s a big part here. I got you. I know I’m asking a lot of questions just about logistics, but again, just super curious about what this process might look like. Are you testing over multiple days or a specific time period or only a couple hours at a time or a couple of minutes at a time, how’s this?Dr. Jen: I always start with the idea that I could get an assessment done in one day and that’s my goal always. Many of the kids are coming from far and it’s not convenient for them to come for multiple days. So I try to get in and out as quickly as I can. I’m just always open to rebooking them for other days if I need to. I just start with one and then I go from there. And that’s another benefit of these kids being seen in their schools because the school psychologists have a much better ability. They do it over three days and pull them out [00:21:00] of their classroom for an hour at a time. Then that probably works better for everybody. And they get to see them over multiple days, which is really nice too. If I happen to get them on a day that they didn’t sleep well or something, then I may be getting a different picture.
Dr. Sharp: Yeah. I want to make that explicit. I think this will be in the introduction as well. In case people aren’t picking up, that’s a big part of your job, right? It’s like helping other professionals do these assessments in… and what’s the word I’m looking for? …research-supported way or best practices, which is great.
So we’re talking about your practice, but you also help clinicians do this when you can’t get to them or when the kids have to travel or can’t do it or whatever it might be.
Dr. Jen: Yeah, exactly.
Dr. Sharp: Okay. Just to make that super clear. So, we’ve got some logistics and just background for [00:22:00] how you might set up the assessment. Are there any considerations around how you set up the room or what the setting looks like that may be different than typical?
Dr. Jen: Yes, definitely. So one thing that I do really differently in this population of kids is I will almost always have a caregiver in the room with me. Kids who have significant impairment of cognitive-communication motor impairments, many of them would be understandably wary of strangers. And I just put myself in their shoes and think, I’m put in a strange room with a stranger that I don’t know. I can’t get up and leave because I’m in a wheelchair. I can’t even speak and tell them I don’t like something. And they’re asking me to do things that I have no idea what they’re asking me to do. So I can just really imagine how that might [00:23:00] feel for a child and how uncomfortable they may be with that.
I would have the parent. Occasionally an EA has come to an assessment, an educational assistant who is somebody who works really well and gets this child gets the best out of them every day. And they are able to come to the assessment and help me get the best out of them. So that’s been occasional, but usually, it’s the parent. So that’s my number one.
But also it’s the physical setup of the room. So if the kid’s in a wheelchair, obviously we need to be wheelchair accessible. We have special tables here that go up and down and you can fit a wheelchair under them. So, if you look under your desk and if there are any legs there, up and down, vertical legs, then you can’t fit a wheel tray under there. It’s going to be hard to do material work at the table with the child. So in that case, if you’re in that setting and I sometimes do tests out of my main office, then [00:24:00] most wheelchairs will have a tray but parents don’t always bring trays with them. So you just have to tell the parents, bring your tray for the wheelchair because then at least you’d have an ergonomically correct place to work with the child. Those are the big setups.
A lot of kids have difficulty with trunk support. So hopefully, they will come with their appropriate seating and their wheelchair. It’s often some kind of strap that would help them stay upright. And this is something I found really striking when I started this work is appreciating that kids with these neuromotor conditions, they’re putting a lot of effort into all kinds of things. People who don’t have neuromotor conditions don’t even think about breathing and upright trunk support, walking, talking, these things take really quite a lot of effort for some kids, and then how much effort do they have leftover to do the other tasks that you’re asking me to do? [00:25:00] So that’s why we do need to be aware of all of those things to optimize.
Dr. Sharp: Right. I have two questions out of that just going all the way back to the beginning with having someone else in the room with them. How do you get around, and I know there’s a precedent for this and maybe other approaches, but just test security, that sort of thing?
Dr. Jen: Most of the time when I’m doing these assessments, I’m not doing level B assessments. It might be more of a developmental assessment or a psychology unstandardized assessment, but sometimes I am doing standardized tests. You’re right. But it’s something that we know that we’re not going to get a good assessment of a kid.
Sometimes a preschooler or a child who’s very anxious needs their parent in the room or they’re just not going to stay with you. [00:26:00] Like you said, there are precedents for this and your test security obviously, that’s always good to mention to people that everything should be kept in the room. But the main thing that I think if you are doing standardized tests and this is a hard part, is to not get them to help their child because they do want to check your notes. I always tell parents, I know your job normally to help your child and jump in there and give encouragement, but for this part, I know it’s really hard but I’m going to ask you to only get involved when I need you or ask you to, and that can be difficult for parents.
Dr. Sharp: Right. Yeah. The most experience I have with something like that is during the earlier modules of the ADOS where the parents are supposed to be in the room. And even though we always say, don’t guide them, don’t tell them to do anything, it’s so hard for so many parents [00:27:00] not to jump in, “Hey, what about that? Did you see that? What about this toy? Hey, look.” It’s tough. I can only imagine in this setting it’s just as challenging if not more.
Dr. Jen: But sometimes I need them actually because the child won’t bond me and they will only respond to the parent. So I do need sometimes the parents to do things with them so that I could see what they can do.
Dr. Sharp: So I know again, just getting in the weeds, but there are so many specifics that are interesting to me about this type of assessment. What do you say to the parent? Is there a disclaimer at the beginning? Is there a, “Hey, don’t say anything unless I ask you to like”? How do you navigate that with the parent in the room?
Dr. Jen: Yeah. If I’m doing standardized testing, I do spend a little time talking to them about that saying, I know it’s really hard, but I’ll ask you to… [00:28:00] for example, if they get up and you want to help settle them back at the table, that’s fine. But the normal things you do with your kid are to help them and encourage them and help them to learn, today is not about that. Today is just about you get doing what they can with them.
I sometimes will give them a piece of paper and a pen and I’ll say, do you have anything you noticed that you want to tell me about like, oh, they normally would know that one or something like that. I’ll just say, write it down on a piece of paper and then afterward let’s talk about it so that they’re not like interrupting at the moment to tell me those things. And some parents have no problem. They just pull up their phone and they don’t even look up the whole time. And other parents, like you said, it’s really tough. And I think I try to tell them I appreciate it. I know how hard this is but it’s important that [00:29:00] we try to keep it just the two of us doing the activities.
Dr. Sharp: Right. And I wonder I could be generalizing here, I’m sure I’m generalizing, but you have mentioned a lot of these assessments, a lot of these kids are working with intellectual disabilities or cognitive delays. So I’ll try to formulate this question the way that I’m thinking it, which is, is there any amount of emotional management with parents coming into an evaluation where in a way, like how bad is my kid? Like how bad off is this? I think if I were a parent would be like, I don’t know if there’s a whole lot of hope for “strengths” or obvious they Excel necessarily. And again, maybe a gross generalization, but I’m just thinking about it from a parent perspective what that might be like.
[00:30:00] Dr. Jen: Yeah. I’ve seen parents come with so many different perspectives to this, some who are just very realistic, like this is what my kid’s able to do. I look at all these they’re lovely things they have about them. But I know that learning is hard and I know they’ll always need help and support and very frank and other parents who are not there and they are able to really try that…I guess the most difficult times are when parents are telling me that they just know their kid understands everything and really gets everything, but they can’t tell me why they know that. So I do ask a lot of questions about those things. Like, give me examples of when you’ve seen them do this or how they understand. I’m getting a bit off track, but [00:31:00] my approach is always to meet parents where they’re at. I’m not here to surprise you or shock you with some information.
I always take the time, in the beginning, to know the parent’s own perspective about their child and I want to work with them during the assessment, towards the end of the assessment, so that nothing I say at the end is really a surprise to them. They’ve come along with me the whole way. So that by the time I get to the end and I say, you remember how we talked about this? And you gave me this example of this and what we call that is intellectual disability. So we’ve already brought them along and there shouldn’t be any really big surprises by the end of it.
Dr. Sharp: Yeah, I think that’s always the hope. It seems particularly challenging in this population. I think it’d be [00:32:00] tough for some parents.
Dr. Jen: For sure, for some parents.
Dr. Sharp: Well, let me pivot a little bit and back into the assessment process. I wonder if we might look at it starting top-down, like the macro to micro. It sounds like you do start with or try to do a standardized assessment as much as possible. So I wonder if we could walk through a sort of, how might I phrase it, like a decision-making model? How to know when to break standardization or not even attempt a standardized measure. Does that fit for you?
Dr. Jen: Yeah. So I don’t want to make any presumptions to start. So I do want to always have in mind that I would like to try to get a standardized IQ test done if I can. [00:33:00] And what I want to think about is, what are the things you need to do to do an IQ test, like the blocks and block design- you have to be able to see the blocks and move the blocks, right? And then what are the child’s characteristics? So what are the communication and motor and visual and hearing characteristics that might impact on that? And so then I’m trying to think of what’s a good fit for that child. And also developmentally, I should’ve already, by the time I need the child, to have a sense that this is a child very significantly delayed or maybe somewhere in the middle or not. If they are very significant, if I think there’s a chance that they’re under 3 developmentally, that’s where we have the hardest time using our standardized tests.
Some standardized tests do like the Stanford–Binet goes down to 20 and the DAS down to two and a half. Those are my [00:34:00] best bets for the kids who are lower functioning in terms of their general cognitive abilities. So if I can, I would do a DAS. That would be the best because that is both developmentally appropriate and age-appropriate so that I can move them anywhere from their age level all the way down to two and a half developmentally on that same measure and still get a standardized score out of it. That’s ideal.
But some kids can’t manage the demands of something like the DAS. So they maybe can’t look at a page with a bunch of pictures and understand that I’m asking them to choose something. They just don’t understand that concept. And in that case, I might go first to the Stanford–Binet because it starts with a hands-on task where they look at objects and answer questions about objects. So that’s kind of my go-to. And if I can’t do that, then I’m probably moving [00:35:00] away from standardized testing and doing something a little bit different.
The other class would be the Nonverbal tests, especially for kids that have significant communication impairments or are unable to understand or speak. Some of the non-verbal like just though not all of them can be easily adapted to kids so they can respond by making a choice in some other way, other than a moving thing. So like the lighter, you’re supposed to match on the lighter to move a little block from one spot to another, but you can think of other ways somebody can indicate that by pointing or by saying the letter that goes along with it.
Dr. Sharp: Sure. Yeah, I’m just struck by the need to have many instruments at your disposal and the ability to really think on your feet and just [00:36:00] say, okay, this is not working. Let’s pivot.
Dr. Jen: Yeah, exactly.
Dr. Sharp: I hear you. So outside of IQ testing, are there other options? I’m just trying to think of other options for assessing, like, are you trying to even dig into something like the NEPSY or more like other domains, memory, and things like that?
Let’s take a quick break to hear from our featured partner.
The Feifer Assessment of Writing or FAW is a comprehensive test of written expression that examines why students may struggle with writing. It joins the FAR and the FAM to complete the Feifer family of diagnostic achievement test batteries, all of which examine subtypes of learning disabilities. Using a brain-behavior perspective, the FAW can identify the possibility of dysgraphia as well as the specific subtype. Also available is the [00:37:00] FAW screening form, which can be completed in 20 minutes or less. Both the FAW and the FAW screening forms are available on PARiCconnect- PAR’s online assessment platform, allowing you to get results even faster. Learn more at parinc.com\faw.
All right, let’s get back to the podcast.
Dr. Jen: Well it’s again, for the kids who are maybe very motor-impaired but depending a lot on their language, then I would like to test their language abilities and understanding receptive and expressive and verbal memory, absolutely essential. There are also visual memory tests where they can respond with like saying a number. The WRAML is my normal memory test that you need motor skills. You’d be able to use a pencil to do WRAML [00:38:00] for the visual memory. So other ones, for example, the ChAMP, where you just choose from an array. So if they’re able to speak but not use their hands that’s an option for sure.
There are probably tests in every domain if you wanted to that are motor-free and motor-free visual perception tests. There are motor-free attention tests. But I would say that I personally do those too often. I’m usually pretty pleased if I can get like a digit span to measure attention and working memory. The other thing about.. this isn’t obvious. Nothing is true of all of the kids with neuromotor conditions, but often they are not going to last for long assessments. Their attention span is like I said, they’re working hard just to keep their trunk control. And [00:39:00] speaking can be slow and labored and asking them to do that for hours and hours on end, I’m trying to condense my battery to the real essentials that answer the question that they came for. So I’m not often going too far off there, but there are exceptional cases for sure. Sometimes kids come in with more discreet motor problems that you’re answering more neuro-psychological questions about them. And then you need to pull in a whole battery.
Dr. Sharp: Sure. Are you working with kids where you are trying to assess motor skills at all? Like, are you doing like a Grooved Pegboard or something, or is that a foregone conclusion? We’re not even really concerned about that aspect?
Dr. Jen: The Grooved Pegboard is part of my routine in my neuropsychology job. I do it with essentially all the kids, but when they’re very motor-impaired, [00:40:00] it’s not necessary. Honestly, it’s going to be still painful and slow and difficult. They’re going to be off the charts and there are ad scores. It’s not going to be particularly interesting to you as a finding. But I do look for things like, I will often do the VMI because it’s very functional. And I want to know about their writing. I’m looking at their pencil grip and what kind of writing instruments they’re able to use. And then I can try different things with them too.
So it’s more like I’m thinking about recommendations and what I need to know about this kid because for example, pencils, you need to put a bit of pressure on to make the marks, and often these kids with motor weaknesses or difficulties with their hands, they can’t put enough pressure on the pencil to [00:41:00] make marks properly. And a typical ballpoint pen actually needs to be held at a certain angle to work. And sometimes they can’t hold it at that angle and they can’t do it properly. So using a felt tip pen can work and I’ll use that often in my assessments. I’ll let them do the right thing with a felt tip pen better than a pencil. I’m not doing motor testing but I’m doing things like that in my assessment.
Dr. Sharp: Yeah, it sounds like there’s a little bit of ecological assessment almost during that time to see what actually works. That makes sense.
I’d love to dig into this area of non-standardized assessment. So again, I think for a lot of us, this is the wild world west. Like once you go off standardization beyond like testing limits or extending the time [00:42:00] limit or whatever, it’s like, what are we doing here? How do I interpret these results? And is this meaningful? So I guess I’ll ask a general question just around, where do you even start with non-standardized assessments, and then we can take it from there in terms of interpretation.
Dr. Jen: Yeah. So hopefully I’ve tried something in a standardized way, say I’ve completed a WISC but the raw scores are all zeros and 1s, you could make IQ scores out of that but it doesn’t really tell you much. It certainly doesn’t tell you what they can do. So then I’m thinking about what else I can do.
Well, what I’ve seen sometimes when people say this kid is not testable and then they write it up, not testable, intellectual [00:43:00] disability based on adaptive functioning, end of story. And if you have made a really good argument for that, I know our community services here, if you are very clear about the child’s history and you said what you tried in your testing and why doesn’t it work, they probably accept that. It is accepted without an IQ score. They’ll look carefully, but that would be okay.
But in terms of best quality service, I would want to go a little bit farther. So, my next step is to try something that’s a bit more structured than just observation and that would be a developmental assessment. So these are the kind of assessments I didn’t learn in grad school, but I learned after when I started this work at the hospital is to do the developmental [00:44:00] assessments, like the Bayley, the Battelle, the Mullen. Usually, people have their one favorite and they do them. Switching between them isn’t great because it’s lots of parts and moving pieces and you have to know them really well. So what we use most here at the hospitals is the Bayley, just out with a new version, which is quite nice. So I try to get a full Bayley, or at least at the very least of the cognitive measure on the Bayley.
And if I can, many of these kids can make it through essentially a full Bayley. And then what do you get out of that? Well, you get your observations or structured observations of their developmental skills, like object permanence, cause and effect reasoning, and things like that. And then you can get an age equivalent and we know age equivalents are psychometrically problematic for sure. And I think if you’re using age equivalence, you need to be super aware of all of [00:45:00] those limitations of age equivalent.
Dr. Sharp: Can I interrupt you real quick? Just in case there’s someone out there who’s like, why are there age equivalents psychometrically on sound, can you give us the by-size version?
Dr. Jen: Yeah. So age equivalence, I think we say them, we don’t really know what they are, where they’re calculated from. So that’s important first of all. The age equivalence is the age at which that raw score is equal to the median or mean raw score for that age level. So like if a 12-year-old has a raw score of 5 and 5 is the mean score for three and a half-year-olds, then their age equivalent is three and a half.
One issue is that they’re ordinal meaning age 3 equivalent is less than age 4 equivalent. The gap between them is different. So what’s the age equivalent [00:46:00] between the 6th-month gap between 3years and 3.5years, how much different that is, is totally different than between like 7yrs and 7.5years. So you can’t compare age equivalents even across sub-tests on the same test. Many of us have used the Vineland age equivalents and gone, the standard scores higher on this, but the age equivalent is higher on that. It doesn’t really make any sense. How does that even…?
So you can’t compare across subtests within one test. You can’t add them together or average them or anything like that. And they’re also less meaningful, I think, as kids get older. So if you have like a 14-year-old and you’re getting an age equivalent of 2years, are they’re really like a 2-year-old? It just becomes complicated that way in lots of different ways.
The way I think of them is that they’re one more piece of [00:47:00] information, just like my observations and my interviews and my records review. And they should never be turned into your test results like therefore they have an intellectual disability, but I see them as one more piece of information in my broader assessment because our assessment is more than our test results. I like to just think of them in that context, but I do find that useful. And I do find it can be useful when sometimes it’s hard to argue for an intellectual disability without a full-scale IQ because of the way community services are set up and having more structured “tests” `that you’ve given a kid that comes out with an age equivalent is one more thing you could use to help in that setting.
Dr. Sharp: That’s fair. I appreciate that explanation. I know I interrupted you and I think we were talking about just what you can gain from these measures when [00:48:00] you’re… Go on there.
Dr. Jen: So then I hopefully have seen more things about what the child can do, because while we’re talking often about it, it could be a moderate, severe, or profound intellectual disability. I want to be identifying not only what they can’t do, but what they can do. And so doing these kinds of measures will pull that out. You can do them on your own, the pull-out washcloths, and hide things under them, but doing it on the Bayley is just a standard structured way to do that essentially that gives you some guidance along the way.
And sometimes a kid is able to complete a Bayley and you ended up getting age equivalence and sometimes they are not. There’s a lot of parts to that, like motor wise that you just cannot put blocks in a cup and you ask them to put blocks in a cup, like no getting around that. That’s not supposed to be a motor test. It’s supposed to be a cognitive test. And it just may not work. [00:49:00] So in that case, you may be really just pulling pieces from the Bayley and doing what you can just to see what they can do. Do they have object permanence? They’re playing with something, and then it falls on the floor, and do they go look for it? It was just an observation on object permanence, then you go, well, I know object permanence develops around 9 months therefore probably they’ve built that far along. And so I’m then I’m looking for those things throughout the assessment.
Dr. Sharp: Sure. Is there another step beyond that? So, if the Bayley doesn’t work, then is there even more non-standardized assessment to be done?
Dr. Jen: Yeah, so this is the third step when I go to step three. And sometimes you’re not using one or the other you may be using all three of these. So like the standardized assessment [00:50:00] because like I said, maybe it gives you all zeros, but you’ve tried it, you’ve got a score and then you move on to do the Bayley and maybe you get a score or not, but maybe it still didn’t give you enough. Or, there are definitely kids who can do almost nothing from the Bayley and they’re not responsive to command. They don’t have any way of saying yes, using words to express themselves. So in this case, I’m looking for more play-based interactions and maybe use what I can from the kits and stuff. But sometimes it’s more like I need to find what their things are.
For example, I had a kid who didn’t speak and was in a wheelchair and the only thing I could see that he really loved would the spoon he carried with him and he liked the feel of it and he just holds it on his face and touch it and maybe bung in a little [00:51:00] bit, I had his father take it away from him. And stand behind him and ask him, do you want the spoon? Can I give you the spoon? Do you want it? And he didn’t respond or answer, but I knew this was something he really loved and wanted, and even when the father, I have said, can you just say it in one word, spoon, spoon? And even with that, he didn’t respond.
And then as soon as the dad put it to his space, he lit up and grabbed it and he was so excited. So that tells me that, I know he can hear. Hearing is not an issue, but he did not understand even a word for something that he loved and knew so well. So that’s the non-standard assessment. And so a lot of times you do need to focus on that kid. What drives that kid because that’s what’s going to drive them to show you what they know.
Dr. Sharp: Right. Yeah, I know I said this just a few minutes ago, [00:52:00] but again, there is another point that I’m struck by. You really going to be on your toes with this kind of thing. You going to paying attention. It’s not your typical route. We start with block design and go through the WISC. You got to really be in it. I guess that’s an indirect way. I’ll just say it directly that I admire the work that you’re doing and recognize the investment on your part to make sure you do what needs to be done and everything that you can do to honor a kid’s functioning.
Dr. Jen: Yeah. Well, it’s creative, but that could be fun too, to think outside the box a little.
Dr. Sharp: Yeah. Got you. So then is there a step four or it’s step 3 and that’s where we end up?
Dr. Jen: That example of his spoon might be about the end of the line. You’re thinking maybe something that you know that they care about [00:53:00] and trying to do some interaction with it because most of the time we’re using our materials and asking them to do something with it.
There are two people who have thought about this and put together structured ways of reaching kids. They are often around deaf-blind. Kids who are deaf-blind and or very severely impaired. And one thing that I really liked about this model of coherence of helping them to understand the world around them and what they do with them is helping them to… first, they have to just get them activated and interested in something, interact with it, understand it, and then move towards like labels for things. So this about assessment interacts with intervention here and it’s best done [00:54:00] long-term by people who work with them and not me and my one-time setting, but I do try to.
You asked about the last step. The last step would be just, sometimes they come to me, if we really don’t know what interests them, it’s nice to have on hand something from every sense. So things that light up, things that make noise, things that vibrate, things that are hot or cold so that you can try these things with them. And it’s almost like a, wow. The parent may be like, “Oh, they like things that vibrate. That’s new. I could use that.” And then that’s something they can work on to help them along their way in their learning journey. I’m playing a small role there to start that but I am only able to see them on time so I can’t follow up with their intervention.
Dr. Sharp: Sure. That makes sense. So I’d love to [00:55:00] talk with you about the report writing process. And maybe even actually before we get to that, it’s the interpretation of the data that then feeds the report. So I’m not sure how you might want to tackle that but how do you make sense of the data and then communicate your findings, especially when things are non-standardized? What are some considerations there?
Dr. Jen: I think one thing that I get a lot of questions about it I think that’s can be really tough it’s adaptive functioning because it’s something we, especially in these cases of nonstandardized assessment, we’re going to rely on a lot on adaptive functioning measures. What are they like out there in the real world? What are they able to do for themselves and their classroom and their home and their community? And as you think about the items on those measures and a lot of them are dependent on motor functioning or vision or hearing. So brings their dishes to the sink after eating, for example, What if you don’t [00:56:00] use your arms, right? How do you even think about that in terms of adaptive functioning, right?
And then some things are going to develop at different rates, like for kids, with visual impairments, identifying money and bills is just a much harder task than it is for typically developing kids. So they can learn it and they will if they have the skills by adulthood, but it’ll just take them longer to learn those things.
In terms of adaptive functioning, some tricks, or you probably need to look pretty carefully at the item level and think about the child and for example, why they can’t carry a dish to the sink. So you want to see some evidence of impairment on things that are not motor-related as well if it’s a motor issue and you probably need to look at the item level. The social skill, especially it’s good for that and the communication skill as [00:57:00] long as they are able to communicate verbally is also good because those are less reliant on motor skills.
But we also know that the social development of kids with disabilities is different. They’ve had different opportunities for playing, different opportunities for friendships. Their friends may be more like helpers than friends. So that all affect how they get rated on these things and how they develop. So that’s really tough but that’s our job to try to pull that apart and figure it out. And sometimes we may see them when they’re younger and we may not be ready to make that diagnosis yet. They may need more time to come back in the future after they’ve had more development. So I think we might feel unsettled to make a diagnosis for any kid when they’re young for a good reason.
Dr. Sharp: [00:58:00] That makes sense. Yeah, sometimes it’s okay to wait and see or say it. I don’t know.
With your thoughts about looking at item-level endorsements, does that lend itself to say more follow-up interviews with parents or educational staff to really dig into some of those items or do you find yourself doing that more than you might otherwise?
Dr. Jen: Yeah, I would say it’s pretty important that you do this in an interview format, the adaptive functioning for these kids. I almost always do the Vineland interview, and I do the Vineland interview for all ages at different times. Sometimes it can be long and difficult but for kids who are lower functioning, it’s actually not that long to do because you’re not going through the whole item set. You’re looking at the lower items set.
I think it’s really [00:59:00] important that we are aware of what handing these forms to parents feels like to them when we ask them to fill out questionnaires where they are having to write zero, zero, zero, No zero over and over and read questions that are inappropriate for their child, that doesn’t make any sense for their child. And I know that we have to do that, right? Sometimes we just have to do that but it’s important that we at least recognize that to parents that I realized these questions don’t really seem to apply and sorry, sometimes I just got to ask questions even if they’re not relevant. But yeah, Vineland’s interview is definitely the way to go.
Dr. Sharp: So you are making a disclaimer of sorts to parents during the process just saying like, Hey, I’m sorry. This might get tedious or however, you phrase it.
Dr. Jen: Yeah, if I have to have them fill out, I don’t usually do questionnaires like the BASC and BRIEF [01:00:00] for this population. It is just not relevant and parents would get very frustrated having to answer questions that like, do they look both ways before crossing the street? They are like, I push them in a wheelchair. They don’t have to. It seems really silly. I read the room. I think sometimes I don’t need to give too much pre-emptiveness or if I feel like there’s weariness, I’ll definitely explain why I’m giving questionnaires, but it just depends on the situation. It’s good to know how to say that if you need to.
Dr. Sharp: Of course. So how do you translate this to the report and actually, are you writing reports in your setting?
Dr. Jen: Yes, just because it really needed for the school, for the access to community resources. Yeah, for sure, I do write reports. The report format I like the best for, I’ve tried all different [01:01:00] things over the years but when I’m doing these non-standardized assessments, the format that works well for me is where I integrate everything together. So the same things like brief background and what I did in the assessment, any accommodations or things that were done differently.
And then my results section would be a combination of my observations, my interview, my records review, and any standardized tests I did. So like I might say, communication. I might divide into expressive and receptive? And I may talk about what was on the adaptive functioning measure, what the teacher told me, what goals they’re working on in school and what I observed in my study altogether in one spot. And if not long. There’s like half a paragraph, small paragraph, four sentences or [01:02:00] something. You try to pull it together and not a ton of detail. And then I do that for each domain.
And then the very end I do a pull it all together brief paragraph that gives the diagnosis and summarizes briefly the rest of the assessment. And then I always like to in the report in each of those sections, not just say expressive language is impaired, receptive language is impaired, but I say what they can do. So to communicate, they use facial expressions, gestures, and grabbing for what they want. They do not use any word approximations for spoken words. They tried technology for assistive communication and it was not successful. So that might summarize it so that I said what they’re not able to do and what they are able to do in one section.
Dr. Sharp: [01:03:00] I see. I like that that integrative model. It seems like it is maybe more important when you more often don’t have standard scores to fall back on the report, right?
Dr. Jen: Well, standard scores are like the lowest possible. Everything is below the first percentile. Like that’s not useful.
Dr. Sharp: That’s not helpful. Yeah, absolutely. And then recommendation-wise, I assume you’re making some recommendations.
Dr. Jen: Yeah. I would say the big three for this population are communication, sleep, and behavior. And some families may come with most things all wrapped up and they have the goals they’re working on and are working on them in a good way. And they don’t really need much for me but we do need to formalize these intellectual disabilities so they can access supports and services. Great. Okay. I can do that. You guys are good.
And then other times there is [01:04:00] just an incredible amount of things to work on, and it’s about prioritizing what to work on first. And oftentimes communication is a big one because communication can need behavior problems and other things, and in terms of their child’s own the ability to make decisions for themselves and feel good about themselves and have like a voice in the world. Like if they’re not able to even say when they’re in pain, that’s really tough. And you can see all things. Self-injurious behavior, lashing out physically at others, no cooperation and it can be really tough. I always get into detail on all of those three and see where they’re at.
Sleep can be an issue. A lot of these kids have sleep issues that are part of their condition that are just built into their biology. And it can be super difficult for [01:05:00] the family because they’re up often in the night a lot with these kids, and then you get the family stressed. So whatever we can do to help manage sleep can be really beneficial for everyone too.
Dr. Sharp: Of course. My gosh. I’m trying to think what else. I’ll turn it back over to you. I know our time is winding down, but what have we not covered or what feels like we need to say more about it before we start to wrap up here. We’ve covered so many points. I feel like this is just…
Dr. Jen: I think we didn’t talk too much about the difference between adaptations and modifications in testing. So maybe just to briefly mention that and the people could think about it and look in too. The schools use those terms, adaptation, and modification. Well, here is what the terms mean. Adaptation is when you need something in order to be successful and do the same work [01:06:00] as other kids, you just need them adaptation like braille instead of print or extra time or whatever. But modification is like a completely different program. And in terms of standardized testing, I think of like, for example, an easy adaptation would be instead of pointing to the answer, you say the letter attached to that answer.
But some things like say somebody you want to try block design. Maybe the kid tells you how to put the blocks. Oh, put the red corner down in the white corner up and then put a white one, to me, that’s a modification of the test. So for adaptations, I’m pretty confident that we can use the standard normative data. But we have to think carefully because there goes from easy ones, like saying a letter instead of pointing, and ones that are quite different. Like, [01:07:00] moving the blocks for the child. That’s something. To somewhere in the middle, like on picture span on the WISC. There are letters under there if they said the letters.
So in picture span, they see like 2 or 3 pictures and on the next page, they have to pick out those pictures in order. And instead of pointing to them they could say the name of the picture or point or say the letter because they can’t use their fingers to point. That’s an adaptation, but it does change the tasks. It makes it harder. They have to hold more in mind. So it’s subtle out of somewhere between an adaptation and a modification. And if it’s all you got then fine, you can try it. To me, because that makes the task harder, if they do well on it, I’m okay with that. But if they don’t do well on it, then I’m more cautious about interpreting that results.
And [01:08:00] handwriting, giving them a felt pen to write to me is an easy adaptation, you should be able to use the standard normative data. Typing as a little bit more of the adaptation. Some kids can’t handwrite, they only type in their everyday life. Typing is a modification but it’s a reasonable one. You just have to know that you’re comparing this child’s ability to type to every other child’s ability to write by hand.
And then another level of adaptation would be dictation. And so if they’re dictating to you to write down, that’s a full modification, right? Which is really oral expression instead of written expression. So you need to think through where on that spectrum of adaptation to modification. Am I confident in using the normative data for that?
Dr. Sharp: Right. Yeah. It just requires us to be Very mindful and cognizant of what we’re [01:09:00] actually looking at and what we’re trying to measure and what we’re going to use it for. That’s such a good point. I feel like there’s a lot more we could talk about. There were so many questions, that I did not ask as we were discussing but the takeaway from all this for me is just that it is doable to work with kids who may not fit the mold. And it’s like you said that not to be content with could not test intellectual disability by default or whatever, by virtue of no data. Like that’s just not doing these kids justice. And one thing that may go without saying but we’re going to say it anyway, I guess, is all these adaptations and modifications that you might make, I assume you’re documenting those in the report.
[01:10:00] Dr. Jen: Yeah, of course. And I would say that if you’re using modifications, especially if they’re tools, they should be tools that the child is already familiar with. For example, somebody asked me once, oh, I wonder if I could try iGave with this kid who doesn’t, and I was like, well, do they use iGave normally, because if they don’t, then you cannot because that’s something you need to train any kid. They need to learn to do iGave stuff. So just being in good touch with their team who knows them well, it’s an easy start.Dr. Sharp: Right. That’s awesome. Well, thank you.
Dr. Jen: You’re welcome, my pleasure.
Dr. Sharp: This is lots of information. I feel like it was a firehose of information, but like I said at the beginning, we’ll put it in the show notes. I mean, you have an ebook I think that is really helpful in thinking through a lot of these points, and [01:11:00] you’re doing a lot of good work to help folks get more familiar with these approaches.
Dr. Jen: Yeah, I appreciate that. And it is open access, so anyone can check it out without any payment.
Dr. Sharp: Always nice. Well, Jen, I’m so appreciative of our time today. Thanks so much for being on here.
Dr. Jen: My pleasure.Thanks for having me.
Dr. Sharp: All right, y’all. Thank you so much for checking out this interview. I hope that you found it as illuminating and informative as I did. This was one of those interviews where I could have asked questions forever because Jen is so knowledgeable and so kind, and this is just a fascinating and necessary area to know about.
Let’s see what’s coming up on the horizon. Next business episode, we’re going to be talking about transitioning from [01:12:00] insurance to private pay. And then we have some clinical episodes on dimensional diagnosis and assessment, as well as the Mullen family of instruments. So stay tuned. If you have not subscribed to the podcast or followed the podcast on Spotify, now’s a good time to do so you won’t miss any upcoming episodes.
I’m glad to be back with you again. I hope everyone is doing well. Until next time.
The information contained in this podcast and on The Testing Psychologist websites is intended for informational and educational purposes only. Nothing in this podcast or on the website is intended to be a substitute for professional, [01:13:00] psychological, psychiatric, or medical advice, diagnosis, or treatment. Please note that no doctor-patient relationship is formed here. And similarly, no supervisory or consultative relationship is formed between the host or guests of this podcast and listeners of this podcast. If you need the qualified advice of any mental health practitioner or medical provider, please seek one in your area. Similarly, if you need supervision on clinical matters, please find a supervisor with expertise that fits your needs.