This podcast is brought to you by PAR. PAR offers the SPECTRA Indices of Psychopathology, a hierarchical–dimensional look at adult psychopathology. The SPECTRA is available for paper and pencil assessment or administration and scoring via PARiConnect. Learn more at parinc.com\spectra.
For a limited time, you can get one free administration and score report for the SPECTRA on PARiConnect by calling PAR at (855) 856-4266. Just mention promo code S-P-E-C.
[00:01:00] All right, everyone. Welcome back. I’m excited for the episode today. This episode would belong very well in the International Assessment Series that I did about a year or so ago, but it’s also a fantastic standalone episode. I have Dr. Oxalis Jusino with me today talking all about assessment and practice in Puerto Rico.We dig into a number of different topics. We talk about the importance of advocacy and how Oxalis has worked over the years to bring attention to the needs of Puerto Rico around assessment and psychology. We talk about the general climate of assessment and psychological work in Puerto Rico. We dig into the health care system, the [00:02:00] school system, we talk about cultural factors and concerns and hurdles that have been present over the years as far as doing accurate and valid assessment in Puerto Rico.
So this is a very rich episode. I’m sure as you listen, you’ll be able to tell that these topics are very important to Oxalis and quite meaningful in her life.
So let me tell you a little bit more about her before we transition to the conversation.
So she is a licensed clinical psychologist who completed her PsyD at Ponce Health Sciences University. She’s currently an Assistant Professor at the same university. She has been teaching assessment courses for about 5 years, including cognitive assessment, psychoeducational assessment, and Rorschach class. Oxalis has been in private [00:03:00] practice for about 5 years as well doing all kinds of assessment including some forensic work. Before that, she worked at the public school special education system for about 4 years. She has published about remote testing specifically in relation to Puerto Rico and is currently working on conducting formal research into the remote testing environment there.
So again, this is a fantastic conversation. I hope that you take as much from it as I did.
Now, if you’re an advanced practice owner or a hopeful advanced practice owner, I’ll invite you to check out The Testing Psychologist Advanced Practice Mastermind Group. This group is starting in June. It is a cohort of 6 psychologists all looking to take their practices to the next level through hiring or hiring more, [00:04:00] adopting that CEO mindset and just getting support and accountability in advancing their practice. So if that sounds interesting, you can go to thetestingpsychologists.com/advanced and schedule a pre-group call.
All right, let’s get to my conversation with Dr. Oxalis Jusino.
Dr. Sharp: Oxalis, welcome to the podcast.
Dr. Oxalis: Hi, thanks for having me here.
Dr. Sharp: Yes. I’m excited to talk with you. This is another instance of just small world moments where I was talking with Andres Chou who’s been on the podcast and interviewed me a few episodes ago, and he said, ” You should [00:05:00] really talk to Oxalis and see what she has to say. She’s been fantastic.” And here we are. I love these moments when these personal connections work out. And I think we’re going to have a great conversation. So thanks for coming on.
Dr. Oxalis: Thank you for having me. It’s an honor, actually. And it was really a surprise when Andres wrote to me. And I’m like, “Well, okay, yeah, sure. Why not?”
Dr. Sharp: It’s funny. A lot of people have that reaction. I found that whenever I reach out for interviews, a lot of people all the way from big names in our field or test authors or whomever, like all the way down to just those of us who were doing private practice, a lot of folks kind of have that, “Who me?” kind of thing going on.
Dr. Oxalis: Yeah, the imposter syndrome it’s called, I [00:06:00] think.
Dr. Sharp: Yes.
Dr. Oxalis: It’s like, ” Why me? Yeah. Okay, sure.”
Dr. Sharp: Yeah, exactly. I think it is a testament to how strong Imposter Syndrome can be.
And on the flip side, so many of you all have just this amazing expertise that needs to be shared with the testing world. So I feel fortunate. And I’m excited for our conversation today as we are putting it together and conceptualizing it. I kind of started out thinking about really just like Spanish Language Assessment. But as we talked more, it really feels like we’re going to focus not just on Spanish Language Assessment, but sort of Puerto Rican culture and what it’s like to be a psychologist there and some of the challenges that you face each day. And so I think there’s a lot to dig into here.
Dr. Oxalis: Definitely.
[00:07:00] Dr. Sharp: Yes. Well, let’s do it. So I always ask right off the bat, why is this work important to you?Dr. Oxalis: Here, we have a lot of challenges that we’re actually going to talk about, hopefully. And one of these challenges is how many tests we have that we can use to assess people. And when you approached me for this interview, it was like, maybe we can get this word out. We can maybe work something out that people in the States can actually know that we’re here and that we have these challenges. We have Spanish tests available. But what about the norms? What about culturally [00:08:00] correct tests that we can use to assess kids, to assess adults? And that was my main focus when Andres and you reached out. Maybe we can let people know that we need more. We work here to get more, but sometimes it’s hard when publishers, they don’t really forget about us, but they do. I’m not sure how to verbalize that correctly. But yeah, that’s what I’m trying to do.
And I have this podcast that it’s… we hear it here. Most of my students, because I also teach and some of my colleagues, it’s been getting out the word. And that’s my main focus just to get assessment to another [00:09:00] level here in the Island because sometimes it’s like people just get it over. They don’t give the importance to it that it should have. And sometimes, this is really important. It’s a really important thing to do. And here it’s more therapy-oriented. And so testing psychologists are really rare. Practices that are only do testing, maybe 1,2,3. You can count them with your hand, like maybe single digits.
Dr. Sharp: Jeez. It sounds like, just the way you frame it, it’s like advocacy on an International level almost, right?
Dr. Oxalis: Something like that. Yeah.
Dr. Sharp: Yeah. Well, I just imagine that’s a huge driver. That’s some serious motivation. That’s some serious [00:10:00] meaning for the work that you do.
Dr. Oxalis: Yeah. I was taught from that point of view. One of my teachers became my mentor and I still call her and we still talk and it’s like, you know, we have these conversations and most of them are theoric conversations about what we can do to make it better because our Special Ed program relies a lot on what we do on evaluations assessing. This is my point of view, this is my experience, right? I’m not here to judge anyone, but we could do better. We’re doing a lot of evil evaluations in one day. And I know some colleagues, they can do 16 or more in one day.
[00:11:00] Dr. Sharp: Oh my Gosh!Dr. Oxalis: So we need to get our word out and we need to do something about this because we don’t get paid as much as we should maybe compared to the States. It’s a big difference. That’s one of our roadblocks.
Dr. Sharp: Yeah, there’s so much to unpack already from what you’ve said.
So I wonder if we might just start there with a general discussion of the challenges that you’re seeing with assessment in your area right now? What are the biggest hurdles? What are the biggest barriers that you find?
Dr. Oxalis: So one of our biggest hurdles I think is tests because we rely a lot on the Wechsler tests. We have this WISC-R standardized, [00:12:00] normalized and everything for Puerto Rico. And we used to rely on this test a lot. And to be sincere, it’s probably the best thing that’s happened in psychometrics in Puerto Rico. It was released in 1992. And it was maybe close to perfect. Like, if you want to describe it, it was close to perfection.
We’re in 2021 and sometimes I see people that are still using it. And it’s a 1992 test which was normed in 1989. And I think 1990 because of a hurricane that destroyed us at that time, they had to put it on hold for a little while. So, this test is great, but the WISC-Ris just [00:13:00] the WISC-R, you know? It’s small- the stimulus cards are small, it’s black and white. So, it’s a challenge. Like it has some images that are outdated. You show the kids the image and they’re like, “Well, I don’t know exactly what that is.” You have to know what’s missing, but they don’t even know what it is. So it’s really hard for them to know.
Then we started using the WISC-IV. And there was a lot of research on it that was pointing that maybe we were being unjustly measured by the WISC-IV maybe up to 14 points which is a whole standard deviation. The test is in Spanish so it wasn’t really the language. It was just the [00:14:00] norming. There is actually one research that I read that says that maybe it wasn’t that much of a difference, but it’s just one. All the others were pointing to 14 points.
Dr. Sharp: Can I ask a question real quick?
Dr. Oxalis: Yes.
Dr. Sharp: When you say you were unjustly measured by a difference of 14 points, can you explain to people what that means exactly?
Dr. Oxalis: So maybe if you were going to be, maybe in IQ you should be measuring 115 which is above average, you may be in 101, 100 or less like maybe a standard deviation from the total score- more or less.
Dr. Sharp: And where does that come from? I don’t know if you can answer that question, but is that poor standardization or norming of the measure? How does that happen?
Dr. Oxalis: There are different theories. [00:15:00] One of them is just like it’s a different model and we were comparing this to our last test, the 1992 WISC-R which is a little bit of a bias because of the newer psychological theories and psychometrics, which is pretty valid.
There’s also one of our theories is that the sample that was used maybe wasn’t as represented as… we weren’t as represented as we could have been which was one of the things that people in those times used to tell Pearson because Pearson they own the Wechsler test. And they were saying, maybe you didn’t represent us as well. And they actually tried to do a better job [00:16:00] with the WISC-V, which is the one we’re using now in the US Spanish version, which is different than Spain or Mexico.
Dr. Sharp: Yeah, and I’m sorry to jump in again. Can you, can you explain that a bit? Like when you say it’s the US Spanish version and that’s different than Spain or Mexico, what exactly does that mean?
Dr. Oxalis: So I learned this the hard way and pretty recently. The start of last semester, we were ordering our tests for students. They get a discount and stuff like that from the publisher. And we were changed from US to the Latin American people, which is fine. There’s nothing wrong with that. But then they asked me, “Which test do you want? The WISC -V the US Spanish version, Spanish-Spain version?” And I [00:17:00] was quite confused. I didn’t actually know that it was a different version. Actually, I think we don’t even have canceling in our version. Cancellation, I think we don’t have it in the Spanish US version.
Dr. Sharp: Okay.
Dr. Oxalis: Yeah. And it’s different from the US and it’s actually different from Spain because I think if I’m mistaken, anyone can write and tell me otherwise, they actually do have cancellation. But our test was normed in iPads because it was only going to be available on iPads. And what Pearson didn’t know is that people here don’t like technology a lot.
Dr. Sharp: Oh, no.
Dr. Oxalis: Yeah. So there were like, ” No, we’re not buying that test because we don’t know how to work this thing out.” And then it was like, “Okay, fine. We’ll put it on paper also.” And that’s how we actually got the paper version.
[00:18:00] Dr. Sharp: Oh, that’s a lot. Sorry.Dr. Oxalis: That’s a story I know. Again, there can be different versions of the story but this is what people talk and in the streets. One of my friends, she was in the normalization study and she told me it was all iPad. So the cancellation test couldn’t fit in the iPads.
Dr. Sharp: I see. And the significance then of not having the right version, this is just an assumption and you tell me if I’m wrong, but is that obviously like the Spanish language, culture is different in Spain versus Mexico versus the US. And that’s why this is a big deal that you only have the US Spanish version.
Dr. Oxalis: Yeah. So, like the words we use, they’re all different. [00:19:00] But the test provides the uses of other words, they just put them in parentheses. Like we say, banana in a way, and they say banana and another way, you know. But as long as you know it’s right, it’s fine. But I’ve never had the Spain version in front of me. So I can’t really tell you what other differences it has. I would love to be able to have all these tests available. If I would be rich, I would buy all of the tests and then just to sit down and compare them all.
Dr. Sharp: Of course.
Dr. Oxalis: We’re not Spaniards. We were colonized by Spain. We have a mixture of [00:20:00] Taíno culture, African culture, European, Spain. And then we have US. So we’re like a little mix of people.
Dr. Sharp: Right. You tell me, I’m asking a lot of questions about these tests, but it seems very important. Are these differences going to show up primarily on the say verbal subtests or they’re primarily language-based or are they going to show up on the visual-spatial subtests or other places? Where would you notice the biggest differences between these different versions?
Dr. Oxalis: So I have a very good answer for that but with the Woodcock, which is Woodcock-Muñoz over here, Muñoz is a Spanish word. We call it Batería IV, right?
Dr. Sharp: Yes.
Dr. Oxalis. So at least in version III because we don’t [00:21:00] have a lot of… I don’t think we actually have one study on Batería IV yet because COVID hit when we were actually trying to start all of this. With Bateria III, there’s a really, really good doctoral thesis by a friend of mine. And she found out that in math, all of our students were below average. So we would start off with a disadvantage in the math, this is the achievement part of the Woodcock. So we would be starting out with a disadvantage, right? So most of the kids were like, “Well, maybe they have a learning disorder for math.” But if you didn’t know about the research, you wouldn’t know that we have that disadvantage already.
There’s also [00:22:00] differences in some other scores, but math was one of the biggest ones. And I think that was really good research and very important. It’s a doctoral thesis but it’s published, so it’s really good.
Dr. Sharp: That’s great.
Dr. Oxalis: Yeah.
Dr. Sharp: I see.
Dr. Oxalis: So with the WISC, I’m sure there are differences especially with WISC-IV but the 14 total one, I don’t remember if there was there… I mean, I’m sure there were differences, but I’m not sure which ones, especially.
Dr. Sharp: All right. Okay. So we were just talking about how it’s generally hard to get tests with the right norms and that were standardized with the significant portion of Puerto Rican folks in the sample, right?
Dr. Oxalis: Yes.
Dr. Sharp: So I think that’s where I interrupted and took us down a complete rabbit hole. So feel free to jump back to that if there’s more to say about just [00:23:00] difficulty finding the right measures.
Dr. Oxalis: Yeah, sometimes it is difficult because, for example, if you evaluate a kid today and for some reason, he gets into an accident or something and you have to retest, your choices are kind of slim because what we primarily use is the WISC-V and you’re not supposed to retest before six months, maybe a year. That’s more or less like what they recommend. So, in the States, you have all these amazing measures. And it’s like, “Okay, they gave the WISC, so maybe I’ll get the RIAS or the Woodcock Cognitive that we actually can use, but again, we don’t know how they’re working. We don’t know how [00:24:00] we’re being measured.
And we also have the Cognitive Assessment System, which is, Naglieri Das and Kirby. And that one we can use because Dr. Moreno, which is actually the… she translated the test along with Tullio Otero. They’ve actually worked a lot with that test in Puerto Rico. So, that’s another measure we can use. But if you know that test, you know that there’s a lot of motor skills that sometimes… so there’s this little thing that we don’t have as much available.
And then we have the Woodcock Achievement test but we don’t have another one. We don’t have a substitute for it. So there you give the Woodcock or you don’t. Like you have the, [00:25:00] what’s it called? The KTEA, the achievement? I forget the and tests.
Dr. Sharp: Say that again.
Dr. Oxalis: The achievement test they have for like Pearson […].
Dr. Sharp: Yeah, the KTEA?
Dr. Oxalis: The KTEA. Yeah. I just translated that.
Dr. Sharp: I like yours better. Sounds a lot cool.
Dr. Oxalis: That’s how we call it actually.
Dr. Sharp: Nice.
Dr. Oxalis: And I know you have others like the WRAT something like that which are tests that I’ve heard and I kind of know what they are, but never used them. Some of them are not even available in Spanish. And one of them is I think the KTEA. I can’t even say it now. It’s not translated. It’s not in Spanish.
Dr. Sharp: I see. Yeah, that seems like a big [00:26:00] limitation to not have at the choice of measures that you may need or want.
Dr. Oxalis: Yeah, it’s like, if you don’t like the Woodcock well, too bad, you have to use it.
Dr. Sharp: Sure. Well, it just makes me think of what a luxury it is here in the States to have so many measures to choose from almost to the point that it’s overwhelming. I see people posting all the time about what measure would you use to assess this? And there are like 7 or 8 options and we have to choose sometimes. Y’all have the opposite problem.
Dr. Oxalis: Yeah. I bet it can be overwhelming. But when I read those posts, I read them because I like to learn about tests and what tests are available. And I’m like, “Well, I have no idea what that is. I have no idea. Oh, I know that one but I can’t use it.” Sometimes I say, “Wow, I wish we could have that here.” Like I could just open up a post about, So which measure should I use to evaluate I don’t know what, [00:27:00] ADHD or something like that. And it would be great to have more than 5 measures.
Dr. Sharp: Right. Oh My gosh. So in addition to not having maybe the choice of measures, what do you see as some of the other hurdles or barriers, hard parts about assessment there in Puerto Rico?
Dr. Oxalis: There’s this, I think very personal opinion, but the economy is one of them. When I asked how much you guys charge for assessing over there, it was like, “Wow, I don’t even get paid not even half of that.” So I was shocked. It was like a culture economic shock because I have friends in the States and in the group I also see sometimes like, no, because I’m charging this and I’m like, “Oh wow.” [00:28:00] And I think it can become a very big hurdle because we’re limited with measures. So we’re also limited with the amount of money we can spend on them. Like if I would like to have a Woodcock, I would like to have a WISC, I would like to have one of each, I don’t get paid as much. And maybe not me because I’m a full-time professor and I do evaluations on my other time. But I know people that have their office and everything, and they’re short of measures too because we don’t charge as much.
I know forensic people do. They get paid really well but it’s not as close as what they would be doing over in the States. So it can become [00:29:00] a hurdle or challenge. And that’s why I said there’s no testing clinic per se if you need other things.
Dr. Sharp: Yeah, that makes me curious about the economic and healthcare system as well down there. And this is, My Gosh, there are so many naive questions here, but do y’all use the US dollar?
Dr. Oxalis: Yeah.
Dr. Sharp: Okay. So is it a fair comparison to say that Puerto Rico areas, at least I’m guessing there’s some fluctuation or variability down there depending on where you live but is it fair to compare to say like more rural areas or something here in the US where maybe the cost of living is lower and the rate for services just is lower [00:30:00] to match the standard of living or do you feel like psychologists there are paid even less than they should be based on the cost of living?
Dr. Oxalis: The cost of living here is really high and it’s getting higher ever since 2009 or 2006 or something. We fell into a recession and it’s been getting worse. The hurricane didn’t help, hurricane Maria. The earthquakes didn’t help and COVID is making it worse. Our cost of living is high and our economic barriers are higher. There are really rich people and there are really poor people.
So, yeah, psychologists are not top health [00:31:00] professionals. We’re not rated as that. They actually forget about us all the time. I may be getting into deep waters here, but mental health is not maybe as important as it should be. And testing is way lower than that. So, we don’t get paid as much and we pay a lot for everything. We have one of the highest taxes actually in the US compared to all the States. Our tax is 11.5% sales tax.
Dr. Sharp: Oh my gosh.
Dr. Oxalis: And then we pay for everything that comes into the Island and everything comes into the Island.
Dr. Sharp: Of course.
Dr. Oxalis: Well, the archipelago because we’re actually an archipelago, right? But anyhow, it’s like we have to pay for everything. So the rates are low because [00:32:00] people can’t actually pay for higher rates because it would be unjust for them. It would be really unjust for them. And then the healthcare, most of them, they don’t cover evaluations. Like most of our healthcare, what’s it called? I forgot the name. Medicare covers it and all the advantages. We have a lot of plans that go with Medicare and stuff like that.
And there’s actually some private insurance that covers it but there’s not a lot. Mostly, parents pay privately for the kids’ evaluations and adults do the same. If not, what actually happens is that everyone will go to Special Ed because of the IDEA law. We actually have that here too. And then the government pays for [00:33:00] the assessment and for the services. But it’s not a lot.
Dr. Sharp: Is it like the US where there is quite a bit of assessment happening at school, like a lot of kids can get testing through school even if it might be kind of limited? I guess what I’m asking is, are school psychologists a thing down there where they’re testing kids in school?
Dr. Oxalis: Yeah, they’re, there are a thing. I don’t think there’s enough school psychologist. There’s a law that says that there should be a psychologist in every public school. This year, they actually started trying to get a school psychologist for each school- school or clinical because clinical here to do everything. It’s like, the mother of all the psychologists or something. [00:34:00] But kids get tested. Yes. They get tested through Special Ed and they get tested a lot.
Sometimes what happens is that, instead of every 3 years, maybe 4 or 5 years sometimes,1 they can get lost in the system, but I bet that happens everywhere. But they get tested. But the psychologist doesn’t get paid as much for that evaluation.
Dr. Sharp: It sounds like you’re getting squeezed from both sides. The territory is in a recession so people don’t have the money but measures are expensive and the cost of living is really high. It’s the worst of both worlds it sounds like. That sounds very challenging.
[00:35:00] Let’s take a quick break to hear from our featured partner.The SPECTRA™Indices of Psychopathology provides a hierarchical-dimensional look at adult psychopathology. Decades of research into psychiatric disorders have shown that most diagnoses can be integrated into a few broad dimensions. The SPECTRA measures 12 clinically important constructs of depression, anxiety, social anxiety, post-traumatic stress, alcohol problems, severe aggression, antisocial behavior, drug problems, psychosis, paranoid ideation, manic activation, and grandiose ideation, and organizes them into the three higher-order psychopathological SPECTRA of internalizing, externalizing and reality impairing.
These scores provide a quick assessment of the overall burden of an individual psychiatric illness also known as the P factor. The SPECTRA is available for paper and pencil assessment or administration and scoring via PARiConnect; PAR’s online assessment platform. [00:36:00] Learn more at parinc.com\spectra.
And for a limited time, get one free administration and score report for the SPECTRA on PARiConnect by calling PAR at (855) 856-4266 and mention promo code S-P-E-C.
All right, let’s get back to the podcast.
Dr. Oxalis: I don’t know about it, but sometimes when you become a psychologists, most of us become a psychologist because we want to help. So when someone comes into your office and you really see that they are in need of this is, it’s kind of like, wow, they can’t pay for it. So you go like, well, maybe, I can give you a discount. And [00:37:00] Puerto Ricans are so good at that. We give good discounts for everything. But then we forget that we have to pay for the office and we have to pay for power. And sometimes patients will ask for a discount or clients. It’s like, but maybe you can… you understand that not everyone is as lucky as you are because I consider myself lucky because I’m in a privileged position. But then you say, not everyone’s as lucky as me and they need this. So yeah, you get squeezed from everywhere.
Dr. Sharp. It sure sounds like it. So how would I ask this question? Do you think it’s more of a cultural norm around bartering and negotiating for services?
Dr. Oxalis: Yes, we do that all the [00:38:00] time. This is not psychology-related but my husband owns his car accessories store and people will go there like, “How much is this? $100. I have $80.” Sometimes you go like, ” Well, you don’t go to Walmart and say, I have $80.”
Dr. Sharp: Sure. Right.
Dr. Oxalis: Yeah, we do that a lot.
Dr. Sharp: I got you. So how do you handle that in your practice, like with doing assessment? Do you think most people just accept the offer or are willing to negotiate a little bit?
Dr. Oxalis: Most people will accept the offer. What they usually do is that they just pay in payment plans.
Dr. Sharp: Okay.
Dr. Oxalis: They will give you something ahead and then they will give you [00:39:00] something next session or something like that. And what we usually do is that we don’t give them the report until they pay the full price.
Dr. Sharp: Okay.
Dr. Oxalis: Yeah. And it’s happened that someone owes you $50 or something.
Dr. Sharp: Okay.
Dr. Oxalis: Yeah, it happens.
Dr. Sharp: Right. Oh my gosh.
Dr. Oxalis: So you don’t give them the report. And it sounds kind of harsh. Where I work, you can discuss it with them but then you can’t give it to them.
Dr. Sharp: Yeah, I think we wrestle with that. A lot of us wrestle with that. Like what to do if someone isn’t paying for the service. What about just general practice requirements and training? Is a Ph.D. required to be able to do assessment and testing or what?
Dr. Oxalis: No. [00:40:00] So, people here can work at a master’s level. They don’t need to do a PsyD or a Ph.D. PsyD’s here are really common. I think that maybe we have more PsyD than in the States as a whole. But you don’t have to be a clinical psychologist to assess or a school psychologist. Like you can be a counseling psychologist with a master’s degree and you can go about doing some evaluations. It all depends on what you think you’re competent at. And sometimes, maybe you’re not as competent as you think you are but there’s obviously a board that is looking over us. So, you make decisions based on your competence.
Dr. Sharp: I see. [00:41:00] Maybe I’m a little jaded or cynical, but do you feel like people operate outside their competence or are they self-policing pretty well?
Dr. Oxalis: Well, most of the psychologists here self-police really good.
Dr. Sharp: That’s great.
Dr. Oxalis: Yeah, but there’s always someone, there’s always an industrial psychologist that’s conducting evaluations as a clinical psychologist. I’m giving an example. I don’t know anyone, but giving out an example. People practice outside their expertise area. But overall, we do a good job.
Dr. Sharp: That’s great to hear. Like I said, I’m apparently jaded or suspicious of [00:42:00] people.
Dr. Oxalis: I would be too, actually.
Dr. Sharp: Okay, thanks for validating. I wanted to circle back a little bit and talk about the actual process of assessment. So I think one theme that I’ve noticed in speaking with folks just about culturally and linguistically responsive assessment is a willingness to break standardization or interpretation is maybe a little more qualitative or you make just certain changes and adaptations while you’re administrating these measures. Is that something that you resonate with or that you find you do more?
Dr. Oxalis: Yeah, I would think so. I tell my students all the time with the WISC-V, for example, use the Confidence intervals. Use them [00:43:00] wisely. And don’t think that the score is fixed. And I always tell them, make observations, look how the kid is working, for example, because of the WISC, right? Look how they’re working, look how they process things, observe everything. And when you’re going to give out your clinical opinion, don’t rely only on the test scores. Rely on everything including your clinical judgment because sometimes the tests will say everything’s fine. When you see the kid working, you’ll know there’s something going on.
So yeah, sometimes we do that. We test limits a lot also like maybe the kid can’t [00:44:00] work within the timeframe or something and we give them a little bit more time. I mean, it’s still wrong, right? The answer is wrong, but maybe he got it right but afterwards. And qualitative data is very valuable to us, at least to me and that’s what I tried to teach.
Dr. Sharp: Of course. I wonder, is there any way to operationalize this a little bit in terms of… do you think this is, it almost seems like an oxymoron or kind of opposites, but standardizing the way that you go off script or improvise or look at qualitative data. Is there any way to kind of teach how to do that or is it more just like in the moment you’re going with your clinical intuition and tapping [00:45:00] into the way you’re seeing in front of you with the kid?
Dr. Oxalis: I would think it’s more of a in the moment thing. I don’t think there’s a way of standardizing this.
Dr. Sharp: Yeah. How do you standardize a non-standard administration?
Dr. Oxalis: Yeah. It’s kind of hard to think about that.
Dr. Sharp: Right.
Dr. Oxalis: I mean, sometimes you always do the same thing in your office when you’re doing your thing but I don’t think everyone works like that. Like we have different personalities, we work in different ways. So I’m thinking it’s kind of hard to do that.
Dr. Sharp: Yeah. Agreed. So then when you’re helping or teaching or training someone on assessment, tell me about say the behavioral observations. I mean, are you leaning more on observations in [00:46:00] the moment? And if so, how do you document those? Is that section bigger in your report? I’m trying to get it to some of those… Yeah, just how you […]
Dr. Oxalis: I wouldn’t know how to actually compare maybe that section with other people. But I always try, for example, if you’re doing a psychoeducational assessment, I always tell them, look at how they write, look at what hand they’re using, look at everything. And if you have to ask for the notebooks from the school, ask for the notebooks, see how these kids are working at school, obviously we try to have the teacher’s observations and opinion, try to make this a whole and not to stay [00:47:00] with the test.
I always try to tell them that. And I always try to do that too, to watch how they’re working and how they write. And if they’re writing, left-handed people write differently than right-handed people, right? So just watch them. Look at them. Are they writing the way they’re supposed to? Are they holding the pencil the way they’re supposed to? So that’s more or less what we do. And obviously, behavior, like sometimes the kid goes underneath your table and you’re like, ” Okay” This hasn’t actually happened to me, but I had a student once that ones that told me that she had to assess the kid laying down on the floor. I think it was a TAT or something like that. And it was like, [00:48:00] “Okay, well, if it worked at worked but you have to write it down, you have to let people know that this happened.” And she’s like, “Okay.” Because you changed the whole thing. We usually don’t give out TATs lying down the floor.
Dr. Sharp: Right.
Dr. Oxalis: She was kind of scared that I was going to be all mad. And I’m like, if it worked for the kid and you didn’t get the kid to work it out any other way, it’s fine. We just have to write it down and explain why we think this happened and why we let it happen. There are other tests like the EightOS which is really fun sometimes.
Dr. Sharp: Sure. That’s a great question. No, it didn’t even occur to me to ask about [00:49:00] autism assessment, but how do you handle the EightOS which to the best of my knowledge is not normed in any other language than English?
Dr. Oxalis: Well, I’m not an expert on autism but I’ve helped my colleague who actually assesses autism. And I gave one in English under her supervision because she’s not as fluent in English and the patient’s first language was English. So we gave out that in English. And they were military military folks. So we used US norms and everything was fine. But I’ve actually been with her in some EightOS and she gives them out in Spanish, and it works pretty cool. We sing happy birthday in English, [00:50:00] so it’s fine.
Dr. Sharp: Oh that’s wow! Okay.
Dr. Oxalis: I’m not sure what norms or is there any difference in norms for the EightOS but she uses other measures. She doesn’t rely on me on the EightOS. She hasn’t had a lot of problems with assessing autism.
Dr. Sharp: Okay.
Dr. Oxalis: And there’s this interview… I forgot the name. She always reminds me. …which you give to the parents of autism evaluations and I observed once and it was pretty cool.
Dr. Sharp: Got you. Is that maybe like the ADI-R?
Dr. Oxalis: Yes.
Dr. Sharp: Yeah, that’s true. I don’t know enough about it to know if there’s a Spanish language version, but [00:51:00] that’s a good thing to keep in mind.
So I feel like we’ve touched on a lot of different things and some of the challenges of doing this work. One thing just since it’s topical as we maybe start to wrap up, I’m curious how you all have handled assessment during the pandemic, and if you had options for remote assessment or not, just what that process looked like over the last year.
Dr. Oxalis: Yeah, that’s a really good question. So we actually had this… well, we still have one of the strictest lockdowns all over the world, actually, not only the US but the world. At first, we closed to everything. Everything was closed. You could go out and buy your food and you would come back. So at first, there was no work, you couldn’t make any money.
[00:52:00] And so people started using Teletherapy. And then, as weeks went by, it was like, well, what do we do with testing? Kids still need to get tested. There are some adults that need to get tested. I don’t know if I’ve been focusing on this, but we do a lot more kids evaluations than adults evaluations. So, all of a sudden it’s like, “Wait, what do we do?” And it was like, well, there’s this thing called remote assessment. And everyone was like, this usually happens over Facebook and stuff like that. And it’s like, well, but what is that? Is that even possible? Is that ethical? I’m being a little bit sarcastic, but this is, this actually happened. People didn’t know that you could do that because we don’t use it. We only [00:53:00] measure 100×35 nautical miles more or less. It’s an estimation. So we don’t really need to… we never had this need to assess someone remotely.So associations started giving out their opinions on this. And I remember this letter that came out from the School Psychologists Association, which was what the NASP recommends.
And it was like, don’t test, it’s not valid right now, etcetera.
So, I started reading and researching. And I was actually publishing through my Facebook page. Like, this is possible, and it can be done this way, this way, this way. And teaching.
And then I got [00:54:00] webinars, I signed up. I actually signed up for one of your webinars and so I learned, I wrote, and I published. I was lucky enough to have something published.
And then it was like, wait, maybe this can be done. Not because of what I wrote but generally people were starting to learn. But I don’t think it’s used as much. We’re actually working on some research. But we’re still trying to… we’re still starting. We have very limited time for the research but we want to find out how many people are using it, how much it was used. We’re in a very bad time with COVID numbers right now. So they’re talking about another lockdown. We still need to [00:55:00] work, so maybe with all the information that’s coming out, maybe people will actually start may be using it a little bit more. People are still kind of like, maybe that doesn’t work as much. They’re not convinced yet.
My colleague and I gave out a CE, Continuous Education, and the people that went were like, ” Wow, this blew my mind. I didn’t know this.” And there’s a book out now, Dr. Wright wrote a book. And I’ve been telling everyone, this book you have to buy because maybe educating people, we can get there. But we use Mass, we use the screen, and we use all these other measures. We try our best.
[00:56:00]Dr. Sharp: Of course. Yeah. I think we’re all doing our best. But it sounds like you have a little more of an uphill battle there to convince folks that it is doable.Dr. Oxalis: Yeah, remember I said that not a lot of people are technology-oriented here.
Dr. Sharp: Yeah. Well, that’s okay.
Dr. Oxalis: We showed them Q-global and some of them were like, “Wow, what is that?” Again, I’m getting kind of sarcastic and exaggerating it, but it was like that. Like, “Wait, how do you use that? That’s great.” And I’m like, well, every publisher has a system. They have a platform and it’s HIPAA compliant. You can do this. This can be done. I think we need to teach a lot and we need to spread the word that this can be done.
Dr. Sharp: Of course. Well, I think that’s a nice segue. You mentioned that you have a [00:57:00] podcast and you talked about that at the beginning. So tell folks about this podcast. I was very pleasantly surprised to hear about it once before we started recording. What’s your podcast all about?
Dr. Oxalis: So, COVID has made us reinvent ourselves. And I had this idea a long time ago about educating professionals. Maybe not the general population about psychology, but professionals. My brother has a recording studio and I asked him, “Is this even possible? Do you think people would listen to this?” And then he’s like, “Well, let’s try it.” And I have a very good friend of mine who is still a student, by the way. She’s going to become an intern now, which is pretty cool. [00:58:00] She’s not from my university. We’re just friends from some other source. And I asked her, “Would you like to be part of this project?” And she’s like, “Yeah, I would be honored actually.” And I’m like, “Okay, I didn’t know you would be honored to do this, but fine, let’s do this.”
So we started interviewing experts in their area. My main audience is still students, but I’ve seen more professionals join the cause and become part of it. And what I always say is that we would like to revolutionize the testing in Puerto Rico. Like we would like to make this huge, to make it right, get bad practices [00:59:00] out and more good practices in.
Dr. Sharp: I love that.
Dr. Oxalis: That’s, that’s my main goal. Educate and let people know that we should be doing this right. Actually, that’s the name of the podcast, Assessing the right way, but in Spanish, because personally, I get really frustrated when I see reports from colleagues. And I don’t judge them. Like I just get frustrated and sad that we could be doing this better as a profession, not as an individual, as a profession. We can be doing this better. There is hope we can do this better. That’s what I always tell myself.
Dr. Sharp: I love that. Yeah. Go ahead.
Dr. oxalis: And people here deserve it. We’ve been going through so much stuff. We [01:00:00] deserve it. That’s what I think.
And I was going to add about the remote assessment. We have challenges with technology for patients and the internet and sometimes the power will go out. In my experience, you’re teaching a class, and boom, no power. So those challenges are also present when you’re evaluating people remotely. So a little bit of our reality.
Dr. Sharp: Right. My gosh. Well, I think you have given us a lot to chew on, a lot to think about. I just so admire the work that you’re doing and the podcast, the teaching, everything, all the energy that you’re putting into being [01:01:00] that advocate. And it’s funny, I hear people say like, we want to put our area on the map, we want to put it on the map. It’s like you want to put Puerto Rico on the map for testing.
Dr. Oxalis: Yeah, definitely.
Dr. Sharp: I hope that this might reach some people who can help in that process and continue to kind of further that mission.
Thank you so much for coming on and talking through all of this. We’ll have a link to your podcast and all of the resources we mentioned and all of your contact information in the show notes.
Any closing thoughts? Anything that you’d like to share before we wrap up?
Dr. Oxalis: I want to let people know that this is my personal experience. I’m sure that people will share my experience across Puerto Rico. And I’m [01:02:00] sure there are others that have different experiences. This is my personal experience and I just want to let people know that. And I also understand that sometimes we don’t have a lot of measures because after all, we’re a small archipelago, right? I know that sometimes publishing can be really expensive. So, I also understand that part. A little bit of both sides doesn’t hurt.
And thank you for listening. And thank you for having me here and for doing this, letting people know that we’re here and we’re working and we want to do things right and we want to grow and get better [01:03:00] at this.
Dr. Sharp: Absolutely. And so, I appreciate the work that you’re doing. I’m just grateful that you shared some time with us. It was a pleasure. Thank you.
Dr. Oxalis: Thank you.
Dr. Sharp: Okay. And we’re back. Thank you as always for tuning into this episode. Like I said, this could easily fit in the International Assessment Series, but I really wanted to just release it and get this information out there. It’s clear that Oxalis cares deeply about assessment in Puerto Rico. And I’d invite any of you who are native Spanish speakers to absolutely go check out her podcast. It is linked in the show notes and it could be a wonderful resource for anyone conducting evaluations in Spanish.
Like I said at the beginning of the episode if you’re an advanced practice owner and you’d like to get some support in growing your practice, some [01:04:00] group support and accountability, The Advanced Practice Mastermind might be for you. You can get more information at thetestingpsychologists.com/advanced.
Okay. Thank you as always for listening, y’all. If you haven’t told all your friends about the podcast, I would invite you to do so and continue to grow our listenership and our reach, and spread the word about testing to as many folks as possible.
Okay. Take care. I’ll talk to you next time.
The information contained in this podcast and on The Testing Psychologist website is intended for informational and educational purposes only. Nothing in this podcast or on the website is intended to be a substitute [01:05:00] for professional, psychological, psychiatric, or medical advice, diagnosis, or treatment. Please note that no doctor-patient relationship is formed here. And similarly, no supervisory or consultative relationship is formed between the host or guests of this podcast and listeners of this podcast. If you need the qualified advice of any mental health practitioner or medical provider, please seek one in your area. Similarly, if you need supervision on clinical matters, please find a supervisor with expertise that fits your needs.