Dr. Sharp: [00:00:00] Hello everyone. Welcome to The Testing Psychologist podcast, the podcast where we talk all about the business and practice of psychological and neuropsychological assessment. I’m your host, Dr. Jeremy Sharp, licensed psychologist, group practice owner, and private practice coach.
This podcast is brought to you by PAR. To determine if someone is feigning psychiatric illness, trust the Miller Forensic Assessment of Symptoms Test or M-FAST. This 25-item screening interview helps you assess for malingering. Learn more at parinc.com\mfast.
Hello, everyone, we are back for a clinical episode today. This is a great conversation.
Dr. Laura Anthony is a professor at the University of Colorado School of Medicine, Director of the Executive Functioning Clinic at CU Medicine in the Department of Psychiatry, and the Director of Research for [00:01:00] the Division of Child and Adolescent Mental Health. She’s also a clinical and developmental psychologist. She’s especially interested in developing, testing, and implementing evidence-based practices in the communities where they are needed using the co-development model.
Dr. Anthony is co-author of The Unstuck and On-Target Curricula and Resources, which we are talking all about today. It’s a school-and home-based executive function intervention program proven in multiple research trials. Much of our current research is focused on developing and evaluating this evidence-based program to improve executive functioning, including cognitive flexibility, goal-setting, prioritizing/ planning, and coping skills in neurodivergent children.
The project has received private funding, NIMH and PCORI funding, and through that funding, she’s been able to expand into four age groups in schools, an online parent training program, parent [00:02:00] support videos, and a new CE credit online training model.
So we are talking all about this program, the Unstuck and On Target program. It’s an executive functioning curriculum like I said, and we dig into lots of things related to the program. We talk about what it is, how it was developed, who it’s for, and who can implement it. We spend a good amount of time on the impact for marginalized families and prioritizing being able to deliver the program to those families.
So there is a lot to take away from this conversation, and I think this is a great episode for any of you who work with kids either in the schools or in private practice, or if you’re a parent who has kids and you’re thinking about trying to find a program that can help build executive functioning skills.
And as you’ll hear in the interview, this is even closer to my heart [00:03:00] because Laura is local here in Colorado, and that’s just a cool thing that we’ve got some great research and intervention coming out of the Colorado area.
So without further ado, here is my conversation with Dr. Laura Anthony.
Hey Laura, welcome to the podcast.
Dr. Laura: Hey, it’s great to be here.
Dr. Sharp: Yeah, thanks for being here. I love having local folks. The audience doesn’t know this, but you’re just about an hour down the road at Children’s Colorado. It’s really cool to be talking to another Colorado person. So thanks for taking the time.
Dr. Laura: Yeah, thank you so much for having me.
Dr. Sharp: Well, I’m excited to talk about your program, Unstuck and on Target. [00:04:00] I’m not sure how many of our audience members might already have an idea of what this is about, but let’s start with the question that I always start with, which is why this? Of all the things that you could do in this world with your time and energy and training and education, why this?
Dr. Laura: That’s a really great question. When I was in graduate school, virtually a million years ago, I went to a dual degree program, Clinical and Developmental Psychology. Why I chose that program was because I felt like you couldn’t help kids who something’s going wrong in development unless you knew what development was supposed to be like, right? So my guiding philosophy for my whole [00:05:00] career has been to help kids and their families and their teachers really support children getting to their best developmental trajectory including growing up and then being independent, but also setting your own goals and making your own plans and getting stuff done and getting more of what you want.
And so from that philosophy, I became interested in neurodivergent kids, particularly kids with autism and ADHD, and saw a real weakness in executive functioning. And executive functioning is that brain power to help you get what you want essentially, right? It’s helpful for us to know what our plans are and what [00:06:00] our goals are and to think flexibly so that we can do things like to make compromises and see things another way or see things from somebody else’s perspective.
And so, I got really interested in this idea of executive functioning. I was looking around for evidence-based interventions and there weren’t any. There were a lot of scientific studies and there were supports and suggestions and some really great books out there, but the books and the suggestions for parents or teachers or kids themselves, you have to digest it and figure out what to do. That’s a lot of work. So we made it up.
Dr. Sharp: I love it.
Dr. Laura: That’s what we did.
Dr. Sharp: If you don’t find it, [00:07:00] make it up. Yeah.
Dr. Laura: Yeah, we made it up. I think one of the things that’s really been our secret sauce in Unstuck is that the authors are a multidisciplinary team of people where we all had an interest in executive functioning and different kinds of expertise, but I’m the therapist and developmental psychologist person. We have a neuropsychologist. We have a special educator who’s amazing and she’s the first author on our elementary school version, a school administrator, parent coach, and our occupational therapist.
And so those of us who were there from the beginning were able to really match all of our skills together and our strengths in [00:08:00] very different areas to create something that could just be picked up by anybody in a school building and done with kids to help them. And doing it that way is not easy way. The easier way is to do something that I make up to do in clinic, which of course I did that too and implement it myself, but then we’re really limited in how many kids we can reach with that model.
Dr. Sharp: Right.
Dr. Laura: Yeah. So even though we just made it up, we wanted evidence to support what we made up from the very beginning and didn’t want to release anything without evidence. So that’s been another core part of our philosophy.
Dr. Sharp: Sure. I’m so curious about that process of, how do you go from zero, or I have an idea for [00:09:00] something that does not exist, how do you go from that to now I have a research-supported program that I’m distributing to as many people as possible through a major hospital system? I think a lot of us have those ideas, but the implementation is daunting. I know we could have hours and hours of conversation about that process, but I wonder if you could talk through it a little bit just to give folks some background.
Dr. Laura: Yeah. It is a little bit daunting, but if you’re going to try to stick to your core philosophy, it matters that what you do with kids works and you got to prove that it works first really before you do it with them. So the way we did it was the first one that we developed, the group of us was the elementary [00:10:00] school version, and we wrote together, brought all of our good ideas together, and wrote a lot of lessons. I think the number was something like 48.
Dr. Sharp: Wow.
Dr. Laura: Like really. Yeah. It’s way too many. So we wrote too much and each lesson was also long with lots of activities and lots of worksheets because worksheets are easy. And then we tried it out, those 48 lessons with kids in a school where two of the authors were working at the time. And these were with kids who were all on the autism spectrum. And by lesson two they said no more worksheets, right? And of course, they were right. They were so right. But worksheets are [00:11:00] really easy.
We then very quickly needed to rewrite all of the rest of the lessons because the kids had spoken and they honestly really saved us from ourselves. And so then as that development trial is going on, we collected data on every activity from the implementer and then from the kids themselves. They’re like, no, I didn’t like this. I didn’t like the way this game happened or I didn’t like this activity.
And then at the end of the intervention, the kids made PowerPoint and presented what they felt like they learned from the intervention, what they really liked, what their favorite part was and what they didn’t like, and what they suggested we change. So we all come in and sit down and they do their presentations and again, [00:12:00] we changed everything that they suggested because if the kids don’t want to do it, then what’s the point of having your intervention?
Dr. Sharp: You got nothing?
Dr. Laura: Right. So that’s the way our development trial went, and then we cut out about 20-something lessons and activities and streamlined. And then the implementers, the school people really told us what it was like, they wanted a little more scripting for the concepts that were hard. And that was really helpful. And then that product is what went to our first trial in schools, not where we did the development, but just mainstream public schools.
And by the end of that trial, we had [00:13:00] expected to do a major rewrite to see, out there in the community to make sure that it’s still working because it’s a very different setting. And we didn’t have to do too much of a major rewrite.
There was one concept that was confusing about the heroes, but we integrated that in a different way and decided then after one little trial to go bigger and spread the impact by including kids with autism or ADHD And to translate all the parent materials into Spanish and do it only in Title 1 schools as an addressing disparities trial because poverty can also cause executive functioning difficulties in addition to neurodevelopmental disorders.
So [00:14:00] we went big in that revision, and that’s the one that is now been proven in another trial, a bigger trial. That’s the one that is the basis for everything else that we’ve done.
Dr. Sharp: Yeah. And what was the timeline from the idea in conception to where we’re at now, let’s say?
Dr. Laura: I will tell you, if you’re looking for a quick career, intervention, development, and testing are not the way to go. That’s my advice to you and all of your listeners. We started working together in 2007 and our first version was published in 2011. That was the [00:15:00] elementary school version. We then had a parent manual that came out in 2014. The revised elementary school version came out in 2018. An online parent training also came out in 2018. And then the most recent version of that elementary school one that was optimized for virtual over Covid came out last year.
And now, next year we will have, in 2023, our middle school and high school versions will be launched. Those trials are done and the manuals are ready to be published next year. And also within the last year, we’ve done and tested an elementary school staff training or implementer training. Doesn’t have to be at school, but that is the setting [00:16:00] that we target in the videos. And that is free and was funded by a PCORI grant, Patient-Centered Outcomes Research Institute. And we have a high school version that will be coming out next year of that pre-online continuing educational training that was supported by a generous donor from […]
Dr. Sharp: I see. Wow.
Dr. Laura: Along the way, we got lots of other funding as well because you can’t prove that something works without honestly, a whole lot of money for interventions because it’s expensive especially if you want to prove that it works in the real world. We were doing things like driving around to schools and doing classroom observations. When you’re doing that for 150 kids, that’s a [00:17:00] lot of driving. But if you want to make sure that it’s working out there where the kids need the skills, that’s what you have to do.
Dr. Sharp: Yeah. There’s so much to take from that story. One, like you said, this is not a quick process. I’m sure that there are curriculums and programs that maybe get developed quicker, but y’all have put in a lot of energy and time to doing it right and making sure that it does work like you said. And that’s crucial. That takes a lot of time. And I can tell you that you’ve been very deliberate about that process.
Dr. Laura: Well, the part of the right, it’s not only that it works, but we wanted to make sure that it works for somebody who’s not us because we wrote it, right?
Dr. Sharp: Yeah.
Dr. Laura: So we know what we meant when we said these things in the book [00:18:00] but it doesn’t always read that way if you’re not the person who wrote it. So how it gets implemented in the community is really important and important to pay attention to because that is really how you reach the most kids and help the most kids.
Dr. Sharp: Well, that’s the part that jumps out. I’m glad that you highlighted that and I’ll do it again just to bring a little bit more of a spotlight that that’s the crucial process to me, to make it something that is actually accessible to the most number of kids. This is not a program that is only deliverable by a psychologist. You’re going to talk about this, but I’m going to give a little spoiler that it’s something that almost anyone, maybe anyone could do in a school setting, in maybe other settings. It’s translatable. So that’s [00:19:00] important.
Dr. Laura: Yeah. [00:19:00] And that’s really important because there aren’t that many psychologists around. Like when you think about reaching kids because we’ve got kids in every school who have executive function difficulties. Every school across the country has kids. Even if you’re really small, it’s so common that you’ve got kids. We can’t have psychologists in every school doing this intervention. That’s why we were multidisciplinary from the start and why we embedded it in schools, because most kids in the country are in school, not all, but most.
Dr. Sharp: Sure.
Dr. Laura: And if you’re not going to do schools, the other place that most kids go is to primary care. Not all, but most. But very hard to embed an intervention in primary care that would take this long and it’s not where the skills are required. Schools [00:20:00] are where these executive functioning skills are really required.
Dr. Sharp: Right. Well, I know that we have taken a deep dive here right at the beginning. So I’m wondering if we could back up a little bit and give just some basics of this program. I think folks are probably intuiting what this is all about, but I would love to hear from you a description of what Unstuck and On Target is. Let’s start there.
Dr. Laura: Executive functioning skills, it depends on age, but they encompass a wide range of abilities. Some of them are really not very included in our intervention. So things like working memory, a very important skill and is a real pivotal executive [00:21:00] functioning ability for kids, but that’s not really a part of our intervention. Organizing your notebook or your stuff, and cleaning your room, is not really a part of our intervention.
We picked two of the areas that were particular difficulties, especially for kids on the spectrum, which are cognitive flexibility or getting stuck on things, sometimes it’s called perseverating, but it’s not as nice of a word to me, but having trouble moving on or thinking flexibly, keeping an open mind, thinking about things a different way, and especially how to solve the problems that you have. If your first way of solving it is not working, how do you come up with another way? That’s a really important life skill for getting through the world and [00:22:00] getting more of what you want.
We chose cognitive flexibility as well as planning and goal setting because again, that’s another crucial skill. It’s harder. Both of these skills are very higher-order executive functioning skills, which are higher order in and of themselves for sure, but they do rely on other things like working memory or organization of thoughts or materials.
Dr. Sharp: Can I ask you a quick question?
Dr. Laura: Yeah.
Dr. Sharp: How did you choose to exclude working memory or any of the other executive functioning skills for that matter, but since you led with working memory, I’m curious, how did you decide not to include that or focus as much?
Dr. Laura: So it’s not that it’s not important because it is.
Dr. Sharp: Of course.
Dr. Laura: At the time, that was not a particular area of [00:23:00] weakness for our kids on the spectrum. In fact, some of them have really amazing memory. But it is for some, so again, it’s not that we aren’t thinking that it’s important. At the time, there were several computerized working memory training programs out on the market and those are still out on the market. They do improve working memory in the ways that they’re targeted in the computer program but are not great about generalizing outside of that, right?
Dr. Sharp: Yeah.
Dr. Laura: And so, part of why we took these higher order flexibility and planning goal setting is that they also rely on lots of other [00:24:00] skills, so to accommodate the working memory issues, we do more accommodations and unstuck around working memory than direct interventions.
You write stuff down and have visual supports. You have keywords that enter into the cognitive domain of the kid’s head that can help make, say, a switch. So something like my plan A isn’t working, what’s my plan B, right? If they have learned that in a calm moment, it’s possible when they need plan B, they might be able to rely on that wrote a script that they have learned and worked through and practiced when they really need it in the real world.
So it’s difficult because, I’m [00:25:00] not really sure why I picked working memory, but it’s working for me, so I want to stick with it. If you think about us as adults, if something goes really wrong in your day and your stress is very high, your working memory stinks, right?
Dr. Sharp: Absolutely.
Dr. Laura: Many years ago, I was in a serious car accident. And when that happened, I needed to call my husband and was able to do that because it was on speed dial. The other people or the police asked me what my phone number was, the same phone number I’d always had ever since I first got a phone, I couldn’t recall it. So was it the stress or was it [00:26:00] because I hit my head? I don’t know. I did get a pretty serious concussion. But when you have these skills, that are sensitive skills, in times of stress or pressure or struggle, it is harder to bring up those skills- all of the executive functioning skills.
Dr. Sharp: Of course.
Dr. Laura: Yeah, it’s harder.
Dr. Sharp: Gosh.
Dr. Laura: Yeah. So we want to do things that become habits or automatic so that we can rely on those skills. So like I could have a speed dial version for these kids where they don’t have to rely on any skills. They’ve got this, kind of like having a cheat in a game, something that they’ve practiced, they know really well, and causes this automatic process in their brain that they can rely on even if they’re on their way to getting really upset.
Dr. Sharp: Sure. [00:27:00] That’s great. Yes, this is a program aimed at building executive functioning for neurodivergent kiddos. And tell me a little bit about the delivery and anything about the content that you’re willing to share in terms of what a lesson might look like. Who’s doing it? Is it in a group? Is it individual? I’d love to hear some of those details.
Dr. Laura: The project that we’re working on currently that we developed the online training for that is free, as part of that, we did a trial of the training and looked at how implementation was happening across schools in Colorado and Virginia. So we took two states and they are very different states, but wanted to capture [00:28:00] that diversity.
And so in that, we watched people doing Unstuck out there in the real world which we had done for our previous trials, but this one was more removed because we had given them some training in our previous work. And so it was really cool.
It’s a flexible program. We designed it to be a Tier 2 small pullout group in schools. So Tier 2 is another little part of our special sauce or secret sauce because there are almost no evidence-based, school-based Tier 2 interventions in general. There’s just not that many, much more prevention, Tier 1, and then some great programs that are Tier 3 that are really comprehensive and effective and needed [00:29:00] for that Tier 3, the tip of the triangle that took care of it.
Dr. Sharp: And just to provide a little context for folks, when we’re talking about tiers, we’re talking about the MTSS process, like the Pre-IEP or even Pre-504 intervention for kids. Is that accurate?
Dr. Laura: Yeah. Tier 1 would be prevention and universal for all kids. Tier 2 would have kids with 504s or IEPs but not necessarily depending on whether the pullout group is during academics or not. To pull a kid out from academics, you generally need some special education and to do that. And then Tier 3 is some kids who are in a special class or a special school who have more intensive [00:30:00] needs.
Dr. Sharp: Thank you. I’m clearly not a school psychologist. Thank you for straightening me out there.
Dr. Laura: Yeah, so that’s Tier 2, where it’s kids who have spent some time in the mainstream or Unstuck but get pulled out maybe once a week or sometimes twice a week for an unstuck group. And in that case, it can be run by anybody in the school, anybody who has time, which is generally the biggest issue. And it’s usually the same kids in the group for the whole school year, for the whole duration, which is about 21 lessons. The lessons take about 30 to 40 minutes each at most. Sometimes they repeat lessons if they want to go into something more in-depth or have extra fun with something.
So that’s the way [00:31:00] we designed it. What happens out there in the real world, we suspected and discovered everything from an individual model where people do it individually with a kid at school. And that could be maybe because the student can’t really function well enough in a group with other students or something about their schedule to work out, so can’t combine kids. It’s harder to schedule than you think for some schools.
And then we also saw full classroom models of Unstuck where they did it with the whole class. We saw both whole class and special education, but more often whole class mainstream education where there are some kids in there with particular executive functioning problems maybe, but [00:32:00] also maybe not. So they are really thought of by some of the implementers as just crucial life skills for independence and happiness.
Dr. Sharp: I like that. Yes. These are crucial life skills, right? You don’t have to have weaknesses in those areas necessarily. I feel like all kids and many adults could probably benefit from having these skills.
Dr. Laura: Yeah.
Dr. Sharp: Give me a little taste of what a lesson might look like. Is it mostly, I don’t know, lecture-based is the wrong term, but you see, like delivering information. Is there an interactive component? Are there worksheets at all?
Dr. Laura: There’s no worksheets.
Dr. Sharp: Completely get rid of that. How does this look?
Dr. Laura: The only worksheets are occasionally when there’s a game or something that involves them.
[00:33:00] The first thing happens in a lesson is a review of last time’s concepts and that’s done as a lightning round. So questions, and then the kids raise their hands or hit a bell or whatever. So it’s fun. It’s quick and they earn points for getting things right, for knowing the answer. And teachers are instructed to make sure to spread them out, spread out the opportunities. And then there’s not a lot of didactal because…Dr. Sharp: That’s interesting.
Dr. Laura: Well, there’s more didactic with the older kids. As you’re getting older, especially into high school and transitioning out into the world and you’re older and able to [00:34:00] handle more of the didactic piece, there is more of that there, but for the elementary school version, it is much more interactive. So there’s gains or fun activities like in the beginning to say why be flexible? What does it get you by being flexible, even when it’s hard sometimes.
They learn the story of Silly Putty and how it was invented, and then they make Silly Putty or sometimes the teachers who don’t want the mess will buy Silly Putty for the kids to take home to remember, like sometimes your plan A is not the best way. And the person who invented it actually didn’t intend for it to be a toy. They intended it to be a replacement for the rubber shortage during a [00:35:00] World War. It just didn’t work for that at all. It was too gooey but it’s pretty great as a toy and somebody else picked that up and figured it.
So, there are lessons there, but they are much more interactive. So all of the content is the kids and the teacher; the kids talking to each other and the kids talking back and forth will be with the teacher and practicing. So actively practicing all of the skills in a calm moment when they’re not in high demand. If you practice, practice, practice, then they’re more automatic for when you need them without having to give too much thought once you’ve created those new habits.
Dr. Sharp: Yeah. Can you give me an [00:36:00] example of a skill that might be practiced and how a kid might practice it in that particular lesson?
Let’s take a break to hear from our featured partner.
Determining if a person is feigning psychiatric illness is critical in certain evaluations. It’s essential to have an instrument that you can trust and provide fast and reliable results. The Miller Forensic Assessment of Symptoms Test or M-FAST is a 25-item screening interview for adults that helps you assess the likelihood that an individual is malingering. Learn more at parinc.com\mfast.
Alright, let’s get back to the podcast.
Dr. Laura: Yeah. So when kids are learning about the value of making multiple plans upfront. So if you have a goal, it’s important for you to [00:37:00] know why you want to reach that goal, right? And this is why the kid wants to reach their own goal, not why the teacher wants the kid to reach the teacher’s goal, or the parent wants the kid to reach the parent’s goal. It has to be the kid’s own goal or else it doesn’t work. What’s the why? What’s the motivation? And then in our process, you make a plan A and a plan B at least. And then you see how it went to evaluate your plans and see if you need a new plan after you try it out.
To learn the value of a plan A and a plan B, we have things like watching a Pink Panther video where he’s trying to cross the street and make track, that is on a piece of paper that they can also do it on the board, or it’s not exactly a worksheet. As they’re watching Pink Panther try across the street [00:38:00] in all of these ways, that’s busy street, it’s not working, then they see how many plans he gets through. I don’t remember the answer. It’s something like, K.
Anyway, it’s a funny video and a fun way to, that’s not copyright protected anymore, so we can include it as a suggestion in the curriculum and kids are learning in a way that’s active, but it’s also fun.
Dr. Sharp: Yeah, I like that.
Dr. Laura: So then they see the value and eventually you get to your goal and that’s what you’re trying to do all along. You’re trying to get to your goal and there’s sometimes more than one way to reach that goal. So fun there.
Another thing we do if they want is the endless obstacle game, where the kids come up with options for something and then [00:39:00] the teacher might get more and more ridiculous about the obstacles that are coming. Like, you might end it with aliens come and I do not know, whatever, take over the planet. And it’s like, well then you won because you got through so many obstacles. And so it’s a fun out loud activity.
We want people to try to make it fun because otherwise, it’s just hard. Kids don’t love that, the computer programs for working memory, they don’t think they’re tons of fun. So the kids told us to make it fun, so we really tried, and then they told us which ones were fun and which ones were annoying. You got rid of the annoying ones.
Dr. Sharp: It’s so valuable to get that direct input from the kids themselves. I don’t know how many [00:40:00] programs do that.
Dr. Laura: That’s amazing.
Dr. Sharp: Yeah. That’s great. I just have so many questions about this, so forgive me for peppering you with questions here, but then is there any translation to home? So if the kid is doing it in school, does anything happen at home? Is there any coordination with parents or is it siloed in the school setting?
Dr. Laura: That’s a great question. Every lesson has a home practice sheet and a classroom practice sheet. Parents and teachers said, don’t give us stuff to do and don’t make it like homework for the kids. There are also a few “report cards”. Well they’re not getting grades, but they’re saying, here’s the things that your child really picked up on and liked the best and did the best with. So try this stuff at home.
[00:41:00] During Covid, we got some extra money from PCORI to create some free videos for parents. So we have 13 videos. No, I’m sorry. We have 16 videos now because parents asked for a little more. Those have been through trial and those are free on our website and on YouTube. Five of those originally were in Spanish, but we are now in the process of getting all 16 in Spanish and their parent tip sheets. And so that’s a light touch. Your kid doesn’t have to be doing unstuck for those videos to give you little strategies.And then we have a more intensive, we have the book that’s for parents or caregivers for applying things at home. And that has things like example, IEP goals and things that parents might find [00:42:00] helpful. And then we have a more intensive online parent training that also went through trial that’s very individualized for each parent. Or they pick what they want to see and then build things that parents get worksheets that are electronic that they are working on around their own understanding accommodating and teaching these skills for their own kids particular area of difficulty.
Dr. Sharp: Yeah. So that addresses one of my other questions is, can it happen at home? Can kids or can parents implement this intervention on their own? Could we implement this intervention in a private practice? So that broader question of can it happen outside the school, it sounds like.
Dr. Laura: Absolutely. And there are a few more studies that are happening in private practice or in clinic-based that [00:43:00] show that it’s also very successful. And here at Children’s and at University, we do unstuck groups here as well. But they’re being done by people around the country. And it doesn’t have to be part of a school.
Dr. Sharp: Got you. Well, and it just calls back to one of the things we talked about earlier, which is this very deliberate way of writing it so that anyone could deliver it. I feel like there’s so many interventions, for better or for worse, where you have to be “certified” to do it. And that’s not the case here. So it is just another step of, I don’t know if decolonizing is the right word, but you know what I mean. Like decoupling it from having to pay for a certification, holding that behind any number of [00:44:00] obstacles or hoops for people to jump through, so that’s very valuable.
Dr. Laura: Yeah. If we had had evidence that we needed to do that, then we would have.
Dr. Sharp: Great point.
Dr. Laura: It’s not how we designed it. We designed it to be able to be done by anybody, but if we had found out that you needed to have near-perfect fidelity in order for it to work, then we would have needed to require near-perfect fidelity. But that’s not what we found. We found that actually the people who are doing it are doing a great job with it. And when they make adaptations to make them fit better in their school, or with this particular student, or with this particular group, the adaptations are […].
They are really consistent with the purpose of the intervention and just make it [00:45:00] work better. It’s how we designed it. If we didn’t have a choice, we would’ve absolutely done it a different way, but especially like psychologists or social workers or people in private practice, they can really just pick it up and do it.
If they want more, they can do the free online training. And they can get a certificate, and they can get CE credits, but they don’t have to. We don’t have any evidence that you need to do that in order to do Unstuck well. Really, we just wanted to give people options for what way they learn best or what way they implement best and to just try it out. Just maybe feel more confident with like, oh yeah, I can do that.
Dr. Sharp: I love that. It’s got me thinking about all of the [00:46:00] trainings or certifications that are out there that may or may not need certifications. Now, I don’t want to necessarily go down that rabbit hole, but I’m just verbalizing out loud. Oh, that’s really interesting. I wonder how many of these interventions actually need that? I don’t know.
Dr. Laura: Yeah. When we were doing some online coaching to evaluate how the trainings were working, we did have questions from implementers, like, I don’t want to mess it up. So we have a coaching session that’s also free online now. People can look at it. You can’t mess it up. It’s hard to mess it up. Other than forcing your own agenda on a kid, like your own goals or telling them that they don’t feel the way that they feel. Not using the coping strategies that are in Unstuck. There’s [00:47:00] those big ones, but other than that, it’s pretty hard to mess it up. We have no evidence that anybody messed it up at all.
Dr. Sharp: My safety net. Geez. In a world of high achievers wanting to do everything right, that’s a blessing to have a little leeway to not mess it up. That’s great. Well, you mentioned earlier, I definitely want to talk about this whole idea, the attention that y’all put toward helping or serving marginalized kids, kids in lower-income schools. I would love to just dive into that and translating to different languages. It seems like I put a lot of energy into making this accessible and making sure that it is helpful for a wide range of kiddos and families. So could we dive into that for a bit?
Dr. Laura: Yeah. When we were writing this grant to extend the reach to [00:48:00] more kids whose families live in poverty or to Spanish or even to ADHD, in writing that grant, I’m like, we’re going really big because we had no evidence that we couldn’t, we also had no evidence that we could, so we went big. And I have to tell you, I’m passionate about what we do, but I am also very skeptical as the science part of my brain.
This is just a secret. I never thought it would work. I never thought it would work the first time. I didn’t think it would work in the next trial. I just never. I’m skeptical because as a psychologist, I know it’s hard to get people to change. [00:49:00] I do. So we went big and we were told by some of our consultants that it’s going to be really hard to get these kids who are experiencing multiple disparities.
In that trial, it was just around the DC area. DC Northern Virginia, only half of our families spoke English at home. We spoke primarily English at home. Right? We had a great range of income but we thought it would be harder to get those kids and their families to change and maybe even their schools. What we found was that was so not the case. We [00:50:00] had families and schools clamoring to get into the study because they wanted something to do to help, and families loved it that their kids could be in an intervention at school that was specifically designed to help them.
And we found that parents, just by the nature of having their kids in Unstuck, felt less stress and burden. And they felt it was helpful for their kids. In fact, another impact parents could, when we said how much did Unstuck help your kid at school or the other intervention, which was a behavioral in a traditional behavioral intervention, how much did it help your kids at school? Parents reported that on that one question. And it was highly correlated with our masked classroom [00:51:00] observations of the kid, how much we thought they were helped over the year at school which was pretty cool, really shocking. We didn’t think we were going to get that.
So what we found was not only a real welcoming of new interventions in the schools that are serving kids who are experiencing disparities. We also found a great effect and a powerful effect size that was not influenced by income. And that never happens. In this country, income determines just about everything for a family or school. It’s really powerful.
Dr. Sharp: That’s is incredible.
Dr. Laura: And [00:52:00] income was not an influence. I could retire now because of that finding. We were an addressing disparities trial and we addressed disparities. I’m so proud of that. I’m so proud of our team that we were able to create something that did that.
Dr. Sharp: Of course.
Dr. Laura: I don’t think I am going to retire now, but I could have. It’s like there’s just no bigger gold star than that.
Dr. Sharp: Yeah. That’s pretty incredible. There’s lots of incredible elements about that but one that jumped out is, at least as far as I know, there’s a fair amount of research in the ADHD world that you’ll get pretty big differences in parent rating of improvement versus teacher rating of improvement, which I’m guessing is classroom observation. So to have those two be fairly [00:53:00] pretty highly related is impressive in and of itself. And then of course, the income component, you should be proud. It’s amazing.
Dr. Laura: Yeah. So proud. I think probably the next grant I write, I think I’m probably not going to be so skeptical because of those findings.
Dr. Sharp: Yeah, it gives you some confidence.
Dr. Laura: Yeah.
Dr. Sharp: I love that. And there’s so many, gosh, we could talk so much about that, just the way that y’all designed the program so that it didn’t feel like another burden or obstacle for folks to take on and congratulations.
Dr. Laura: So one of the things that we’ve done with the Colorado Virginia implementation project,is we also got some extra money to do a cost estimate. And because it’s in [00:54:00] schools, we’ll be estimating a dollar cost, but more importantly, it’s a time cost. How long does it take? What we found, this isn’t published or peer-reviewed yet, we’re just analyzing the data, but it looks like it takes, as you’re implementing about an hour or two a week, you implement it.
So the other thing is that to be getting effects with a small amount of burden on the school. And that was for the first time that they run it, if they run additional groups all or the next year, if they run it the next year, it’s then like no preperation time at all.
Dr. Sharp: Yeah. That decreases.
Dr. Laura: Like, you don’t even have to like read it in advance because you just remember it. So [00:55:00] that’s a low dose, something else I didn’t think was going to work. That’s a low dose. That is not a lot of intervention especially if kids are really struggling.
Dr. Sharp: That’s fantastic. Tell me a little bit more, I should have asked this earlier, but I’m curious now that we are getting into it. What kind of outcomes are you looking at in the classroom that are improving with the program?
Dr. Laura: Here’s another thing I never thought would work. I know. I said, we’re just going to drop into classrooms. That’s what we’re going to do. Could be any class. It just needs to be academic. They can’t be doing a standardized test or that kind of thing. So you drop into an academic class, you need to be there for at least 15 minutes. And you need to see at least one transition, either from one class to another or from one activity to another, like [00:56:00] from seat work to group work or something like that.
So those are the requirements and you have to be able to hear the kid, but other than that, stand away and watch quietly. And then we’re coding things like how does the kid transition? Do you see the kid getting stuck on something, repeating something or getting stuck? Do you see any signs of negativity or overwhelm? Like anything where you see the kid looks overloaded or they’re expressing negative things, are they reciprocal? Are they participating? We have six behaviors. Was that six? I should have been counting. Sorry.
Dr. Sharp: I wasn’t counting either.
Dr. Laura: I know. I think the one more is […]. Are they following the classroom rules and are they doing what they’re supposed to be doing? [00:57:00] And so it’s simple and it’s basically just, yes/no. We started out with a rating scale that took about an hour and had 30 or 40 items on it.
Dr. Sharp: That sounds familiar.
Dr. Laura: You can’t help it sometimes if you’re a psychologist because there’s all these things you can see in the classroom, but then teachers told us what they really cared about. It was interesting. Kids also told us what they cared about. They didn’t want to be feeling negative at school. They want to participate. They don’t want to have meltdowns. They don’t like them. Nobody likes them, but the teachers and the principal, they don’t like them either, parents certainly don’t like them, but the kids don’t like them either.
Dr. Sharp: Makes sense.
Dr. Laura: Yeah. [00:58:00] So we picked the ones that we thought were higher level and mattered the most to schools and kids. And so those were the ones.
Dr. Sharp: Love it. Yeah. Keep it simple. And I know we mentioned the Spanish language components. Is it translated into other languages or stuck with Spanish so far?
Dr. Laura: Well, it’s not totally translated into Spanish yet. We’re working on it.
Dr. Sharp: Nice.
Dr. Laura: Hopefully, we’ll be having a trial in Latin America and maybe Central America. And we have a publisher that’s interested in doing that. We also have in process Unstuck in Italy and in Chinese. So Italian and Chinese in those [00:59:00] countries. And we are asking for a cultural adaptation process when necessary. And you should try it out. Not maybe so much like we do because that is super expensive but we do want people to make sure that we don’t need to do any major cultural adaptation. But it is true; our heart is really prioritizing Spanish because we have such a large population here in the US.
Dr. Sharp: That makes sense.
Dr. Laura: I want to reach them but I’ve got to tell you doing this in the responsible way, it is hard. And so [01:00:00] we can only work so much and we have to prioritize sticking to our process of keeping things evidence-based by being responsible but we can’t do everything. And that has been a struggle for us ever since we started working together in 2007. We bring in new people that are going to help with the special populations, like for our preschool version, which is in a development trial right now, we brought in a speech and language pathologist because with that population, we needed to add in some expertise there but it is a little difficult to not be able to do everything everywhere, all at once. Right. That’s a moody or something.
Dr. Sharp: Right, always.
[01:01:00] Dr. Laura: And it is a challenge, but hopefully we will have more people doing those adaptations for us. I have had a K awardee, Kelsey Dixon, who is a psychologist in California and she is just finishing up her K award where she’s adapting Unstuck for community mental health centers, which was great. They said they didn’t want to do a group model, but it’s much easier for them to do an individual model and to target certain things. So what’s great is that when we can get other people involved and speeding up this process for their populations or contexts of special interest including other countries.Dr. Sharp: Yeah. That’s amazing. When you talk about the things that you have to leave on the [01:02:00] shelf in a way, that makes me think about exclusionary criteria, I suppose, or any situations where it’s maybe not appropriate to try to implement this program either on a system level or an individual kid level. Are there any things like that that we need to be aware of?
Dr. Laura: Yeah. So I would call them, they were our exclusionary criteria, so that means we haven’t tested at all in these groups, right? So we haven’t tested with intellectual disability. We haven’t tested with kids who are not speaking or can’t use language or AAC to communicate. And one of the things about Unstuck in general is that it is very language [01:03:00] based.
Dr. Sharp: Okay.
Dr. Laura: These higher-order executive functioning skills rely heavily on language; self-talk and out loud. It doesn’t have to be out loud. It could be through a device or typing, but they really rely heavily on language. It would be great to come up with a way to do this without language, but we haven’t been able to crack that nut yet. We have a lot of visual supports, but without the concepts really being embraced by the kid with language and self-talk that they can use when they need it, that’s a real area for us for reliance.
Dr. Sharp: Sure.
Dr. Laura: So like the preschool version, our exclusionary criteria is, at least the language of [01:04:00] the three year old. So simple sentences, at least, in some way. We have seen it being done with, as I said, kids without any neurodivergence at all, or not that they know of. We haven’t tested it specifically with general students. We’ve seen it, a few lessons being done in kindergarten classes also, universally which is really, really cool, but we haven’t tested it.
Dr. Sharp: All in due time maybe.
Dr. Laura: Maybe.
Dr. Sharp: I saw your face. You’re like, ahh, maybe.
[01:05:00]Dr. Laura: Yes. It’s hard to get everyone included. And though that is our core principle, in some ways it may not be possible to get everyone, especially not using any language would be, I think it’s the hardest for me to imagine anyway. I think an adaptation for intellectual disability would be maybe high on our list or next. But we’ve also been asked by many people to do adult self-versions that maybe includes an app or something on a training for themselves online, like our parent training. And that I think is also a really great idea. There’s a whole bunch of ideasDr. Sharp: There’s never any shortage of [01:06:00] ideas. Yes. Well, maybe that’s a good segue into what you do see in the future. What are the priorities over the next 6, 12, 24 months? I’m not sure how far out y’all are projecting.
Dr. Laura: For next year, our priorities are getting the middle and high school manuals out and published. And if you’ve never published a treatment manual with a publisher that has a very high standard with co-writers who also have very high standards, it takes a lot of time to get it right and it’s really painful if you get something wrong, like if there’s a typo in there or something, we don’t love that at all. But they [01:07:00] happen anyways often. So that is our number one priority so that they can be purchased and used because until the manual’s out people can’t implement it.
Originally, we had wanted to make the manuals free and then we found out what that meant: How much of our time would be spent maintaining the website and disseminating materials ourselves and keeping things updated and having a help desk for when people can’t download it because it’s so huge. So our manuals are published by Brookes Publishing and they’ve been a great partner, but there is a cost to them and that is unfortunate but is a way that we, I mean, couldn’t reach their network. I can’t imagine how I would’ve ever [01:08:00] gotten 25,000 people to buy the book, right? How would I have done that?
Dr. Sharp: Sure.
Dr. Laura: That’s not what I know how to do. I can write grants. I do not know how to do that but we are on our website also over the past few months and now over the next year, we are trying to put as many things as we can on there for free, including the elementary school training is on there and we are working right now on the high school version. And we’ll be doing a little test of that and then putting that on the website also for free.
So when we can get the money to cover those costs and we don’t have to maintain something ourselves, then we do want to make as much as we possibly can free or low cost or very low [01:09:00] cost so that it can reach as many people as possible.
Dr. Sharp: Love that mission.
Dr. Laura: And then I think probably the preschool version will go to for a real trial. It’s just really in a development trial right now, but we’ll go for a real trial next.
Dr. Sharp: That sounds great.
Dr. Laura: Yeah. I do not know, I’m tired just listing off these things. It’s very exciting and it’s great that we’ve added additional people to help us get the work done- some younger people. So pass the torch as we are feeling more and more fatigued.
Dr. Sharp: Right. Yes. Bring some fresh energy in and have others take that.
Dr. Laura: It’s really necessary.
Dr. Sharp: Yeah, I hear you. Well, at least you will get a little break over the holidays, hopefully. And then…
Dr. Sharp: Definitely
Dr. Sharp: …hit the ground running in the New Year if you choose to.
[01:10:00] Dr. Laura: Yeah. A very important executive functioning skill is to take vacations and holidays and not work to give your brain a break from that part, makes you much more efficient when you come back. So that’s my personal advice to all of your listeners.Dr. Sharp: I hope everybody listens to that.
Dr. Laura: I will try to follow up myself.
Dr. Sharp: Yeah, exactly. I talk a lot on the podcast on the business side about taking breaks and making sure that you have time to have some spaciousness and actually breathe a little bit.
Dr. Laura: And reflect.
Dr. Sharp: And reflect.
Dr. Laura: And reflect on who you are, what you’re doing and why you’re doing it. We can hold onto the why even when things are really stressful and we’re tired, then just like it does works with our kids, it works with us too. Keeping sight of that purpose.
Dr. Sharp: Well said. That’s a nice note to end on. [01:11:00] Thank you so much for coming to have this conversation. I know we could talk for much longer about the program and the work that you’re doing, but I think there’s a lot here for people to take away. I’m very excited about being able to access this program and we’ll make sure to include all the links in the show notes for folks to do that if they want to look into it more and perhaps even implement some things. Thanks again, Laura. This has been great.
Dr. Laura: Thank you so much for having me. And you ask great questions. It makes it really fun.
Dr. Sharp: Well, I appreciate that. Yes, it was good time for me as well. I hope our paths cross again soon.
Dr. Laura: Oh, me too.
Dr. Sharp: All right, y’all, thank you so much for tuning into this episode. Always grateful to have you here. I hope that you take away some information that you can implement in your practice and in your life. Any resources that we mentioned during the episode will be listed in the show notes [01:12:00] so make sure to check those out. If you like what you hear on the podcast, I would be so grateful if you left a review on iTunes or Spotify or wherever you listen to your podcast.
And if you’re a practice owner or aspiring practice owner. I’d invite you to check out the Testing Psychologist Mastermind Groups. I have mastermind groups at every stage of practice development, beginner, intermediate, and advanced. We have homework, we have accountability, we have support, we have resources. These groups are amazing. We do a lot of work and a lot of connecting.
If that sounds interesting to you, you can check out the details at thetestingpsychologist.com/consulting. You can sign up for a pre-group phone call and we will chat and figure out if a group could be a good fit for you. Thanks so much.
The information contained in this podcast and on the Testing Psychologist website is intended for informational and educational purposes only. Nothing in this podcast or on the website is intended to be a substitute for professional, psychological, psychiatric, or medical advice, diagnosis, or treatment. Please note that no doctor-patient relationship is formed here. And similarly, no supervisory or consultative relationship is formed between the host or guests of this podcast and listeners of this podcast. If you need the qualified advice of any mental health practitioner or medical provider, please seek one in your area. Similarly, if you need supervision on clinical matters, please find a supervisor with expertise that fits your [01:14:00] needs.