Dr. Sharp: [00:00:00] Hey everyone. This is Dr. Jeremy Sharp, and this is The Testing Psychologist podcast. Welcome back. I have another episode today talking all about remote assessment and administration of primarily behavioral checklists. I have two fantastic individuals from PAR Incorporated to chat with me about this topic.
I will introduce them in just a moment, but first a little disclaimer. So as we have been home during the COVID-19 pandemic, I have, of course, been recording podcasts from home and you may hear some slight variations in sound that maybe aren’t always there. You may also hear my dog barking or the wind howling outside. So just a little apology for those things if they get picked up on the microphone.
Without further ado, let me talk to you about my guest today. Dr. Carrie Champ [00:01:00] Morera is a project director at PAR. She’s a licensed psychologist, nationally certified school psychologist, professional counselor, and board-certified telemental health provider with 19 years of experience in the mental health and education fields.
She’s provided psychological supervision and consultative services in schools and outpatient settings. She has at PAR, assisted in driving product strategy, defining products, and bringing them to market for the clinical assessment product line. She is the primary content specialist and project management and psychometrist for the development of print and digital assessment products as well.
So prior to joining PAR, she led the admissions assessment function for a residential school in Pennsylvania, where she designed, implemented, and managed the evaluation and review of applicants to maintain the student body and the school program. So Carrie gets it from both sides, I [00:02:00] think, and brings a very unique perspective to this interview.
Her partner here, Daniel McFadden has an MBA and is the director of customer support for PAR. Daniel has been with PAR for more than 19 years; started as a customer support specialist and then led the technical support department before taking over as director of customer support.
He’s responsible for ensuring that PAR provides unparalleled customer service and tech support for all the products that they sell. You have probably experienced this if you’ve had to deal with PAR over the years. They’re fantastic. They try to work to answer every call live with the goal of delighting every customer in every interaction. He’s also an internal product champion, which means he’s responsible for gathering customer feedback and providing recommendations for improvements and also delivers monthly webinars on products.
Daniel has a bachelor’s in [00:03:00] psychology, and like I said, an MBA from the University of South Florida. Prior to joining PAR, he was the assistant station manager at WBUL-1620 AM at the University of South Florida.
Carrie and Daniel make a really dynamic team. And I think this is a fantastic interview that really shed some light on remote administration that was new to me and highlighted some topics and areas that I haven’t thought about in administering behavior checklists online. So I hope that you’ll take some valuable information from this interview.
Without any further delay, here’s my conversation with Carrie Champ Morera and Daniel McFadden.
Hey, everyone. Welcome back to another episode of The Testing Psychologist podcast. I’m Dr. Jeremy Sharp, and like you’ve heard in the introduction, I am here with two fantastic folks from PAR, Daniel McFadden, and Carrie Morera. We’re going to talk all about online administration of behavior checklists, and so forth. So welcome to the podcast to both of you.
Dr. Morera: Thank you for having us.
Daniel: Yeah. thanks, Jeremy. We’re happy to be here.
Dr. Sharp: Sure. So I have to get the first, probably most important question out of the way right off the bat. And that question is, is it PAR or P-A-R?
Daniel: That’s a great question. I’ll take this one, Carrie, because I’ve been around forever. It depends on when you started with the company. I think if you’ve been around for a while, you call it P-A-R and I know that’s what the owner of the company prefers. But we’ll certainly answer to PAR as well. [00:05:00] We’re not too particular, I guess.
Dr. Sharp: Okay. That’s all I needed. Historically, I have said PAR. I don’t know why exactly, but that’s what I’ve decided to use. So you validating for me.
Daniel: You got it.
Dr. Sharp: Nice. Well, now that we got that question out of the way, we can move on to some of the more important stuff. I know people are really into the remote assessment world these days, perhaps a little bit more than usual, and I’m excited about our conversation.
When we started to plan the conversation, I was curious, I think like many practitioners, I’ve been administering online measures for a long time, and then a part of me was like, what else is there to know? But as we talked, it turns out there’s a lot to know that I think will be illuminating for folks. So I’m excited to dive into it here.
Let’s start out just with [00:06:00] considerations for using these products via telehealth. And maybe we could even back up and just start with, when I say these products, what measures are we talking about here that are available for online administration?
Dr. Morera: We have a variety of measures that are developed for online administration. We have behavioral rating scales, and we also have a few performance-based measures as well. We have over 70 instruments that are available on our PARiConnect platform.
Dr. Sharp: I got you. Yeah, I think, for me personally, I tend to dwell on 2 to3 rating scales, but there are a lot of measures available. And when you say performance-based measures, do you know two of those off the top of your head?
Dr. Morera: Yes, we do. We have the RAIT and the TOGRA that are available. And we [00:07:00] also have the RIAS-2, which is available as well. And we’ve done some equivalent studies on those. So with the RIAS-2, they weren’t the norm using digital standardized administration, but the RAIT and the TOGRA were.
Dr. Sharp: I got you. And I know that those have come up in discussions in my Facebook group and on other listservs as measures that are particularly well suited to remote administration, cognitive measures, I should say, which is a unicorn these days to find those.
So I want to talk about these considerations for using the products via Telehealth. Can you just go through, you named three considerations we need to keep in mind when we are administering these measures over Telehealth. What are those?
Dr. Morera: Sure. PAR developed some considerations for using our [00:08:00] products via Telehealth. And just to back up a little bit, just some general statements when doing any remote Telehealth, it’s really important for practitioners to be familiar with relevant state laws and national practice standards and really follow them. And I know that APA recently put out some updated guidelines regarding Tele-assessment during the COVID 19 crisis. So I really think everyone should start there first.
But before we get into the considerations that PAR developed, there are two bases around them, and they came from two codes and standards that we use when developing our assessment products. The first one is the Ethical Principles of Psychologists and Code of Conduct that’s put out by the APA. And then also the second one is The Standards for Educational and Psychological Testing. These two guides really drive the basis for how [00:09:00] we came to develop the guidelines and the considerations for Telehealth.
So if we go ahead and jump right in, the first one to consider is the copyright standard. So really in response to the whole COVID 19 pandemic, PAR is granting the customers limited permission to administer PAR assessments using different telehealth technologies. So as we get more into the podcast today, we’ll explain in more detail how we can do that in a safe and ethical manner.
And also regarding copyright standards, as long as our copyright materials are not photocopied, recorded, reproduced, published, or electronically stored, we can go ahead and use them. We don’t want people purposely photocopying the materials because we need to maintain the copyright standards. And that’s also in line with professional ethics and responsibilities, [00:10:00] and we need to protect confidential client information that is available on the forms and protocols that we’re using. So that’s the first standard.
Then the second one is test security. So this is really understanding that the security of the test must be maintained whenever we’re using it, especially in the telehealth technology way. And the security of the materials is really important to PAR. We really consider them to be trade secrets and protect them to the fullest extent of the law.
And some people think, well, what really are the test materials? That really includes everything that comes with it, for example, the manuals, the instruments, the protocols, the stimuli, all of the tests and answers, our profile forms, and any other items that could accompany the tests such as manipulatives or overlays. Again, the [00:11:00] test materials are confidential and we do take a lot of steps at PAR to ensure that confidentiality throughout our development process and after publication.
So for example, during our development process, we have expert reviewers, bias reviewers, and beta reviewers, and they’re involved with helping to create and review the products and give us feedback. We have each of them sign non-disclosure agreements with us. So that way we can help protect the security.
Another way that we can help with our test security is we only sell our instruments to individuals who are qualified and also obligated by professional and ethical standards so that we can protect the materials. PAR has different qualification levels that help us with that. A, B, C, and S and they range from [00:12:00] having no special qualifications needed for some of the products that we have to those having advanced degrees, licenses, and appropriate training. And we have these qualification forms that are available online. You can either download them or complete them.
And then the last area related to testing security is that when using telehealth technology, we need to make sure that customers maintain the security and integrity of the test at all times. And we can talk a little bit more specifically about how to do that through telehealth.
Dr. Sharp: Sure. Yeah, I would love to do that. I think that’s a primary concern for a lot of us with this process. As much as we try to control the environment where the person is taking the measure, sometimes we don’t know, right? So have y’all thought about [00:13:00] particular considerations for telehealth and how we might protect test security?
Dr. Morera: Sure. And that leads to the third area, which is the measurement fidelity, our last consideration. And so we’re really looking at the administration and scoring procedures that have been used to standardize and validate the assessments. They are documented in professional or technical manuals.
So really when we are looking at using these on Telehealth, we really would like to encourage our customers to go back and read the sections of these manuals for the instruments, and then look at what are the limitations of using them in Telehealth? What are the benefits? Is it feasible to do so, and is it worth it? Do I need to do the assessment now? Can it wait for another time?
And like we had mentioned earlier, most of PAR’s performance-based tests were standardized during an in-person examiner-client interaction. [00:14:00] And before administering any of those performance-based tests using this type of telehealth technology, again, we need to carefully review the standardization procedures because any time we deviate from those standardization procedures, we need to really make sure that it’s supported by the clinical situation and context that we’re in. We really want to try to preserve the test administration as much as possible whenever we’re doing telehealth.
Dr. Sharp: Right. So can you say a little bit more about the ways that you suppose standardized or those equivalency studies, particularly with the cognitive measures? And then if you can compare that process to how it worked with the behavioral checklist. Can you speak to that at all?
Dr. Morera: Sure. With the equivalent studies for the [00:15:00] behavior rating scales, there is a really excellent review by […] and colleagues that was published in 2015, that really looked at a lot of different rating skills. They took rating scale data obtained from electronic platforms, and then they compared those to paper administrations.
And so what they did when they were looking at all of these previous studies that were done, they were looking at different types of modifications that were completed moving from paper administration to electronic administration. And so there are three different types of modifications that you can make.
The first one is considered minor modifications. That’s where you just simply have paper and pencil rating scales, and you can put that on the text screen in some way without significantly altering any item [00:16:00] content or response for the item. It includes going from multiple items from your paper format page to one item per screen. And so this is an example of what PAR does with our paper-based rating systems. So it’s really only a minor modification which has been suggested to be okay for research in terms of equivalency.
The level of evidence here required to show equivalency is just a minor modification is cognitive interviewing and usability testing. So we’ve tested it out through equivalent studies to make sure that it’s consistent from paper to electronic format.
If you also look in the literature, there are what’s also considered moderate modifications of moving from paper-based format to electronic format. And so this type of modification can result in the [00:17:00] perceived meaning of the assessment items maybe being altered.
Some examples would be having items and responses on different screens. So it’s not altogether. The order of the items might be a little different, or you might have to scroll down on the screen when you’re reviewing all of the responses. So here the user then, the meaning may be misinterpreted because it’s not altogether in one location so to speak. Here, if we want to demonstrate equivalence, we need to have some type of quantitative equivalence testing that would need to be done as well as usability testing.
An example here would be, if you’re using a text-based to an interactive voice response system, then here you’re really changing the modality from a visual to an auditory. And so the modality has just changed. [00:18:00] Does that make sense?
Dr. Sharp: It does. I think that’s one of those things just to highlight, we don’t have to get in the weeds with that, but I think a lot of folks just assume that it’s an easy translation from paper to online for a behavior checklist, but there’s a good amount of research that goes into making sure that those are actually equivalent administrations, right?
Dr. Morera: Yes, there really is. And the last one is just a major modification. So that’s just when all the wording and response options are changed. And really in that situation, that should just be treated as a brand new assessment and you should do full psychometric testing, but generally, statistical results of equivalents are demonstrated by using Pearson correlations, mean difference correlations, or intra-class correlation coefficients. So those are really the types of statistics that are used.
Dr. Sharp: Okay. All right. Well, anybody who’s listened for a long time knows that I’m not a statistician, so I’m not going to [00:19:00] dive into that and embarrass myself in short order. So thank you. I’ll trust you. Others may have more detailed questions, we’ll see, but for now, we’ll leave that.
I think people are probably really interested in what we need to keep in mind and what concerns there are for administering these measures remotely. Now, this is something that has come to the forefront now remote administration is really our only option in many cases, but these concerns are always present, whether there’s a pandemic or not, when we send these checklists over the internet. So what are some of the concerns and things we just need to look out for when we’re doing this?
Dr. Morera: Sure. So it’ll be a lot of concerns. I think the best way to minimize concerns, first of all, if I could just speak to that for a moment, is to be prepared especially if you’ve [00:20:00] not engaged in telehealth prior to the pandemic. Many people are now realizing, oh, I can’t just jump in right away. And you really shouldn’t. Like I said earlier, you should really read and adhere to the Telehealth guidelines, attend webinars, take courses, consult with others, and do whatever you can first to educate yourself about the process.
You really need to make sure that you and your client have access to all the technology needed and you both know how to use it. Explain the process to your client and really make sure that both of you know what to expect on both ends. So I just wanted to preface that before I address some of the concerns.
Dr. Sharp: Awesome.
Dr. Morera: So one of them is distractions. Always consider the environment. Distractions can arise from the client side and also on the practitioner side. With the client-side, there might other [00:21:00] people in the room or going through the room, there might be loud noises outside, and the television might be on in the background.
Really what you want to do is try to meet the client where they are and work on what they have to make them feel comfortable. If there are other devices in front of them or in the room, have them turned off, and work with your client to figure out, is there a particular time of day that works better for them for telehealth given what’s going on in their current environment.
What is the best location that’s going to give them privacy in the home? Not everyone has a lot of space, so where can you go? I’ve had situations where the only people that have had privacy was in their cars. I mean, if people need to get away and that’s where they need to go, then that will be fine as long as there are no distractions.
You really want to provide clear expectations though from [00:22:00] you expect from your client. And it’s really not a time for them to be multitasking. I’ve once had a situation where I was doing therapy and a client decided to go out and try to go shopping. So at that point, we decided to stop the session. They need to be focused.
Dr. Sharp: Sure. I think one of my clinicians told me a story two weeks ago that she started a Telehealth session with a client, and this person was literally riding on a lawnmower at that time. Then they figured out quickly that that was not going to work.
Dr. Morera: Yes.
Dr. Sharp: So, a note to clinicians.
Dr. Morera: Definitely. And then another just distraction too from the client-side is just other adults or other people that are present in the room. So you really want to try to make sure that only the people that you need to speak with a [00:23:00] person is there. And if at all possible, have others be removed from the room due to privacy concerns.
Also, we can look at distractions on the practitioner’s side. So we really want to be cognizant on our end and maintain as much as possible a professional appearance. So I like to say, try to mimic as much as possible your in-person setting as much as you can. Remove distractions in the background. Try to hide your dirty laundry or whatever else is behind you.
I know that there are a lot of platforms now where you can put up a background, so maybe put a neutral background behind you, but again, I would encourage it not to be a distracting type of background because you don’t want your client paying attention to whatever it is going on behind you. So try to keep it neutral.
Dr. Sharp: Hey Carrie, can I jump in and ask you a question [00:24:00] about something you said a minute ago when we were talking about people being in the room or not preferably not being in the room?
Dr. Morera: Yes.
Dr. Sharp: You may not know the answer to this and feel free to pass if that’s the case. But I do know that there are some cases, both in-person and remotely where we’re administering checklists to kids and they have trouble reading it. And so the parent may have to help them with reading it. Do you have thoughts on how that impacts administration or validity?
Dr. Morera: Sure. So if the parent is in the room with the child, and the child is completing a rating scale, I would first from the practitioners and try to work with that child in some way to have that interaction be between the practitioner and the child and remove the parent from the situation because you can have instances where the [00:25:00] parents may encourage the child to respond in a certain way, or the parents may rephrase the question and then the original intent of the item is lost.
So I think there are tools and tricks that we can do through telehealth as we’ll get through today, where we can really place that responsibility on the practitioner’s side and alleviate the parent from jumping in because that will affect the validity of the results that you obtain.
Dr. Sharp: Right. So, yeah, maybe we’ll talk about the ways to do that with technology, but off the top of my head, I’m thinking that may be just a good rule of thumb to either have the manual or have a paper record form in front of you if, for example, you need to read the items to the kid so that we can get the parent out of the equation.
Dr. Morera: Yes, that’s good.
Dr. Sharp: Okay. Thanks.
[00:26:00]Dr. Morera: Sure. I do have a comment though since you had mentioned about people being in the room, so observers, that can really affect the outcome of assessments online as well. So there are third-party observers. So we really need to think about how will their presence really impact the client’s performance. There’s what we call observers, and there are also facilitators. So we may need facilitators to help with the telehealth process. And so they may be there to help facilitate participation, reports, and communication.So, for example, a child that you’re working with that may have separation anxiety or autism spectrum disorder. The parent may need to be physically present in that room to encourage them to participate and help them to feel more comfortable. [00:27:00] Similarly, a facilitator that can play that role is an interpreter. So if you need a foreign language interpreter or a sign language interpreter, they will be necessary to help facilitate that process and may be needed at times.
Dr. Sharp: So let’s see, we were talking through distractions and accuracy. Have we touched on the accuracy of the result, or were there more distractions that we should be aware of?
Dr. Morera: There could always be lots of distractions and it’s really difficult to anticipate all of them. Technology can also be a distraction. Daniel, I don’t know if you’d like to talk a little bit more about the technology piece in terms of distraction?
Daniel: Really, it’s less of a distraction, although certainly, if we’re talking about having applications running in the background when you’re setting up [00:28:00] something with a client. If it’s emails that are popping up or chat messages. There are thousand different things that can happen with all the technology that we’re using these days.
But I think probably an even bigger consideration is the inconsistency we have with technology. When you’re sending something to someone who’s remotely, you don’t know necessarily what their device is. And it’s a good idea to always dig into that and make sure that they have an acceptable device. If you’re asking them to do an assessment remotely, that they have appropriate webcams or whatever it’s that you’re going to use if you’re wanting to do some telehealth.
So, everything from screen size to the performance of devices, different operating systems, obviously, internet speed does pretty consistently these days, but you just never know what’s going to happen. And certainly, reliability can be a question. So I think all those are definitely all considerations that you want to take into account.[00:29:00] And it’s always a good idea to use a few minutes to just go through that and just make sure that there’s good quality on both sides.
Dr. Sharp: Sure. And where could we find those guidelines for each of the measures in terms of technology requirements or what screen size they were standardized on, if that available?
Daniel: On our website. It isn’t necessarily by assessment but PAR in general, we do have a list of the requirements. Fortunately, it’s not real lengthy as far as what you need to be able to use the platform itself, really just any modern web browser will work just fine. If you’re doing an assessment with someone remotely, we really would say anything smaller than a regular-sized iPad, you probably would not want to do that just for the validity of the assessment. But there is a list on our website as far as which web browsers are acceptable. If you’re asking [00:30:00] about webcams, I think that there are no specific devices, just checking the quality and everything.
Dr. Sharp: All right, Carrie, let’s see. Are we back to accuracy at this point?
Dr. Morera: Yes. Accuracy of results. Telehealth assessment in general, as we know, needs to be interpreted with caution. It’s really best practice to discuss in our reports, the limitations and deviations from any standardized procedures. However that being said, results can still be accurate.
In terms of accuracy results, especially during this time, many people may be asking, how does the pandemic affect the results of testing? Just with any testing that we’re doing in general right now because it’s probably fair to say that the majority of our psychological and educational assessments that we’re using today were not standardized during any type of pandemic. We’re really in unprecedented times right now. [00:31:00]I don’t think we can fully understand at this point, to what extent and for how long we’re going to see the impact of psychological testing.
So, for example, if we just take something as simple as anxiety measures, will we see a rise in standard scores that are in the clinical range, for example? Perhaps. We need to look at the results cautiously. So whenever we’re getting the results back on any standardized assessments, especially during this time, we need to not only look at the scores, of course, they’re important, but we need to go deeper and evaluate the individual item responses like we would do in best practice anyway. And if we really feel that the results are being affected by the pandemic, for example, we need to talk about with the client how they’re feeling and think about how it’s impacting their scores.
And this can affect not only rating scales but also academic testing and [00:32:00] cognitive testing as well. If we really think about it, financial stress and depressive symptoms, anxiety, are all weighing in on people perhaps more than ever during this time.
Dr. Sharp: I know a lot of people have asked about that, just the standardization and how we can even interpret test results through the lens of this pandemic. And you’re right. I can’t find any good data on standardization during other pandemics. That’s just not out there. I’ve seen a number of additional disclaimers floating around that people are putting in the reports just to say, this evaluation was conducted during COVID 19 pandemic and results might be impacted as a result, and so on and so forth.
Dr. Morera: Yes. And related to that, just thinking about the psychological effects of quarantine. There have been some studies that have come out there recently. [00:33:00] There was a meta-analysis that was done by Brooks and colleagues earlier this year that was looking at the psychological impact of quarantine. They found that post-traumatic stress, anger, and confusion were really common. No, surprise. And within that same research study, some researchers reported that there have been long-term effects that have gone on after pandemics which include vigilant hand washing and avoiding crowds that have gone on for many months after being in quarantine.
Dr. Sharp: Sure. I think it’s just important to highlight that, especially the example that comes to mind right away is if you have a kid who is developing compulsive cleanliness behaviors right now, just to put that in context, maybe not OCD or PANDAS or anything like that.
Dr. Morera: Yeah. And it’s really important to look at was [00:34:00] this happening before the pandemic?
Dr. Sharp: Of course. Great. So the accuracy of results, we have to keep all these factors in mind. And I think that may be a nice segue too. We discussed some cultural factors, SES, familiarity with technology, and computers. I think all of those are important as well. So I wonder how you might speak to that in this context of remote assessment?
Dr. Morera: Culture is a really broadly defined term and can include lots of different things, such as race, ethnicity, spirituality, religion, gender, age, et cetera.
So just thinking of some examples, there have been some studies with the indigenous populations, for example, that have shown that the indigenous populations have expressed concerns about privacy and confidentiality in general, as it relates to Telehealth [00:35:00] and overall feelings of just being uncomfortable with it in nature.
And that they struggle with the communication technologies in Telehealth when we compare it to Westerners, for example.
And then additionally, when we look at culture and religious beliefs, some studies have shown that rural patients generally prefer practitioners who are familiar with their culture. And so as a practitioner, we need to really take into consideration and be mindful of those differences. And sometimes individuals may choose not to engage in Telehealth because of those reasons.
And then as we look more specifically at some socioeconomic factors, including poverty, for some low-income individuals, it’s difficult to pay for telehealth if the costs are too high. There are some public programs that may be free [00:36:00] to them, but not available on the Telehealth platform. So that could be a reason why they may not en engage in Telehealth.
Additionally, those who generally lack transportation or have limited education opportunities experience poverty, in general, studies have shown that it can negatively affect their help-seeking behavior and willingness to receive psychological care in general. So that can be more difficult to have them involved in Telehealth.
One thing I just want to add to that is I really think that relationship is the key and practitioners need to really focus on developing that trust and rapport to help account for some of these cultural differences. There are studies out there that suggest Telehealth can be equivalent to face-to-face services. For [00:37:00] some individuals, they may need that initial direct in-person contact at least on that first visit, so to speak, before moving to Telehealth services, to even get to that level of comfort.
Dr. Sharp: Right. I think this is just such a big can of worms to dive into. We touched on it two podcasts ago with Jordan Wright and now today. It’s hard to tackle in one podcast, but just to be mindful and to know, at least for us, we’ve started to add questions around computer literacy and privacy and just familiarity with technology into our intake where we’re actually checking in with parents about that before we try to do Telehealth with their kids or with the… Go ahead, Daniel.
Daniel: I was going to say, it’s interesting when you say familiarity with computers, I think most people [00:38:00] probably the first thing that pops into their head is older populations struggling with technology. But there’s an article that they ensure our parent colleague out there and can speak to this.
We always assume kids are super technology-friendly and they know exactly what to do, but then you hand them a laptop versus the iPhone they’ve been using for the last two years, and they don’t have any idea what to do. And I think a lot of parents are struggling with that by trying to educate at home right now. So you can’t just assume that just because kids have had technology around their whole lives that they’re fully used to the technology that we’re talking about using in this particular case.
Dr. Sharp: Yeah, I’m so glad that you highlighted that because it made me think of my own kids where they are bright. I know this because I’ve practiced our tests on them. And so I know that they’re bright and resourceful and so forth, but they go to Montessori, and part of the Montessori philosophy is not a lot of technology. [00:39:00] And so compared to even some other kids in public school, we got them a laptop to do “homeschool” during this pandemic and they didn’t know what to do with it. We had to teach them how to use an actual computer.
So, just put that out there that even, I think socioeconomically fine and doing well, but it’s not just like low-income populations or those that we might assume that kids don’t know how to use laptops a lot of the time unless they’ve been exposed to it. That’s a great point.
So let’s see. Are there other factors to consider, cultural issues, and technology issues that we want to make sure and touch on before we move to a different area?
Dr. Morera: Language barrier is another area to address. So really thinking about what impact does [00:40:00] this has on the communication between the practitioner and the client and the relationship. Things can get lost in translation. So considering what is the best way to incorporate in interpreters and how does that third-party relationship affect the dynamics? And really should the third person be involved?
Dr. Sharp: Right. And again, I know there’s a whole literature out there on interpreters and how they might be involved in an evaluation, just to highlight that that is important to keep in mind.
So let’s see. We’ve talked through some of the equivalency studies. We’ve talked about the number of products you’ll have. Maybe we could move to some of the technical aspects and what this actually looks like and what you’re doing on y’all’s side to make sure everything is secure, [00:41:00] and so forth. Who would like to jump into that technology side?
Daniel: Sure. I’ll try to do that without putting everybody to sleep instantly. I don’t want to crash your ratings, Jeremy.
Dr. Sharp: Fair enough.
Daniel: I’ll make it as short and sweet as I possibly can. Well, so really with PARiConnect, in case there’s anybody out there who doesn’t know, PARiConnect is our online assessment platform. You can administer assessments either by email, or on-screen, or you can manually enter data if you had done paper and pencil, and then, of course, you score it and generate the report. It’s all online. There’s no software that is downloaded or anything special on devices. […].
With PARiConnect, there are really probably two options that you have for remote administering rating scales. For the majority of assessments, if you don’t need to be present, you don’t need to be proctored. If you were sending a teacher for an assessment, for example, [00:42:00] you can use the email option. So with PARiConnect, you have the ability to email the link directly to the person who’s going to be taking the assessment, then complete it, and then you get an email notifying you when it’s done, you can log back in and generate the report.
The other option, obviously we recognize that times are challenging and assessments that previously you might have done in person you’re trying to get done remotely. So with that in mind, there are certain assessments that really should be proctored. And if you’re in a situation where you’re comfortable doing that and you feel like it’s the right decision for your particular case and your particular client.
I actually did a video that’s up on our website that just walks you through one of the options that might be a good fit, which is, that you can actually launch an on-screen assessment within PARiConnect, and then use a Telehealth tool like Zoom, for example, which is the one I used on the demo video. And then instead of sending them the link, you’re actually [00:43:00] sharing your screen and then actually giving control over to them. And I think the real advantage to that is just you have complete control over the administration.
We talked earlier about technology usually being pretty reliable and everybody has pretty good wifi these days, but stuff happens. And if you started the administration that’s on someone else’s device and then your video link goes out or they do something nefarious and turn off the video, you don’t know what’s happening on the other end at that point.
So with that method, if you need to do proctoring, I think it’s probably a good option that allows you to have complete control. Even if that video does drop out, you could take control back, you can stop screen sharing, whatever it is that you need to about the assessment.
And then I don’t want to spend all day talking about data security, but I think that it’s something that we both know sometimes is really important. So worth talking about some of the things that are built into PARiConnect. Obviously, it’s designed to be HIPAA compliant. [00:44:00] All the data is encrypted using fancy algorithms and stuff that most people don’t care about.
But one thing that I think is probably worth noting is that I suspect a lot of people don’t know, we actually stored the data separately on servers. We don’t save reports. There are no reports on PARiCconnect, everything’s actually generated on demand. For example, the demographics from an assessment are stored on one server. The responses are stored in a separate server. And then the report logic is stored on a separate server. And so, we have all kinds of technology that’s protecting the data.
But even if somehow somebody did get into one of those, they wouldn’t be useful data. It wouldn’t be something that identifies who the client is. So I think it’s something that a lot of folks don’t understand of what lengths we’ve gone to make sure that your data is safe and secure.
Dr. Sharp: Sure. Yeah, that is something that I was not aware of. I mean, [00:45:00] we just assume, I think that it is secure but it’s nice to hear you talk through it and know that it’s not just “secure,” but there are a lot of measures taken to make sure that that’s the case.
Daniel: Yeah, absolutely. That’s great that people will take our word for it, but I also like to about some specific measures. And you like to actually dig into the real issues. Jeremy, I’ve heard your podcast before. We’re not going to just skim over that. Let’s really get into it.
Dr. Sharp: Sure. Well, I have to hold myself back. I have enough of that tech geek in me to want to ask a lot of questions here, but I’m not going to. I’m going to be mindful of the 99% who don’t care about that, but it is totally secure. And one thing that y’all brought up in our pre-podcast chat is this idea of protection for kids that are under 13. And that is something that was not on my radar at all. So could you talk through that?
Dr. Morera: Sure. [00:46:00] I’ll speak a little bit about that. So for the protection of children under 13, there is The Children’s Online Privacy Protection Act (COPPA) which Congress enacted in 1998. And so this is really to protect the safety of children in an online environment for those that are under age 13.
So if you attempt, for example, to administer an assessment through a link through PARiConnect, you’re sending it out there for a child who’s under 13, you will get a popup, which will not allow you to do this. And the reason being is we want to protect children and the information that they go online and fill out. And so it’s really to keep them safe on their end. So that way they are not providing personally identifiable information that can be [00:47:00] used against them or used inappropriately. So that’s why we do not allow email links to be sent out for individuals under 13 to complete them.
Dr. Sharp: Sure. So then the alternative solution, I suppose, is maybe solution like Daniel was talking about was to have it on your screen, but a allow them to manipulate your screen and answer the questions that way, if it’s a self-report measure or even ask them the questions. Are those options?
Dr. Morera: Yes, I think that would be the most appropriate way to do it because then that way you are there as the practitioner and able to answer any questions that may arise while they are filling out the instrument. And you’re also able to reiterate the instructions to them and make sure that they are filling it out in an appropriate manner, rather than sitting off somewhere by themselves [00:48:00] trying to figure out how to do it independently.
Dr. Sharp: Okay. Gosh, we’ve covered a lot in terms of remote administration concerns and guidelines and what we need to keep in mind. But I know there’s also this whole side of, there are a few other things that y’all are doing to try to support us as we’re going through this process. So I hope that we could talk a bit about that as well?
Daniel: Absolutely. I think we at PAR recognize that these are extraordinary times and that they’re a lot of challenges. We’re trying to do everything we can to make it easier for everybody to do their jobs and do the most important thing which is to help your clients. So probably the most important thing that we’ve done is that we were actually offering [00:49:00] e-manual for prior print purchases.
So, if you purchased a manual in print form, either it as part of the kit or just by itself from PAR in the past, and it’s a product that we publish, which is not everything of course, but most of the products we sell are ones we publish, if you call or email our customer support team, we’ll send you a free manual version of it. Because of social distancing, you can’t access something that you had purchased. If you’re working at a school and you can’t get into where the manuals are right now, if you contact us, we’ll be happy to give you an e-manual for free. We’ve had a great response to that. A lot of folks have taken advantage of it. So that’s certainly one thing that we’re doing.
I did make also a demo video that just shows you how to download the e-manuals as well as some of the cool features that are built-in. There are things like highlighting, bookmarking, and then searching, which if you’ve ever tried to find that one page in the manual that talks about whatever, the search feature is really cool and handy. So there is a demo video that goes through all of [00:50:00] that that’s on our website.
There are two things that I think are worth mentioning as far as limitations. Those e-manuals are downloaded to a specific device. So you can’t share it out to Google drive or anything like that. You can’t print from it. Obviously, we’re just protecting the material itself. But I think it’s really handy to have those available.
We’ve also developed a COVID resources page on our website that has tons of resources linked and practices information. Actually, I’m pretty sure at least linked to one or two of your podcasts is on there. I think the one with Dr. Wright is linked on there and we’re constantly updating it. There are always new things being added to it. Obviously, most of it’s focused on practice information and guidelines that are out there from state national, and federal organizations.
We also recognize that everyone’s mental health is important. And obviously, these are strange times for everybody. So these things that are helpful [00:51:00] for practitioners from a daily positivity checklist that we put up, even our marketing has been, we’ve stopped marketing products for the most part. And we’ve sent out daily sunshine emails, just trying to bring a little bit of light to everybody’s day, and hopefully people like those.
We’ve heard a lot from folks we did a survey to say, what else can we do to help? And one of the things that we heard a lot was just education resources. So with that in mind, we’ve been doing webinars. There’s a recorded webinar for PARiConnect that’s available 24/7 in our training portal. I do live webinars once a month and I’m actually doing another one next Friday, just to walk you through all of that.
But I also don’t want people to feel like they have to wait for a webinar. If you’re using PARiConnect now, and something’s confusing to you, if you call our customer support team, we can do a demo for you immediately. Just ask. We can actually do a screen share [00:52:00] within 30 seconds and show you whatever it is.
We also have a training portal that’s totally free. There’s no cost or anything in the training portal. If people are trying to learn new assessments or they feel like they have a little extra downtime and you want to brush up on stuff, there are tons of products that are covered there. They’re interactive training portals, trauma products, and the Feifer Assessments of math and reading, PAR, RIAS, and BRIEF-2. It goes on and on. There are author videos featuring, Cecil Reynolds, and Dr. Abbo, and it’s free. You don’t need to pay for anything. You just need to have a par.com login and then of course have to have submitted your qualification information to us.
And then the last thing I would mention, it’s been a challenge for us at PAR as well, just to maintain our current levels of the standards that we’ve set. But I’m happy to say that even though most of our team is working remotely, [00:53:00] we’re still shipping orders the same day you place them in our distribution center. Although they’re taking all precautions, they’re still shipping packages. Our customer support team still answers every phone call live. We haven’t had any downtime during this pandemic.
And if you’re talking about PARiConnect, when you place an order, whether it’s on the website or on the phone, that inventory is delivered to your account pretty much immediately. Not to get salesy, but those are just some of the things that we’re doing to try to support our customers and all the important work that they’re doing in these really odd times.
Dr. Sharp: Absolutely. Well, and I’m glad that you highlighted that. It doesn’t sound salesy, just for the record. I think it’s important to highlight those things especially in this era, I mean it’s super important otherwise, but especially now when people are confused and overwhelmed, the last thing that they [00:54:00] want is to not get guidance when they need it. So having people that answer the phone for customer service and getting products quickly, when you realize you forgot, but you have an assessment in five minutes, that’s really important. So I’m glad that you highlighted that.
Daniel: Thanks.
Dr. Sharp: I have to say, of course, that we’ll link to all of those resources that you mentioned in the show notes so that people can access those. I’m personally excited about the training portal. That’s one of those easily overlooked resources that I know was in an email somewhere, but you just never pay attention to it, but now people are home, maybe they have some extra time.
Daniel: You just made our marketing team cry, Jeremy. Now they’re like, how could someone not know but you’re right. It’s the kind of thing just when you’re in the normal day-to-day, it’s easy to overlook that kind of stuff, but it’s a great resource. [00:55:00] And again, we don’t get anything out of it, except that we want people to know about the great assessments we have.
Dr. Sharp: Right. And I can definitely say that is just personal experience with y’all over the years before having any connection with you as a company has been great. Y’all have really set the bar high for customer service, quick delivery and so on and so forth. So it’s just good to hear that is consistent and you’re putting energy into that.
Daniel: Thanks, Jeremy. Appreciate you saying that.
Dr. Sharp: Yeah. So let’s see. We are getting close to time. I want to be respectful to y’all. I know we’ve packed a lot of info in here. Any closing thoughts, any resources, anything else that people should be aware of as far as remote assessment, or really anything that we talked about during our time together today?
Dr. Morera: I’d just like to highlight one other resource that I think would be very useful for practitioners. On [00:56:00] the resources page using PAR products in Telehealth, there’s a link that takes you to a PARiConnect product listing of i-Admin capabilities. And it’s a really nicely organized chart which has all of our products that are available in PARiConnect listed in alphabetical order. And it lets you know which ones are i-Admins, which ones are score reports, which ones have interpretive reports, and which ones have multi-rater reports. So it’s a really nice go-to reference sheet.
So if you’re looking for a specific type of measure, you can just find it right there. And there’s even a direct link that will take you to each of the product pages. So you can read more about the assessment. So I think that could be really helpful for practitioners.
Dr. Sharp: That’s fantastic. Probably that’s another thing and I’m sure has been out there for years and we just don’t search for it because we never need it, [00:57:00] but here it’s. Awesome. Well, thanks, Carrie.
Well, I’ve really enjoyed talking with y’all and again, I just appreciate your willingness to come on and take time out of quarantine to chat with me for a little bit about remote assessment and its importance these days.
Daniel: Thanks, Jeremy. It’s really been a pleasure to join you and hopefully, we shared information that is helpful for folks. I appreciate everything that you do.
Dr. Sharp: Thanks. Of course.
Dr. Morera: Yes. Thank you for the opportunity.
Dr. Sharp: All right, thanks y’all for tuning into this interview with Dr. Carrie Champ Morera and Daniel McFadden from PAR. Like I said during the interview, I just appreciated that they were able to highlight some of the things, the lesser-known aspects of remote administration of behavioral checklists and other measures. But that information in particular was helpful to me. And like you heard toward the end, they are [00:58:00] putting in a lot of time and energy to make sure that we are supported as practitioners. Those e-manuals are available and there is a great resource list in the show notes for you to check out. So please do that.
At this point, this episode is due to release in late April and I am opening up or rather I have 2 to 3 spots open in my beginner practice mastermind group. It will run on Wednesdays from 16:00 to 17:00 Eastern time. And I would love to have you in there.
If you are just getting started with your practice, give me a call and we can do a pre-consulting call to see if you’d be a good fit for the group. We, like I said, have two spots available. These groups have been really dynamic in the past. So I hope to add two more energetic and enthusiastic psychologists to my beginner practice group on Wednesdays. So if you’re interested in that, you can [00:59:00] go to thetestingpsychologist.com/consulting and book a call to talk about the group and see if it’s a good fit.
As always, take care. Try to maintain your sanity and your health during this time that we’re in and tune in next time. Thanks, y’all.