Dr. Sharp: [00:00:00] Hello, everyone. Welcome to The Testing Psychologist podcast, the podcast where we talk all about the business and practice of psychological and neuropsychological assessment. I’m your host, Dr. Jeremy Sharp, licensed psychologist group practice owner, and private practice coach.
All right, welcome back to The Testing Psychologist, and welcome to 2021. I am ready to get started with 2021, and this episode is launching the year with a bang. So my guest today is someone that I am guessing many of you have heard of, or know of. Dr. Yossi Ben-Porath is a professor of psychological sciences at Kent State University and a board-certified clinical psychologist. He did his doctoral training at the University of Minnesota. He’s been involved extensively in MMPI research and development for the past 35 years. So he is the co-developer of the MMPI -2- RF, MMPI -A- RF, and MMPI -3. He’s also the co-author of numerous test manuals, books, book chapters, and articles on the MMPI.
[00:01:12] Yossi served as the editor-in-chief of the journal psychological assessment, and he is a member of the APA’s committee on psychological tests and assessment. His clinical practice involves supervision of assessment at Kent state’s psychological clinic and consultation to agencies that screen candidates for public safety positions. He also provides consultation and expert witness services in forensic cases.
So like I said, many of you have probably heard of Yossi. He is relatively active in The Testing Psychologist Facebook community as well. So I am just so grateful to be able to sit down with him and really dig into his experience with the MMPI instruments over the years. So we go way back, we talk about how he got involved in the MMPI and his progression from a grad student assistant all the way up to now a co-developer of MMPI instruments. We talk a lot about the transition from the MMPI -2 to the 2- RF, and then to the MMPI-3, what that was like both from a development perspective, but also just from a professional standpoint and what that was like to have such a big transition and encounter some criticism around that. And we talk about a lot of the updates to the MMPI -3 and how they continue to improve on the family of instruments. So, if you utilize these instruments at all, I think this is a great episode. You’re going to take a lot away from it, and we just had a fantastic conversation.
So without further ado, I bring you my conversation with Dr. Yossi Ben- Porath.
[00:03:24] Dr. Sharp: Yossi, welcome to the podcast.
[00:03:26] Dr. Yossi: Thank you. I’m very happy to be here.
[00:03:29] Dr. Sharp: Yes, I am very grateful that you are here. This is one of those moments where I’m interviewing someone on psych. Wow, this is really happening. So, thanks for the time and for the interest here. I’m excited to talk with you.
[00:03:42] Dr. Yossi: Well, I appreciate the opportunity. Thank you, Jeremy.
[00:03:45] Dr. Sharp: Yeah, I know we have a lot to talk about. There’s the MMPI-3. That’s the big topic or agenda today. But I would love to start with a little bit of your origin story as it were if you’ll be willing to talk about it. I’m just so curious, how does one even get into something like this? Can you go way back and think, how did this even start?
[00:04:15] Dr. Yossi: Well, you’re correct in referencing way back because it does go back a while. And when I was preparing for this conversation, I was thinking back in fact and ask myself, when was the first time I ever heard about the MMPI and it was in my first year as an undergraduate. I was 22 years old at the time. I’m from Israel originally. And so after high school, I did 3 years of military service and then took a year off to just clear my head, and so started college at the age of 22 and the year was 1981. So I guess everyone can do their math now and figure out my age if that’s of interest, but in any event, it was 39 years ago.
And the reason I remember I think, first hearing about or first awareness of the MMPI is that our textbooks were in English. We didn’t have Hebrew textbooks, and I was fairly fluent in English because I’d spent some years in the United States as a child. My classmates knew this. And I remember one day, one of my classmates coming up to me and asking me, what does this word inventory mean? That was in our introductory textbook and it was in the context of the MMPI, and so I explained to her that basically you’re taking inventory of someone’s personality and translated that then. And that is my earliest recollection of the MMPI. I didn’t really think about it as much for the remainder of my undergraduate years. When I first started as a psych major, it was with the intention of becoming an industrial-organizational psychologist. I was thinking that would be an interesting area, and I had been planning to go to graduate school. This was at the University of Haifa Israel, by the way, I should have mentioned.
And so in my last semester of undergrad, I took an abnormal psychology class and the professor who taught that class was a brand new faculty member who had just arrived with his Ph.D. fresh from the University of Minnesota where he had actually worked with Auke Tellegen who of course is my coauthor on the MMPI -2- RF and an MMPI 3. But now we’re in 1984 and I had already decided that I was going to take a year off before applying to a graduate school and some research experience that would be helpful with my application. And I just really fell in love with clinical psychology and in the field of abnormal psychology and psychopathology in that class that I took with Michelle Maguire was the name of the professor, someone I still work with and also have a now very long-term friendship with. And I started working with Moshay who as I said, was a student of Auke Tellegen, and what we actually worked on as a research project was translating a psychological test into Hebrew, but it was not the MMPI, it was Tellegen’s multidimensional personality questionnaire, which is a normal personality inventory that is not used at this point in applied settings, so most people are not familiar with it, but in any event, that was the personality measure that I cut my teeth on.
We did the translation and eventually wrote up a publication. But the other thing that happened was that Moshay made some introductions for me, wrote my letter of recommendation and I wound up at the University of Minnesota in 1985 as a graduate student in the clinical training program. My plan had been to work with Tellegen which I did, and of course, still do till this day. But I wasn’t a US citizen at the time which made me ineligible for many of the training, scholarships, and fellowships that were available but required US citizenship and Tellegen didn’t have search funding.
[00:09:12] So I was kind of poking around and looking into possibilities of working as a research assistant. One of my professors, still at the University of Haifa, probably gave me the best piece of advice any professor ever gave me, which was if you’re planning to go to the US and go to graduate school, learn how to run statistical analyses on a computer. We’re talking about the early 1980s here. That meant computer mainframes, not on your laptop or even a desktop, and that will be a very good skill set for you to have available.
And as luck would have it, right at that time in my second quarter, they were quarters back then, when we took the MMPI class which was taught by James Neal Butcher, that was just when the data were starting to come in for the MMPI-2 revision that was still under construction at that point. And he happened to ask me whether I knew how to run statistical analysis on a computer because he needed someone to work as a research assistant with the data that were coming in, and I said, sure I do, and within the next few weeks I was processing the data that were coming in for the MMPI-2, started working with James butcher, not just as a research assistant but he wound up being my advisor and working with him on a number of the MMPI-2 projects, among other the construction of the content scales of the MMPI-2.
And at the same time, Tellegen was also involved, he was another member of the MMPI-2 development project. And so with him, I worked on developing the VRIN and TRIN scales of the MMPI -2 and the uniform T scores that have been used since the MMPI -2. And that’s basically how I wound up at that point immersed in the MMPI-2 world. And then when I got my first job and I’ve only been in one academic position since completing my graduate studies, it was at Kent State where Jack Wieland another, of course, MMPI -2 legend if you will, was another member of the development committee. He was on the faculty here at Kent. And so when I came to Kent, I continued to still work with James Butcher with Tellegen on some projects but also began working with Jack Graham. And so, now we’re talking about the early 90s. That’s essentially how I wound up working on the MMPI.
[00:12:12] Dr. Sharp: And then you’re in it from that point?
[00:12:15] Dr. Yossi: I have been in ever since.
[00:12:17] Dr. Sharp: That’s wow. I know that a lot of us have the experience especially in grad school of stumbling into projects that may or may not be a good fit and then we realized, this actually does fit. Did you like statistics? Did you like running all this data or was it one of those things where you got into it and then it grew on you over the years?
[00:12:47] Dr. Yossi: I liked running the statistics but I’ve always viewed it as a means to an end, not an activity that I enjoy in itself. What I was really fascinated by was the notion of assessing and measuring and quantifying psychological characteristics and variables. And obviously, the statistics are necessary both when you’re constructing measures, but also when you’re studying them and evaluating the utility, validity, and so forth. I started to become interested in that part while still working with Alma Gore and MPQ at the University of Haifa. And then having the opportunity to work on the MMPI, actually get paid for it as a graduate student, and then continue to work in scale development and validation. I just learned that that really is what I have a passion for in terms of my research interests.
[00:13:52] Dr. Sharp: Sure. And at this point in your career, I’m just personally curious, I think we all know you for the MMPI series, right? Do you work on anything else at this point or is the MMPI kind of all-consuming in your research?
[00:14:09]Dr. Yossi Ben- Porath: I’m a one-trick pony, Jeremy. As far as my own research is almost all MMPI. I do a lot of work and get exposed to a lot of work and assessment because I’ve been for the past 6 years the editor of a psychological assessment, the APA journal psychological assessment. So in that capacity, I have an opportunity to read all of the submissions that come in. I can’t say that I read every one of them to the very last letter because I have a very Productive team of associate editors who do a lot of the work with journalists as well, but it does give me an opportunity to get exposed to a very broad range of work in the area of assessment which I think is helpful to me in my own work.
[00:15:06] As an MMPI researcher, I also get exposed and get involved with a very broad range of work that assessment psychologists do because the MMPI is such a versatile instrument. Obviously, some of the work that I do is in the area of psychopathology working with mental health populations. But I’ve also worked and I’ve been very fortunate to have collaborators in the area of health psychology and use of the MMPI with medical patients, for example, in pre-surgical evaluations. We’ve done a lot of work in that area.
And also in the forensic area, I myself have done a lot of forensic work evaluations in the criminal court area, but also I’ve done some research on uses of the MMPI in forensic assessments and also with psychologists who work in the police and public safety domain. So although it’s one test, it’s really many different applications. And I think that has helped keep me interested in a broad range of various psychology and assessment psychology.
[00:16:26] Dr. Sharp: Yeah, sure. When you talk about it in those contexts I suppose that’s true. There are so many applications and different ways that you can use it. It’s hard to get bored I would imagine it.
[00:16:41]Dr. Yossi: This hasn’t bored me. That’s for sure.
[00:16:43] Dr. Sharp: That’s great. I know people are just so curious about the new version, and I think we can probably dive into that.
[00:16:56] Dr. Yossi: Sure.
[00:16:57] Dr. Jeremy Sharp: If you’re ready to do that. But I have just a general question that I think will hopefully lead us in a productive direction, which is, when you go from one version of a measure to another, how do you know when an update is needed? That’s a very naive and general question, but hopefully, it takes us somewhere.
[00:17:23] Dr. Yossi: No, it’s a great question actually. And I can tell you specifically, describe the evolution of, at least my thinking about this. So, as I mentioned, when I first started out as an assistant professor at Kent State, I was working with Jack Graham, and the hot topic back then when the MMPI – 2 first came out, there was some controversy about whether the coat type literature that had been developed with the original MMPI or empirical correlates that were the foundation for interpreting the Kotex. Could that be applied to the MMPI -2 because the norms have changed, and as a result, someone who may have been classified with one code type with the old norms may not actually produce the same code type with the new norms? And that was a fair amount of back and forth about that with some concern expressed that we addressed with empirical studies that show that yes, you actually could apply the MMPI literature to interpreting code types on the MMPI 2.
But what Jack Graham and I did was to initiate the first large-scale new study of empirical codes of the now MMPI-2, collecting some data at a community mental health center in Akron, Ohio, Portage Path Behavioral Health, where we collected a very large sample data from a large sample of outpatients who had received services at this community mental health center were tested with the MMPI -2 on admission as part of the intake process, and when we obtained information from the intake workers and then later from the therapists who saw these individuals, but before they actually were exposed to their MMPI so that we would not have criteria and contempt contamination.
And then, with this data set, we did a very large-scale analysis of the correlates of the code types but also the individual clinical scales. This wound up getting published as a book that we authored along with our post-doc at the time, John McNulty. And I can tell you that when you do that kind of work and again, do the statistical analysis and go underneath the hood as I got to do, what I discovered was that the clinical scales, some of the Kotex made good sense. And some of the correlates of the clinical scales made good sense and others not so much were very counter to what you would expect on the basis of the constructs that were to be assessed by the scales. A really good example of that is the PD scale 4. What we found was that the things that you would expect with that we would be correlated in a mental health sample with higher scores on scale-4, things such as, acting out behaviors, substance abuse, possible involvement with the criminal justice system[00:20:40] were all correlated with scale-4. But so were things such as being unhappy and disgruntled and dissatisfied. And those are not the things that are prototypically associated with elevated scores on a scale- 4. And there were many other examples of that.
So to me, that was the early indication that things are not quite as clear-cut as they may appear to be if you just look at the textbook now. At the same time, one of the members of the MMPI-2 development the re- standardization committee called Auke Tellegen had actually already begun to work on a project that eventually produced the restructured clinical scales, the RC scales.
[00:21:36] I actually remembered hearing Tellegen talk about the need to update and deal with some of the psychometric deficiencies of the clinical scales as a graduate student while the MMPI -2 was being developed. He had actually proposed that to the committee and the committee decided that updating both the norms and the scales at the same time would be too much of a change at once, and that, the two needs and need for new norms was more press sensitive. So they essentially decided to carry over the clinical scales to the MMPI -2 without change.
But Tellegen then shortly thereafter began working on this project that he at that point called streamlining the clinical scales. And he and I had continued to communicate. I think it was in 1995 that I first saw him present some of the data on these scales that he was working on. And one thing that was very clear was that what you get in terms of correlates with these streamlined scales that he was working on was much cleaner in terms of their discriminant validity. So what came to be the restructured version of clinical scale-4 had all of the acting out correlates, but none of the other unexpected correlates having to do with being unhappy and dissatisfied.
And I think it was at that point that it became clear to me that there really was a need to at least update the clinical scales. And I joined Tellegen in that effort and together we finalize the RC scales which were pretty much complete in 1998. They weren’t actually published and added as another set of measures to the MMPI -2 until 2003. We spent several years studying them and making sure with a lot of datasets that we had available at that point that they worked as intended or that we could develop some empirically grounded with solid foundations guidelines for using them in the context of the MMPI -2. But that was really when the realization set in when I had the opportunity to look under the hood and see what really correlates with what. And so that probably was the turning point, at least in terms of my perception of the clinical scale was of the MMPI.
[00:24:09] Dr. Sharp: Sure. And then how did that process come into play this time around where… I mean, in my understanding, and you, of course fact check me anywhere you need to, is that the MMPI-3 is pretty solidly correlated with the 2 RF, right? I’m not sure.
[00:24:30] Dr. Yossi: They’re very similar.
[00:24:32] Dr. Sharp: So how do you know this time around?
[00:24:35] Dr. Yossi: So there’s one more step that I haven’t described kind of in the evolution if you will. So in 2003, the University of Minnesota publisher added the RC scales to the MMPI -2 as another standard set of scales. But as we were studying the RC scales, Tellegen and I, it became very clear to us that it wasn’t just the clinical scales that could benefit from restructuring, but the entire instrument. The restructured clinical scales were never designed to assess everything that could be measured with the MMPI-2 item pool, but rather major distinctive core components of the clinical scales. And we knew that there were many other constructs that could be reliably invalidly assessed with that item pool of the MMPI -2.
So once we completed the work with the RC scales, we began restructuring the entire instrument and that’s what produced in 2008, the MMPI -2 RF, the restructured form. And what we did with the MMPI – 2- RF was essentially applied a similar methodology to what Tellegen had done with the RC scales but restructure the entire instrument. But we limited the restructuring in the revision to existing MMPI-2 items. And in standardizing the MMPI – 2-RF, we use the MMPI-2 normative samples. So we didn’t collect new norms. We didn’t write new items. And that’s the answer to a question that I sometimes, rather frequently was asked when the RF first came out was why didn’t you call it the MMPI-3? And the answer was because we did not write any new items or collect new norms. It was a restructured version of the MMPI-2. And doing it that way, had the advantage of allowing us to use all of the existing data sets that had been accumulated and not just us, but anyone who had an MMPI -2 data at that point could do an MMPI-2- RF study. Was it possible to go back in and rescore your data as the restructure form and that allowed for a lot of investigators to immediately begin doing MMPI -2- RF research without having to collect new data, and contributed to the fact that we now have close to 500 peer-reviewed publications on the MMPI -2-RF not just from my group, but just across the board.
But the flip side of that is that the two things we didn’t do were to update the item pool or update the norms. And those were the two primary objectives for the MMPI-3: to pick up where the MMPI -2-RF left off, take advantage of all of the psychometric improvements that we had made in the restructuring project, but then fill in some gaps, areas that the MMPI -2 item pool doesn’t cover and therefore, were not adequately or at all covered with the restructure form. And a good example of that is there are no items on the MMPI-2 and therefore no items on the restructured form having to do with eating disorders or any kind of problematic eating behavior which of course is a significant problem in mental health today.
[00:28:13] And there were other areas where the item pool was relatively lacking. Not many items having to do with grandiosity on the MMPI -2 or compulsive behavior. And so, our objective then was to explore adding new item content to the MMPI and update the norms. The norms for the MMPI -2-RF are the MMPI-2 norms. Also, the data that I was working with as a graduate student in the mid-1980s, those data were collected in the mid-80s. And so they’re more than 30 years old. It was a normative sample that represented the adult population of the United States in the mid-1980s, which was very different from the population today, demographically and experientially.
One way that I sometimes tried to illustrate that is to point out that not a single member of the MMPI-2-RF normative samples had heard about the internet, let alone social media. We live in a very different environment today. And of course, demographically, the adult population was over 80% white in the mid-1980s. It’s now, the latest estimates for the 2020 census are 62%. And of course, individuals have of Hispanic origin, that’s a much larger proportion of the population today, only 2.9% of the MMPI-2 normative sample identified as being of Hispanic origin. And so, demographically, the normative sample was becoming increasingly inadequate and we wanted to expand the item pool content-wise not lengthwise, but expand the coverage of the item pool. So those were the two main objectives for the MMP-3.
[00:30:20] Dr. Sharp: Got you. Thanks for talking through that. It is interesting to trace the history. And I don’t know that I really thought about it in that stark of terms to think the normative sample had not been updated since the 80s. That’s a long time.
[00:30:39]Dr. Yossi: Yeah.
[00:30:39]Dr. Sharp: That makes sense. So I feel like I have to ask, I know you have been asked this, but am curious just about that big decision to go from sort of the structure of the MMPI-2 to the RF and abandoning I suppose criterion validity for the content. Well, am I getting that right?
[00:31:07] Dr. Yossi: Well, the criteria and keying, yeah. one of the questions that has been raised by moving away from the clinical scales that were constructed on the basis of the criterion King approach the empirical, in the approach, are you not abandoning the origins of the test if you will? And I think the answer to that is no because we still use the items that came out of that process of criterion King or, empirical King Stark Hathaway, and Charlie McKinley used in constructing the original clinical scales by contrasting the responses of specified diagnostic groups with a set of nonclinical individuals who were called the Minnesota normal who were not in a part of the treatment process.
And I think that Hathaway and McKinley wound up assembling a remarkably rich and robust item pool, but they were applying the methodology and technology of the late 30s and early 1940s when they constructed the clinical scales. And we’ve learned quite a bit both about personality and psychopathology and test development since then. And so, our goal was to apply more modern test construction techniques. And this is in developing the MMPI -2-RF to apply more modern test construction techniques to that item pool that they assembled and see if we could address some of the psychometric problems of the clinical scales. Now interestingly, the empirical King didn’t work. The folks who will bring that up sometimes ignore that part of it.
[00:33:11]Dr. Sharp: Can you elaborate? When you say it didn’t work, what do you mean?
[00:33:15] Dr. Yossi: So the idea was that the MMPI would be administered to any new patient coming into the University of Minnesota hospital, where the test was developed and be used as a direct differential diagnostic indicator to indicate whether the patient had one or more of the eight most commonly occurring diagnoses in the patient population at the time: hypochondriasis depression, hysteria, psychopathic deviance, paranoia, psychasthenia which today we would call anxiety disorders, schizophrenia, and hypomania. Those were the eight original clinical scales. And the idea was that if a patient produced the profile that was two standard deviations above the norm, a score that was two standard deviations above the norm, say on the schizophrenia scale, then that patient had schizophrenia just like any other medical test if they exceed the cutoff, then the condition is present.
It turned out that you couldn’t use it that way. The vast majority of the patients who scored above 70, T score 70 was the cutoff used at the time, and the schizophrenia scale didn’t have schizophrenia. And so if you use the test as intended, the false positive rate would have been excessive and unacceptably high. And so very early on I had to make a complete shift, a paradigm shift. This was the first paradigm shift in the evolution of the MMPI instruments and it was actually one of Hathaway’s former students who was a graduate student at the time that the MMPI was being developed and then actually, shortly thereafter became a faculty member at the University of Minnesota in the mid-1940s, a fellow by the name of Paul Miele. And it was really Miele who led this paradigm shift and focus and looking away from the scores on the individual clinical scales to looking at patterns of scores or combinations of scores of clinicians using it.
I think there was such a strong need for a measure like the MMPI that didn’t exist before it was introduced that clinicians began using it. And though they couldn’t use it as intended as a direct differential diagnostic test, they found that certain patterns of scores, certain combinations of scores on the clinical scales tended to reoccur and that the individuals who produced those scores had certain features in common, certain personality, characteristics, or symptoms of psychopathology or behavioral tendencies.
[00:35:48] And what Miele led was then an effort to study this systematically and collect empirical data on these patterns of scores that came to be known as the code types. And so MMPI interpretation really from that point on was no longer all dependent on the empirical King approach. The way in which the scales work was were constructed had no impact on their use when code type interpretation became the primary vehicle for interpreting scores on the MMPI. So when I hear that we’ve abandoned the richness that came from the empirical King, the richness that came is represented by the items, not by the methodology itself.
[00:36:41] Dr. Sharp: Yeah, I think that was the major reaction criticism, I suppose.
[00:36:49] Dr. Yossi: That’s one. The other one relates to what I was just talking about, and that is the code type. So, another criticism that we’ve heard and that has been expressed is, well, you’re abandoning the richness of the code types because the code types very much didn’t have a rich empirical foundation that helps support their interpretation. And that’s how the MMPI Clinical scales have been used over the years, rather than looking at scores on the individual scales. But the code types themselves were an ad hoc solution to the problem that the clinical scales didn’t work as intended. And the reason the code types worked was that they basically looked at and solved some of the challenges of the clinical scales, all the overlapping items, and the construct overlap.
Going back to the example I gave you earlier of the PD scale-4, the fact that we find scores on the PD scale to be correlated empirically in clinical settings not just with acting out, but also with unhappiness and dissatisfaction, well, if you look at the rest of the profile, not just scale- 4, but the rest of the profile, and along with scale 4 you have an elevation on scale 9 and 4-9 codetype what we find empirically to be correlated with a 4-9 codetype, what we find in therapy will be correlated are the acting out behaviors and all of the things that we typically associate with PD. On the other hand, if another individual has exactly the same score on the PD scale but rather than an elevation on 9, they also have an elevation on scale 2,so a 2-4 codetype, [00:38:51] well, if you look at the empirical correlates, and we did all of this with the data that we collected in the 90s if you look at the empirical correlates of the 2-4 code type, that’s where you find the unhappiness and dissatisfaction.
So you really need to look at all of the other scales in order to make sense of each individual scale. And so the code types really provided this kind of ad hoc solution. What we did in the restructuring project is basically go back to the building blocks, the items and see if we couldn’t apply these more modern test construction techniques and develop scales that didn’t require all of this additional information. I want to point out that this was not some kind of a unique insight of ours. I think Tellegen and I obviously, we’re the ones who did the work, but with the restructuring of the clinical scales. But, if you don’t mind, Jeremy, I want to read something to you.
[00:40:04] Dr. Sharp: Of course.
[00:40:06] Dr. Yossi: You can just quote, and it’s a quote from Paul Miele who was Hathaway’s student, and of course we came, one of the most famous clinical psychologists of the 20th century, and this is from a chapter that he wrote in 1972 which was part of a book. The book was edited by Jim Butcher. And the book came out of a conference that was held to honor Starke Hathaway and his contributions. And one of the themes of the conference was, how would we go about improving the MMPI? It’s been out for 30 years now at the time and a number of different authors presented at the conference and then written chapters and then Miele wrote the final chapter in the book. And what he’s commenting on here in this chapter is what I was just talking about. So I’ll read it really quickly here.
He said, ” Unfortunately, one can achieve a moderate and sometimes rather high elevation on scale 4 without being a sociopath. Not surprising when we look at the items scored for this variable. At an increment of two or three T-score points per Ross core item shift, it takes less than 10 items in the combined areas of family strife and post institutional troubles to achieve a T score of 70. We all recognize today that this kind of thing happens and is one source of error which we attempt to “correct” for mentally by taking the patient’s situation into account as well as looking at the rest of his profile. But it would be nicer if such errors were eliminated from the PD key entirely. As a factor analyst once complained to me during a heated discussion on criterion King, internal consistency, scale purity, and related topics. If you Minnesotans are going to eyeball the profile and do a subjective factor analysis in your head that way, why not let the computer do it better at the stage of scale construction,? Not an easy argument to answer.” He wrote.
And that’s basically what Tellegen and I did. We did some factor analysis looking at each of the scales and some other items we don’t need to get into the methodology in detail here, but that was the idea. And my point here is that this wasn’t a new or novel idea. It was something that people who were very familiar with the MMPI had thought about including Paul Mielle who probably other than Hathaway thought about the MMPI more than anyone else.
[00:42:50] Dr. Jeremy Sharp: Right. That’s a great quote. I want to dig in a little bit here because I’m just imagining what this might have been like for you, and I’m doing a little bit of projecting here, of course, but I would think if this were me and I had been working on this project for 15ish years, you go through this major revision doing something that I, in my mind, I would be saying, we’re making this better. We’re really honing this and applying these modern statistical methods, and this is going to be an upgrade, and then the RF comes out and people are like, what are you doing? That’s terrible. All this criticism. What was that like for you as a researcher and a person to go through that process? And am I projecting?
[00:43:42] Dr. Yossi Ben- Porath: No, I think those are great questions. And it is a process. And it’s sometimes can be complicated, sometimes it can be challenging. I remember early on, I do a lot of trainings, a lot of workshops, and of course was doing a lot of trainings after the MMPI -2-RF came out and at one of these workshops, I had been talking about some of these issues that we were just discussing and someone raised their hand, and actually stood up and said, “I really don’t understand, what did you do to my PD scale?” And I said, “We fixed it.” And there was a little bit of laughter and so forth, but then, of course, I went into the details and explained why and how we did so. But there were. I think it’s natural for many. Clinicians had been using the MMPI- 2 for years and the code types, then developed a fair amount of expertise in doing so. And you really needed to be an expert interpreting the MMPI -2 with the clinical scales and the sub-scales and supplementary scales and the code types and so forth was a rather complex process. And some people became really, really good at it and wrote books about it.
But most clinicians, in my impression and experience and then because I do so many trainings, I get to talk a lot with clinicians. And that has been one of the most important sources of ideas and feedback for me. And one piece of feedback that I remember hearing when I was still doing trainings on the MMPI -2 is, “I have a lot of other tests that I use and in my practice and in my battery that I administer, if it’s someone who does a batteries of tests and I see clients or patients for therapy, and I don’t have the time to do a Ph.D. on every single instrument that I use. Does it really have to be this complicated?”
And so one of our goals in the restructuring was to make the use of the test less complicated without sacrificing at all its reliability or validity. So, while I certainly was hearing from some of the experts who I think were experiencing the sense of the loss with the introduction of the MMPI -2- RF, they were by far outnumbered by clinicians who gave me very positive feedback about how helpful they found the MMPI-2- RF and how much easier it had become to integrate it into their practice. And the fact that it was shorter was also a benefit. So more efficient not just in terms of interpretation, but also administration time.
And the positive feedback certainly by far outweighed some of the criticisms and complaints. And as many Minnesota PhDs are empiricist in my approach to questions such as this, and the empirical data were quite compelling and I felt quite comfortable with that. And again, we worked on this for many years and studied this test inside and out. And so I was very comfortable that what we were making available to people was from a psychometric perspective of very useful and well-constructed instruments.
I think when you change something, a test that has almost iconic status, in fact back in the 70s and maybe even the beginning of the 60s there was a group of people who used the MMPI who called themselves the mult-cult, and I think the word cult is maybe, appropriate to some extent. So obviously, some members of the multicultural were not happy with the restructured form, but the vast majority of users and the feedback that we’ve received from people who use the test in all of the settings that I mentioned earlier has been quite positive. The criticism is out there. We respond to the substance of the criticism when it appears in the literature. And I think that’s the way to deal with it.
[00:48:42] Dr. Sharp: Sure. Well, spoken like a true empiricist. If you can fall back on the data and know that you can be solid, I think that would be really helpful in that process if I were in your shoes. I remember, I started grad school in 2003 and we took a personality assessment that very first semester with Dr. Chuck David chauffeur, and he was one of those folks who seemed to use the MMPI-2 as a crystal ball. It was magic as far as we were concerned and the nuances that he could pull out. And we were just in awe of that whole process. And knowing that there are other folks out there like that, I can se, it does feel like there’s some loss wired.
[00:49:33] Dr. Yossi: It required a great deal of experience and expertise to do that. That’s what Miele was writing about when he talked about doing a factor analysis in your head. That’s basically what these experts do. They’re deconstructing the clinical scales if you will. And they’re doing it by looking at all the other pieces of information that come in and of course taking advantage of any collateral information. But it doesn’t have to be that complicated. And if it doesn’t have to be that complicated, why not make it a more user-friendly instrument is really the thinking behind that.
[00:50:10] Dr. Sharp: Sure. So I want to ask some more specific questions about the MMPI-3 of course, but on that whole process. Do you feel like moving from the criteria and keying to the sort of the new structure, were there any trade-offs? I mean, were there any compromises sacrifices that you had to make?
[00:50:33] Dr. Yossi: Not really. And again, I think, as far as the criterion King is concerned, I don’t think it’s a coincidence that the clinical scales of the MMPI were the last set of applied assessment measures constructed that way. And this was again in the late 30s and early 1940s. It’s a methodology that has some limitations. And the way it was applied had some additional limitations. So I don’t think, and again we were able to still take advantage of the richness of the item pool that Hathaway and McKinley had assembled which I think is unparalleled. The reason why the MMPI has survived all these years is because test users are able to glean some clinically relevant information from the responses people give to those items. How you organize those responses into the scales is really what we’ve focused on.
But your question is a very good one because one of the things that we focused on very much while developing the MMPI -2-RF was the question, is there anything missing? Is there anything in not just the clinical scales, but all of the scales that had been developed over the years for the original MMPI and MMPI -2, are we missing any important content?
And I’ll tell you a little story here. It’s not one that you’ll find in any of the books. It turned out that we were missing some content because when we thought we were done constructing the MMPI-2- RF, I was already at that point doing trainings on the RC scales. I had just done a workshop for the Louisiana psychological association, and one of the people attending that workshop was a neuropsychologist who I had known previously Kevin Grieve who’s in New Orleans, and a few weeks after the workshop this is while the MMPI -2- RF materials were still being developed and produced, but the RC scales were already part of the MMPI-2 and Kevin had done a pre-surgical evaluation of a spine surgery candidate, and he sent me the profile which included both the clinical scales and the RC scales of restructured clinical scales.
[00:53:14] And on the clinical scales, the individual had a very clear 1-3 code type, and on the RC scales there really wasn’t anything at all showing up and Kevin said, “If I use just the RC scales, I would’ve missed something very important here.” And my response was, “Well, the MMPI-2-RF has additional scales.” So he sent me the item responses. I was able to score the MMPI-2- RF and really still didn’t find what 1-3 code type was picking up on. And that sent Tellegen and myself back to the data. And what we found was that there’s a set of items that appear on those scales that were not adequately represented in any of those scales we had developed for the restructured form. And that led us to develop the final scale of the MMPI -2-RF which was the Malays scale, which is one of the specific problems scales and a very important scale that would have been very important item content to have missed or left out of the test.
Fortunately, we were able to stop whatever was already ongoing and make some changes and move some items around, re-replace some items, and include the Malays scale. And so clearly it was possible and it’s still possible that there’s some important content that’s missing but we were pretty systematic and so I didn’t think we had missed much. But what was very clear was that we were constrained by the MMPI -2 item pool for the restructured form. We had decided not to write new items which would have necessitated collecting new norms. And so we always knew that we were going to explore as a next step if the MMPI -2-RF proved to work well using that as the starting point for expanding the content and that really as I mentioned earlier was one of our two primary goals for the MMPI -3 was to add item content, not that was missing from the RF but was missing from the MMPI -2, things like, eating disorder-related items and the others that I mentioned earlier.
[00:55:35] Dr. Sharp: Yeah. What were some of the highlights from that in addition to eating disorders? Can you remember some of the…
[00:55:46] Dr. Yossi: So I mentioned earlier that there’s a surprisingly limited number of items on the MMPI -2 dealing with grandiosity.
[00:55:53] Dr. Sharp: That’s right. Yes.
[00:55:54] Dr. Yossi: And impulsivity and compulsivity. And there’s really also a limited range of items dealing with anxiety-related experiences, for example, very few if any, that would be associated with Panic attack and those kinds of experiences. So what we did for the MMPI-3 in developing the MMPI-3 was to go systematically through. The first thing we did actually was to survey experts, MMPI -2-RF researchers, and clinicians who use the test and ask them, what do you find lacking? What areas would you like to see strengthened? And then we looked at other measures and looked back to, I’ll give you an example when we were developing the MMPI -2-RF, we wanted to develop two separate measures, one focusing on stress-related experiences and the other on worry, but we didn’t have a sufficient number of items to develop two separate scales. And they were highly correlated with each other.
So we constructed a single scale called stress and worry. For the MMPI-3, we wrote additional items dealing with stress-related experiences and worry. And our goal was, and what we did in fact do was to develop separate measures of stress and worry and write additional items dealing with anxiety-related experiences. Some of the items dealing with Pescatore beliefs and the scale, the restructured clinical scale 6, RC6 which is called ideas of persecution, most of the items done that scale are quite extreme. They deal with very significant substantial Pescatore beliefs. And as a result, all it takes is responding to a small number of those items in the key direction and the T score is already in the clinically elevated range.
So we wanted to write some items that weren’t quite as extreme. Dealt with a sense of being mistreated and singled out for criticism, but not necessarily somebody who’s trying to kill me level of persecutory beliefs. So we had a range of objectives for writing new items and exploring the ability to add them to the MMPI-3 to expand the item coverage of the test.
[00:58:30] Dr. Sharp: Right. Thank you for digging into all of that. I know people are curious, how is this different and what changed of course. We sort of crowd sourced some questions from a Facebook community, and there are a few of those hanging out there that I want to make sure I ask. A lot of folks who are curious about the forensic applications of the MMPI-3. Can you speak to that at all? And if it’s going to be useful or valid?
[00:59:04] Dr. Yossi: Absolutely, you know, anytime a new version of the test is introduced, forensic practitioners, and as I mentioned, I for many years did hundreds of competency to stand trial and insanity pre-related evaluations. I would spend my Fridays at the summit County jail in Akron Ohio doing these assessments. So I have a lot of experience both doing this work and research that informs this work. And anytime a new version of a test is introduced, the practitioner faces a dilemma. If they use the previous version of the test, they can be challenged for using an old outdated instrument, and if they use the new version of the test, they can be challenged for using this new unproven device.
It’s an adversarial system, anything you do can and will be used against you in a court of law. The cross-examiners task is to weaken the impact of your testimony and so the introduction of a new version of a test opens the door wide to that. There’s only one way to avoid that situation and that’s to never update our tests. I don’t think that’s a satisfactory solution. So we do need to update the tests from time to time. We shouldn’t be using technology that’s 70 years old, I don’t think. And so I think what the practitioner needs to do obviously is become very well informed about the nature of the change and the potential impact.
[01:00:51] Now in the case of the MMPI-3, we paid particular attention to not necessarily in the forensic context, but one of the things that we did that I think is very relevant to forensic practitioners is, as we were updating the MMPI -2- RF scales, most of the MMPI-3 scales are on the MMPI-2-RF as well, but as we were updating these scales, we were looking very closely at the data to make sure that we weren’t making changes that reduce their validity. In some cases, we substantially increased their validity. We shortened some scales. So you want to make sure that by shortening the scales to make room for new items, you’re not sacrificing validity. And we paid very close attention to that and included in the technical manual for the MMPI-3 there’s an appendix E that has I think over 38,000 correlations just in this appendix alone.
And what these correlations do is compare the correlates of MMPI -2-RF versions of the scales with the MMPI-3 versions of the scales using collateral information that we collected in different sites in mental health settings and medical settings and others. And what these data demonstrate very clearly, and they’re out there for everyone to examine and see for themselves, is that the correlates that have been identified of the MMPI-2- RF scales apply to the MMPI-3 as well, which means that the literature, the peer-reviewed literature that’s accumulated with the MMPI -2- RF can be relied upon when interpreting scores on the MMPI -3. And we go through and explain and provide all the data to support this. As a result, a forensic practitioner using the MMPI-3 doesn’t need to rely only on the validity information that’s in the MMPI-3 manual although there’s extensive validity data there as well.
[01:02:57] Another appendix, Appendix D with another set of 38,000 empirical correlates, we can continue to rely on the now close to 500 peer-reviewed publications on the restructured form and the studies that are beginning to appear in the literature on the MMPI-3 so that the answer to the question, are you using a new unproven devices? No, I’m using an updated version of the MMPI -2- RF that is proven, that does have solid empirical data validity data. When questions of admissibility come up certainly in States where the Dobber criteria are the ones that govern judges decisions about admissibility. It really boils down to the scientific validity of the technology or technique that you’re relying on. And I think there’s a very solid foundation that practitioners can rely on when using the MMPI-3.
[01:04:02] Dr. Sharp: I appreciate that. And I appreciate you giving an example of how you might respond if kind of a question is there. I think people are really curious about that. Another question that came up. I don’t think this is specific to the MMPI-3 or the MPI series, but personality measures, in general, seem to struggle to accurately represent folks with autism or autistic folks, depending on what you might call yourself. Just given the true/false dichotomy and sort of interpreting questions literally, did y’all take that into account at all? Or do you have any thoughts on that particularly?
[01:04:52] Dr. Yossi: That’s a great question, and in fact, that’s one of the areas that we contemplated when we were looking at adding items. Can we develop a measure that would be helpful in assessing manifestations of autism spectrum types of difficulties? And we’ve studied this very thoroughly. It was something that we definitely wanted to do if we could. We wound up concluding essentially the same thing that you just said. It’s difficult to assess autism spectrum-related manifestations by self-report. I don’t think it’s the true false structure of the response options so much as the capacity for introspection and insight and self-awareness that at least some of the variables that are related to the autism spectrum disorder entail.
And that’s not to say that there isn’t information on the MMPI-3 that could be helpful. I think there is. There hasn’t been a lot of research, unfortunately. There’ve been a couple of studies looking at the RC scales, with individuals with an autism spectrum disorder. And not surprisingly, the one RC scale that you do find elevated is RC-2, low positive emotions, dealing with anhedonia- that lack of positive emotional experiences. Although it’s by no means diagnostic in itself, but it captures features of autism spectrum disorder that is found in some not by no means all individuals with that condition. And some of our interpersonal scales disaffiliatedness on the MMPI -2- RF and, and social avoidance, I think also have some relevant information.
This brings up what I think is an important point is that the scales of the MMPI-3 really our measures of what we call Transdiagnostic constructs. These are psychological variables that are not unique to one diagnosis or another, but maybe relevant in assessment of functioning of individuals with various types of conditions and disorders. So if there are some Transdiagnostic constructs relevant to autism spectrum disorder that can be assessed by self-report, then I think we have some scales at least on the MMPI-3 that are relevant to that, but others that really don’t lend themselves well to assessment by any kind of self-report at least from my perspective.
[01:07:40] Dr. Sharp: That’s fair. I know we’d maybe meant to talk about sort of that shift to the more of a hierarchical model. Our time is short, but I wonder if there’s anything more to say on that that we haven’t already talked about.
[01:08:01] Dr. Yossi: I guess what I would say related to that, Jeremy, is that it’s interesting with the MMPI -2-RF that was introduced in 2008, was introduced as a hierarchical model essentially with 3 levels of measurement represented by the various scales. And probably the most significant development in the field of psychopathology research in the last 15 years has been a recognition in the literature that it has nothing to do with the MMPI that psychopathology is perhaps best understood and ultimately diagnosed in the context of hierarchical, dimensional models. The most prominent model being the high-top model that has gotten a lot of attention in the literature. And so the MMPI-3 is very consistent with that movement in the field towards assessing constructs that are both dimensional and transdiagnostic, not unique or specific to any one diagnosis, but also hierarchical. Some are broader and others are more narrow just as are the scales of the MMPI-3.
[01:09:15] Dr. Sharp: I’m glad you mentioned that. I heard Dr. Katherine Jonas from HiTOP on the podcast maybe a year ago, I want to say. And it was fantastic. I like seeing things moving in that direction, I think.
[01:09:29]Dr. Yossi: Interestingly in the article that they published, the first introduced the HiTOP model to the psychiatry community, they had a table that listed tests and measures that assess the HiTOP constructs in the MMPI -2- RF. I don’t think it was a coincidence. It was included in that list of measures. There’s a good synergy there..
[01:09:53] Dr. Jeremy Sharp: Absolutely. So one other thing that those folks were asking about is the, how would I phrase it? Gender dynamics, I can’t quite capture it but hopefully, you’re nodding, you see where I’m going here with the MMPI-3. And I think we’re living in a time where I don’t know if gender is literally more fluid or there’s more coverage or exposure to fluid gender dynamics, but did you all think about that realm?
[01:10:29] Dr. Yossi: Yes, and it’s manifested in several ways. Beginning with the MMPI -2- RF, we’ve been using what we call non-gendered norms with RF and now with the MMPI-3 rather than the traditional gender-based norms of the original MMPI and the MMPI-2 which of course require that gender be specified in order to score and convert the raw scores into T scores. That’s no longer necessary with the MMPI -2- RF and the MMPI-3. And one of the challenges that clinicians faced was that in order to even using Pearson software, for example, score the MMPI -2 you needed to specify male or female gender. The software wouldn’t work without that. That’s no longer necessary with the MMPI-3. There’s no need to specify gender in order to score the MMPI-3 because gender is not a factor in converting the raw scores into T scores.
The other thing that we’ve done with the MMPI-3, we have interpretive report for clinical settings. And that interpreted report by default uses gender pronouns. But there is an option if you’re printing the report to produce what we call a gender-neutral version of the report that does not use gender-based pronouns at all. And that certainly would be consistent with individuals who are non-binary or gender fluid. I think that would be the more appropriate version of the interpretive report to produce; it doesn’t change the interpretation, just the language used.
We still have a lot of work to do in this area. I think this is obviously a population that’s been under-researched including with the MMPI instruments. There actually is some research ongoing at this time with the MMPI-3 and transgender and gender non-conforming individuals. And I Iook forward to seeing that published in the not-too-distant future.
[01:12:53] Dr. Sharp: Right. Yes, I think we all have a lot of work to do in that area. It’s nice to hear you talk through the considerations that you made though to this point. I feel like we’ve covered a lot of ground in this conversation and I really appreciate you digging into some of the personal components and sharing the story. I really enjoyed getting to hear some of that. Is there anything else before we wrap up that may be important for folks to know or just things that you want to highlight? And if not, that’s totally okay. But I always like to give that opportunity.
[01:13:33] Dr. Yossi: Well, I guess I’ll take the opportunity to extend an invitation to anyone listening to our conversation today. When I do training, I always make sure to include my email address on the handout and I tell people that that’s an invitation to contact me if they have questions or if they encounter an interesting case that they would like to discuss. And I’d like to extend the same invitation to our listeners here today. I’m very easy to find. If you Google my name, you’ll find my email address. There aren’t that many psychologists named Yossi Ben- Porath, and so, please do feel free to write and I always try to answer questions. And I’m happy to hear from MMPI users and non-users what questions they might have.
[01:14:31] Dr. Sharp: Fantastic. I’ll make sure to put that and a bunch of other resources that we’ve talked about in the show notes so that folks can access that if they’d like. And I know you’re in our Facebook community as well, The Testing Psychologist Community.
[01:14:43] Dr. Yossi: Yeah, I’m happy to respond there. I try to respond. If someone tags me in that community, then I know to respond and I try to do so.
[01:14:51] Dr. Sharp: Fantastic. Well, thank you so much. This was a great conversation. I really appreciate it. It was great to talk to you for a little while.
[01:14:58] Dr. Yossi: Thank you, Jeremy. I really enjoyed it.
[01:15:01] Dr. Sharp: Okay everyone, thank you so much for listening to this episode. I hope that you took a good bit away from it. Yossi’s fantastic interview, super knowledgeable, kind, articulate, and I really enjoyed being able to talk with him. So everything that we mentioned during the episode is in the show notes. You can check those out.
Otherwise welcome to 2021. Here we are. I hope that this is the beginning of some better times for many of us. So I will be back with you on Thursday. We’re going to resume our EHR review series. And I hope you enjoy. So take care and we’ll talk to you next time.
The information contained in this podcast and on The Testing Psychologists website are intended for informational and educational purposes only. Nothing in this podcast or on the website is intended to be a substitute for professional, psychological, psychiatric, or medical advice, diagnosis or treatment. Please note that no doctor-patient relationship is formed here. And similarly, no supervisory or consultative relationship is formed between the host or guests of this podcast and listeners of this podcast. If you need the qualified advice of any mental health practitioner or medical provider, please seek one in your area. Similarly, if you need supervision on clinical matters, please find a supervisor with expertise that fits your needs.