HBO Max Persona Documentary Discussion

Keith McCormick and Joe Arrigo discuss the latest HBO Max Documentary ‘Persona’

The following is a minimally edited transcript. Apologies for the mistakes. Authenticity in dialogue is vitally important.

Joe Arrigo – INTJ
Yes, we are live. Super excited for this. I’m Joe Arrigo. I’m an INTJ. I am joined by Keith McCormick he’s an INTP. And I need him here for this because for the people that saw our last interview, I’m not a man of science. But Keith is a data scientist, and he’s gonna help kind of talk about some of the aspects of this persona, HBO max documentary. The way that we got here was kind of serendipitous. We had an interview, just scheduled to talk about some of the critics critiques the memes that are pervasive in The MBTI community. And this kind of these stats that keep coming up, you know, for example, the 50%, test, retest the attraction that’s like, well, it’s not reliable, it’s not valid. And just so happened. Like, as we were setting that discussion up, we learned that this persona HBO, Max documentary was coming out. So we’re like, well, we got to talk again. And we both watched it. It came out on March 4. And the response from The MBTI community has been overwhelming. I didn’t realize I was gonna see so much on Twitter and so much on LinkedIn, what is the response? Man? What have you seen Keith? And also thanks for being here?

Keith McCormick -INTP

Oh, you’re well, I’ve been looking forward to this. You know, I saw I saw some of the activity on Twitter, but mostly I’ve been chatting with, you know, my friends in the Association of psychological type. That, you know, I’m sure you’re familiar with most people are was founded by Isabel Myers, daughter in law, who you know, who married you know, Peter, anyway, no, no, a number of folks on that, because I’ve been doing the type thing for quite a few years.

Joe Arrigo
I saw like Roger Pearman did an article and it was very good. And I think another lady Doris Fullgrabe, but I think sorry, if I butchered that name, but she’s like in Germany, and she wrote a very long thing on it. So I think there’s sort of a unified front. And not like necessarily circling the wagons type deal. But definitely, just like, we all watched it. There were some things that were Yes, we all agree on. And then there was a lot of it that we were just like, I think we need to discuss that part of it, which is maybe some of what you’re going to talk about. And I think one thing that we should start with, you know, Kathy O’Neill, you’re very familiar with her work. I know, you’ve kind of chatted with her a little bit. At the beginning of the documentary, she kind of mentioned something about this idea of weapons of mass destruction, and how it applies to all aspects of like, discrimination or like secret cabal type blackbox algorithms, can you kind of go into next I know, that’s your specialty. And I know you have a really good point to make.

Keith McCormick
Sure. So, um, to clarify, I don’t know Kathy O’Neill. Personally, I’ve seen her speak. She’s given keynotes at conferences and conferences where I was a speaker, you know, as well, you know, and I’ve assigned her TED Talk, as a discussion kickoff. In my UC Irvine predictive analytics course, I’m quite familiar with the, you know, with the arguments that she makes. So, you know, what we’ve talked a little bit offline is that I think the two-thirds of the film that really has Kathy O’Neill, at its center is very compelling, I think, at least now, if Cathy O’Neil and I suddenly found ourselves in some kind of a panel or something like that, I might not agree with every single, you know, point that she makes in the book. But the basic notion of what she describes, I think, is very convincing. And it’s this that if you’re going to have an algorithm that is going to affect your life, your ability to get a mortgage. Something like the length of time that you’re on parole after prison, ability to get a good interest, credit card, and so on, that those algorithms should be transparent, that we should be able to see what they are, what they’re doing. Now, owners of algorithms like that don’t like to do that because they’re proprietary, and they don’t like to release those details. But when it’s impacting someone like that, I don’t think he really had much choice, there has to be transparency. So she gets a tremendous amount of credit for making that point. And starting that discussion. Where I might differ with her slightly is that she’s left the field to concentrate on things like algorithm audit, so she doesn’t spend a lot of time talking about how we can, you know, fix such things. But if you read her book carefully, you can tell where the real problems are. But the other thing that’s so relevant to this Because there’s the sexism, racism theme that goes throughout the whole movie, is that Kathy O’Neill is very specific about how sexism and racism can manifest themselves in these algorithms. In fact, this is the main thing that she talks about. And again, I find it very compelling. So she thinks, hey, it’s math. Math is inherently objective right now. subjective, its objective, she goes, but watch out. If the data on which it’s based is historical data. And there’s been historical bias, then you’re going to continue to have a problem. So for instance, take something like recidivism, frankly, I wish the whole movie had been about Kathy O’Neill, because then it would have been a movie, you know, but take something like recidivism, um, you might be told, there might be a company develop an algorithm, and the company that’s developing that is told you simply can’t use race as part of the algorithm will say, Oh, no problem. I’ll use a nine-digit zip code. I’ll use neighborhood, I won’t use race, I’ll use that. Okay. And that’s a real, that’s a real problem, right? Because they get around it, or what they do is they say, do you have any family members that have been in prison? So now they’re kind of what you know, particularly if you’re going to let the previous generation? You see what I mean, you know, or do you know, anyone in your neighborhood that’s been to prison might be like a parole question, let’s say, Oh, well, you’re not going to, you’re not going to get parole? Because you know, you know, people. Right? You see, you see the problem? Yeah. So the issue that I have with the other 1/3 of the movie, you know, which was about The MBTI, is they make this leap into the racism and second sexism, but they never explain the mechanism by which that occurs, there’s a lot of innuendoes. But if you listen carefully to Kathy O’Neill, particularly if you bring the contextual lens that I would bring it, somebody that’s built these models, and is trying to be very careful to make the models good models. If you bring that context with you, you can see not only why they might do it, but how they might do it. But all that detail is completely missing on the other third.

Joe Arrigo
I think so too. It’s funny how you and I have I mean, obviously personify plays about so many different things caught my attention. And I think that I’m glad you’re giving credence to that, like what her nuanced stance is. And I think the broader picture of what you’re saying is that, you know, her belief is that sometimes, like this personality credit score, like your social credit score, will be something that is almost like being on a no-fly list at an airport, where it’s like, this person is going to apply to a job through LinkedIn, or indeed, but this back end software is going to say, Oh, this person scored a red flag area in this for this job. So thus, you’re never going to get this job ever again. And there’s no way to access the company to fix your credit score. So there’s going to this is going to plague this person for their life. And that would be a very, that’s a cause for concern. So her her stance throughout the documentary is fine and valid. Where I think someone who’s wasn’t was the Lydia I forgot her last name. But just the kind of I think she ended up kind of being a caricature of somebody of a critic where it was just like, type is inherently not able to see its own racism in western imperialism in its construction, with a lot of ad hominem attacks against Isabel and Carl young. Some Like, this doesn’t feel like I understand the emotional play. But I think it just played itself, in terms of it doesn’t it doesn’t actually carry a lot of weight.

Keith McCormick
I left the movie being a Lydia brown fan, actually. So this might be like our Siskel Ebert moment.

Joe Arrigo
Okay, okay.

Keith McCormick
I know I can, I can see what you mean, because through editing, they would always go back and forth between the two. But I don’t recall what your brown ever saying anything about type. I’m quite certain that she never did. Right? Yes. Although they were always going back and forth. Right? Because she was always talking about there was one particular moment I recall where she was talking about, you know, something being normed on straight white college-aged, college-educated males, right. Yes. But think about what that means, right? Almost certainly what she’s describing there as a bell curve, because remember, she’s a she’s an attorney. She’s coming up this and there’s a scene if I recall where She’s in a meeting preparing for the congressional testimony with Kyle Boehm dad. Roland? Right. Right. So what would she mean about it being normed in that way? It seems to me that she’s describing a bell curve. In other words, she’s basically saying is if you know, if you do a whole bunch of Americans, you know, but by Americans, maybe it’s predominantly straight white college-educated males. Right, right. And then you’re looking at notice, notice, too, that Kyle was scored red on a particular scale, right? That indicates to me that what’s probably going on is you’ve got a bell curve. And that Kyle came out something like two standard deviations above the mean, on what is almost certainly neuroticism on the on the Big Five. Right, right, right, trying to get into some kind of battle rail with with the big five here, I’m just saying that all the language that’s used, and that two-thirds of the movie about Kyle Boehm and hiring is all very normative bell curve, you’re far from the center kind of a thing, right? So the point that Lydia Brown is making, I actually think it’s quite compelling. Now, she’s definitely taking no, take no prisoners style, I think that’s why I found her so charming. But, um, the way that she’s describing it is you have this center that’s described as abled white. Male, right. And if you’re distant from that center, you’re not the norm. And by the way, we’re not going to give you a job.

Keith McCormick
Okay, um, I think most people on the tight computer community would find that argument pretty compelling if it was framed in that context. And it’s not at all clear to me how The MBTI could even if it was used for that purpose, how it could be used for that purpose. Because right, nothing about the instrument that serves that purpose.

Keith McCormick
There’s no bell curve. Right?

Keith McCormick
Right, which is the critique that the Big Five folks make of type. And again, I’m not trying to get into what we’re trying to do, but the movie, not this big, you know, battle between MBTI and Big Five. But the whole idea of the Big Five is that most people are in the center. But some people are out at the tails of the distribution. Oh, and by the way, they’re strange in some way. Right? You know, it’s getting it’s getting a pathology in a way that type does it.

Joe Arrigo
Well, you’re right. And I, maybe I’m talking about so I remember Lydia, she’s the lawyer, but there was a gal. There’s an Asian gal that talked about how she was she felt like because she was autistic. She said she was she’s the one that I’m referring to that became a caricature. So maybe they both have the same name, or I’m getting her name wrong.

Keith McCormick
We’re talking about the same. Oh, we are okay, one. Yeah, she was a young Asian. Yeah, you know what, cystic a woman but I believe she’s also an attorney.

Joe Arrigo
Gotcha. Okay. Okay. So the some of the things that she said that I I’ve kind of found paradoxical, or, like, you’re selling MBTI. At the same time, it’s like, You’re, you’re saying it’s negative. So she said something like, you know, they’re able to sexist racist classes. young, black, college-educated, straight white man with no known disabilities. But then she goes on to say, it’s a good tool, if you’re on a journey of individual self-discovery. But when someone uses it against you, it’s dangerous. And I think no one would ever, that’s not a criticism. Like, of course, if they use it against you in a hiring or professional standard, it would be can be used negatively, but most people use it, and you’ve set it. It’s usually one on one. So it’s for personal growth, where it is most effective. And she’s just Yes, it is. It’s very effective there, but not but part just, I stopped listening because like, well, that’s why people use it. Yeah. You’re selling me here.

Keith McCormick

I find her very compelling. So I listened pretty carefully. I don’t think she was ever talking about The MBTI. No, I get I get what you’re saying when she said self-discovery. For those of us that are fans of Type that goes Oh, she must be talking about type,

Joe Arrigo

I guess. Yeah, I think she was talking about personality in general terms. I think that she’s basically making almost precisely the same argument that Kathy O’Neill is, which is when it’s low stakes, that’s up to you. But when it’s high stakes, that’s a different matter. I could be mistaken. But I have a feeling that Lydia brown isn’t particularly knowledgeable about or interested in The MBTI. But she does care deeply about advocating for populations that might be biased against by a normed instrument. You know, and then have that label, follow them. You know, so I put her in the Kathy O’Neill camp, but I could be mistaken because it’s all done with editing, right? So with editing a lot of fast cuts back and forth.

Joe Arrigo
You and I have different perceiving functions. So we’re both going to take in data differently and have a different determinations. That’s why, you know, it’s better to have an INTP in here rather than INTJ. Because you’re going to bring a different angle. So I don’t think the interpretation is wrong. So I could be wrong to let me go back to the beginning. Because there’s a couple notes to the bullet points i was i was furiously taking notes like, Oh my gosh, I had to stop and I had to keep stewing over something. So I like how the, I think, in the first part had a really good trajectory. Like it started off really good. Had Frank James, had Lindsey Johnson (or Lijo), who’s pretty big in the type community. And it kind of was like, and then even more of Merve Emre came in there at the beginning. She’s like, I’m an ENTJ. So like, I was like, this is an interesting intro, because it’s showcasing people that are popular. And Frank, James probably got like, I don’t know, 10 minutes of time, maybe less, but he got a pretty good section time. And he sold it for me, he was like, I was an INFJ. I didn’t know there were other people in the world that understood the way that I think it felt the way that I felt. And it was just like, you know, having an outlet? And I was like, Yes, yes, this is it, like you’re selling it. And then like Joe said, some of the same thing. And then Megan Levota, she’s in the thread right now. She was, uh, she got a little bit of screen time in there as well. Um, and then they kind of, you know, had a quote from Carl young saying, Man itself, man itself is the danger, we are pitifully unaware of ourselves. Like, yeah, that’s, that’s why you create instruments. So that took off for me, like, there, I can see where they’re gonna take the, you know, they’re gonna, but here’s the evil dystopian future. And I do think some of the points where they brought in David Scarborough, the creator of the unit, unit crew, or unit crew, and then they brought in one of the guys for HireVue, who does the AI facial technology. That’s where the like dystopian future really came in, where it’s like, this is such a big platform where we’re, you know, David Scarborough was talking about, you know, we were doing 70 to 90,000 people a year of all job titles, all from like, lowest level, the highest level. Like, that’s where I can see that being, I mean, so pervasive that there is a credit score assigned to you. Well, you blame, but go ahead. And what are your thoughts there? Well, I’ve

Keith McCormick
got a bunch of things on my mind at the moment, but yeah, the fact that there’s only two or three companies that do that, in that there does imply some tracking, it seems, you know, I’m, I did some research back in the 90s, on the LSAT and, and type as some people that are actually on the call are familiar with. And a lot of people probably don’t remember this, but the essay t items used to be secret, you know, so other than memorizing them, when you took the test that was it. So you know how you can go to a bookstore, go online and do practice questions, you could be that you couldn’t do that it was absolutely secret, right? And you could get into trouble trying to memorize a question, like, share it with somebody or something that, you know, it was it was super proprietary. While the problem was until they started releasing old questions, there was no mechanism by which people could correct mistakes in the questions. So that’s why I kind of have a long history with this. If the models not transparent, it’s gonna problem. So if you combine the lack of transparency with the tracking, you know, I’ve got some issues with that. And, you know, for those of us that are fans are tight, I think we run into, we run into trouble with that, because I think most of us would agree that it can be very powerful for team building, you know, but if we don’t speak strongly in one voice about how we feel about it not being used in hiring. We, you know, we endanger its ability to be used for team building after hiring, which I think we would all, you know, find very valuable. So, I want to, I want to circle back to what you were saying about your friend James, and I know that you interviewed him a couple of weeks ago, and that’s kind of how I first learned that name. Anyway, so I know, he’s super popular. He’s got hundreds of 1000s of, you know, followers. I think he’s, uh, you know, I think he’s great, you know, this kind of comedic take on type and everything. But, you know, they happen to grab a quote, where, you know, it’s this classic, you know, the indicator knows me better than me, you know, kind of a thing, which, you know, is all true. But wouldn’t you agree that most of us outgrow that pretty soon. I mean, in other words, that, in other words, I don’t think I needed the indicator to know that introverted thinking is something that I do. So after that initial like, oh, wow, you know, that’s really powerful. I think what keeps us interested in the indicator for decades, those of us who’ve been doing this for a long time, is what we learn about the functions that we don’t, that aren’t that conscious for, and about others, you know. So I think, again, through the editing, there’s somewhat playing to stereotypes there that you take the indicator, even though the indicator is worth was, you know, in their opinion, you read the description, and you find the description so compelling that you’re still entranced with it years or decades later, almost like it’s some kind of hypnosis, you know, going on or something. And that, that wears off pretty quick, what makes it powerful, is when you realize that through this language, you can connect with someone that you didn’t naturally connect with. But now you have a way to build a bridge there. And also to give an idea of where you might be when you’re older. What with what are some areas, you know, for growth? So when did it really click for you? I mean, for me, just reading a couple of paragraphs description, it’s not like, you know, the clouds parted. And you know, there was singing. For me, it was like the first time you sit at a table with nothing but six or eight iron teepees that you’ve never met before. You know, you go wow, this is pretty, this is pretty powerful. You know, and after 45 minutes or an hour, you’re called back to you know, reconvene with a conference or whatever it is, and you kind of don’t want to leave that table because you’re it’s really powerful. See you soon. I mean, I think I think they’re playing to it. Even though print James was very effective in the in the film, I think, advocating for it. They’re, they’re just playing to the same stereotypes, you know, that we all read these descriptions. And, you know, it’s the whole Barnum effect thing, and we’re completely eaten by that. But that’s not that, for me, at least was not the moment when it completely clicked. And I was persuaded It was the first time I was in a type of like, group, like all four letters, like, right on trivial size, which is why I think what Joyce Mang does is so powerful.

Joe Arrigo
Absolutely, yes, that’s a great project she’s endeavoring on to showing just the spectrum of type and how they actually represent themselves. That’s a that’s an awesome thing she’s doing. You kind of alluded to something that I fell in is this has been criticism as well, Merve Emre said something, like she focused on the seductive language of it with the dichotomies, give you like, it’s the seduction, the cult, the cultish theme of it is she kind of alludes to like you’re hypnotized by type and then you get sucked into this like, even though there’s no like David Koresh type leader in type, like, I guess Carl Jung would be that guy, but not even close. But there’s an idea that, that is, these crazies in the type community are just, you know, the first time they read INTJ they’re like they’re drawn in, and then you can’t get them out. And you can’t use facts with them. And that’s another critique that I felt was also in there as well. Because I kind of alluded to I think Katherine had the, the home the home lab, as she called it. Yeah. So like, yeah, that’s, I mean, the way that I feel like she didn’t realize how that would be taken and calling it a lab, you know what I mean? But, uh, so that there are some valid critiques there was like, well, it was a lab where they, like, was it ethical studying, was it forced, like, that was also a pervasive kind of critique throughout, as well. Yeah, I

Keith McCormick
remember the line that you’re referring to, she says that, you know, well, what is it about this language that people find so seductive? In other words, she’s basically saying, how is it that this many people have been duped? You know, it’s basically what she’s saying. But then she goes, and then the language won’t let you go. And then there’s even a slight turn to the camera. You know, it seemed very staged. And then that, that that moment, there were quite a few moments like that. I mean, during the interviews with Lydia Brown with certainly with Kyle Boehm and Rowan Boehm with Kathy O’Neill. It was just the cameras rolling and they’re just, they’re just talking very naturally. But with the scenes with Merv, am rain I don’t want to sound overly harsh, but they always seemed very staged to me. You know, rehearsed lines, slight turns to the camera, you know, dramatic pauses, lots of music. You know? Yeah,

Joe Arrigo
very ominous. And there’s something so okay. Surprisingly, Merve Emre I don’t think she’s like, doesn’t become the focal point of the story, even though I think like it’s based off her book. I felt like that. I thought there’d be more of her, but move into a different point that I want to talk to you about because I did not even realize this would come up But I actually highlighted as one of the main things I want to talk about. So Richard Thompson is the Senior Director of Research at the Myers Briggs company, the foundation, he talks, he kind of represents he’s the face of MBTI, in this documentary, and he says that it was a big mistake for a lawyer to not copyright the letters. And I was like, well, thank God, because then there being that you couldn’t even talk about it in public without being certified. You know what I mean? So, for me, that was like a, that seemed like a negative comment from the corporation. Yeah,

Keith McCormick

I didn’t know what to what to make of that. But you know, I think there’s also confusion around the relationship of most of us with an instrument, in fact, I would love to have, I would love to have a little study done, where you have a group of individuals that are, you know, going through a feedback session, let’s say, and you skip the instrument, I’m not trying to be overly controversial here. But there are there are a number of folks that will do workshops, and not use an instrument, right? And I’m not sure if people are, you know, aware of this, you know, but, but it’s quite common. And, you know, what’s the kind of thing that people talk about? You know, when a PTA, so let’s say you have, it could be, it could be any number of reasons that you do it. But it seems to me that the only argument for an instrument when you’re doing something like a workshop, is that it reduces the amount of time it takes for folks to conclude what their type is. But you know, so if I’m doing a feedback session with somebody, I’m doing, like, almost like a little type club, but at work at the moment, and I had one on one sessions with a with a half dozen folks that are in this group. And if they take an indicator beforehand, I can usually do a feedback session in about 45 minutes. If there was no indicator, I’m sure it would probably take more like three hours. Okay. But what people forget is, once we’ve taken the indicator once or twice, we never take it again. It’s not like, sometimes I think that someone that is, you know, taking a critical point of view, like, Oh, you know, like that, and the personality brokers is a lot more obsessed with the instrument than us. For us, there’s a there’s a famous one in vidkun Stein (?), he’s one of my favorites, where you climb up the ladder, and then you push it aside, because once you’ve used the ladder to get up, like to the next level, you don’t need it anymore. Okay. For most of us in our personal journey, that’s what the indicator was it kind of helped start the conversation, and then we don’t need to take it again. There’s, um, there’s a question by Megan that I want to get to, as well. But I don’t know if you go…

Joe Arrigo
So I’ll read it. So I have a question for Keith. I’m an IO psych master’s student. And going to the SRP conference in April, when my professor sent out a message to our program about the documentary, she said that it was unfortunate the valid instruments for personal selection are being lumped in with the invalid MBTI. When I reached out and mentioned that they were there are other uses for The MBTI, like improving team building communication, self-development, she asked for resources. Does anyone have any? Yeah, so

Keith McCormick

to two quick things, right. And we can put, we can put something in comments. So yeah, we did the talk. Last week, I posted the reliability and validity numbers for the forum am just in case people want to read it. Now, not everybody is going to know how to read that. But maybe you absolutely will, right, because of your background, but it’s not going to be the percentage, it’s not going to be the percentage retest because that’s not how psychometricians do it. It’s internal consistency, usually as a Chronbach’s alpha, right, so we’ll put that link in there, all you have to do is is Google reliability, validity form m MBTI. And you can find this, it’s on the publishers website, but you can do it, it’s all the numbers, right. But the other thing that you should do is check out the form m manual, right, which is substantial in size here. And what we want to do too, is you want to check out the whole section on the form M and other instruments. So for instance, you’ll find that the Ei scale on the Big Five correlates with the Ei continuous scores on The MBTI point seven. So it’s kind of crazy to say that the big five is like this amazing, perfect thing and that The MBTI is nuts, you know, in the you know, was bogus, because there’s no way that a bogus instrument and about what instrument would be correlated 2.7 right. So but there’s no reputation with an IO psych has been you know, weak for some time, but people really aren’t doing their homework. You know, the other thing that people have been talking about lately, you know, like below the movie premiere and stuff is you know how it was, you know, the whole kitchen table in the 30s thing. It’s just kind of crazy. The forum am was started by Bill and 43. And I don’t want to take any credit away from Katherine. I think she obviously introduced type to the family and the whole nine yards. But you know, as far as the instrument goes, Catherine did not have a huge chunk of the instrument now, right? I get to develop Isabel’s. I heard Peter when he was still alive, give a talk, you know about this. And he was actually saying Katherine sometimes didn’t get enough credit. But that’s because he was talking about an environment where Catherine was almost never mentioned. And it’s because it really was Isabel Isabel that wrote the instrument. Because it’s just simple arithmetic. Katherine was born in 75. The form a was written in 43. Katherine was an elderly woman when the bell began the process of writing the instrument. And it really didn’t become a sophisticated instrument until Isabel met Henry Chauncey in 57. And she was hired as a consultant by ETS and started working with the psychometricians. In ETS, by the way, the big five, what was it was the development on that started in 62. So this whole, like the big five is contemporary. And this was the kitchen table in the 30s. It’s just people just haven’t double-checked the facts. Isabel meets Henry Johnson in 57, they begin working on the 62 manual right away. She has fans at ETS, but she has this real enemy at ETS, David Stricker, who was in his mid-20s at the time and just had a chip on his shoulder, you know, for Isabel and stuff. But that’s really when you get the form f and form G, which somebody like me would have encountered when I was much younger, you know, like in the 90s. And then the form M was done in 98, using completely different items, and a completely different test design approach than Isabel used in 43. That’s why I have a problem with the embedded racism, right? I’m not saying dismissing it, because I racing that argument, as it’s made by Kathy O’Neill and Lydia Brown, but because they explain the mechanism by which this can occur, right. But there is no such talk of the mechanism when it gets to the other side. And sometimes it’s just not true. Like I’ve got my old form G. Can you with these are, are different never seen them most people never had, can you see the little, you know, I’ve met Yeah, that’s I’ve never seen that are used to put these this is like 80s era stuff here. You used to put this over the number two pencil stuff to self-score it.

Keith McCormick
And you’ll never be able to see this on webcam, but I’ve got the TF, male and female, right. Okay, and you probably can you see that, you know, where you’d be able to sit down here, if you were holding up to the sunlight now, you’d be able to see that they’re almost identical. But the reason that Isabel had to make slight differences in the two was Mary used to always describe it this way, it was kind of like a cultural wind. That was kind of pushing. So for instance, let’s say,

Keith McCormick
men, right? You were to come out on the fence between INTJ and INFJ is about what argue take a real close look at INFJ. And there’d be a couple of reasons, you know, for that, right? Okay, we’d be trying to figure out your Auxilary, not your dominant, right. And the cultural wind would be such that as a guy, you would be there’d be cultural expectations that would favor tea, and we’re only talking about a 6040 split anyway. So the reason that Isabel Myers had to struggle, and it really was a struggle to figure out how to address this. It was really just a tiebreaker issue and a couple of items that just didn’t seem to overcome this cultural wind business, but Merve Emre is quite explicit a movie and this is just objectively false. I mean, she’s just wrong about this. She said that Isabel’s belief was that women were biologically unable. I don’t know if you caught that phrase, but that’s a phrase that she uses in the film, right? But that wasn’t at all what was going on says they actually the opposite of what she was arguing there. And she felt that whenever a woman took the indicator, and was on the fence between t and f, that for those women, they probably favorite tea. Right? Because the cultural wind was pushing them towards F in the reverse would be true for men. So this is something that they always struggled with. Mary was just saying, Well, is there some other way to address this? How do we do it, but it was really trying to increase the reliability and validity of the T f scale was the reason to do it. But it was always an issue of debate. And then in 98, when the form M was done, it was addressed in a different way.

Joe Arrigo
Interesting, yeah, you have so much history, that’s just it’s so needed. Now, you know, my generation to know. And I think her daughter, Isabel’s daughter, got a little bit of screen time. Hughes,

Keith McCormick
I think, Great great, granddaughter,

Joe Arrigo
granddaughter, and give gave context on like, all, First of all, World War Two, and tolerance coming out of this era of disharmony. And that, you know, Sally had to make the point that Isabel didn’t like Hitler, and wasn’t a part of this Ubermensch ideal. So like, he was trying to promote tolerance. So like, actively against the things that Lydia would say, formed the framework of it. It’s like, even when I interviewed john Beebe, he’s like, you have to understand the time that was created was trying to preach more tolerance and understanding. But yet the people say no, this is like some people like take this extremist approach that it is an extremist framework. So it’s like, that part of the history of what it is vitally, as vitally important as any other data science point you want to point out in my mind.

Keith McCormick
Mary, when Mary would talk, you know, there’s, um, there’s, um, there’s a quote in the personality brokers. And as you know, I knew Mary quite well, and I was quite young in 87. And then she died around year 2000. So, but between 80 No, no, she, she died after that. Why can’t I remember? But, gosh, what was I gonna say? But oh, you, okay. Mary would be quite explicit about this, when she would give talks, she really felt that the indicator, if it was a good widespread use would promote world peace. You know, I mean, I think, I suppose somewhat of a stereotypical INFP comment, perhaps, you know, and, and it’s easy to try to make, make light of that or make fun of that. And I and I think in the book, it comes very close to, you know, to doing that, but we’re where that quote appears where speech of Mary’s is, you know, was pulled out, as she’s talking about two geniuses. And in the quote, it’s never clarified who the geniuses are. And I think we’re left to suspect that it was, you know, Isabel and Katherine, you know, which is, which is fine if we’re supposed to conclude that, but I think it was because it was in this, it wasn’t really because it was an accurate quote, it’s actually not accurate. It’s because it was in the, in the service of this narrative that it was about these two women that had these beliefs, you know, in the 19th century, and then up through the 30s, that we’re still living with now, because those beliefs are somehow embedded in items even though the items have all been rewritten, right? So right, the quote is there in the service of that narrative, but who Mary man was Isabel noon.

Joe Arrigo
I see. Okay, well,

Keith McCormick
alright, because Mary used to talk about that quite often in the reason that she felt that, that Isabel was a genius. I mean, one she knew Isabel directly. So it was it was her conclusion to draw, but also, you know, spending decades wrestling with this thing. And I know another thing that Isabel found impressive is that, you know, working on the format, through the form he or whatever it was between 43 and 57, that when Isabel was working with the psychometricians at ETS, Mary would always relay that Isabel had intuited a number of the concepts that the psychometricians you know, had learned in school but she had done it by basically persistently wrestling with this thing and trying to get it to work. And I think that would be true of you know, so many folks that make amazing discoveries through just incredibly hard work and persistence. Randomly just thought of something like a Gregor Mendell, you know, who’s who was a monk, you know, the wrinkled pea smooth p guy.

Joe Arrigo
I don’t, but

Keith McCormick
but he’s, um, he’s, he’s famous and he’s famous and heredity, and it was almost he was almost forgotten because he learned about some early hereditary mechanisms long before we knew how DNA worked and stuff like that, but he was a monk that was growing peas and the bag yard of the monastery, you know, but wrestled with these ideas for a long time. So as Mary would relay it anyway and right, I never met Isabel, you right. So I can’t say this firsthand. All I know is that if you saw Marian when she was telling these stories, it was certainly a sincere thing that she felt that Isabel was a genius. Because she wrestled with these ideas, and came up with solutions. Even though she had to come up with these solutions on their own. We forget to that when Isabel went to Swarthmore, around 1920 or so. There were no departments of psychometric psychometrics, they didn’t exist, they did not exist. An interesting biographical detail of Henry Chauncey is that he went to Harvard. He was classic white patrician, you know, kind of a family, you know, very interesting. Guys spoke to him briefly on the phone once. And when he was like, 100. That’s an interesting story in itself, because I asked her about why why he overrode his these younger psychometricians that hated Isabel and 57. Right. And it was, it was partly because she was trying to do something that didn’t involve pathology. And he was very, you know, he was very taken with that. But when he was away after being an undergrad, he was administrative faculty at Harvard. And then he went on to do grad school. He did grad school at Ohio State. Why? Because Harvard didn’t have psychometrics. Harvard didn’t have psychometrics at that, at that time, right. But Ohio State to this day is famous for their psychometrics program. But this whole, you know, they weren’t formally trained, no one was formally trained, no one was formally trained in this stuff at the time. Now, by the time you get to 57, you did have people that had degrees and the stuff, but they were in their 20s. And you’re gonna imagine what a white guy in his 20s who just came out of an Ivy League program, thought of Isabel in her 60s. Right, they did not get along.

Joe Arrigo
This is my main, this is like my main point of anger with these Vox articles, this appeal to elitism and no real psychologists or no, no trusted institution, or magazine, or, or publisher would ever really take MBTI seriously, it’s like, Where do you think this came from? It didn’t like it didn’t come out of the elite schools. It didn’t come out like he came from the Pennsylvania housewife. Right? So it’s like, now they’re appealing to their own? Well, thanks for starting this thing at the kitchen table. But we’ve got it now. Because we’re elite and we fixed all the stuff that you messed up, and we don’t have any prejudices at all, were post prejudicial,

Keith McCormick
that you know that that reminds me of something too, right. So you have to kind of picture how this all went down. So you got the 20s and 30s. What, what Katherine Isabel were doing together, this was the true collaborative part, you know, as I understand it, right, because certain one thing that Maria Marie certainly gets a lot of credit for, she’s read more family letters and correspondence on the Katherine side than probably anyone else, you know, because she found the Catherine cook, you know, letters, but that’s why, you know, I’ve mentioned this before. That’s why if you look at her book of, you know, going out here, all the way until I’m we’re talking chapters one through 12. When you get to 12, you finally get to that ETS period, which to be honest, I there’s not, there’s not a lot there. But you know, two-thirds of the book is on, you know, was on Katherine. And really, as far as they got at that stage before Katherine was elderly is they basically had this kind of little focus group really is what it was they figured out the types of their friends and family. But that experience allowed Isabel to try to develop items. So let me briefly describe how items work. And when I do this, I’m talking about items generally now, not just MBTI. And a little bit about the PSAT too, because remember, in the 90s, I was researching the two of these together. So yeah, so I had to learn quite a bit about the LSAT. So what is about what have done in 43, as you would have had to do a lot of items. I mean, a lot of items, hundreds and hundreds of items. More than 1000 is my understanding then Okay, so that’s the famous three by five cards, right? So why do you need more than 1000 items if the form f has to died at 180, or something like that, and the form q or la 260? I mean, know that Form K? I’m sorry, the Form K is the expanded analysis. You get the subscales. Anyway, there’s all these things to remember. Yeah, but why do you need Why do you need more than 1000 items if you’re only going to publish a couple 100? Well, this is where people don’t Understand how psychometrics works. You can’t use science to write the item, there is no science that will tell you what words to put in a row on a three-by-five card that requires human creativity, right? I think it’s just in our intuition will tell us that right that. That’s why you have to come up with an item, but most of the items won’t work. That’s where psychometrics comes in. So think about what’s happening in 57. Then when Isabel meets Henry Chauncey, she’s got all these items, but she’s trying to figure out which items are going to pass the psychometric tests. So the choosing the items was not done, not done over the kitchen table in the 30s. The selection of the items and the format, and the form g was out of a pool of hundreds of items that Isabelle had done this beta testing on basically, yeah, but that were finalized in the format from the forum. And that’s what people forget. So when I was learning about all this stuff, when I was much younger, I had this conversation with a guy at ETS. And I said, you know, it seems to me that coming up with a sh t question is kind of a creative act. You know, you can’t really science your way through that. You just have to kind of be creative and come up with the item. But then the psychometricians get their shot at it, and they figure out if it’s a good item. He said you might not realize how correct you are. He said my nephew was interning here as a sophomore, undergrad. And he helped write some items. And one of his items made it into the made into the SCT. Okay, think about what I’m describing here now, right? I’m saying that when you write the items, when you’re really doing is you have to make up for it in volume, you have to do a ton of items, because a lot of them simply won’t pass muster. psychometrically right. Okay. So when I was talking to this guy, he asked, and I was starting to get a feel for this, you know, he was one of the psychometricians. At ETS, I was saying, it seems to me you have to come up with a gazillion of these items, and then only a small fraction will make it through. He said you don’t realize how Right you are. My nephew was here for his like summer internship. And he was writing items. And he said one of them made it into the test.

Joe Arrigo
Okay. What’s your point is that that’s negative or that’s…?

Keith McCormick

what I’m saying is, is that’s what Isabel was doing in the 40s is she’s iterating items. But they all say that there was no training, there was no psychometrics involved. Not and it’s not really true. Because Isabel during the form A through E, when she was experiment, the bulk of what she was really contributing was spending years generating items. Yes, items that passed psychometric muster were chosen, starting in 57, working with the PhD psychometricians. At ETS, there’s two distinct phases here. There’s this creative phase where you come up with the items, and then you have to decide which items work. And it gets a little technical, but the whole idea of a chromebox alpha, which is when Megan was asking about the what she could show, you know, the faculty, what a chromebox alpha is, when you look at all the items, do they seem to go together? Do they belong together? Do they make sense? Because Mervyn Emery will suggest that Ei is simply sociability, which is not true. It’s actually multiple subscales, right? You have dozens of right, different things going on there. But that’s the point that I’m trying to make is you’ve got these two distinct phases, the generation of items, and then the choosing of which items pass muster psychometrically. And this collection of items in the form f and g were largely done in that period between 57 and depending on when you want to start there was kinda Isabelle had a falling out with ETS, and they didn’t officially stop until 75. But basically between 57 and 75, and it went through a number of phases during that period. That’s when you really get the form up and gee.

Joe Arrigo
I reached out to Merve Emre’s Team. She was too busy to be interviewed by me. So you know, too busy so I would this would have been great to have her like to discuss that part with her. However, she’s an ENTJ. I don’t think she would back down.

Keith McCormick
Well, in her defense, she was on a podcast today because I just happen to check the Twitter feed just to see if there were reviews and things to to check in. She was on you know, she’s on UK time and she was she was on a podcast. I wasn’t able to catch it live. I’m hoping to catch it and recording because it was it they might have. They did they did announce the time so I think they did do it live but it wasn’t in it was Interview podcast. Yep.

Joe Arrigo
Well, I just want to say, I just want to say for the audience that a lot of these comments that I’m not able to get to Keith and I will definitely respond to them in-kind later. So like, you’re not gonna miss out just cuz I didn’t feature that on the live right now. There is so Merv m re ends, essentially, like as they’re kind of concluding the documentary, she says, we need to walk away from the individual and think about to live in a world with with a sense of community, a world where desires and needs are more collective. That sounds nice. Sounds great. Is there something sinister about that?

Keith McCormick
there’s something strategic about why she’s saying that, but I couldn’t quite, you know, figure, figure it out. I think she’s trying to end on this. I think she is trying to end on an ominous note. But this whole idea that basically, The MBTI ultimately is to keep us in our place. You know, that there are there are people that are born to be leaders, most of us are born to be followers. And that The MBTI was developed to enforce a social hierarchy.

Joe Arrigo
Because he did say, I don’t think it was Ashley Murphy. I think it was Lydia, we’re saying that. We still have eugenics as our mainstream science, but we’d cloak it in personality. Like that’s, that’s funny. That’s just makes me laugh, because like, that is so untrue. But it’s like, it’s such a provocative point. That now people have to consider it like, Wait, do we actually have this? We’ve just reclassified eugenicist as personality type and I’m like that I just had I had a pause and laugh.

Keith McCormick

Well, but again, I’ve left the movie being a Lydia Brown, you know, fan? I think it would be like for instance, there’s skepticism about a polygraph right. I’ve never researched that. So I yeah, I couldn’t tell you how accurate a polygraph is. But you wouldn’t want you know, some kind of situation where, oh, in fact, there’s an interesting, um, they really missed an opportunity because I love Susan Cain. You know, I think you know that. And there’s a section in Susan Cain’s book where she actually talks about failing a polygraph. And she was being honest on the polygraph, but the whole process made her very anxious, you know, he failed the polygraph as a result. And I would imagine, you know, as somebody that was, in the army, when I was a, I was younger, every once in a while, I’ll get approached to do you know, some kind of project where I have a look where I have to look at data, it’s the kind of data that you have to see as a clearance probably sounds super mysterious, like you’re being some kind of a spy. It’s usually something very innocuous, like, the range of ammunition or something like that. But there’s certain things where people are allowed to say, or like the top speed of an aircraft or something that’s usually the stuff that’s classified, it’s not as mysterious as it is. But I could imagine having a failed polygraph somewhere, in some record, in never being able to be able to get, you know, a clearance, you say that that’s what I think Lydia Brown is talking about. Okay. Now, obviously, with eugenics, it’s trying to decide who can reproduce. Right, right. But the notion that somebody would be permanently unemployable, I’m on I find, you know, I find that argument, you know, compelling You know, I think again, I think these models have to have transparency but that’s because I was associated What do you black Brown with the Kathy O’Neill part of the narrative? I will think what are your Brown has much knowledge of nor much of an opinion about The MBTI I could be wrong about that. But I think she was simply being included back and forth in the editing.

Keith McCormick
There’s an interesting moment to where Merve Emre says almost the exact echoes the almost the exact same phrase that what are your brown does so Dr. Brown, you know, rattles off ablest sexist, and so on. Right. But again, he’s talking about the hiring business. And then towards the end of the movie, Murphy, M. Ray says that almost exact phrase. And I could be wrong about this, but I strongly suspect that the Lydia Brown was recorded. They were already you know, trying to put the film together. And it was, wow, this would be such powerful bookends of Merve Emre comes in, you know, representing the other part of this narrative, you know, the narrative that begins in 1875. With Katherine is that the MBTI narrative and echoes almost the exact same language as Lydia Brown. So in other words, I think to some degree, as an audience, we can be manipulated by the editing here because staged it well, because I mean, what is the likelihood that that exact raise almost like in the same word order would be used. I think that was added after the Lydia Brown, you know, was there to create greater cohesion than otherwise would exist because two-thirds of the movie that was on the hiring narrative was powerful and the other 1/3 could not possibly have held up the weight of the documentary on its own. There just wasn’t anything there. Okay, innuendo.

Joe Arrigo
So, let me give my real name is I want to say two things. One, I would say this is the best documentary ever called the persona on HBO Max has ever been released. So I can I can definitively say those words. Secondly, I know personally, people in the type community that were followed around for days recording their like a live typing session, neuroscience eg scans and that were totally dismissed from the document or not even given a little bit of credence to in talking to them, it’s like, well, now I can see why they had a they had the here’s what we’re going after, if we don’t get it, we’re just gonna dismiss all of it. So I felt like these prominent people in the community. I have to say Joel and Antonia and Dario Nardi, they were interviewed for it. And then all that stuff was dismissed. So it’s like, Well, clearly, you couldn’t even give credence to a little bit of it. So that was disingenuous and not intellectually honest. So I just want to say that forgot about that point. But some of the Twitter action has pointed to Hey, I was interviewed, and we were talking to them all day. They didn’t put one second of our stuff in there. So therefore, I think it’s disingenuous.

Keith McCormick

Yeah, so I wish I wish they had made a whole documentary about Cathy O’Neal’s book, because Kyle, bam was just one I would give, I would give that two-thirds of it a pretty good rating, I think, actually. But the thing is, there’s about a dozen chapters in Cathy O’Neal’s book. And some of those other case studies are really powerful. when I’ve had students at UC Irvine, exposed to some of those case studies, most of them will actually pick the, the teacher evaluation one as one of the most powerful where teachers were, you know, were fired, you know, because then an evaluation and again, there was a lack of transparency. And there would be requests made of the, of the algorithm authors, you know, how does this work? My evaluations were really strong. Um, you know, other faculty would sit in the back of my room, and everybody thought I was an excellent teacher, but now I’m fired. And I can never work again. Right. Um, some of that stuff was really, really powerful. I think they should have made a whole documentary about that.

Joe Arrigo
Yeah, I want to include this year, Megan said that I was interviewed, and I mentioned that I had an INFJ for Africa had reached out to me and I said, instead, I changed her life. They even tracked down interviews, the subscriber ly Joe, Lindsey Johnson also said they interviewed one of her subscribers from Iraq. So like, I didn’t even know that till right now. And I’m like, you couldn’t have included that? Like, I don’t know. So I want to say it was during the Omni wanna hear that it was edited for time. So when it changes, like, that’s how I got into type, it changed my life, and I’m a stubborn INTJ. So when I see like that it can apply, and there are benefits for it. I just think you got to show both sides and let the audience decide. And that that was not present here.

Keith McCormick
Yeah, you know, there’s something that was a powerful experience for me years ago, I used to love to go to the kept conferences. Yeah, it would be every other year because they rotated through there were four of them. I don’t think I’m gonna remember all them. But there was like a type in counseling one, but the one that I wanted to mention was type and culture. And what was so powerful about that is you would learn about different, you know, there was a moment I think, her name, Doctor, she’s a PhD, but I think also attorney, Dr. agenne. Want to remember. Oh, yeah, an American woman that was, was, you know, also part of the congressional testimony. She made, she made a comment, which I believe to be quite accurate, where she said, you know, if you normalize things in an American population, you run into trouble and translating The MBTI has always been a tough thing. So I remember one of the panelists at the typing culture conference talking about introversion extroversion in Russia, as opposed to in the US, you know, things that most of us in the audience wouldn’t have had first hand knowledge about. Just kind of a funny thing. They were mentioning a scene in Dr. Zhivago where they shake hands because they’re going to be parting or something like that said Russians would never be that way to be crying and being and you know, things you know, but talking about that was Very, very interesting because Japan was one of the first cultures, one of the first countries where they did a proper translation of The MBTI. and Japan is generally thought of as a culture where introversion is treated very differently than here in the US, Susan Cain, perhaps even you know, talks about this. So that’s fascinating, right? Is that what happens when you go to measure something like introversion, extroversion and another culture, having grown up in that environment, type wise, in other words, in this environment of learning type, those are the kinds of events that I was going to this whole idea that you have a like embedded race or racism in the instrument. I’m, I’m open to the argument, I’ll listen. But I’ve got to hear a mechanism for which That’s true. And I don’t I don’t get how if it was, you know, anyway, I’ve already made that point. Right. But you already know me. I know how the racism can be embedded when Kathy O’Neill was talking about the algorithms. If you’re going to make the same claim about The MBTI, you got to tell me how you got to kind of walk me through, you know, the data scientist, I mean, wants to hear how this is happening. Right? You can’t just share a letter from the 20s. and convince me that 100 years later, that it’s embedded in the instrument, right. What have been my experience?

Joe Arrigo
No, that’s the angle we want. So I think we, you know, I think we have given a lot of info to the audience in this hour, and one minute and 28 seconds we’ve gone here. I think we’ve covered a lot. I do want to probably end the show now. Because I do want to get to these comments. I think there’s so much more to be said. But I think I’ve gone over everything I need. Is there anything else that you feel like we still need to discuss?

Keith McCormick
No, I hope I hope the conversation continues. And I would.
I’ll say one final thing, because I do sincerely hope that I get a chance to chat with Margaret Emery at some point. Maybe she made a comment that I found fascinating because she basically made a claim that personality is performative. You know, or that personality is a performance. Right? I could be mistaken. But I believe that she’s making a tangential reference to Judith Butler there. And if she were to pursue that Judith Butler is famous for saying the gender is performative, right? It’s very famous academic. Okay, if I’m right, and that’s what Mervyn Emery was referring to. I think now, I wouldn’t necessarily agree with her conclusions. But that’s something interesting. So rather than just kind of leaving that hanging there, I think she should probably just be more forthcoming with what she thinks is really going on, and just kind of put it out there. So that we can talk about it, because that would be fascinating. In fact, if she could make that claim, it probably would have been a more interesting book than the personality brokers, if she could take that, that notion, that personality is performative. and pursue that at book-length, it probably would have been a much more interesting book. So perhaps up in an interview, at some, some point not necessarily with us or with whatever form she chooses, it’d be really interesting if she would pursue that.

Joe Arrigo
I mean, we’re, we’re fine. We would, we would be great to interview I would love to get her and I will, I’ll work on it again. I’ll talk to her guy, also named Joe. I will try to get in touch them again, to say like, what would it take to get you on here? Because like, it’s not like we’re gonna tell her she’s terrible. And no one read her book, like we would treat her fair. So I’ll work on it. I’ll try to reach out on Twitter

Keith McCormick

questions in advance.

Joe Arrigo
I told him that much. Yeah, I get all that. So Keith, up this last couple of weeks getting to like understand your side of MBTI why like you strengthen the arguments and that you really helped me in some of the things that I fell out because I’m not a man of science. And I there’s some arguments here that I really need to use this kind of AI to just like a weapon against the the people that talk trash about type, but also just to have a better historical viewpoint of where types have been, where it’s going and to not dismiss and have some contextualization of what it’s about. I hope our audience here today really understands that. Like, we don’t believe that there’s any sort of embedded negativity or nefarious notion in type and that everyone can use it as a tool for self-discovery. That’s pretty much where the show ends! So I appreciate it. Keith, and if something more component comes up, I’d love to do another live with you.

Keith McCormick
Sure. That’d be great. I would enjoy it.

Joe Arrigo
And what’s my audience? Sorry, go ahead.

Keith McCormick
You’re just gonna say what’s hope the conversation continues in the comments below this when it is No hits the LinkedIn fee.

Joe Arrigo
Absolutely. And thank you to everyone in the comments for keeping the discussion going, I hope I hope it was worthwhile. So, talk to you all soon be good thanks!

To book me for MBTI consultations–find my Calendly on the sidebar or here!

For more of my content, you know where to find me:

1. Youtube
2. LinkedIn
3. Blog
4. Etsy