Episode 12: Is Your AI a Personality Match for You?

Brian Milner (00:23) Hey everybody, welcome in. we're back here for another episode of People Over Prompts Podcast. This is a place where we're trying to talk more and more about the the future of teamwork. In this new age of AI. I'm your host here as always, Brian Milner. And I'm excited about this week's episode. I think we've got a lot of good stuff to talk to you about. before we dive into that stuff, I just want to give a couple of quick kind of community notes, reminders, point pointers here to you about a couple things. Just don't want to want to make sure you don't miss out on any of these things. I am going to be speaking at the regional scrum gathering that's in Central America this year. That's coming up actually really quickly here. That's November fourth through sixth. it's gonna be happening at a fabulous place called the the Porta Hotel Antigua. it's really a nice kind of a luxury resort hotel there. And to be giving a keynote speech there. So I'm looking forward to to meeting with everyone. I'm also teaching a class on agile OKRs there that you could sign up for in advance and we can spend more time doing that. But if you're there, please do come up and say hello. say hi to me while we're there. I'd love to hang out with you and have a drink and talk more about this topic. Obviously I'm passionate about it and if you are too I'd love to hear what your thoughts are on it. So if you're interested in that you can go to the Scrum Alliance website. It's Scrum Alliance.org and look for the regional scrum gathering in Central America. You can sign up for it there, or you can even sign up for the Agile OKRs class that's going to take place that same time period. The other thing I wanted to let everyone know is I do offer my own classes that I teach online that anyone can sign up for. Usually have them in central time zone here in the US, but you know that can change from time to time. And just added a couple new classes that are going to be Be taking place the first couple of weeks of November after that conference. So I'm teaching an ACSPO class, an advanced product owner class, and an advanced Scrum Master class. So if that's interesting to you, then please do sign up and let me know when you come into class that you heard about it on the podcast here. But let's get into today's topic, this week's topic, because there's been so much AI news. And I, you know, I've I've been bombarded. With it, which I'm sure you have as well. And there's just been this non-stop coverage of is AI gonna kill us? And you know, I I don't mean to make light of that. Obviously, there's there's some important things to discuss and talk about there, but I kind of feel like that's covered to death everywhere else. So I I'm not making that the focus here because it doesn't really align as much with what it is we're trying to to cover, which is an area that I don't think is covered very much in other places, which is this Interaction between humans and AI and how that fits into the workplace environment and more importantly, how that fits into how a team works, the structure of our team, our processes, our practices, all those things I think are going to change and adapt based on this new paradigm. And there was a kind of unheralded study that maybe got lost a little bit this this last few weeks in the shuffle of this all this other AI news that's come out. But I really thought it was an important study. It had a lot of really important information in it. It was a joint study from John Hopkins and MIT, and it was published in the proceedings of the National Academy of Sciences. And it was really this this interesting kind of study that I'll I'll get into a little bit more of the details about here in just a moment. But the the idea it was about AI and personality. Specifically the personality of an AI and how that matches up with the human personality that's interacting with it. You know, we're programming these AIs to be human like, even though they aren't human. and one of those things that they they inherit is some type of a personality. So you know, we we we tend to think most often about AI as w which model is the smartest or the fastest or you know what's the secret prompts to do this or or those kinds of things. What tool has the most features. But the other variable is the personality of the AI that you're interacting with. Unlike a a human teammate That you might have on a team, you can change an AI's personality. It's not a fixed point. it's n I would argue it's not a fixed point with a human either, because we we change and grow and adapt. but we can program it with an AI, which is not really something we can do with another human. and the the the the research was really based around this this thing called the big five personality traits, which I'll I'll get into a little bit more in depth here. If you're not familiar with that, don't don't feel like I'm gonna leave you behind. I'll I'll kind of explain that a little bit. But the big five personality traits are something that's that's really kind of used quite a bit in in psychology. it's a little Deeper, I would say, or more scientifically grounded than something like a Myers Briggs or Enagram or something like that. little less horoscope-like, if you will. But the w the way they did this was they they paired humans randomly with these agents that had different of the big five personality traits, and they really wanted to kind of see the combinations. how did a human of this this particular personality trait match up with an AI of of you know a different personality trait or the same personality trait. one of the the authors of this study, some this Harang Jew said you can't manipulate the personalities of humans, but we know we can change the personalities of AI agents and get results. And that's I think going to be an area of exploration as As we team up humans to AI, is how does that matching of personality from one side to the other actually affect things like performance? I mean we we we know personality matters on human teams, obviously. you know, you can have two very capable people, but they can be awful together. they just may not work very well together. And if that's the case, then You we we we treated AI mostly as a kind of a capability, when we've thought about it in the past, but human personality traits might actually be just as important as is one of the things this research is showing. So let's talk a little bit about the research itself, because I you know, if you listen to any other episodes, you you know I I tell you most of the time, be skeptical of research. Make sure you you kind of figure out where it's from and what methodology was used because there's a lot of sketchy research out there that people quote and that's one of the things I try to do is always link to it here in our show notes so that you can go and and look at it yourself. you don't take my word for it, right? but the the this this wasn't a people rating whether they liked different chat bot personalities. So it wasn't about their preference. they actually did work With these AI agents, and the output is what's evaluated in this study. So it was a pretty wide-ranging study. They had 1,258 participants, and the AI agents were randomly assigned different combinations of the Big Five personality traits. Human AI teams then working together. produced in this case for this study display ads. And they they were real display ads for a think tank. They they worked together using chat plus synchronized text and image editing tools. The the work was independently then rated and researchers then tested the ads in the real world on Twitter or what was Twitter and it was now X. The the the field experiment generated nearly five million impressions of those ads. And the researchers measured more than whether people enjoyed the interaction. They they actually examined things like teamwork, productivity, the text quality of the ad, the image quality of the ad, the likelihood of someone actually clicking on that ad. And then actual advertising performance of the ad. So this is a really interesting study, I think, for that reason. It's not opinion-based, right? It's not asking someone how did you feel about this? Did you think it worked better or worse? It actually judged the results of that combination. And that's that's to me what's really interesting about this. Human rated quality predicted real world results in this case, a one standard deviation increase in text quality, increased click-through rates by 5.9%, better rated images per reduced cost per click by 31 cents. So this is where it starts to get really interesting, right? If we're tweaking the personality, does it really have an a direct impact? on the quality, on the effectiveness of the work that we are created. so it it's it's interesting because the research was trying to figure out, you know, if there were very specific pairings of personality traits that worked better than others. And were there some that were complementary? Were there some that were more destructive or you know anti-uh productive? It it's it's more than just I enjoyed working with this AI then, right? It's it's more about this AI helped me produce better work. And that's what we'll look at throughout this is the comfort may not equal effectiveness as we go through this. this was advertising work, so it's important to understand the limits of the of any research, and this research was specifically with advertising work, that may not translate, right? so you know, it it's not provable that then that would be the same thing in let's say software development or healthcare work or product management or whatever, right? It's specifically in in advertising and ad work. so you have to kind of make that inference on your own. but you know, there there's some bigger findings in this that I think are are are hard for us to just dismiss and bypass. Who the human is changed which AI behavior worked best. And that's something I think we'll have to really examine over the years. So first of all, let's set the table about what this big five thing is, because I think it's important to understand what we're talking about that with the big five. It's it's not they aren't personality types. So it's not like you are a this or that, right? The the big five is intended to be understood as everyone has all five. You just have different levels of all five. So being a pretty geeky kind of guy, I I think about something like you know, a DD character sheet, right? Where you have things like intelligence, charisma, strength, wisdom, right? All those kinds of things. everyone has them all. Every character has them all, but They have different scores on different ones. And that's kind of how you can think about the big five a little bit, is that we all have all of them. We exercise or display some of them more other times than than some other times. but we we probably have have a tendency towards being higher or lower in these these five different traits. So the the acronym that you often hear used for this is Ocean. so again, just setting the table for this so you understand it when we talk about it in the study. so the first one, the O is is openness. Openness is talking about someone's curiosity, imagination, their their interest in ideas and new experiences. that's what openness means as far as a scale is concerned. the C is conscientiousness. this is when we're talking about kind of the organization, the the the Discipline, planning, kind of nature of people, responsibility, right? That's conscientiousness. E is extroversion, and extroversion here is sociability, assertiveness, energy, kind of positive emotion, right? agreeableness is the A. this is talking about cooperation, compassion, trust. generally kind of the willingness to get along. And then the the N here is one that gets a bad rap because the word actually has much more negative connotations in real world dialogue. But N is neuroticism. And what it means as far as the big five is so subjectivity, distress, uncertainty, risk, negative emotion, that sort of thing. And someone can be, you know, highly open and highly conscientious and highly agreeable. It's not that you're, you know, an INFJ like a Myers Briggs or you're an Enneagram type, whatever, right? you're not one thing. You're you're you're a mixture of these things. so that's important to understand in this research because they they weren't trying to just have have it pegged one thing. and nothing of the other things. They were trying to have different combinations. So we you know, in this study they the AI would move those sliders, they would move those sliders on the AIs deliberately to try to get different combinations. and you know find that maybe certain things would need more conscientiousness or some things would need more openness. so so the the the question wasn't really which one of these things does my AI have, but what combination of these things or what levels of these things, what where should I set the sliders for this particular situation, that sort of thing. So I want to go through each one of these and talk about when the human has this high level of this particular one, how that matched up with different matchings of AI. So let's start with the openness one. Right? If a human really displayed a high level of openness, well again We're we're saying a curious, imaginative openness is people who like possibilities. They they enjoy possibilities, new approaches. they enjoy experimentation. they're they're they're comfortable with that unknown, with exploring new ideas. And, you know, you might hear someone who's who's really high on openness say, Well, what if we tried something completely different? Right? What if we went in this other direction? So for humans that had a high level of openness, here's kind of the interesting thing. Conscientiousness, an AI with a high level of conscientiousness improved quality. Improved quality of the work. Agreeableness in an AI showed what the researchers referred to as a jagged effect. What they mean by that is that there was better image quality, but there was worse Text quality, which I I find fascinating. it it does kind of make intuitive sense to me though. An open person might say something like, What about this? Or you know, here here's another possibility, right? conscientiousness in an AI potentially provides something different that's needed. you know. it it might respond like, Great, let's let's turn that into something. it's it's complementary to to the openness. So it it kind of begs the question: maybe we shouldn't be making AI more like ourselves. Maybe we should be making it different, right? but you'll see here that it's not as clean as that. It's not as clean as, it should always be the opposite of what we are. so let's talk about conscientiousness. when when a human displays that trait, you know, these are the kind of people who like checklists, who you know. Confirm the deadlines, organize files, notice when something's missing, right? They they find the details. So for those high humans who have high levels of conscientiousness, if they were paired with a high level of conscientiousness in the AI, that it improved their text quality. Openness in the AI improved image quality. Agreeableness in AI decreased text quality. and click likelihood, which is is very interesting. The paper also reported that conscientious humans benefited from actually a disagreeable AI and overall quality results. neurotic AI, impaired aspects of teamwork in conscientious humans. So this is maybe one of the most useful findings in the study. Because agreeableness sounds good. Who doesn't want an AI to be agreeable? But if I'm already disciplined, if I'm deliberate in how I do my work and convinced I'm going in the right direction, an AI that just kind of responds to me by saying, Absolutely, great idea. That may be the last thing that I need. maybe what I need is friction in that scenario. Have you considered this? I I don't think that assumption follows. I think you're making a leap there. That evidence actually goes against the conclusion you're making. this ties closely to what people talk about a lot with AI sycophancy, you know, kind of feeding into your the ego. And for that conscientiousness in in humans, it it really is kind of a a bad combination, right? we we spend a lot of time and energy making AI pleasant. But pleasant and useful aren't synonyms. And I think that's true here as well. you know, I I have a little bit of a musical theater background, and one of my favorite composers there was Steven Sondheim, and he has a show called Into the Woods that has one of my all-time favorite lines in any show, where one of the characters says, nice is different than good. And I think that's kind of applicable here, right? pleasant is different than good. it's not synonymous with being useful. Pla if pleasant and useful aren't the same, well nice and good aren't the same here as well. So we may not need something that just is affirming us in that situation. let's talk about extroversion. you know, we we probably this is the easiest one for people to understand of the big five because We all i if you're not an extrovert, you know extroverts and you you can kind of spot them. they they get energy from interactions. they're they they the the the person that walks in the room and starts talking through the problems out loud, right? So if you pair an extroverted a high high extrovertedness human with an AI that's the same way. That improved text quality, but extroverted human to extroverted AI combinations were among the better performing performing combinations actually in the study. So that's kind of an interesting thing. Agreeable AI paired with this. Increased productivity. But that's an interesting thing to call out there as well, right? If you pair extroverted with conscientious AI, It actually decreases quality across the text, image, and click measures. So kind of interesting there. extroverted humans paired with conscientious AI produce the lowest quality ads of the combinations highlighted. so that that challenges our simple assumption of the opposites attract. It's it's not really that same thing. For openness, complementary traits seem useful, but here. Similarities seem to work best. Extroverted plus in extroverted kind of worked really well. so that challenges our assumption that you know your AI should be the opposite. It's it's messier than that. there's maybe something also interesting about momentum a little bit in that. you know, an extroverted human pair with an agreeable AI may produce more work. But, you know, we've talked about this in Agile for a long time. More isn't better. you know, just because we produce more of something doesn't mean that's a higher quality. Doesn't mean it makes as big of an impact. Right? if you're, you know, this this might have a leadership analogy here as well. If you're you're already a strong assertive voice in the room, do you need another person simply turning your ideas into action? Or do you need someone capable of saying, hold on. Are we sure that's the direction we want? So kind of important things to think about there. let's talk about agreeableness. If if the human has agreeableness, well, first of all, what are we talking about with that? We're talking about the person who notices tension in a room. and and they they always try to smooth it out, right? They're the the empathetic people. they're interested in harmony. They're they're less inclined towards conflict. they want to smooth over conflict. And this may be the most interesting productivity quality trade-off, actually, that came out of this study. So for agreeable humans, openness in AI produced or actually increased productivity, but decreased image quality. And neurotic AI decreased productivity, increased text quality, improved communication, coordination, mutual support, and effort. So the authors in this study. They describe this as a productivity performance trade off. Al almost every conversation about AI that we have begins with how much more productive will we become as a part of it. But what if what if the AI combination that makes you most productive isn't isn't the combination that makes you produce the best work. You know, productivity doesn't always equal better, like we looked at in the last combination. For for agreeable people, neurotic AI slowed them down. But quality went up. So let that sink in. Slow them down, but quality went up. That's fascinating. It it it it it's fascinating and maybe kind of shows that people benefit from AI that worries a little bit about what what might go wrong. Are we missing something? So maybe it has us think a little bit differently about something like AI friction. We we talk about AI friction as a bad thing, but maybe it's not inefficient, maybe it's quality control. And that's the way it should be looked at. Maximizing maximizing our utilization and and output rather than value is is not the right direction. AI could could make that mistake vastly worse. that's what this combination I think shows. So let's talk about the last one, neuroticism, because again, this one gets a bad rap because it sounds insulting in everyday speech if you say someone's neurotic. but what we we mean for the big five is Think about a person who hears we're launching on Friday and they immediately ask, what could go wrong? have we tested the backup? What happens if our server fails? Right? They're there are people who really think about the worst case scenarios. And that's not a bad thing. that's something that's needed. We need to plan for the worst, right? Hope for the best, prepare prepare for the worst. So, for humans that were high in neuroticism, extroverted AI improved their performance. Conscientious AI worsened performance. And conscientious AI also decreased perceived click likelihood, which is kind of interesting. So, this is another case where intuition and common sense might actually lead us astray a bit. You might think that somebody who worries about mistakes should. Work well with an AI that's extremely careful and conscientious, but the research just doesn't support that conclusion. perhaps adding more caution on top of caution doesn't actually create balance. It it's two negatives canceling each other out. The study it's important to understand here that the study, while it does tell us what happened. We gotta be cautious not to make the leap because it it doesn't necessarily tell us why. And that might take some future research. That might be other studies that we'd have to look to for that. So interesting findings, but we may not have the whole story as to why. So I th I think the the point here or one of the the things to to take away is that the biggest finding may not be that there's any one particular perfect pairing. it may be that there isn't really one best pairing overall. the research calls some of the effects jagged. you know, where where one thing goes up and the other thing goes down. The the effects actually in the research weren't limited to personality either. The researchers found differences based on other human characteristics. Things, for example, like extroverted AI improved quality for workers born in Latin America, but reduced it for workers born in East Asia. So there's there's cultural dynamics that come to play in this as well that we're not really even touching on here in the limited time we have this week. but th this does kind of push against one of our favorite ideas, I think, about AI. we kind of had this this mentality of set it up once, personalize it to me, and tweak my settings and and now it's my AI. But I think the research is is really kind of r ringing the alarm bell to tell us that that's that's wrong. there may not be a you know perfect AI that's a perfect personality for you know for me for Brian. There may be a a perfect personality for when I'm brainstorming. There may be another one for when I'm I'm writing. I may need a completely different one for when I'm performing research or I I'm trying to make a decision, or planning. So it it it's not really a a a match to one human to one personality type. Right? It's it's kind of personality type plus scenario that I think we have to think about. it it it it appears to be about configuring AI personalities, but it I think it's also about really the harder the harder conclusion is about understanding ourselves, right? Because it it suggests that that human AI performance isn't an interaction. It it's it's not simply the better AI equals better results. We have to understand our own personality and what we need in that moment. And that's where I think we have a wide open field to explore. Right? we've been trying to understand ourselves for f forever. And I don't think that that's ever gonna change. I think this is just another sliver, another refraction through the you know, through the prism. That we're we're using to view ourselves. And I think it's it's maybe a tool to help us better understand ourselves, to become a little bit more self-aware, you know, to ask questions about how do I naturally approach problems? you know, where where are my strengths and weaknesses? Where do I need help in? where do I tend to overdo things? We're where we're scenarios and situations where I need extra support. And and that that may ultimately be less about giving AI personality and more about understanding what kind of teammate brings out our best work. Because I think there's a broader lesson there about teams as well. I don't I don't want just one personality on my team. Right? There's not a uniform personality that I work best with. I need different personalities, right? I need some personalities on one day and other personalities in other situations. I need a range. And that may be where we end up on with some of this research is really trying to narrow that down to say, hey, Maybe if I'm a human and I'm working with some AI agents, I need to have actually not only programming into them their skills, their abilities, what they can actually handle, but their personality. So that when I am using that agent, I'm in this research mode, I need this sort of personality trait to be higher in that scenario. But when I'm brainstorming over here with this other AI agent, I need a different set of personality traits to really push me in and challenge me there. So just a fascinating study that I'll link to in our our show notes. I hope you find it as fascinating as I do. the this kind of research I think is is really on the cutting edge. And I'm really interested in in seeing where this goes over the next few years. if this kind of topic really interests you and it's something that you find as fascinating as I do, then I encourage you to join us every week here for the show. We try to put out an episode just about every week. great way for you to do that is to hit subscribe and whatever podcasting platform you you found this on, or maybe you're watching this on YouTube. You can subscribe to us there, like us there. I ask that as a personal favor because I don't advertise or do this anywhere else. I I'm not gonna buy ads on any any other platforms. I'm just counting on you. So I I'm hoping a word of mouth will spread this. If you have a colleague at work that also is interested in this, just tell them about it. Just say, hey, I found this episode. This was really interesting. Maybe this is this is kind of what we were talking about the other day. There's some interesting research around this. Point them in my direction and the show's direction. I really appreciate it if you would do that. You know, so so subscribe, like to whatever in whatever platform you use, tell a friend about this. And the other thing I I ask from you, my other my third ask is just to to email me If there's any feedback you have from me, if there's anything you want me to know, topics, research you want to point in my direction to have me talk about, or guests even that you want to have on the show here, I like to occasionally bring guests on here as well. Although, as you can tell from our episodes, there's going to be a plenty of them with just me sharing some research with you. but I I will have different guests over periods of time. So if there's someone you want you think would be good to come on the show, that's talking about. This stuff as well, then please just email me Brian at agilityevolved.com. That's my website, Agility Evolved. And it's Brian with an I, so don't put a Y in there, it won't get to me. B-R-I-A-N at agilityevolved.com. you can send me anything you want there. send me questions, send me anything, and I'll get back to you on it. So I hope everybody's week is going great, and thank you for joining us on this episode. I look forward to talking to you next week on another episode here of People Over Prompts podcast.