Episode 13: What Happens When You Invite AI To Your Meetings?

Brian Milner (00:24) Hey everybody, welcome in. We're back here for another episode of People Over Prompts Podcast. This is where we try to discuss all the ramifications of what happens when AI joins your team. Or put another way, the future of teamwork in the age of AI. Where we're we trying to find different topics that are in and around this area. And today we have a good one for you that I I think a lot of us can probably identify with, sympathize with, because it's it's pretty much a part of anyone's life who is in the workplace today. And so what we titled this is what happens when you invite AI to your meetings? Because that's happening you know pretty pretty regularly now. You may not even know it. we're talking about, you know, when you have that ghost person that shows up to you know, your Zoom call or you know, that's just an AI note taker. or maybe you don't even know that it's there. Maybe, you know, I know Zoom has a note taker that it can take notes and transcribe. And without much notice. you you you may not even notice that it's actually going on in the background. so as always I'm curious about the human dynamic of this. there's there's a lot of reasons I bet we can think of that we can say that this might be a good thing. There might be some things we would look at that would say Be cautious or this might be dangerous if those kind of scenarios. And that's kind of what we wanted to dive into. Now, usually on the show, what I try to do is find one big study that kind of drives the entire conversation. not doing that this time. So I've got a bunch of different studies, I've got several things from different areas, so there's not just one main thing. I'll try to list all these for you in the show notes as we go through. But let's start with why we would do this. Why why are people finding an advantage in recording? Why do people think it's a good idea to record and transcribe meetings with AI? Well, there's a couple things here. There's a Microsoft study that found that people if they have an AI transcription of the meetings, people catching up on missed 35-minute meetings with kind of a copilot trans transcription. Did it in 11 minutes and 13 seconds versus 42 minutes and 34 seconds without it. So put it another way, about three 3.8 times faster. So there's a speed bonus. I think we can all probably understand that. That's kind of common sense. if if there's somebody who is very good at taking notes that can share their notes with me after the meeting, in this case an AI, then I I can probably catch up much faster. how many times have you been at a meeting that you thought, man, I wish I could just read the summary of this afterwards? that's kind of what that that's telling us. There there's an accuracy trade-off, but it's very small. That the Microsoft studies found that the co-pilot group captured eleven of fifteen key details versus twelve of fifteen in the control droop group that wasn't using AI. So yes, there's a small accuracy trade-off there, but it's minor. I think it's fair as well that we we kinda make the distinction about just regular office meetings and areas where it's extremely important, like legal matters or medical records, those sorts of things. Right? That there's a there's a difference there. And what we're talking about here is more your your stip your typical run of the mill kind of meeting that people would have with within their offices. the study also found seventy percent of co-pilot users said they were more productive. Seventy three percent said they completed tasks faster when they had AI on in the meeting, to to kind of make notes about it. the The the survey was of eighteen thousand one hundred workers. So not a small sample size. Eighteen thousand one hundred workers. Seventy-six percent wanted tools to handle meeting admin such as note taking in that that group. and I I think there's some obvious upside to it. I think there's a clear upside that we can pull from that. I know I've used it for several things, you know. sometimes do interviews, people to try to assess how things are going within an organization. Boy, it's really great for that because I can then scan across the multiple interviews to see patterns, things that that maybe I missed in a conversation, things maybe I should ask about in the next meeting. So there's things like that. There's there's searchable memory. this can now become part of the collective organizational memory. And that can be used for for good. you you might even have the ability for some people to skip meetings. Do you know how many times do people come to meetings that's they're just there for informational purposes? They have nothing really to t to contribute to that meeting. but they want to make sure they don't miss anything. Well, those people wouldn't need to go to those meetings then. They could just read the transcript afterwards. And do that, as we said, in about three point eight ti times faster. Also just the the you know if you've been one of those people who have taken notes in meetings, the the frantic kind of scribbling, I I know I s I kind of out of habit do that. Anytime I'm on a a phone call or in a meeting, just frantically kind of scribble down things. And, you know, sometimes I'm a better note taker than others. sometimes I capture details that are unimportant. Sometimes I capture exactly what I need to know. And AI is more consistent with that. It'll it'll you know, catch Most of the important details there. Also gives you the ability to catch up later, to refresh your memory about things. So there's there's some obvious upside to doing it. And I can see lots of reasons why organizations in some cases might even make this a policy kind of scenario. Because there are cases, there are situations, scenarios where companies are saying, we're gonna go all in on this. We want to transcribe every meeting and we want those meeting notes to go into our collective kind of database, our our our place, right? where we can then go back and search them and and and capture that information for our organization. But but here's the thing, here here's kind of the next big point that I wanted to go over, and that is that there's there's research or studies that show that when you invite AI to a meeting, it changes behavior. And here's the analogy that I'll make for you. you know, I come from the scrum world, from the agile world, and one of the the main things that you do in that kind of world is is a daily meeting called the daily stand-up. And in the daily stand-up you can talk about what you need to do for that day. Kind of make a plan for for what you need to accomplish in that day with the team. So the team can help each other out. The team can say, hey, I need help on this. Great, I can lend you help there. Or I just finish this up. I'm available now if someone needs my help on this. So it's a you know kind of a planning meeting. And what we found is that when a boss attends the meeting, it can have the chilling effect that people don't talk as much. So this is just an everyday scenario I'm trying to bring up to you as an analogy to say sometimes when you have a certain person in a meeting, it changes the behavior of everyone in the meeting. And that Daily Scrum one is really kind of an example of how a human might change that meeting. But what happens when AI is in the meeting? What happens when AI is actually transcribing the words and and making summaries and recording this for posterity? Well, there's a couple key things that the research has shown that people do differently. People speak differently when they're in a meeting, and they do that in order to help AI understand them. Right? So it's it's it's almost like You there's someone there who you know English is a second language, and so I'm gonna speak more deliberately, more clearly. In this case, people are doing that in order to to kind of clue in AI as to what's important. So they actually change how they speak and the precision with which they say things because AI is transcribing it. People also tend to want to manage how they appear because this is now going to be more of a part of a permanent record. so they're going to use more deliberate wording. They're going to be concerned about in some cases organizations might have things like sentiment scores. so they might be concerned with something like that. They're gonna have greater awareness that that boss, that leadership, may may come along and review what was said. They may not be in the meeting But they might as well be in the meeting if they're gonna read over every word that was said in it, right? And potentially it could create this scenario where they're more people are more guarded with their words if they're being transcribed. there's also really kind of an interesting side effect that participation patterns can change. there's some evidence that there's there can be more balanced if it's guided by AI. but there also can be some just anomalies in how people speak in a meeting that aren't there if the AI is not recording. So there's a case study for this that really is what got me on this topic in the first place. It's from a company called PLO, PL P-L-A-I-T-O. And they have an internal AI that they call PLO, like the philosopher. and They were using that to transcribe everything. So they were embedded in the meetings. They were, you know, it was embedded in Slack. It was embedded in Jira, in their Salesforce program, it was embedded in Confluence.Anywhere that there was dialogue and collaboration going on, AI was there to capture that for posterity. Now they were doing this, you know, for all the benefits that we we said before. They wanted to build institutional knowledge. And they did some research on this and found that employees actually changed how they spoke because they knew the transcript would later be processed by the AI. What they found is people started to to they they repeated certain action items, they added explicit content. They they reformulated comments. And the interesting part is they left what's called breadcrumbs, verbal breadcrumbs, right? Because they knew that the AI was looking for specific hallmarks for specific moments within the conversation based on the outline of the the meeting or anything like that. And so they would they would kind of try to give those references, give those breadcrumbs to say, you know, here, you know, AI, pay attention. Here's where I'm talking about this thing, so don't miss this, right? They would kind of do that verbally so that AI wouldn't lose out on something that they thought was important. so that was kind of an interesting finding from it. There was another one from a company called Read AI. They they looked across 159,000 meetings, 159,870 to be precise. And it was over 30 industries over a span of about two months. And One of the interesting things that came up here, again, I don't know why these kind of topics keep coming up in in our podcast. I'm I'm not trying to make this a focus, but it just is something that kind of keeps sticking out. in those meetings where AI was present, women accounted for roughly nine percent more speaking time than men relative to their representation when a the AI note taker was play in place. So women speak more. They feel freer to speak more in an AI recorded session. Now you can debate about why that might be. Again, I'm not going to get into that in this this podcast, but it is an interesting finding that it does change behavior. And for whatever reason, when AI is there transcribing, it's it's not a huge amount, it's nine percent, but that's over, you know, kind of a margin of error. Right, it's it's statistically s significant. Nine percent more women speak up during meetings where AI is present and and and you know, transcribing it. they separate they separated out peer reviewed virtual meeting research. Women used meeting chat more in these kind of situations. Men spoke verbally more than using chat. they also found that it was really important that psychological safety affected how people participated, especially for lower status employees, people further down the rungs of the ladder. so the the question is is AI merely recording the words, or is its present becoming part of that behavior? Is it actually moving the behavior, triggering behavior? I can I can completely understand why this might take place. if I am being recorded, if every word I say is being recorded, well, I I'm gonna think about the consequences of that. who is going to access that later? And what is the purpose for it? For example, what if what if HR was reviewing every meeting transcript? What if HR had an AI agent that was going through every meeting transcript to see. If anyone said anything inappropriate. Right, that that might be a good thing because it could cut back on people saying inappropriate things in the workplace. But it might also have a chilling effect. Right? It could also have this effect of people who now I don't know if I can speak up, if I'm free to speak up, what I say might be used against me. Or let's not even say let's not even talk about inappropriate things, right? let's talk about overpromising. what if, you know, I'm I'm in a meeting and I say, hey, that that project I'm working on, I think I'll wrap that out up in about, you know, six weeks. Well, now that's in the transcript that I set on this date that I'm gonna wrap it up in six weeks. Or or I actually said I think about six weeks. But when it that six weeks rolls back around, the A AI is gonna flag this and say, Hey, you know what? Brian said he should be done with that thing in about six weeks. It's probably about time to follow up with him and see where he is on that. Well, if you follow up with me and something unexpected happened during that time, what happens? In that company, do they kind of accept that there's unknowns? And those unknowns might stretch things to be longer than it than it was first thought it would take? Or is in that company, is that seen as Now you have a punishable offense. Right? You said six weeks back five weeks ago. Now you're telling me you're not gonna make that. And I look at it as a deadline. Right? If that's the culture, then now this is weaponized. Right. Now the transcription is weaponized. And if I am in that meeting, I'm probably gonna have to be more guarded in things I say, right? And the way that I say them. So that kind of leads us to the question of whether AI is being used used as sort of collective memory? Is this corporate knowledge so that we don't miss anything or lose anything over time if someone leaves, right? Whatever project knowledge we have that available to us, or is it more court reporter? This is evidence. I feel like I'm under oath. I said this stuff. Will it now be used against me? That's a very important distinction to make. And I think it's a very important boundary that we need to establish if we're doing this. Right? I think we need to understand why. I mean, even the Plato case, they were automatically assigning meetings with a sentiment score, a one to five sentiment score. And meetings that were three or below, they were surfaced to leadership. Well, you can imagine how that. affects the behavior of people in the meetings. employees began you know who became aware of that, they they started trying to influence the sentiment score in the meeting. you get what you measure for. And if they're measuring for a certain sentiment, then hey we can pepper in the right words, right? The right tw triggers To make this just squeak by or get to a level where we know it's not going to have any further scrutiny, but now we're being performative. We're not being honest. And my argument there is we're wasting time if we're doing that. I'm not saying it's wasting time to be nice. Right? I'm not talking about being rude to people and and and generally not being you know kind. But I I am talking about Going going overboard, going out of your way to to say too much that's not needed. Right? We we could get to something faster, but I better pepper in these certain keywords that are gonna raise our sentiment score. That's more court reporter, I think. And I think we have to be cautious about that in how we're using this in our organizations because there's the unintended consequence. Right? The intended consequence is we have all this great stuff now. The unintended consequence is now people are going to change their behavior. So are you aware of how it changes their behavior? And I think a lot of it has to deal with how we actually end up using this stuff. there was another study, it was 50 employees that were across this company that was based in US, China, and Germany. And they they kind of identified five tensions that that rose out of this AI analyze meeting notes. One was privacy versus transparency. they wanted transparency, but they we need to preserve some sense of privacy. So where do we strike the balance? It's not saying it's one or the other. We have to have a dividing line. psychological safety versus reduced safety. This, yeah, we wanna re- we don't want reduced safety, wanna lean on the side of psychological safety. Learning versus evaluation. When I'm speaking, am I allowed to say things that are wrong? if I'm brainstorming, am I gonna can I come up with a bad idea? Right? That's a learning kind of based approach. First evaluation, am I gonna be evaluated? Hey, you said that was an idea. That was a terrible idea. yeah, but I was just brainstorming. employee control versus management control. Who controls the stuff afterwards? Is it there for employees to go back and find information from meetings they've been in? Or is it there for management to oversee and kind of assert authority? trust in AI versus trust in people. AI gets things wrong. So do we trust the AI over someone's word in a meeting that AI captured that in incorrectly? I think that's an important thing to try to understand as well. so you know, couple other things here. Tech the technology here isn't the culture of the organization. It's it's a tool. And the culture is is going to either continue the way it is or it's going to shift or change. But the it's more going to be amplified by this than it is going to be changed by it. i it it depends on what's already there, right? if we have transcripts from our meeting, is that going to support learning? Is it gonna support better memory? Fewer meetings? Is it gonna support accountability without blame? Or is it going to be about surveillance? Is it gonna be about scoring employees? About gotcha kind of accountability? Right? that's what we need to determine, and I think we need to be on the same page with that. So I what I encourage is trying to be deliberate about mapping these things out. Try to be deliberate about stating these things. Who controls this information? Who has access to this information afterwards? What can it be used for? and what can't it be used for? Really importantly, right? Because I think a lot of those things will can establish those boundaries that we need that will give that safety to say, hey, can we continue on in a normal way in our meetings? Can we continue on in the most effective way? Versus, do I have to guard myself because now I'm I'm under the eye of somebody else? And I know. I've gotta always be minding my P's and Q's, as it were, because this could then be seen by my boss or leadership, and I may not get that promotion six months from now because we had a disagreement in a meeting. Right? Conflict is gonna happen. you know, if if we're scanning things for sentiment analysis and we find there's conflict in a meeting, guess what? You gotta have conflict in meetings. Now there's a difference between healthy conflict and and destructive conflict, right? But anytime we're in a meeting and I say to you, hey, I think I have a better idea than yours. You think you want to do this, I think we should do this. We're in conflict. And w if we can handle that in a safe way, then that can be a constructive great thing within a team, within an organization. Because the best ideas will surface. But if we're worried in our meeting that, hey, if I if I kind of speak up and say, hey, I think I have a better idea, now I am injecting conflict into our meeting. And previously that may have been fine, but now we're being judged on sentiment analysis of our meetings, and we might get a lower score because. We had a conflict, we had a disagreement. It was healthy, it was fine, but AI just ticked the box to say, hey, conflict here, and let's let's let's chalk that up as as being you know a a lower score on the sentiment analysis, right? So I think we have to establish these great almost a social contract with it. If I'm using AI, what am I using it for specifically? what is it not going to be used for? I'll give you an example. When I do the interviews with with organizations when I'm doing assessments, one of the things I'll say you know right up front to anyone who I'm interviewing, want you to know I'm transcribing this meeting, but I will promise you that I will never share this transcript with anyone else in the organization. This is for my purposes. This is so that I can see patterns across the different conversations, and it's also so that I can be here present and listen to you. rather than furiously try to scribble down notes as we talk. I try to establish that safety line because I don't want anyone feeling like, well I better be guarded in what I say. I need to know what's really going on. So I establish that boundary to say, hey, you're safe, right? You're safe to talk about this. And that way we can move forward, right? We can do what we need to do. so I think that that social contract is important in organizations as well, is to establish you know Who can search this? how long does it live? Does it stay forever? Is it are we purging this after a certain period of time? can this be used for things like performance reviews? Can a half-formed idea come back and haunt me at some point in the future? D do people get to challenge what the AI says happened? Or is AI always going to be the source of truth? I think these are the things that, again, it's on that fringe of this AI bubble to say, have we really thought this through? Have we thought the ramifications out here? Because this is human AI interaction. And as you can see, it's changing the behavior of people in meetings. And there's some good in here because we can catch up and we can do things faster. But there's also maybe some unintended negative side consequences. And I just think that organizations need to go in with. open clear eye view of this, right? To understand, hey, there there's potential upside to this, but there's potential downside. And we need to try to put things in place, barriers, guardrails in place, so that this doesn't have those unintended side effect kind of follow-on effects, right? well, I hope this topic has been interesting to you. We we try to find these kinds of topics here on the show really, as I said, on that bubble, it's not really Pure psychology, it's not really pure technology. It's kind of where the two meet. And that's what really interests me is how does this actually change how we work together? So if you like this and want to come along with the for the ride with us, please stay along with us. we try to have an episode out here just about every week, except for maybe some holidays or other things. But we'll try to have an episode out for you every week. tell a friend about it. If you like this episode, if this is something you've been talking about with a friend, or maybe you've seen these effects and think, wow, we we kind of have this problem and need to do something about it, point him to the episode. I'd appreciate that. Just kind of spread this by word of mouth. We're not advertising anywhere. Your word of mouth is the only only way people find us. So I appreciate that that always when people kind of just tell a friend about this or or co coworker, right? A colleague about this. If you have any suggestions for how I can do this podcast better, if you have suggestions for guests or topics you want me to cover that's within this this realm, then please send me an email. My email is Brian with an I at agilityevolved.com. Brian with an I at agilityevolved.com. So B R I A N. just want to make sure that no one no one transcribes that literally, Brian with an I. no, it's B R I A N at agilityevolved.com. send me an email and I'd love to hear from you. I always love to love it when people write into me. So if there's something I can do or answer for you, I'll I'll do that as well. But that'll about wrap it up for us. So I I hope you are having a great week and we're gonna reach out to you again next week on another episode here of the People Over Prompts Podcast. Have a great week, everyone.