Episode 5: AI Is Not a Productivity Tool. It Is a New Operating System

Brian Milner (00:21) Welcome in everyone. Welcome to People Over Prompts Podcast. We're back here again, and this is the podcast where we talk about the future of teamwork in this new age of AI. I'm your host here, Brian Milner. I am the CEO of a company called Agility Evolved, and I'm here to talk about a recent TED talk that I saw. I thought this was an interesting video and I thought it was worth us examining here on the show because it really delves into some deeper areas of the kinds of things we're talking about here on the podcast. It's by a gentleman named Florian Bancoly. Florian is the chief digital officer at Bosch Mobility. he's got a lot of credential behind him And being a TED Talk, it's one of those things that kind of throws out big ideas and challenges your your way of thinking. So I I wanted to kind of dive into it, talk to you a little bit about it, talk about some of the big main ideas in it, give you my take on it a little bit, and you know, we'll put the link to it in the show notes so that you can find it and you can watch it yourself if you'd like to. But it there's some some questions I'll just throw out throw out at you at the start of this. one, what what happens when effort is no longer the bottleneck in how we build things? Next, what what if the organization was built around constraints that are disappearing or perhaps even no longer there? And maybe kind of the most important question, what if AI makes planning less valuable than learning? So I want to dive into this TED talk from from Florian Bancoli and talk about some things that he he kind of brings up in this episode. So one of the first things he says in this is like many people, I thought of AI as mostly productivity tools. Useful, very impressive, but still just tools. He said they they had not simply given Their programmers and assistant, though, he found out, they had changed the way they created value through AI. And that's a good foundational element here that we're building this around. the the distinction between improving a task and redesigning the system. that produces value is is kind of what he's diving into here. most conversations that we have around workplace AI still center on things like how can AI write this faster or how can it summarize this meeting or how can it automate the handoff that we have with this other third party or other, you know, internal system. How can it help this role? even become more productive? And it's not that those questions are wrong. It's just that maybe they're too small. What if the real opportunity is not making our current way of working faster? That's assuming that the current way of working is right. What if it's recognizing really that our our current way of working was built to handle limitations that that maybe no longer apply? That's what really drew me into this talk, is that that foundational kind of element. It's not really a talk about AI tools. it's about the assumptions that are underneath organizations that are around how we structure and build our systems. we take a lot of those assumptions that we have for granted in in our our organizations and say that's just the way it's always been. And so how do we just adapt this for AI? But what he's proposing here in this talk that I found really interesting was it's about a new kind of operating OS, if you will, right? A new way of working. a new OS for our system. So I want to go through some big ideas that he had on this, talk and talk to you about what he proposes, and then give you my own take on these. maybe even have throw out some research for you around each one of these things. So his first big idea was that organizations have this invisible sort of operating system. What he says in the video is many people believe businesses operate according to their organizational charts. They do not. They operate according to invisible rules. How complexity is handled, how work moves between departments, who is trusted, who makes decisions, and what gets recognized and rewarded. He says that invisible layer is the operating system that holds an organization together. AI is beginning to reweave that operating system. It's causing us to and Going out on my own here. It's causing us to think a little bit around questions that maybe we we haven't really thought in this way before, but things like who can make a decision without permission in the organization? who's allowed to challenge a decision in your organization? How much work sits and waits for approval? And that's something to really think about in this age of AI because AI is doing work. So, how much of the the work in AI is waiting for approval? what mistakes get punished? Right? W what kind of punishment system is there? Reward, carrot, and stick kind of thing within the organization? What behaviors earn promotion? And who is trusted when facts are are uncertain? I think those are all things we have to re-examine in the light of AI, because The the challenge here is that most companies are installing a new application, if you will, if I'm gonna carry this analogy, while trying to preserve the old system, the old operating system. But, you know, if your iPhone gets upgraded to a new system and you try to install a new application that's made for the new operating system, it's not gonna work. And the same thing happens with our organizations. we add these AI tools all over the place. But they we retain the sort of the same approval chain. We retain the same job boundaries and and planning cycles and the the flow of our work through the system. even sort of the measures of our productivity and the assumptions about who holds expertise, all those things a lot of times don't change, even though we're changing some fundamental things that that kind of underpin what we're doing here with AI. So you know in that sense AI sort of becomes trapped inside this system that it that it could have actually helped replace portions of. not just you know change something out, but actually change the the underlying OS underneath it. So here here's kind of the question that I I kind of propose you to to wrestle with on this first point is which parts of your organization exist because They actually create value. And which parts exist because information, expertise, and execution used to be slow or expensive, which exists because old assumptions that that are no longer valid valuable or valid. I kind of feel like this demands us becoming all systems thinkers of some kind and and looking at the entire system because i i it it's kind of asking us to challenge all those assumptions, right? To examine each one and say which ones are still valid. So that was his first big point. so you can s kind of see where how this is this is going. The second one that I thought was really interesting was that hierarchy Was a solution, not merely a problem. And I know a lot of times we think of hierarchy as being sort of this obstacle to overcome. But here's what he said in the video. For a long time, hierarchy was not only about power. It was also a solution to complexity. Information and decisions moved upward, authority gradually became concentrated at the top, while instructions moved downward. That was not merely bureaucracy. It was a solution. Hierarchy was once an answer to complexity. And I think he's right here to call that out. I think it's important for us to understand that this isn't just bureaucracy. This isn't just systems that were built for the sake of self preservation or, you know, a justification of layers of management. They were built to solve real problems. It didn't appear because managers wanted control. It it solved real problems. Now, I think you also have to say it created real problems as well, right? But it was there to solve problems. And I do think that's an important distinction for us to keep in mind as we re-examine our systems. when the information that we needed was difficult to gather, people near the top could could see more of the organization. expertise had to be concentrated in certain ways. Coordination required central control. Trade-offs had to be brought to someone with a wider view. And instructions had to move back down through the organization. So hierarchy enabled all those things. It was an information processing system, if you will, right? But AI changes the cost. Of processing information and comparing alternatives. That doesn't mean that hierarchy disappears. It means that hierarchy changes its purpose. And if we understand a different purpose for it, then that's gonna reshape that hierarchy. So Baikoli in this video goes on to say a shift from a hierarchy based on information to a hierarchy based on judgment. And I love that emphasis there that he's saying that it's it's moving from this this distribution of information to where does judgment accumulate? Where does judgment need to accumulate? who holds the judgment? that's that's worth exploring, I think, in in your own context, in your own organizations, you know, asking those questions. in the past, authority rested on. I know more than you do. And information flowing up allowed that to actually happen. But I think future authority may rest on more on I'm accountable for deciding which problems matter, which trade-offs are acceptable, or which outcome we're willing to own. Right? So so AI may flatten the hierarchy of information, but it It could also increase the need for clear hierarchy of accountability because that becomes really the structure here. Judgment means accountability. There's got to be accountability for the judgment. So who owns the consequences? Who determines whether the risk is acceptable? Who's responsible when the AI is wrong? These are now the new questions that we have to struggle with. And our I think our hierarchies are gonna be built around this. I've been diving into some research on this, in preparing for some different things I have coming up. And and one of those papers I found recently was really interesting. It was on high-stakes military decision making using AI. And, you know, if there's one place you want to get the decision-making process right, it's you know, in military-type decisions. and it was really interesting because that traditional command responsibility becomes harder when when cognition and decisions are distributed between people and machines. The the the answer in this paper was not to eliminate authority, but it was to create very clear oversight. to to understand decision rights, to to provide more traceability. And accountability. And I think that's the kind of system that doesn't really exist in our organizations today as as much. It's not as clear. we had accountability, we had racy charts and all those sorts of things, but we we didn't AI kind of threw a wrench into it because we were we're using AI to help us make decisions, but AI doesn't really own accountability. you're not going to fire the AI, right? You're going to fire the person who presented the bad idea or said, you know, we should go in this direction. It doesn't matter if they use AI as a tool to do that. So I think our hierarchies are going to be built more around these kind of levels of authority and decision making. So AI may ultimately distribute the ability to make a recommendation, but it doesn't Distribute responsibility. So I think that that kind of leads us to understand that we're in this buyer-beware time, right? we need to understand that you know our usage of this stuff is on our own necks. And we need to structure our accountability within our organization in our hierarchies to really understand that, to take advantage of that and responsibility for it. His his third big idea. Now these start to build, and I I he he has a really good way of putting this together in the in the talk. But he says leadership moves from answers to conditions. And his quote here is the leader of the future will not be the person who has every answer. It will be the person who creates the conditions in which better answers can emerge more quickly while maintaining alignment and accountability. That does not mean less leadership. It means a different kind of leadership. I absolutely agree with him on this point. I think he's nailing this. When answers become easier to generate, which they do with AI, a leader creates value by by shaping the environment in which answers are generated, challenged, compared, tested, verified, connected to organizational goals and converted into. Accountable actions. Leadership starts to move away then from being the primary source of knowledge. It doesn't need to be anymore. It moves toward system stewardship because the the organization with the better system wins here. Right? Information becomes easier to come by. Knowledge and expertise becomes easier to come by. So having the system in place to surface it and find out reality, that becomes what wins. A leader can't simply create the conditions for answers to emerge anymore. AI can produce endless answers for us. The leader has to create conditions in which the organization starts to be able to distinguish between an answer and a good answer, right? Speed from Actual progress. Speed doesn't necessarily mean progress. Activity from value, confidence from from accuracy, alignment from obedience, and learning from just noise. we have a lot of signal, but we we have to be able to distinguish truth. And I think that's one of the areas leadership is going to have to take hold of in these coming coming years. there's another study that I read recently that was on really interesting is on human AI teaming. And by the way, I'll I'll upload all these research papers into our show notes on my website at agilityevolved.com that you can go to and and dive into yourself if you're you're so interested. but one of the interesting quotes from this study said AI autonomy is only perceived as beneficial when user goals are not undermined by the AI. when the the human establishes the goal and AI doesn't detract or take away or steer you off of that goal. it it means that more autonomy is not automatically going to be better. Faster answers are not automatically useful. The system Must remain aligned with the actual human goal. And I take it even a step beyond that to say we've got to be certain about what our goal is. Right? I've worked with a lot of product leaders over the years who, when I say, Why are you building that thing? the answer is not clear. They they can't really say, Well, that's to forward this business goal. So I think we have to be super, super clear on what business goal we're trying to forward when we build something. there's a tension here. I think it's worth exploring. I think leaders have to encourage AI use while preserving the ability to challenge that AI at the same time. It can't just be rubber stamped. in fact, that's a a huge problem that we see with AI is people kind of letting things slip through without challenging it. maybe we're we kind of give it a human quality of trust that we say, you know, we we've it's done things right 99 times, the hundredth time I'm sure it will be correct. Well, no, it has the same percentage chance of being incorrect each one of those hundred times. So trust is something you give to a a person, to a human. It's not something that that can be earned by a machine. it It has a percentage chance. It's all mathematical. it it's it's harder to do this than it actually sounds, because the the the more convenient and capable the system becomes, the easier it is for people to just stop asking things like, where did this come from? And which assumptions did this make when it made this this decision or or presented this information? What context did it have? Or maybe more importantly, what context did it lack? In this instance. did it exclude alternatives and and why? Why did it exclude those alternatives? What would make this answer wrong? sometimes I I will do that with AI. I'll just say, tell me all the reasons why this would be the wrong choice. And that can be a really good way to process through some information if you're trying to make that judgment call. I think this kind of leads to that the leader of the future may not need to know every answer the way they did in the past, right? They may not be that funnel point of information in the organization, but that leader does need to build an organization that knows how to question an answer and how to find truth, right? Because we're flooded with information. His his fourth big idea was the effort is no longer the constraint. And this is a big one. So here's what he says. When AI supports execution, coordination, analysis, and decision making, the bottleneck changes. The constraint is no longer effort. It's the ability to coordinate. The question is no no longer who works the hardest. It's who designs the best system. And I I would just say most management systems that we have in place today were created to organize scarce human effort. That was the expensive cost, right? Because it took human effort to do all the things that we were building. A worker, a human worker can only write so much. A human worker can only analyze so much or build so much. They they can only communicate with so many people. That's one of the reasons we we would have small teams and kind of agile teams is because it's like spokes of a wheel, right? You can only have so many lines of communication, and the larger your team gets, the harder it is to maintain those. AI doesn't have that problem. also manage so many active tasks. A worker can only manage so many active tasks, but AI doesn't have that problem. The the organization divided. Previously, labor specialized roles. It created cues and coordinated work around those limits. A lot of our practices were designed to handle those specific limits. And I think it's important for us to kind of analyze and realize AI changes the amount of work one person can initiate. But generating more work is not the same as creating more value. And this is something that's becoming Painfully apparent to a lot of organizations today. The system can become overloaded even though production becomes easier. We can still have bottlenecks, even though we removed maybe some of that constraint around building, around actually producing. Bankoley in this talk says coordination becomes the bottleneck. So here's where I don't that I would say I differ from him. I would say it's quite It's kind of more of a yes and. I agree that coordination does become harder. And I do think that that is a bottleneck. But I'm not sure that coordination is the deepest constraint. I think the bottleneck may be moving even further upstream into product judgment. When when development accelerates, the question no longer is: can we build this? Or even how long will it take to build this? The question is, should we build it? Who are we building it for? And how do we know? How do we arrive at that conclusion? If developers can build in days what it used to take months to do, then product discovery has to move just as quickly. Otherwise, the organization creates more output than it can. meaningfully evaluate, right? And we could end up at this situation where the the developers are kind of on hold, right? They they're waiting for more things to do because the product area hasn't kept up with product discovery. There's a research paper that I read on this one that was about accelerated experiential design. And there was an interesting quote here. It said that AI should be a muse. Injecting novelty and searching for areas of weakness to break the human out of over-exploitation induced by cognitive entrenchment. Now that's a powerful sentence, and there's a lot that's in there. I know some big words in there, but basically what it's saying is yes, humans are creative, but we get stuck in ruts. And we we start to get these blinders on us of having a limited perspective. And what AI needs to be able to do in the product discovery area is inject novelty. It needs to help us search for our blind spots, our areas of weakness, and challenge us in those ways to think along these different lines so that we don't get entrenched in maybe just one standard way of thinking about something. The the human provides domain expertise in this area. AI searches outside that expert's normal thinking, and the human evaluates those ideas. That's the structure that we want to put in place. And that's a structure we can repeat over and over. That directly ties to the the fifth big point. And this is kind of the home run, I think, of this. Faster work may make planning less valuable than learning. Now that that bears repeating. Faster work. May make planning less valuable than learning. What he said in this talk was the systems are faster, but decisions remain slow. The tools are more intelligent, but the structures remain old. when building something required months of scarce human labor, it made sense to invest heavily in deciding what to build before starting it. If I'm going to build 50,000 washing machines, right, planning and and investing heavily in planning up front makes sense because It's really hard to change production once you start it. So that becomes a high cost center. So we want to plan and make sure that we have the right thing before we make 50,000 of them. We planned in those areas because mistakes were really expensive. And that translates all the way into building software. We would do that very early on in software where we plan a lot up front. Because it took forever to build software. And so that way it meant, you know, we we better invest a lot in the planning so we make sure what we're building is the right thing. Even estimation, we estimated because capacity was scarce. Think about, you know, even things like story points or anything like that. That's all to manage a scarcity of capacity and to try to help us to plan and forecast based on that scarcity. We prioritized because we could only build a small number of ideas. And so we invested heavily in prioritization techniques and trying to surface what's most important. But AI is lowering the cost of generating and testing alternatives. So that poses the question of what happens when it becomes cheaper to just prototype more ideas or simulate several scenarios, or build you know, a limited experiment, test competing messages, generate multiple designs. Or compare working versions. When it's cheaper to do that than hold several meetings predicting which idea will work and which idea is best, or conduct some long exhaustive study, then maybe doing and judging the aftereffects of it becomes a better alternative than upfront planning and investing heavily in that upfront planning. So this makes me think about things like the Demming cycle, right? Plan, do, act, change. And we've lived with things like the Demming cycle for years and years. And I'm not here to say that's no longer valid, but I I do think it's worth engaging in this thought experiment to say if you think about those four areas: plan, do, act, change. Do was costly, right? Doing was costly. And so that made the weight of planning heavier to kind of balance that doing. It needed to be heavier because doing was so expensive, and we didn't want to get that wrong. But now change the equation because doing is now getting cheaper and cheaper and cheaper. So planning. Needs to come down in size as well to counterbalance that. We don't want to overweight the planning or remain heavy in the planning area when doing is now so much cheaper. So maybe planning in this new world becomes less about requirements, estimates, sequencing the work, predicting. Maybe it's more about what uncertainty are we trying to reduce? What's the cheapest experiment we can conduct here? What's the smallest proof? What what evidence would actually change our mind? And what customer behavior are we actually measuring to determine what success here? So maybe the the purpose of planning was never to produce a plan like we we maybe assumed, but maybe it was to reduce uncertainty before we spent money building something. If AI makes building dramatically cheaper now, how much uncertainty do we still need to remove before we start building? I think that's the important question to ask. I mean Marketing has been practicing this for years and years with things like A-B testing. Right? It became faster to put those multiple things out and let the kind of evolution of which one is the healthiest survive, and and constant refinement to occur. They've moved from planning up front to doing and and letting the strongest win. And I I see that as being something that's gonna translate into the product area as well. As AI becomes more capable, We're gonna have to move at a faster pace. Right? We're gonna have to be able to design systems that enable discovery and discovery at a much more rapid pace than we have today. And and I believe that's really gonna be the focus in the next coming years. So just to wrap this up a little bit, again, I'll put the link to this talk in our show notes if you'd like to listen to it. I think it's highly worth a listen and and worth your time. But the core paradox, I think, is that the tools are now more intelligent, but the structures that we have that we are fitting these tools into still remain old. The systems are faster, but decisions that we make and the systems around making decisions remain slow. AI is becoming more capable, but the organization does not necessarily become clear. And The speed at which we produce something does not necessarily mean that we're going to see more profits or we're going to be able to produce more value or we're going to be able to move the needle with our customers. Right? That's a different equation. And I think we have to re-examine our assumptions. AI may may actually reduce the cost of producing work. It can reduce the scarcity of information, the time needed to generate options. Or or the amount of human effort required for execution. But that does not eliminate organizational problems as a whole. It it moves them, it transitions them to different areas. And we're I I believe we're gonna see systems of work now that are are built around the new constraints that we're gonna have. Things like coordination, judgment, verification, alignment. Accountability, trust, and ultimately learning. So I I'll wrap up with just some questions that I I would leave you with to think about here. that these are things I'm struggling with well, things I'm I'm trying to think about. What parts of your organization or our organizations in general were designed around the scarcity of information? What parts were designed around the high cost of execution? What happens to management when neither of those constraints works the way it once did? And what happens when producing an answer becomes easy? But knowing whether it is right the right answer starts to become the real work. AI doesn't give us A faster version of the organization we already have. It gives us a reason to question every part of our organizational system and really challenge those assumptions and ask where where are our constraints? Where are our bottlenecks now? It means challenging each assumption we've taken for granted for years. And ultimately, I think it. It's gonna get back to three words that I talk about quite a bit in in classes I teach: transparency, inspection, adaptation. Do we have a clear view of reality? How do we question and examine why that's the reality? And what causes that to be the reality? And what do we do about it once we recognize the truth? Do we maintain? Do we change? Do we add? What do we do about that reality? I think that's the OS for the coming years. How how much we ingrain that in our culture, in our systems, in our organizations. Well, I I appreciate you joining me on this this solo episode. I'm gonna have these from time to time because there are things that I I just want to bring to your attention. There's so many great things that are out there that are popping up every day, and they challenge my thinking. And I I I'm hoping that they can challenge your thinking as well as we talk about these things. So if you like this episode and you you know we're we're a new podcast, we're this is only episode five. We're trying to grow our listener base here and get found by some other people because we want to create a community around this. I I feel like there's there's a a need for this conversation out there. So if you if you find this to be interesting and valuable, then what I'd ask from you is a couple things. One, please do like and subscribe in whatever podcasting platform you listen to this on. that really helps boost our ratings and and people will find the show much more easily. And it only takes a second to do that. That way you also don't miss an episode. But but also we depend on you to tell a friend. Tell somebody that you know about this and tell them how much you you found value in this. Maybe it's just an episode here or there. That's fine. Just send them a link to that episode. And you know, kind of search that out. If there's anything from this episode that you want to find out more information about, try to give you show notes on my website at agilityevolved.com. Also have my list there of upcoming courses. I teach certification courses for Scrum, Scrum Masters, product owners, mostly focusing on the advanced classes. So if you already have one of those certifications, I'd love to see you in class. We'll we'll talk for a couple of days about these topics as we talk about how that fits into our systems of work. If you have any feedback for me about this episode, I'd love to hear that as well. Please you know, send me an email. you can send it at Brian@agilityevolved.com. again, Brian@agilityevolved.com, and it's Brian with an I, just in case you you didn't know. send that to me, but that that's a great place to give me any feedback you have on the episodes, anything you want me to do differently, topics you want me to address here in this this this podcast. or even guests that you want me to have on. I've started to get some recommendations and got those in the pipeline. Really excited about having a couple people that are to come on the show in the next few few weeks here. So keep those coming. I really do appreciate it when when we get those from you. That'll about do it for this week. So I hope you enjoyed the episode and you know I hope you are having a good week, a good month, and we'll talk to you next time. On another episode of People Over Prompts Podcast.