AI in 2025: why 90% of Gen AI projects fail

David Sweenor 0:00 543, welcome to the databases podcast, where we explore the real world impact of AI Data Science and Analytics with some of the brightest minds in the field. I’m David Sweenor, your host for today’s conversation. Today, we’re thrilled to have shell Carlson, head of AI strategy at domino Data Lab. Shel has been in this business for quite a while now, he has a ton of experience, and he brings a unique perspective on AI, combining his expertise and governance strategy. And really, he has a knack for separating hype from reality. So shell, welcome to the show. Thank you for having me. So today, we’re going to talk a little bit about, you know, the biggest questions in AI today. You know what’s AI going to look like in 2025 you know, how do we govern it responsibly? And you know what’s real and what’s not and what’s all the buzz about agentic. Ai, so shell, can you just tell us a little bit about yourself and what’s going on over at Domino? Sure

Kjell Carlsson 0:52 thing, yeah. So I have one of the best jobs in the world. I get to spend most of my time talking to people about artificial intelligence and machine learning, talking with very advanced teams about what it is that they’re doing. The challenge. That they’re doing, the challenges that they’re facing, the successes that they’re having, and going in and getting attention for their best practices, their successes. But I also get to be a little bit of a therapist. It’s also the

Kjell Carlsson 1:16 don’t worry, you’re not alone on this. This is how other folks are tackling this, and, and, and it will get better, as long as, as long as you keep at it and go in and separate the what’s real from what’s not real. Alright. Well, let’s, you know, it’s time flew this year, and, you know, next year, so less than two weeks away, I’m looking at my calendar right here. That’s, I don’t know where the time went, but you know what are the predictions for AI and for 2025 you know what’s, maybe, what trends are over, hype or under discussed. You know what? What’s, what’s 2025 in usher in for us?

Kjell Carlsson 1:51 Yeah, well, I have to apologize. My biggest prediction for 2025 is actually my main prediction for 2024 which was that, you know, like 90% of Gen AI projects are going to fail to deliver transformative value. So, I mean, some of them will be successful. It’s just not transformative. They are they are improvements. They they delivered what they were, were supposed to, but they’re not changing the business as we know it. And unfortunately, we’re really set up for the exact same situation in 2025 most organized. It’s not that, you know, Gen AI isn’t ready, that the technology isn’t mature enough. It’s that our organizations are not mature enough, and we don’t have the people process and above all technological capabilities to really go in and properly identify, design, develop and deploy and certainly govern these Gen AI applications in a way that really moves the needle for the organization. So it’s not to say, you know, don’t invest in Gen AI. No, you’d be crazy. You have to. But it is really a lot harder than folks are thinking. And a lot of the beliefs in magic bullets that, you know, we were just going to go in and put these, put these llms in the hands of end users, and we were going to spark a productivity revolution. It’s like, well, I mean, there are certain areas, but even things like for code generation, it hasn’t been a slam dunk. There was a productivity improvement. In some places there wasn’t, and in many instances, it’s kind of plateaued. So really, organizations need to be really moving beyond the the oh, well, we just got co pilots embedded in our business applications. Or we, you know, have an A a hosted version of a private instance of an LLM, and really take the next step into, okay, well, what are the Gen AI products and solutions that we need to build that are custom for our business and that will really move the needle for our business? So it’s

David Sweenor 3:48 not as plug and play as as one might might think. Is that what I’m hearing, you know, in the case of, like, just, you know, you have a hammer and everything becomes a nail. It’s like, we have to use this. And is it a new way of doing things for companies, or they just have it they’re not thinking through the use cases, or is it just the complexity of deploying something across an enterprise? It’s

Kjell Carlsson 4:10 a little bit of all of the above. But the thing that I noticed from the organizations that I’ve spoken to where they have achieved incredible success. So thinking of bolt the European ride sharing company and where they are able to use llms to offload some giant portion of all of their customer chats. Well, it’s because they were already an advanced ml data science organization to begin with. They had plenty of experience creating data and machine learning products. They had the teams, they had the platforms, and were able to build on that when, when llms came on the scene and execute that really, really quickly. Or thinking about various bio pharma companies probably told them six so far that are leveraging generative AI for protein, developing faster protein of. Based treatments. And again, these are were already very sophisticated data science teams with ml engineering and ml ops capabilities, and this became this new additional tool in their toolbox that really unlocked the world of unstructured data and allowed them to do new things that they couldn’t do before. But it was all building on what they already had. So yes, you can go in and get very generic improvements. So I mean, think of this as like internet search, right? You know, internet search comes along and everybody becomes a lot more productive with internet search. But your business doesn’t sort of automatically transform, because everybody in your organization is using is using Google all of a sudden. All right, in order to be able to use that technology meaningfully, you need to go in and do all of that hard work to get your enterprise search to work. And that how many organizations today have are happy with where they are on Enterprise Search? They’re

David Sweenor 5:59 all terrible, as far as I can tell. Yeah, anything around here,

Kjell Carlsson 6:04 yeah, and it’s, it’s part of it’s a technology problem, but a lot of it is a people in process and incentive problem and and that doesn’t go away, because we have a new tool, no matter how powerful that tool is. And don’t get me wrong, it is incredibly power. Gen AI is and transformer models are incredibly powerful. They’re just not usually in the way that people think and not in and it doesn’t mean that they’re easy, easy to use with your data and your systems.

David Sweenor 6:36 Unfortunately, wish it was which it was different. All right, so on the prediction shell, how about jobs? Am I gonna have to take my job? Curious about your prognostication on that one? Yes.

Kjell Carlsson 6:48 And so it certainly will be the case that a lot of layoffs will be blamed on AI. They already have been there big tech companies who are saying that, you know, the 10s of 1000s of people that they’ve laid off or because, well, they don’t need them because of what they’re doing on AI and at least in developed countries, I think almost all of these are lies. There are almost there. They are a way of calming down existing customers who are worried. You know, if you’re firing all these people, what’s going to happen to your support for me or for investors. If you find all these people, where is growth going to come from, or boards, well, AI becomes the convenient scapegoat for in those cases, a different, different story. When it comes to to developing countries, if you are in the outsourced customer service center industry in in the Philippines, or in back office, the process back office automation, quote, unquote automation. A lot of it is delivery manually. This could be terrible. This could be devastating for for local economies. It probably is already happening. But if you’re in if you’re in the US, if you’re in Europe, most of those jobs had already been outsourced or automated in some fashion. So, I mean, there are, there are certain professions that are, that are likely going to be very different. I mean, this is not the time to go in and plan for that career as a voice actor. You know, if you are marketing, content writer, I don’t hear about that necessarily, layoffs, but a lot of, well, we’re just not back filling these, these roles, and we’re not increasing hiring in the same way that that we want or or occasionally it’s a you’re not getting the budget in marketing that you are because somebody has decided that you should be more productive with AI. So there, it’s not to say that there aren’t jobs that are, that are impacted, but it’s really small in the developed world, at least in 2025 if, if the technology dramatically changes, you know, who knows, in two to three years, but I’m skeptical, because we’ve been talking about that, oh, this would go in and change, change jobs, even before Gen AI came about, right? 2017 2018 when Uber was going out and saying that, you know, they were going to be replacing everybody with fleets of Robo taxis. And, yeah, I’m sure there are some Robo taxis in a couple of places. Minor detail that, in actuality there, there are humans in a remote, in remote centers, who are actually taking control of these pretty, pretty regularly. So, yeah, okay, I’ll believe it when I see more evidence of it. You’re

David Sweenor 9:36 an optimist shell. I like this. It’s, I think on the content side, I think, you know, like, is productivity the right measure of of a person? You know? I think there’s, and you’re right. I don’t think they’re going to hire as many people, but that just puts the people doing the job, they have to do a lot more, a lot more task switching. And I think it’s, I think it’s going to people are going to get burnt out. I think I. I don’t know this is just something I’ve noticed, and then with all these ways of communicating, I’d love to get your perspective on this. There’s like, we got slack, we got email, we got text now, now. And when I try to find something in an organization, I can’t find it because I don’t know if it’s in my slack, it’s in my email, if it’s in my text messages, it’s a nightmare.

Kjell Carlsson 10:18 Yeah? I mean, we can make a lot of that better with with Gen AI, right? We can go in and create assistance and search functionality that can now go and look across all of these different types of content much more effectively. Distill it. Help us find it. Help us use it than ever before. But just because we can doesn’t mean we’re going to and, you know, wouldn’t it be nice if there were caught solutions out there, from like, say, I don’t know, the folks who providing our, most of our business applications, and if, wouldn’t it be nice if they were to embed Gen AI effectively in those and sold that for us, they haven’t so far. And there’s not a whole lot in their incentives, really, to go in and do that. And there’s a lot to suggest that when it comes to Gen AI in particular, that so much of the of the value is in your own data, in the integration with your own systems, and of the alignment with your processes, that just not really seeing a whole lot of off the shelf solutions for Gen AI, even for the ones that should be the most generic and common across organizations. And so there’s not, not a whole lot of optimism around that there’s going to be more for you to buy for Gen AI that you know you can just plug and play. It’s more okay. We’ve seemed to be collecting more battle scars and more suggestions that now this is going to be even more of something that, like all of the rest of machine learning, that that you need to actually build up your own capabilities around.

David Sweenor 11:48 Sadly, all right, well, let’s, let’s talk about one more prediction. Let’s talk about, uh, AI engineering. Is this going to become mainstream?

Kjell Carlsson 11:56 Oh, well, I mean, it’s already been the case that the AI engineer job profile is, is like one of the one of the hottest new titles on LinkedIn. Now, obviously there’s a lot of confusion about what. So what is an AI engineer? Is this just an ML engineer that we’ve rebranded and is AI engineering just ml engineering, or is this entirely different? Most people have approached, been approaching it from the point of view that, no, this is entirely different. This is just Gen AI, and this is all about leveraging your LLM effectively, and it’s an entirely different way of working. Now that is true when you start with them, with with the experimentation phase, but over and over again, I’m seeing folks realize that, well, actually it’s the LLM when you’re looking is, is just a small component to this. It’s about, okay, well, at the beginning it’s a Okay. Well, now I need some I need glue code and beginning of it to manage my prompts. I need glue code at the end of it in order to add guardrails. And I need glue code in order to integrate with my production systems and my business rules engines that interact with those Oh, and now I also need to figure out how I’m going to be working with my with my my vector stores and getting information from those. Oh, and now I need to be doing a lot more on the data and feature engineering that’s feeding those vector stores. Oh, and now that I look at it, actually, this single LLM is much less effective than using multiple LMS in tandem. Oh, and almost always, it’s far more effective to decompose the problem and have a bunch of predictive ml models in this pipeline as part of this. And before you know it, this is well, ml engineering with a couple of bespoke, smaller components for for working with and managing generative AI models. So, so, so, so, I guess, sorry, that was a long winded way of saying. It is, there’s similarities, but it is different. It is an evolution. It’s not the same thing, but it is, but your their foundations are all the same.

David Sweenor 14:00 So we have aI engineers, we have mL engineers. Are data scientists dead? Or do I just need to go update my profile to say, you know, ml or AI engineer?

Kjell Carlsson 14:12 Yeah. I mean, it’s a tough one, because the data science moniker was always unclear, right? That was the power and weakness of the term data scientist was, well, we know we need somebody smart who can figure out a whole range of different technologies and, you know, understand the business problem and deal with our messy data and things. What is that role? Right? It’s not a, you know, it’s not applied statistician. It’s not a, is it a Data Miner? Is it a business consultant? Is an analyst. Let’s just call it a data scientist and and we’ll figure it. We’ll figure the figure out later on. And that’s my personal belief as to how the data sciences term came out. And then we created other terms, like ML engineer, which trying to focus a little bit more on the operationalization side of things, but never really defined. Very well. So yeah, I mean, there’s going to be the continued title proliferation and data scientist. Doesn’t sound as sexy as it as it did before. But what is an AI engineer other than a particular flavor of data scientists, the data scientist who has gone in and spent more time with with getting llms to actually work. And some folks, especially former other industry analysts, have have gone in and said, well, oh, but you don’t, you don’t need data scientists. You don’t need people to understand these, these. These are people aren’t trained on these things. So why would it? Why do you need a data scientist? Just go hire a developer, and the developer can go in and figure it out. And I have not seen that work. I’ve seen a lot of POCs that got created that way. But there’s a whole lot more in creating a data oriented product, and data scientists are usually the ones who’ve had to deal with that, who understand the nuances of that, of working with all the messy data sets from different locations, having to be very what’s the right term, not opportunistic, agile, flexible when it comes to their methods, and trying out a whole host of different methods, and working with the constraints of the data and the infrastructure and the technologies that they’re using, and being iterative with a business partner who never really knows what it is that they want until they see it. And so, you know, they’re pretty well set up for it. That doesn’t mean we couldn’t do a better job training that role. And I think the training will be a little bit will be a little bit different, and it’s going to be, for better or for worse, a lot less statistical rigor Okay, going into it

David Sweenor 16:41 all right, so that makes a lot of sense. So let’s talk about shift gears a little bit, and maybe another prediction, governance. You know, I hear a lot about guardrails and governance, so you know, how do organizations really strike that balance between governance or control, and I will say utility or innovation, whatever, whatever phrase you want to use, yes,

Kjell Carlsson 17:04 and I mean, there’s a strong connection to both of those. And there is certainly plenty of instances where where governance has been the enemy of innovation. But it doesn’t need to be that way. And I think there is a recognition that until we if we don’t fix governance, we don’t get very much in the innovation side. So this holds true if you’re just new to governance. So think of folks who haven’t been regulated before, folks in like retail or manufacturing or not regulated when it comes to data science and machine learning. That is where they’re realizing that the biggest barrier to to innovation is getting this approved and adopted in by the business, and in many cases, the business is very right in being skeptical of that these solutions are actually going to be reliable, that have been tested, that they’re going to perform in the way that that they’re supposed to, because the organization does not have governance processes in place in order to go in and identify what those risks are, identify what the performance metrics are, go in and restrict access to sensitive data or expensive compute or risky, risky libraries, and go and Ensure rigorous testing, validation, auditability and continuous improvement. So in instances where you didn’t have governance, the lack of governance is a is a barrier to innovation with AI. And similarly, on the other side of the spectrum, there’s highly regulated industries, financial services, insurance, pharmaceuticals, public sector. There the existing governance designed around regulatory compliance is so heavy handed, manual, manual intensive, labor intensives and slow that then that becomes its own blocker to to innovation, both within the existing regulated use cases, but as we think about applying governance to the other other use cases that aren’t as risky. We just can’t really if we try and take those processes and apply those elsewhere, we’re going to be, we’re going to be even more of a pickle than we than we already are, and more delays than we already are. So I would say it’s, it’s it’s not, it’s not that there is an innate tension between governance and innovation, there is a it’s a question of, it’s not governance or no governance. It’s it’s good governance, it’s effective governance, it’s streamlined governance and innovation. If you have bad governance, you’re going to get risk and delays without any innovation. If you have good governance, you are able to drive more, more innovation and less and less risk. But that begs the question as to what good governance is, though, so we can delve into that. If you well,

David Sweenor 19:47 you know, it’s sort of like, maybe it’s, you know, I think there’s some terms like adaptive governance, but, you know, it’s appropriate governance for the right use case. But on the risk side of generative AI, i. Did some research yesterday, and I was I asked, uh, look at some scientific papers. How many, how often do these things hallucinate or confabulate and make up stuff? And the stats were between 2% 8% up to 15% in some cases, 60 to 80% of the time, depending on the domain. Can the risks ever really be eliminated or just reduced to an appropriate level for whatever risk tolerance the organization has? I don’t think they can be completely removed, but I’d love your perspective on that.

Kjell Carlsson 20:40 Yeah. I mean, it’s a is it a bug or is it a feature? We we appreciate these models because they can be creative, because they can fill in for the lack of direction and guidance that we are providing to them. It’s a corollary to that, that since we haven’t told it, what it is that we that we wanted to do, that it will come up with things that we didn’t tell it we wanted it to do. So as you say, there is no way that I’m aware of of just taking something that’s off the shelf and then removing all of those risk factors from it, but there’s an incredible amount that you can do in order to minimize those risks. And above all, we as humans are terrible, right? I mean, we screw up all of the time. So the appropriate benchmark isn’t, okay, well, can I get rid of all risk? It’s a can I get rid of I can I, can I reduce risks to at least the level that I currently have, and ideally even further. And there I’m trying to think of an instance where I haven’t, where people haven’t been able to do that. I think in every case, you can get it to be less risky than a person. The only thing is that you’ve got to do a lot of work in order to make sure that it’s less risky as a person. And you got to remember, in most organizations, we do do a lot of work to prove to minimize the risk of us as humans like again, thinking back to that to customer outsourced customer service, so text messaging with with chat bots, and moving that from humans to to to Gen AI bots, when people are like, Oh, but what happens If I ask the bots to give me 1,000% refund on what I bought, sure, or what happens when I just continuously ask for for refunds every time I order something? Well, you can try and do that to a person, and if the person tries to do that, there’s a set there’s a business rules engine afterwards which prevents them from doing that and implementing that in their systems. So what is it that clever organizations who are using Gen AI chat bots for customer service automation are doing well, they’re just reusing that same rules engine. They’re just plugging into that so, so the LLM has a very constrained set of options that it’s allowed to choose from, if it tries to move beyond those ones and feed that into the rules engine to go in and execute on it, it is prevented from doing so. So there’s already a whole lot of guard guard rails. So not guardrails in terms of or built into the LLM, but guardrails around it that we have in our processes, that we seem to forget exist then that we couldn’t, just that we wouldn’t, somehow, why wouldn’t we think about doing those when we’re doing, using, using an alarm? Well,

David Sweenor 23:26 you know, that’s that’s a interesting perspective. I want to maybe poke at that a little bit. You know, if we so, if we take, like, ml ops or just a standard predictive model to if you look at data or model drift, whatever term you want to use, it’s pretty easy to understand. If things are changing, right, we have a distribution and we can go look and so there’s a known method to do that. But when we’re talking about creating text, images, audio, video, whatever, at scale, is there any way to monitor that? You know, like, what would be the approach to monitor something like that? I don’t know if the math is there or how you would even do that if you have this thing pumping out, you know, hundreds of emails a day. Yeah, if you ever truly monitor something like that at scale, it’s easy with numbers. I think it’s harder with unstructured data. Yeah,

Kjell Carlsson 24:23 well, I mean, I think you’ve hit the nail on the head in terms of both the opportunity and the challenge when it comes to generative AI, it’s that we’ve moved from a realm of structured data to unstructured data. We have now this great tool at our disposal, both for analyzing unstructured data, as you say, text, voice, images, video, machine logs, combinations of all of structured data and unstructured data, you name it, we can now do at incredible scale with incredible ease, and we can now create that data with it with incredible ease. Now there’s a minor detail that we since we haven’t been doing that in the past, we don’t really know how. How, how we should be going in and preventing that from measuring quality, preventing the creation of fraudulent fake data, making bad decisions around it. It does require, and you know, what’s the f1 score out of, out of something like this, when it’s very difficult to know for certain, is this a is this a false positive, false negative? What is? What is a false positive or false negative? On an on on a college application essay? Right? Where do we begin with that? And so it’s really on us to define what those what those performance metrics are, and it forces us to do what we should have always done on the structured data, traditional machine learning side, which was really to think about performance much more from the point of view of what the business needs, versus the easy statistical methods that we that we had before. And you know, we were his we’re all hoping that AI would do the thinking for us. And instead, it turns out, well, you know what? What is the most important thing to use artificial intelligence effectively? Well, it’s more human intelligence, around around AI and around around expertise. So you actually need your experts even more to go in and provide that guidance as to what are good answers, what are bad answers, and be involved in this on an ongoing basis, and have tools for them so that they can they’re not inundated with false positives, for example. And you also, and you need a lot of human expertise when it comes to designing these experiences, designing these systems. So I wish there wasn’t there was an easy solution. There is a solution, but it’s the it’s, well, it’s hard work. It’s, you know, how do you evaluate performance of when a person is doing the job?

David Sweenor 26:59 Good point. Good point. Okay, let’s talk a little bit about AI agents. It’s all over the place, and, you know, it’s AI that acts on its own. And from what I’ve read, AI agents are the panacea of productivity, and they’re gonna automate everything. I’m not sure I buy that. So let’s just start with what are AI agents? You know, I might have been like, is this like the daemon in Linux, or is this something more more involved than that? Oh, it was.

Kjell Carlsson 27:28 It was much more than the than the daemon Linux. It was the daemon and Linux. Who would figure out from you giving it like very vague, high level set of instructions, you have an idea of roughly what you meant, iteratively try approaches to figure out what would work in order to accomplish it. Check in with you occasionally, and then go and execute on it. And that was, that’s, that was the original idea with with agentic AI. So if you think couple years ago there and still, they still exist, baby AGI or auto GPT. The paradigm is give, give one of these systems a pretty high level objective. Go in and dispute my parking ticket. Go in and figure out what’s the best way I should, what’s the best way I should, or, you know, email, do an email campaign in the best way possible for this, for this product, come up with a use the the LLM, to come up with a structured approach to going and executing on this, have it go in and connect your operational systems and test out those approaches and see whether they worked, and continuously improve on that, and then, And then continue executing on that. That was the idea. So lots of autonomy, broad, broad scope and objective, very little, sometimes some human in the loop, but very little in the way. And the multi agent world is when you have these agents working with each other. So okay, so now I’ll have one of these agents which, in a very anthropomorphized fashion, will think about legal compliance, another one of these agents which will think about its approaches to going and emailing, emailing folks, and they will interact with each other. Oh, and maybe there will be another agent there which will also help create, uh, Mark brand specific marketing content that it can then feed into these campaigns. And you’ll have all of these different, very human like agents, very autonomously functioning in tandem, hi. It seems

David Sweenor 29:34 like a recipe for disaster shell. Yeah. It’s

Kjell Carlsson 29:37 sort of like, okay, would, would we are there any instances where I would go in and hire a bunch of very of high school, of high school valedictorians, and put them in charge of my company, right? And say, do whatever you want. Try it out. See what works. Keep on iterating this until you until you succeed. Well, we don’t do that with people. Why would we think we could. Do that in a highly emerging technology area. So there’s a one part is that it’s very difficult to get these things to work, to get to some sort of good outcome and equilibrium in them. Another part is that organizationally, right, you’ve got barriers up in the zoo to going in and trying out any of these things, you have to have a domain, well, either a very an entirely simulated, main, entirely virtual domain. So like a simulation of everything you’re doing. So maybe in manufacturing, where you can create a realistic enough simulation of your processes, you can, you can do something like this, and it’s very good for fraud. So if you don’t care that, you know it’s going to get something wrong over and over again, and you don’t mind going in and destroying your relationships with with hundreds of 1000s of folks, then sure have at it. But there are very few areas where anybody, not only does it work, but where anybody would willing to try it work for that conception of agentic AI, instead, there is a different conception of it, which, which is the, no, this isn’t a very narrow, broad objective function I’m giving, I’m giving my my genit AI based pipeline, some very, very clear objectives, very clear actions it can take define parameters around what those actions are, and I’m bridging the gap to my operational systems so that it is actually executing on those actions. And if I define this pretty narrowly, there are all of these places in my processes which were either manual or they were automated, but really bad, then I can now go in and plug these in. And, you know, the the most obvious area is anything to do with document based processes? Sure, because, you know these, these, we now have these great tools for going in and analyzing unstructured content. Guess what? All those documents and files are unstructured content. And so we can now go in and automate all of these mini steps in those processes, and then take it to the next level and automate a lot of the what I would the tasks that we do as people that really shouldn’t be done as people. They don’t use our human intelligence. They’re exactly the things that we hate doing as humans, whether they be that, they’re very manual, they’re repetitive. They require looking at lots and lots of data from a lot of different sources and making nebulous judgment calls around them. Well, more or not, the nebulous ones actually still require people, but ones where it’s actually very prescribed. You don’t need a whole lot of human intelligence to do it. You just need manual labor, sweat, attention, concentration, and you know how huge portions of what we do are things like that? Sure, that’s a different that’s the agentic AI that is working today. That’s the agentic AI that is going in and having these conversations with customers of when their food is not showing up on time, and automatically processing their rebates, but not if they’ve requested them too frequently. Okay,

David Sweenor 33:02 so shell, what I’m hearing is we’ve got to have a very specific use case, and then with generative AI, I don’t really have to write code anymore. It’s like a prompt. Here’s what I want it to do. So is that sort of, kind of where we are today. I’m trying to understand, like, what’s, what’s happening today, and we’re like, what’s what’s yours away, what’s your perspective? Well,

Kjell Carlsson 33:23 you, I actually don’t know whether or not, at the end of the day, you could almost envision a world where you actually have to write more code, because there’s so much code that you need to write around your LLM, in order to get it to work, work effectively. And all of that is you still need all of the control that you would have with code. And isn’t the prompt, in some respects, just a more free form type of code. It certainly doesn’t seem, you know, a lot of a lot of prompt is certainly not human speech in order, at least if you’re trying to get it to work. So I think there’s going to

David Sweenor 33:59 be more code. I’m hearing that, and it’s code maybe, to manage the different Yeah, Gen AI bots or agents that are, that are, you know, with your organization. I think we

Kjell Carlsson 34:09 need to split the the the way that we’re interacting with llms as part of the experience we’re doing in natural language, the way that we are constructing Gen AI solutions is is in code and then something which certainly doesn’t look like natural language. It’s more that the the weird and arcane world of of prompts. So yes, the llms have made it possible to interact with them in human language in a new way, but it’s it’s forcing us to develop new language. It’s a little bit like when we’re going to Google. We’re not going in and writing out the same kind of sentences as we would write in a text or in an email. We are using Google speak, sure, and we. Are forced to learn a much richer, strange vocabulary around this. And occasionally it’ll understand what it is that we’re we’re asking to do, but separate the the language of engaging with an LLM for the language used to go in and create your gen AI applications. I’m not sure it’s necessarily going to be more code, and to be honest, it’s, I think it’s still being figured out, whether or not we can abstract away more of this. But right now, the surprising thing is just how much incredible amounts of code that you need to actually write in order to get these

David Sweenor 35:31 things to work. Okay, that makes a lot of sense. All right, I think we have time for maybe one more question, and we’ll stay on the AI agent theme. And you know, I’m wondering about risks. And you know, how does this change the nature of AI governance, which we, you know, spoke about earlier.

Kjell Carlsson 35:50 Yes, it’s, it’s a great question. It’s one that is still being worked out. I think, especially when I talk to model risk management teams in finance, this is like their number one question is, how do we go from what we’re currently doing today for our credit risk scoring models, and go in and apply something vaguely similar, governance wise, to to to our general AI based projects? And my recommendation there is to go through and decompose the activities that you’re doing as part for governance that follow that life cycle of development and deployment, and when you start looking doing that, you find that those activities are are similar in terms of what they’re trying to achieve at the beginning of projects. What are you trying to do? You are trying to you? Are you going through and identifying risks? You’re identifying risks? You’re identifying risk metrics. You’re identifying the stakeholders who can provide input on risk. You’re creating risk mitigation plans. You are documenting all of these. You are getting appropriate approvals. Now, do those individual things look different for a predictive AI model analyzing structured data than a Gen AI based one they do, but the steps are, what you’re trying to achieve is similar, or the second step in this data and infrastructure environment access. Those are, yes, they’re different sources of data, but you still need to go through and identify, well, what are the risks in accessing these different types of data and writing to these types of data and using this infrastructure and using these third party, expensive models, or then on the ongoing basis, as the project progresses through development and deployment, ongoing monitoring of how is the project evolving, relative to my stated risk metrics and performance metrics going in and managing the process of remediating those risks as they as they come up, and ensuring that they’re remediated and that they have been signed off on, or when these are going into deploy and deployment, that these things have gone through rigorous, broad testing, that they are auditable and reproducible, so that you can undertake a continuous improvement those steps that you’re taking and the foundational capabilities that you need in order to do those don’t look are very similar between what you do for Gen AI versus versus not. So if you take this bottoms up approach to AI governance and think about what it is that you’re trying to achieve when in what you want to do to govern the development and deployment, there’s a lot of similarity, and there’s a lot that we can build on. The problem is, is that a lot of folks are going the opposite, top down Route, and saying, Okay, well, let’s start by creating an AI governance council. Let’s start by with our principles and our and our governance framework. Let’s do an audit. And those are, those aren’t inherently bad things. Those are good things to do, sure, but you run into this big cliff where it’s like, okay, and now what we’ve done these things, but we haven’t really changed what’s happening on the ground. We haven’t changed behavior. So what have we done? We’ve, you know, spent like, six, six months or a year doing this, and we don’t have a whole lot to show for it. And then the the willingness to go in and continue this effort peters out. And, you know, you don’t want to be in that situation. You need to be in some kind of virtuous cycle of, okay, we can, we can improve governance, improve trust. We got things into production faster, and that helps us invest more in our governance capabilities. And to do this even better, more rigorously, more and more in a more streamlined fashion.

David Sweenor 39:13 Okay, well, there’s, there’s a lot to think about for organizations. And I guess maybe to wrap things up shell, you know, there’s this field is just moving so fast. There’s, there’s new things out every day for the technology leaders out there, you know, what advice would you have for them to you know? How can they perhaps get started? What would you recommend to them?

Kjell Carlsson 39:37 Well, it would be move fast, but moving make sure you’re moving in the right direction

David Sweenor 39:40 and walk, walk with purpose. Walk

Kjell Carlsson 39:42 with walk with purpose. And a lot of this is leadership. A lot of this is it’s not going to happen on its own. That direction is not going to happen on its own. There’s a lot of well intentioned belief that, let’s encourage broad based experimentation. Let’s get put this in the hands of everybody, and they will. Figure it out, and there’s not a lot of evidence for that working versus let me go in and find the individuals that are ready my organization, who have the experience, who’ve been working around this. They haven’t been making the sexiest, broadest claims, because they know better than to do that, but they have been working towards existing machine learning and predictive AI use cases already. See what they’re doing, empower them, maybe up level them, move them into leadership positions, if you don’t have them, hire folks externally who have those, those kind of experience delivering business value with AI machine learning, and give them the mandate support funding to go in and build up those capabilities, create a road map and a portfolio of successful use cases and really spark that virtuous cycle. But you know, it requires leadership, and you don’t have to be in to spark this. If you’re a Chief Data Analytics officer or a CIO or a CEO, you don’t have to be the AI expert yourself. You just need to be going in and and hiring those people, promoting those people, supporting those people being involved making sure that they are aligned to value and that they are delivering and, you know, holding them accountable to do so. And you know, if you do that long enough, you will actually become an AI expert, at least a leader, an AI leader in the process of doing so.

David Sweenor 41:18 Okay, well, amazing. Well, thank you for that advice. Shell, and this has been a great conversation. I really enjoyed it. I want to thank you for joining, and I think our listeners are going to learn a lot from that. So appreciate

Kjell Carlsson 41:29 it. Oh, my pleasure anytime. All right, cheers.

David Sweenor 41:32 Bye, bye.