AI agents: state of the union

David Sweenor 0:00 David, Hello everyone, and welcome to the data faces podcast where we bring the stories between data analytics and AI. I’m your host. David Sweenor, founder of tonic TinyTechGuides. On this podcast, we go beyond the technology explore the human side of data driven innovation, how it’s transforming businesses and shaping our world today. We have a very special guest. I am honored to be joined by Sanjeev Mohan. He’s the principal at sanjmo and former VP of at Gartner. We spent years helping organizations around the globe understand and adopt data strategies. So in this episode, we’re going to talk about AI agents. We’ll be exploring the current landscape of agents, what’s real, what life and where it’s headed next. So Sanjeev, welcome to the databases podcast.

Sanjeev Mohan 0:50 Thank you so much for having me.

David Sweenor 0:54 Can you tell us, for people who are familiar with your work or what you do, and a little about sanjmo,

Sanjeev Mohan 0:59 thank you so much for asking. I am a former Gartner analyst, like you said. I was part of the data and AI practice, and I was responsible for defining the agenda. Having a great time. Gotten is an amazing place. But about three and a half years ago, I decided it’s time for me to just go strike it on my own and just explore the bleeding edge of technology, something that you know, lot of Gartner clients don’t want to be at, and I wanted to be at that stage. In fact, it was amazing for me to make that journey at that time, because lo and behold, very soon, generated AI blew up in our face cases and so now I’m literally my job is to research, analyze some new technologies, changes that are rapidly transforming both data and AI space, and then try to connect the dots, explain to people what’s going on. I help a lot of my clients with with their offerings. I write blogs, I record podcasts, so basically, I’m, to be honest, having time of my life. So it’s a

David Sweenor 2:21 great time to be in tech, for sure. So before we begin, Sanji, tell us about Davos. I read your medium article. It was amazing, maybe just a minute or so on kind of what your overall experience was there.

Sanjeev Mohan 2:32 So Davos is an example of what I get to do now, because I have no boundaries, I have no bosses to tell me what to do or what not to do, right? So I decided, Baba said, been on my bucket list for for ages, and I decided, you know, why not just go and explore? So I went with lot of trepidation. I had no idea what to expect, because unlike other conferences where you have a prescribed agenda, here, it’s all self directed, and the hundreds of events going on, some you are invited to. Some you’re not. Some you have to pay for massive amounts, and some that are free. So you get to pick and choose and build your adventure. And I have to say, now that I’ve been there, it it was beyond my expectations, the kind of conversations, the people who show up. It’s you gotta have some gumption to show up. It’s not, it’s not a cheap you know, everything about ours is, is difficult, getting into a tiny village, hobnobbing with billionaires and government leaders. So the ambience is great. I learned a lot, made some really good friends. I didn’t really go for business development. I went more for discovery. Next year, if I do happen to go, my plan is to actually run my own event and has some industry leaders executives talk about what’s going on in IT space. Wow, that

David Sweenor 4:08 would be amazing. I’ll make sure to link to your article in the in the show notes. It was a great article, and thank you for sharing that with us. So let’s talk about AI agents. It’s it’s maybe even more hyped than the generative AI craze that’s out there. Can you give us a simple definition of what AI agents are? Yeah.

Sanjeev Mohan 4:29 So first of all, you know, when I went to Davos, I had so many conversations with people, and everybody was hooked on to agents, and they’re like, I want to build an agent to do X or Y. And I was asking about how you going to do it, and they’re like, Well, we haven’t figured that out. So my point is that the whole AI agent space is mired in confusion and maybe over hyped to some extent. But to answer your question, so when generative AI or. First became really the most, the biggest topic we are all talking about these days. The whole purpose of a large language model, or any foundation model could be even a vision or video image, is to predict the next token. I’m using this whole idea of vector search having a vector embedding of all a world’s knowledge, you tend to predict what comes next. Sure, in agent, it’s different. It’s not about what comes next. It is about about action. It’s about understanding the context from various different sources, making some sort of doing some reasoning, coming up with a plan of action and then executing it. So for example, I, by the way, I have a frequently asked questions also in the same location on my medium site, where I frequently ask questions on AI agents, where I give an example of of, you know, you get all these emails every day if you have an agent that can read the email, because, you know, life language models are very good at understanding language and even extracting sentiment, even knowing who this email came From, what is my relationship to that person, and then taking an appropriate action? So for example, if I’m working in a company and my boss has sent me a stinker, then maybe I should escalate it. If it is a prospect asking me to reschedule a demo, then go update my calendar so you see you’re understanding the context, and you are executing some sort of actions. And if we transpose this to a very complex workflow like supply chain, then agents can automate a lot of day to day tasks. And so that’s why there’s so much emphasis on agents.

David Sweenor 7:00 Okay, you know, that’s sort of what I’ve seen. I’ve seen, yeah, you can read emails and order groceries and things like that. So, yeah, yeah. So it was sort of like mad to me. But I think when you get into supply chain, it is actually much more interesting. So I guess, how mature are they today? And do you think they’re living up to the hyper? Are we just at the very, very early stages.

Sanjeev Mohan 7:23 Early stages. I would not have an agent to run my entire supply chain, because they’re still dealing with problems with hallucination. And so I wouldn’t say they are mature, but they everything is moving at warp speed. So, so I may say something today, and I may be eating my words tomorrow. You know, deep sea, just as like, like a brick only 10 days ago, and like, $2 trillion of market cap, 600 billion of Nvidia market cap. It’s just things are moving really fast, but agents are not ready. I would use agents for things that are less mission critical. For example, let’s say you know, you and I are analysts. We do a lot of research, or maybe within a company. I do love competitive intelligence. So I have an agent that is my assistant. It does research on my behalf. So what it does is, every day it goes out on the internet, crawls. It tries to see which companies are potentially going to disrupt me. Cross through their website, scrapes their website, summarizes what’s going on and very neatly, it presents me in a table. Here are the companies that you should be watching out for. Here are their strengths, their weaknesses. There’s some SWOT analysis. I’m just making this up, but these agents are really so like game changer, because I don’t have the time of the day to crawl through and try to understand what’s out there, but an agent can do it for me, and then I can take that input and I can I’m the pilot. Agent is my co pilot, although that may change, and that’s a scary part when agent become the pilot, and you become the co pilot. So that’s also going to happen, by the way, you know. So

David Sweenor 9:26 I think that competitive intelligence example is super fascinating to me. I just actually wrote a custom GPT to do competitive intelligence, pick a vendor, and it’s going to go through. But I feel like I need to apply my expertise, so I have, say, a 10 step workflow, and maybe step one, step two, I want to maybe tweak it a little bit. So how autonomous Do you think these things can and should be?

Sanjeev Mohan 9:51 So the definition of an agent is that it is autonomous, but the level of. But autonomous may vary, like some are fully autonomous, some have some feedback. But the idea for an agent is that it works on your behalf, autonomously. If it’s not, then it’s it’s a chat bot where you are asking questions, giving you a place. It’s not an agent anymore,

David Sweenor 10:19 okay? So that hasn’t made its own decisions, and that’s quite interesting. So what do you think are some of the key challenges that organizations are going to face when they’re trying to, you know, put AI agents into their their infrastructure.

Sanjeev Mohan 10:33 This challenges include reliability, making sure if it’s autonomous and it’s going to make decisions and act upon it on my behalf. It better will not hallucinate, because if it does right downstream is is screwed up, right? So, so that’s so reliability is really important. Second thing is cost is really important. If the cost, if it autonomously, starts doing things that are going to break my budget, you know, then that’s not a good thing. So, so that’s the second challenge. The third challenge is data, privacy sovereignty, making sure it doesn’t do something that’s toxic in nature or exposes credit card things like that. Those things can also happen. Sometimes a model may be trained on Reddit content, and Reddit content may not be all kosher, and then all of a sudden, it’s mimicking what it learned. You know, so, so that’s why reliability is super important. Making making sure that the the data sources are curated, the models are trained on curated data sources. There’s some sort of Knowledge Graph, some ontology, or some way to ground the results, is very important. The second thing that is also happening is that the models are getting far more sophisticated. So this whole concept of chain of thought, where it’s not just giving you an answer, it’s self introspecting and reflecting. Did I have the right answer? Maybe I should give it another shot. I was just reading the open AI’s response to deep sea, something called deep research, and I saw a LinkedIn post by the founder, CEO of HubSpot. He asked for a marketing plan, and he wrote back, and he says, I was blown away with how good it was, but it took 11 minutes. Okay, so, so these are the new models. And the interesting thing that’s also happening is that these the one year ago, we were expecting that the best models are the ones that are trained on most data, so the larger the model, the more information it had to accurately answer my responses. But the the thinking has changed in this year. It’s not about how big your model is, how how small your model is, but it’s trained on relevant data, and also it’s making decisions at inference time. So this whole test time compute is, is a brand new concept. In fact, with inference time computation, sometimes you don’t even need a GPU. That’s why Nvidia’s stock took a hit, right? Because I could be running it on an edge device or my Apple Mac and chip,

David Sweenor 13:50 right? Interesting, you know. And you mentioned a couple times, you know, the the importance of data, and I probably for both of our entire career data has been a problem, and at that time we we’ve only we’re really focused on rows and columns, structured data, you know, numbers and things like that. What is the state of data for organizations like I don’t think they have their their structured data, right? I don’t even know how they’re approaching the unstructured stuff. They probably have mountains of PDFs and whatever other formats they have, and what, what’s your, what’s your sense of how organizations are going to tackle this?

Sanjeev Mohan 14:30 So first of all, 80% of an enterprise’s data is unstructured, and that’s untapped.

David Sweenor 14:37 I feel like we’ve ignored it, though, like for for so long, and now Jenny Oh, well, now we can better go, maybe go figure. So

Sanjeev Mohan 14:44 I think Jenna makes it really easy to to extract the entities or the knowledge which is buried in these, these structured, unstructured PDFs, I know, but structured data is really important, because structured data is where. You get the linkages. For example, I can put a an embedding model towards a PDF, and I can say, extract entity sentiments, and so I can ask questions. But what if I have a lot of PDFs or a lot of different clients, my client information is sitting in, let’s say, an Oracle database, sure, and I’ve got the entire history of transactions. They are not in PDFs, so, so basically, I need to combine that structured data as we know, we’ve been struggling with data quality for a long time, but, but the problem is, irrespective of how my my underlying data is, I cannot be held hostage. I cannot say that. Let me first put in data governance right log. Let me get my bi correct before I can jump to AI. It’s too dangerous because my competitor is already full in into AI, and I’m just struggling with getting my my my data space, correct. So my point is that recognize you have your data is imperfect and you have problems, but don’t delay your dabbling in AI, because in the long term, you’ll end up paying the price. You’ll have the most pristine data in place, but you would lose out on leveraging it through AI. So it’s a delicate game, because if you put AI on data that has poor data quality, then it’ll hallucinate even more, and it’ll come to bite you. But if you so, so my point is that that you have to be you have to take a very practical view of it. It not no longer, no longer, theoretically, let me do everything by the book. If you do that, then you miss about, right, right? Yeah, the days, the

David Sweenor 17:05 years when we had a, hey, let’s do a big Ed W project and, you know, like a three year project, those are done. That’s clear, yeah.

Sanjeev Mohan 17:12 So recognize you’ve got data in different places and and you may not know the right single source of truth, but we use AI to help you with that. The funny thing is, we always criticize regulated industry for being behind times, because they spend so much time right, worrying about Simon Oxley and basil too and all of these things. Well, guess what? They are now reaping the benefits because they spend so much time regular, like cleaning up the data for compliance reasons that now they can jump into AI and not worry about data as much. Oh,

David Sweenor 17:50 they’re very interesting. So I would assume so there’s like mountains of PDFs and whatever other unstructured documents. How does an organization wrestle with they might have sensitive data in there, or things that you might not want to get out into the world. Is there new technologies, guard rail systems, that people are going to need now to to consider like, Is this a new part of the architecture, or is it data governance plus plus?

Sanjeev Mohan 18:17 So not a new part of architecture, it’s with generative AI, unstructured data, we have to do things a bit different, and governance, having these checks and balances, grounding is really important. Companies already doing it, and the way they’re doing it is that they have these they’ve created these documents with Q and A that keeps track of what’s toxic, for instance, like if you ask somebody’s gender, it says these are the right answers. So I saw that just yesterday. I was looking at one of the vendors software. They have a bunch of these. Custom checks actually checks out of the box, but then companies can create custom checks. So, so this is how generally we are so different from traditional software development. Traditionally, I developed the software tested unit test, systems integration test, UAT, I deployed it, and then I moved on to new, new features. I didn’t worry about the software, but in Jenny and AI application, I have to constantly monitor it, because it can go off the rails, even with my best testing. Because it’s predictive. Sorry, it’s probabilistic and so. So that’s why we have to put a constant monitoring, feedback loop, reinforcement learning with human feedback. And you have to constantly evaluate. That’s one thing. The second thing is we also evaluating which is the right model, deep seek, maybe a great. Model for for reasoning, but for coding, maybe Claude is still the best, and for summarization and translation, maybe it’s Google’s Gemini or GPT Floro. So and the the cost changes all the time. So what I’m doing in this. In fact, I call this an agent management system. And I’ve even predicted on the cube that in 2026 Gartner will have a Magic Quadrant on agent management system. And so evaluation, monitoring, all of these things are much bigger part of of the life cycle than it used to be in software development.

David Sweenor 20:46 Yeah, something to think about. I mean, it’s almost like you might have a model, different model per use case. It seems super, super complex. And I get, you know, and the monitoring piece, I’d love to maybe double click on that a minute for a minute like it’s fairly straightforward. With numbers, there’s lots of statistical tests accuracy, whatever. You could say, hey, the distribution is changing, or whatever, and you can stop making predictions. But when we’re talking about generating text or audio or video, is there even technology out there that can monitor that at scale? Or how would an organization even approach this if you’re pumping out 1000s and 1000s of emails or videos or audio files, how do you how do you even approach that?

Sanjeev Mohan 21:34 Yeah, this is the this is where a lot of agent frameworks are falling apart. To be honest, they, you know the I don’t mention them by name, but, but there are so many of these agent frameworks they don’t scale. They’re incomplete right now, this whole space is is rapidly transforming. And the and what, what I’m seeing coming out is like everyday miracles. Every day is like, whoa, okay. Now this change, but so what it’s like? We’re so used to everyday miracles that we are not even bothered. We are like, we expect these things to happen every day. Now,

David Sweenor 22:22 okay? Well, how do you see the role of human decision making evolving or changing? Do we see, you know, maybe some, I’m sure there’s some use cases that will be fairly autonomous, and some, you know, you definitely want a human in there. It’s a spectrum, but you see it being reduced. Or what’s your what’s your thought on that human AI collaboration, I am

Sanjeev Mohan 22:45 I’m very upbeat about it. I really feel that every new technology we’ve had scary scenarios or things just humans getting devalued. Do you know, in 1930s when the telephony industry was taking off at and Ts and Bell Atlantic world in the United States, there was a thinking that there would be 3 million jobs, primarily women, and their role would be of switchboard operators. Remember those

David Sweenor 23:23 plug in an area, on a party line, all that stuff?

Sanjeev Mohan 23:27 What happened to those 3 million jobs when ATMs came? It was what happened to all the tellers? What the tellers became private client bankers or moved on? So I really don’t I think we’re living in the same moment where we think AI is going to take over all our jobs, and what could we do? I actually came across an article cover page of the Time magazine. I have a link to that. It’s very funny. I was just doing a Google search 50. It’s 60 years ago. Next month it came out, and it basically said, human beings will have long leisure time computers.

David Sweenor 24:09 I’m still waiting for that. I feel like I’m more busy than ever.

Sanjeev Mohan 24:13 Well, I don’t see, you know, I did a, published a podcast today with Google Cloud, two of the General Manager, VPS, and they were saying that AI is writing 25% of our code, but it gives us more time to think more strategically. So all these things, did I get the right syntax? And did I miss this? Or how do I write this stuff? All that time consuming stuff is just being automated, but I think this is a type of human intelligence to shine and take us away from busy work. But those are my opinions, and not everybody agrees with that. You know, even people like Jeffrey Hinton and the father of AI are very, very scared of what AI is capable of doing. I just think it’s, you know, even even printing press, when printing press came out, right, you know, in fact, you can even, I’m on slippery territory here, but you know, the Catholic Church was was completely disrupted by printing press. Because when printing press came up, people like John Calvin and all these people were printing their reformation ideas and sticking it on the church door, and it led to the whole reformist movement, usually disrupting to the Catholic Church, but we as humans survived and thrived. You know, we have more books. There’s a very funny thing where there’s some written documents about parents are very upset that the kids are spending all their time looking down and not going out and playing, and what’s going to happen to this generation. And when you think about looking down and all you think about their iPhones and iPads, it was about books, because my generation was illiterate. There were no there’s not a lot they could read. And then the with over a generation, the kids were reading more and playing outdoors less. So. So my point is that it’s a continuum. With every technology, there’s going to be disruption, but humans will try the only thing that will kill us humans is what we’re doing to the environment and nature, but not technology. We will leverage technology to our benefit. Speaking

David Sweenor 26:50 of that, you know, you see like these articles, like, you know, Microsoft and other companies, hey, we’re going to reopen nuclear power plants just to fuel these do we? Do you think that’s going to continue? Or, you know, with maybe small language models? And, you know, you mentioned earlier that bigger might not always be better for the use case. Is that you think that’s the environmental impact of Gen AI, how is that going to maybe, maybe change? It’s

Sanjeev Mohan 27:14 a I’ve, I’ve talked about this question a few times on different podcasts, and I have to say, on one hand, as the the models are getting smarter and better more efficient, so the need to have 10s of 1000s of GPUs to train a model, hopefully is not going to be the case. But what’s what is also going to happen, on the other hand, is the g1 paradox will take place, which is which says that when we make things more efficient, it’s usage goes up so much so. So basically, if every company has 10s or hundreds or 1000s of agents, then the usage of Gen AI is going to go up tremendously, so which means we will need these massive data centers to process it, maybe not as much for training, but for inference. So. So I actually like the idea of Microsoft buying Three Mile Island and converting one of their reactors to power. So to me, it is renewable of nuclear based and order, huge problem, but I don’t see any decrease in the number of data centers. Hopefully they’ll use renewal energy and not coal fired and you know.

David Sweenor 28:46 So let’s assume now. So the technology, it’s the sort of becoming, I don’t say commodity might not be the right word, but everybody has access to sort of the same models and technologies. So what do you think is going to differentiate companies. Is it their process? Is it how they deploy it? You know what? What will be their competitive edge, if everybody has access to these, you know, high powered, you know, the technology systems.

Sanjeev Mohan 29:13 It’s one word data, it’s, it’s a quality or the richness of the data they have, because no one can replicate that. And so the computation may be the same, the models may be the same. So you may use Lama three, and I may use Lama three as well. But if I have, if I’ve distilled Lama three on to my data and sort of post trained it. So Lama, it’s open source. I have the weights so I can do post training, then I have an advantage, because I have high quality data which is not available to others, and so I can now. Do more decisioning reasoning based on my data. So I think data is the biggest moat for any organization

David Sweenor 30:08 anyway, man, having access to quality and knowing where it is and all of that fun stuff, yeah,

Sanjeev Mohan 30:14 and that includes all those 10,000 PDFs that you have or emails you know, messages, Slack, messages, all of that.

David Sweenor 30:28 Okay, what sort of skills do you think companies will need? We talked a little bit about, you know, as the past technology, technological revolutions, and, you know, maybe the jobs change. So what should be companies be looking for right now, when they’re they’re hiring individuals,

Sanjeev Mohan 30:48 the one that, one skill that I also I personally use, and I think it’s so important, is curiosity. I I would, because so much is is getting disrupted. You coding jobs are getting disrupted. But having this, this flexibility and curiosity to learn and not be afraid to experiment, all these technologies are available for free. You can go to hugging face. You can download any model and Agent frameworks are free, but just experimenting, like you yourself said, you know, you created a GPT chat bot or something, right? Yep, yeah, competitive intelligence, yeah, to do competitive intelligence, that’s brilliant. You’re already a step ahead. You know, a company that wants to supercharge their marketing would be like, I need David Sweenor to to help me supercharge what I’m doing, because all ways are not working. So, so the fact that that you’ve done it and others are doing it. That’s what I would look for in a new hire.

David Sweenor 32:06 Yeah, it’s very interesting. I agree with you on that, you know, curiosity, you know, entrepreneurial or experimenting mindset and being flexible. I don’t see those in job descriptions or or I’ve done a lot of interviews, they’re always like, hey, I want these eight skills or whatever is, whatever role it is, but they seem to, I don’t know, I agree with you. That’s what I look for. Like. I always say, you can’t, you can’t coach enthusiasm and things like that. I can teach people a lot of things, but not enthusiasm. But I don’t, I don’t see the job descriptions. They don’t really focus on those things, nor do the interviews. In my experience,

Sanjeev Mohan 32:45 I just made a note from as a Kent coach, enthusiasm that’s That’s brilliant.

David Sweenor 32:54 Yeah, all right, so we’re getting maybe close to the end of our time, but maybe I’ll talk a little bit about just regulation. There’s a lot of questions about ethics regulation. What’s What’s the state of where we are, what our companies need to think about?

Sanjeev Mohan 33:13 So we can close by going back to Davos. It’s so funny because in Davos, I attended a session. It’s by an organization called Open Forum, and it was so negative. It was, I have turned off GPS, I’ve turned off my internet as much as I can. And there was like doom and gloom the whole time, you know. And and and then I went to other sessions where it was like, Don’t worry about regulation. How do you regulate something that is not even baked exchanging, how do you regulate something like that? So and so I saw two extreme ends of it. I am more in favor of experimentation right now, also this notion that AI is, in fact, will dot i dot, M will i am? Yep, he was, I was listening to him talk, and he said, worldwide web, www, is Wild Wild West, right? And, and, but, but the point that I’m trying to make here is that it’s not that there are no regulations in place and AI is completely unfettered and it can just cause massive harm. There are regulations in place. They may not specifically be called out for AI, but they’re there for data Sure. So, so I’m not too concerned about AI being unregulated. I think it’s just nature of something that’s new. AI is not new. We know that it’s been there for decades, but. Almost a century, but generative AI and those aspects are new. I think those regulations will come in time, but there are some basic safeguards already in place.

David Sweenor 35:12 Yeah, totally agree with you. So that was a fascinating, you know, quick, quick discussion on agents Sanjeev. The last one is, you know what? For business leaders, tech leaders out there, like saying, I can’t make sense of this. What’s your sort of key recommendations to them?

Sanjeev Mohan 35:30 I cut out noise. And you know, there are always people in your organization or outside, like, you know, all the analyst firms, for instance, yeah, like, talk to them, but don’t, don’t get bogged down. I mean, there’s so much uncertainty everywhere, politics, monetary, technology, so social media is, like, filled with all kinds of, you know, very toxic content. So my, my advice is, cut it out. Just look, think of what is your what are your business imperatives? How can I improve that? That’s it. Just be very focused on that. Can AI play a role? Maybe not. And if that’s the case, then, then that’s fine. You know, AI is not a panacea for every issue out there, but, but take a be very data driven. What is, what is the problem I’m having, and I don’t know what’s the right solution, because it keeps changing. So I’m going to talk to to people in my my organization who may know or give them a project, you know, go do an ROI cost value analysis and see if we were to put AI to take some of our customer service calls. Is it going to save us money, but more than money, is it going to improve our reputation? You see? So do this analysis, but do it in a structured manner. Use a lot of AI governance products these days have an ability to capture all of these things that I’m talking about. You were asking earlier how AI governance different in data governance, we didn’t really do all this. We went straight to the tactical pieces, but in AI governance, what I suggest is come up with your list of of business strategic objectives and then do an analysis. If you don’t do that, you know what people do these days? They’re like, how can I use Lama three or GPT four? Oh, that’s a wrong thing to focus on, because you’ve jumped to technology when you should really be looking at it from a business side, and do it in a structured manner, see the results, and then take that experiment into production, if it makes sense. Expert

David Sweenor 38:14 Advice from Sanjeev Mohan so Sanjiv, thank you so much for joining the databases podcast has been illuminating discussion. People can go find you on your website, on medium, LinkedIn, you’re everywhere. Hope they give you a call. But thank you for joining the show. This has been a fantastic discussion. Thank you

Sanjeev Mohan 38:33 so much. Take care. Bye. You.