Stop chasing hallucinations — focus on agentic quality

David Sweenor 0:00 David, Hello everyone, and welcome to the data faces podcast that brings the human stories behind data analytics and AI to the forefront. I’m David Sweenor, founder of TinyTechGuides, and your host for today’s discussion. Today, I’ve been looking forward to this one for a couple of months now. I’m joined by John Park. He’s the CEO and Principal Analyst at amalgam insights. Jan’s a legend of in the industry. He spent worked with countless organizations on data analytics and AI and he actually specializes in tech finops. I don’t know him from that world, but I know him from the AI world. So today we’re going to talk a little bit about what we call, he calls agentic quality, and we’re gonna talk a little bit about the term hallucinations and why we might want to retire that term. So let’s get into it. Jan, Welcome to Data faces. It’s great to have you here.

Hyoun Park 0:48 Thanks. My pleasure. Tell

David Sweenor 0:52 us a little bit about you know, you your background, how you got to be the CEO and founder of amalgam insights. And you know, what are they? What are they all about?

Hyoun Park 1:00 Sure, so I’ve been in tech for about a quarter century now. I been an industry analyst for 17 years, and I’ve owned my own firm for eight of those years, amalgam insights. I i would say that going back into the history I first became an industry analyst because I was reading the call it, Gartner and Forster reports in my spaces, in my IT management spaces when I was almost 20 years ago. And frankly, I was not super impressed with them. I looked at these, what they were calling like these, total economic impact reports that had nothing to do with actual economics. I was seeing these magic quadrants where the solutions at the top right were not the solutions that I would choose. And I just thought, wow, there’s got to be a better way to do this. So I I got put my hat into the ring, and I started working at a company called the Aberdeen Group, and started doing industry analyst work a little bit differently, and it’s been a great 17 year ride where I’ve been able to dig into it costs and financial planning, and then digging into some of the data analyst and database world Database Administration work that I used to do as a technical operational guy, and it’s been really interesting to see all of these transitions that have happened across mobile and big data, big lakes and data lakes, analytics, predictive analytics, and now what we’re doing with generative and agentic, AI, it’s been a really interesting ride. And then now I’m about to actually transition again. I’m about to actually move from the industry analyst to space to the product marketing space, the space you know quite well. David,

David Sweenor 2:57 Oh, I better watch out you’re coming after me.

Hyoun Park 3:01 No, no. But, like, what I started seeing is, although there’s a huge need for analysts right now, I like, there’s more confusion in the tech world than ever, and a real need to define topics and define markets. I found all the stuff being built to be super interesting, and I wanted to get my I wanted to get back in the ring a little bit. I started my career at VC backed startups, and I saw all the stuff that was being funded right now, and I thought, Wow, all this stuff being built with AI, not just super fancy llms, but some really interesting, tactical applications as well. And I thought, wow, I just want to be able to help customers to better use this stuff. And I think a vendor position is slightly better positioned than the analyst position for the tactical and ongoing work that I want to do with customers right now.

David Sweenor 3:58 Well, I’m looking forward to see what you’re doing next. You know, you’re one of the sharpest minds in the industry, John, so I appreciate everything you’ve done, and you know all the excellent posts you put out in podcasts. But let’s talk about AI agents. So really, what I’m interested in understanding is, what the heck are people actually doing with these agents? I’ve seen demos of them, scheduling meetings, ordering groceries. That seems a bit ho hum to me. I think I can do that just quite fine on my own. So what are people doing with AI agents?

Hyoun Park 4:28 Yeah, I think the most when I see what is happening with agentic AI, the what I find most interesting is how we are starting to automate things that we think of as workflows, but don’t necessarily have a fundamentally deterministic workflow that can be defined and structured 100% consistently, time after time after time. So sometimes, for instance, you might have a. A research perspective. This is one that actually that box. CEOs Aaron Levie pointed out that you can go to an agent and you can ask it to deeply research a topic, and it will take a couple hours to figure out something like vendor landscape or figuring out the details of how an agentic protocol interface might work at the back end, where you have to do all sorts of data integration and language parsing and all these other things that you don’t have time to necessarily figure out for yourself, or it would you could do it yourself, but it would also take you several hours to dig up all of the websites and all the documents and to summarize this, and the agent will actually do this type of work for you. Or there have been examples of financial institutions that are trying to conduct trades that are more, I hate to say, but kind of vibe based where trying to pull base their actions on sentiment based analysis, but it’s hard to actually define that sentiment in a word or a sentence that would fit into traditional predictive analytics models. They need to look at broader trends and more, perhaps multi, multiple conditions that don’t fit very well into a straightforward logical structure, because it’s so complicated to describe all of the ifs ands and buts associated with geopolitical situations that they’re trying to take into account for A single trade. So instead, it can use something like an agent to figure out the complexities of what is happening and then fit that all into a specific action or set of actions. Of course, because part of the value of agents is that and have agents work with other agents as well to conduct multiple sets of automated actions at once. So the interesting thing about agentic AI is fundamentally about being able to either automate or have some sort of intelligent action that is taking place that you can oversee as well. But it’s going beyond simply as a basic search or a basic meeting invite agent, right,

David Sweenor 7:30 right? And we’re going to get into maybe the the this a little bit more, but I do have a question for you. So you just mentioned a very simple use case of, like, you know, using an agent for, like, deep research, and I use it all the time, and it will go scour 500 websites and come up with something pretty interesting. What do you think this does to the human you know, I’ve written a few books, and part of the research process you don’t know something until you start trying to write a book about it. Like you’re, oh, you’re like, I really don’t know that. And so I wonder, like, what impact this is going to have on, like, the human brain. It’s like, I feel like we’re going to stop learning and become lazy. I’m curious what your take is on that.

Hyoun Park 8:10 I feel like we’ve had this problem for decades. If that’s the challenge that my entire life, I’m almost 50 years old, computers have been smarter than people for almost that entire time, maybe not my Commodore 64 but definitely by the time we started getting up to like, even just like Windows 98 like my computer was able to do all sorts of things much smarter and faster than I was able to do I’ve had a pocket calculator that Could do six digit times six digit. You know multiplication. You know since I don’t know High School. So that’s always been a problem. We’re never going to be better than computers at math. And honestly, we’ve, over the past 1015, years, have probably used our computers for spell check and grammar checks that for things that we’ve missed in our documents and writing, there are all sorts of things that computers are better than people at, but I feel that computers don’t understand judgment very well at the end of the way, and they can’t independently take on tasks very well because they lack curiosity in terms of being able to explore the world, it’s just not how they’re modeled to work, right? But we do have this problem that computers, yes, they’re a lot better than people at a lot of things, and if we if you don’t learn the fundamental aspects of how something works, you can’t then manage the computer to do a better job. And we’re starting to run into this through this problem where computers can do a better job at peop than people at, say, first year tasks, entry level jobs, right? We we have to kind of fit this into this new career path where you have to accept that if you’re you have to hire people, and you have. To accept that they are going to be worse than computers for the first year of their job, because you want the third year version of them, or the fourth year version of them, who will be able to manage AI and manage computer based outputs, and they will do a better job, because at the end of the day, the human brain organizing and orchestrating a lot of efforts can be far, far more productive than AI, but you have to train the person on the subject matter expertise and how a job works, and who to work with, and all these other aspects of the job that are not simply handling the unit task at the most foundational level, because a computer will always be the person at a unit task that can be automated. Sure,

David Sweenor 10:48 yeah, I do. I do wonder what the impact on sort of, you know, early, early career people, but, you know, we’ll have to wait and see. I don’t think we can ever know. But you know, you mentioned earlier this, this notion of, you know, multi agent system. So that brings up something, you know, that, you know, I was thinking about was this, you know, got these things strung together, you know, three or four agents or 10 or it doesn’t matter how many they are, but, you know, if one screws up, I wonder what the impact on the other ones screwing up? Is it like a chain reaction here? So, you know what, like, what’s this notion about agentic quality? And you know, how do you define it? And, you know, how are we gonna, how are we gonna measure this?

Hyoun Park 11:28 Yeah, one of the challenges we have right now about agents and AI is that we tend to think about benchmarking these things on a purely fact based basis. Like, if I give you a question, do you provide the right answer? Because question answer, query, response, binary ones and zeros, that’s the way we think about computing, and that’s the what, how we’ve translated to AI. And unfortunately, I don’t think that’s the right way to think about AI. AI AI is useful to us fundamentally because it takes on questions that we are not great at quantifying and structuring. If we knew how to quantify and structure the question perfectly, we would write it as a query. We would just build the workflow, and it would work perfectly. And AI is great because it’s actually a little bit more flexible. So we have to, if we’re gonna benchmark agents in and AI in a useful way, we have to start thinking about how it does goal oriented reasoning. How does it figure out answers? And that’s something that, frankly, we haven’t even started to benchmark, which is why I take all of these AI model benchmarks with a grain of salt, because I feel like if you’re tuning to a specific type of answer, you’re probably doing something in the back end to cheat them already, just to cheat the benchmark, rather than testing how Good your model is or your agent is, we’re not great at figuring out reasoning and real time adjustments. So I guess the other side of that is, if your data changed in the last five minutes, is your model or your agent able to adjust and give you better or different answers. As a result these types of timely adjustments that AI is actually built to help out with, with the support of things like retrieval, augmented generation, or other types of augmentation of the model, like model context, protocol, like all these things are allowing models to change on the fly, and we don’t really know how to test or benchmark that stuff, and ironically, that’s the stuff we should be looking at, because that’s where AI is actually helping us. But what we do know for now is AI is actually doing a pretty good job of taking 10s, hundreds, 1000s of variables that we may or may not even be considering and bring them all into some sort of interesting output that reflects what we’ve historically thought of as a decent answer for a question. And that’s very different from simply asking, what’s one plus one? That right different types of thought, yeah, you know, just that

David Sweenor 14:16 term though, that people are use using, you know, reasoning. Do you think we anthropomorphize these a little too much? Like, isn’t it just doing? Is it simulation or scenario testing? Like you mentioned, it has a goal, and it’s just trying to find this goal, and it’s just like, looping through possible scenarios. I mean, like, Should we be using the term reasoning at all?

Hyoun Park 14:40 So it I feel like we don’t have a better word for it, like models do not reason in that they don’t work the way that humans brains work. But what they have done is we have these models. They have billions of parameters. I think the largest ones may even have. Trillions of parameters, you know, we’re just talking about, I’m just adding zeros at this point. I don’t even know what really means, yeah, but in doing so, when you put these, you know, trillions of parameters, and you put in all of these tokens, you start having these higher level structures that start coming into place. So in this case, we put in all these billions of parameters, and these models have figured out their version of grammar based on how words typically follow each other, and they figured out some level of semantics and emotional meaning and context just based on how these words tend to work in order or are near each other, and it doesn’t even see the words you know it. These models translate words and numbers into tokens, which are a separate digital unit that it then puts together from a math perspective, and all the model sees is 4679, goes along with 78901, all the time. So these are things that come together and then it spits it out as some sort of output that is emotionally relevant or semantically relevant or topically relevant. It doesn’t necessarily that, but that’s not reasoning the way that we reason it is building things literally from the bottom up, and whereas we tend to be pattern matching in our reasoning as human beings, and we tend to work top down, which is why we don’t all have 600,000 word vocabularies. And every computer in history actually does right? Because it just knows the entire English language from front to back as storage. We think of it as the other way around. We care more about, do we have food? Do we have housing? Do we have, you know, Caveman brain things that allow us to stay alive? And then we kind of move everything out from, from that,

David Sweenor 17:02 yeah, yeah, that’s interesting. Yeah, people, you know, they’re so good at this pattern matching, and it comes back and, like, you think it, it knows things. I want to use quotes here, but you know, like, I discovered, you know, a while ago, like, it doesn’t know calendar dates. Like, August 13 is a Monday. No, it’s not well this year, it’s a Wednesday. But it just got them all screwed up because I was trying to use it for a social media thing. So it’s not really good at the calculations part. And we, you and I have talked about that before, but,

Hyoun Park 17:32 yes, my favorite joke there is that, you know, a cup, an optimist says that it’s half full, pessimist says that it’s half empty a computer says it’s January 2

David Sweenor 17:44 Exactly, exactly. So, you know, related to this, this notion of reasoning. So we discussed that a little bit, you know, you mentioned you don’t like the term hallucination. I’ve actually never liked that term either. So let’s it’s maybe not a great metaphor. So tell us a little bit about, you know, how we should maybe reframe or rethink about the BS that these llms Spit out, you know, quite, quite frequently. Yeah, so

Hyoun Park 18:09 you mentioned before how you’re not a big fan of anthropomorphizing AI, because it is a tool. And I feel like talking about hallucinations, also artificially anthropomorphizes. AI, because to hallucinate, you have to be, you know, basically high on your own supply and being off, inaccurate, off of your worldview, to kind of formalize it, you know, you are when you are hallucinating, you are literally not seeing the world as you normally look at it. I feel like these what we call hallucinations on the AI side, they do reflect how the AI model, or the or the larger agent looks at the world. This is actually what it sees, and we are trying to pretend that its worldview matches our worldview, even though it sees the world extremely differently. It seems the world as a ton of tokens being poured through billions of parameters and all of the model tools that are happening right now. And I don’t think that’s an accurate way to think about it. We have to, we have to, instead think of these. It’s, it’s a feature, not a bug. With these, these, these issues that we see reflect how that model actually works. And if that is a problem, if we think of that as a hallucination, we have to actually take that next step and figure out, well, what is wrong, what is different about this model, and can it be adjusted to our real life world? It may be that you are using a model or a tool that is fundamentally in out of step with your reality, and you. Just been using it because you you just didn’t know the difference, like, you’re just using the wrong tool at the end of the day, yeah, and, yeah,

David Sweenor 20:09 yeah, no, just like, you know, it’s really interesting. So there’s, there’s, you know, this, people talk about rag and how it may be, maybe lowers the hallucination rate or confabulation rate, or the making up crap rate, whatever rate we want to call it. You know, one thing I found is, like, there was a there’s a report I wanted to analyze. In fact, yesterday had a bunch of strengths, weaknesses in there. I’m like, Hey, by vendor, just summarize them across the vendors. And it was just making things up that weren’t in the report. I’m like, I gave it very explicit prompt, only use stuff that’s in the report, but it just couldn’t do it because I think I probably jammed too much stuff in the context window, and it was getting all confused. But how are we going to even detect these things in the future, given we know that as much as we try, they will be wrong, you know, some percentage of the time, what’s a what’s a company gonna How do you even monitor and test, you know, detect this stuff? Yeah, I

Hyoun Park 21:08 think there are. There are ways to constrict outputs specifically through a semantic layer or appropriate metadata and jargon, but it takes work. It is not simply a plug and play capability at this point to do that level of fact checking, because the AIS that we’re using are not fundamentally designed to simply be factual. At the end of the day, they look at relationships between words, relationships between documents, and they they give outputs based on that, and sometimes they are extremely detailed and interesting, but those details are often wrong and based on specific assumptions that lead to other specific assumptions, that lead to other assumptions that’s not being associated with a grounded, fact based approach. So that is a real challenge. I think that also we’re going to find, over time, different types of AI models are we’re already seeing this with some models being better for coding, some models being better for video, some models being better for enterprise use, and some of these smaller, custom models actually being better more useful for task based capabilities, because there’s less garbage out in the model relative to the use case being considered. All of these things are happening right now in AI, and we might find over time that there are certain models that actually help us to be more creative, because they go off the rails in ways that help us to understand the semantic and semiotic and creative aspects of language and video and audio use, but we should never use those same models for enterprise usage, right, right?

David Sweenor 23:13 Yeah, and big guys, I guess you gotta be careful, depending on the use case, you know you’re just editing, you know they might want, might not want to close your financial books, but maybe it’s okay for a piece of marketing collateral or something like that, you know, right? But

Hyoun Park 23:26 even, but like with closing the financial books, there are so many transactions that go into closing the financial books, and we’ve been dealing with this problem of budgeting, planning, budgeting and forecasting in in the financial world for such a long time, but it’s really hard still to bring in qualitative information and try to figure out how that fits into a budget, right? Like, right? You know, just thinking about something like the recent tariff issues we’ve had in the United States, how does that fit into a budget? Like, there’s so much, there’s so much there to, you know, to assume, but yet, every single financial professional has to do that right now, and most of them are not well equipped to think about this from a qualitative perspective, like, how important is it for the United States to win against China in a geopolitical, you Know, economic war, and that’s something you have to take into account. As you know, this financial, you know, fpna, person who is, you know, running 100 person companies like these, are such different worlds.

David Sweenor 24:33 Really, are they really are, oh my gosh, a lot to think about. So let’s so we can all assume that quality is important. So how does focusing on sort of this agentic quality? How does that change? Maybe the remediation playbook for for organizations, you know, I think they’re going to struggle, monitoring things at scale, monitoring multi agent systems. But I’d love to get your perspective. About, how does this change, really, what companies do and how they think about their IT landscape?

Hyoun Park 25:05 Yeah, I was really excited when agentic, this agentic capability, started coming out, because it provided and relatively easy way to structure, model output and further contextualize it within a business use case or an application. So a very concrete examples are how Salesforce and ServiceNow are building agents on top of their own platform. And there’s obviously business use cases associated with their platform. So you already have this semantic layer in place, this data model in place. Of course, everybody’s going to have their own agentic models and platforms and strategies over the rest of the year, so I don’t even think that’s going to be extremely differentiated over time, but I think that one of the challenges is that it’s almost too easy to build an agent Now, if you’ve tried out, say, the Salesforce, Agent force capability. It’s not that hard to build an agent. If you use Boomi or service now, or you know all these agent creators, Agent platforms, Agent controls, towers, whatever they’re calling them, they all provide extremely easy ways to build agents. So the next step is okay. Now we have 1000 5000 10,000 20,000 agents that are working in your organization, and a lot of them might be duplicates. A lot of them may be asking for very similar things, but in slight with slightly different contexts or slightly different grammar, and all of these things need to be updated over time, because you’re going to bring in new data, you’re going to have new processes. Everything changes over time. Your sales process or your marketing process or your manufacturing process will change over the next six to 12 months or 24 months, and the prompts and the agents have to change along with it. So whose job is that right now? Like, yeah, yeah, right, yeah, we have

David Sweenor 27:09 an update of the model. You know, everything breaks, I feel like, and do you feel like, like, there’s like, a parallel to, like, the the BI craze. Everybody had to have their own dashboard specific to the business unit they were in, because they were special, and every organization and business unit is special, and so we had this proliferation of this, and we got the sprawl we could never get a handle on. Do we see the same you see the same thing happening with agents? You’re going to have yours that’s personalized to you. I’m going to have mine, and maybe we’ll never get to talk again. John, I don’t know, like maybe only our agents will interact with each other. It’s a bit sad to think of like like, you think you’re going to see agent sprawl and, you know, I wonder how companies are going to wrestle this. Maybe it’ll be come down to the finances of it. Maybe they’ll get expensive.

Hyoun Park 27:51 Yeah, so we are going to talk together again. I’m sure we’ll talk after this, but I do think that there will be some aspects of B to B transactional activity that will go too far in this direction. And this might be something like, oh, you know, contract negotiation where the salesperson and the buyer never actually talk to each other for something extremely complex, and then they find afterwards, oh, wow, our agents have been making all these assumptions that we you know, superficially, we got our 10% discount and they got their three year deal. So everybody won. But wow, when we looked at the details, the compromises made by the these agents are things that we should not have done as professionals like I can totally see something like that happening. Or you outsource all of your meetings to agents, which, okay, honestly, like 80% of them are crap anyways, and probably should be, but Right, there’s still, you know, you still, you’re still going to have to make a judgment call as a human being on some of these,

David Sweenor 28:52 yeah, yeah, I find myself doing a lot more fact checking of things and actually thinking, you know, it’s My role is certainly changing as well. So there’s a lot of new technology out there. You know, there’s, there’s vector databases. You know, last year they were all the rave. I think maybe they’re, they’re waiting now a little bit, we’ve got synthetic data. These help improve quality. Or what role do they play in the sort of this new AI media, I’ll call it, yeah,

Hyoun Park 29:19 I think synthetic data is actually extremely important, because it can be really hard to have enough data of a certain type of format and a certain type of consistency to be able to test models, especially if you’re doing something like, oh, a healthcare record where, of course, you’re not going to use real people’s healthcare records to test, test any sort of medical thing, because you’re not allowed to. We have all sorts of personally identifiable information rules across the world that do not allow you to touch real medical records. So you’re going to need to create your call it million medical records and then. And test them in a variety of ways to make sure everything works. And if you can have synthetic data that works and has consistent, say, uh, billing information and symptoms and all of these other things, and then being able to translate back things back and forth, yes, synthetic data can be extremely important and useful from a model building perspective, and so that part I’m really interested in. I think there’s still a lot of progress that we can make from a synthetic data perspective, making it consistent across all of the different types of complex use cases that we have, but we’re going to need to keep pushing the envelope. In that regard, I would say the vector thing is more of a kind of table stakes at this point. From a technical perspective, it feels like every large database has their own vector storage. We’ve all established, like the vector capabilities which are necessary. I want to think that the dedicated vector database is important. Like, I love milvis as a company. I love their focus on how they have looked at AI, and they’re more performant than the average vector store. But I’m just not sure how much it matters at the enterprise level, from right, from a practical perspective, yeah, yeah. They’re kind

David Sweenor 31:21 of everybody. Everybody’s got one now. So I tend to agree with you there. So I know we’re, we’re getting towards the end of our time. So I’m gonna ask you a futures question. And I’m not, I was gonna say five years, but I don’t even think we just know what’s gonna happen next year. So let’s, let’s ask, you know what? What is one capability you think agentic AI needs that maybe just you feel like it’s a little out of reach, you know, at this, this moment in time, like what’s sort of this critical, you know, linchpin that we might be missing?

Hyoun Park 31:49 Yeah, I think the biggest one is fundamentally still around establishing trust. How do I know something is real or fake, and how do I know that I can trust what you’re saying or doing. We’re starting to see little bits of annotation that help, especially if you’re asking for something that is research based for an agent. But we’re also going to have to establish that that trust across taking action. So if I asked my agent to buy my groceries for me. Is it going to do it right? Go to the right store, handle that relationship with Instacart or whatever, or, you know, Uber Eats or what have you, you know, all these things that go into building trust. And you know, that’s just consumer level issue, but there’s enterprise level issues as well that are similar in nature. I think that fundamentally building that trust and consistency across variable circumstances, which is something that we do with people, but we have trouble doing with agents, because we don’t know how to judge that yet. We don’t know how to place that trust for non deterministic events and workflows,

David Sweenor 33:09 yeah, and how to build in your thought process or morals, because each culture has a different set of things that are important to them. You know, we can even talk about the trolley problem, but that’s maybe my one, one last question, Jan, and this wasn’t in our list. So you’re a musician, I don’t know, I don’t know if you’ve, you’ve First, I’d like to know like what you play and like what you like, but what, how do you think about the AI music and all the creative side of AI we’ve been sort of talking about corporate. Have you experiment one of those? And what’s your, what’s your initial take on the music or the art ones or Video, Video ones out there?

Hyoun Park 33:46 Yeah. So interestingly, being a musician is a lot like being a coder. In some respects, I’ve done a little bit of both, in that, like, there’s this tactical feel to writing the music yourself or playing the music yourself, which is inherently pleasing. And I think that human aspect is going to continue to be useful, like like, for instance. I think of chess. Every chess computer at this point can beat every Grand Master on the planet. But we still play chess. We still love watching people play chess. We still watch people like Magnus Carlsen, the best Grandmaster in the world, do creative things that we don’t necessarily see computers do, even if computers can annihilate Magnus Carlsen on a one on one. So that part is still interesting, and I think there’s still this creative and community aspect to music that will be useful, but at the same time, I’ve got to admit, if I need 10 seconds of music for ad copy, I’m not really sure what the point is of getting a human to write that and getting humans to perform something like that. But these. A little interstitial bits, yeah, I mean that that’s, that’s definitely going to be computer work, but writing symphonies and hopefully interesting songs and music, I think, is still a human endeavor. I can definitely see pop songs, you know, moving into the, moving into the digital AI, right, right, right. Yeah, people, okay, I’m a tell us. I’m a singer. I love doing what I do, but I know that AI is going to take away some of the easier jobs that I used to get in in that world. Yeah.

David Sweenor 35:36 Well, you know, it can be helpful, like playing around with some of these tools, and you can play along with it, you know? I can have it back. Can have a backing band now that I didn’t have before. So I do enjoy that, that piece of it. But yeah, and this has been a great discussion, I really appreciate you coming on the show. You know, any last words of wisdom for people who are trying to understand this? You know what is sort of your recommendation to them trying to understand agentic AI and AI agents and everything that’s out there, I

Hyoun Park 36:04 think AI is an extremely polarizing topic right now. You’ve got people saying that AI is just copying a parroting what it sees, which I think is too simplistic, given the breadth of model and agentic capabilities that are coming out, and these things are evolving on a month to month basis. On the other hand, you’ve got people saying, well, in five years, we’re going to get rid of 20% of jobs. Well, there’s still a human element and a business element to this. I know people are going to get laid off, but I also think that a bunch of people are going to get laid off, and then businesses are going to realize, oh, wait, we got rid of some really great orchestrators and business experts, and now we’re gonna have to hire them back that that’s just a part and parcel of every interesting technology that has ever come into the business world. But I do think there’s a middle point where we have to learn how to manage AI that that is going to be a part of our world going going forward. I would be shocked if anybody in a white collar position did not spend, call it, a quarter of their time managing AI over the next five years like that. That’s just going to be part of what we do from from now on. Yeah,

David Sweenor 37:17 absolutely just, just another tool. For sure. You know, it’s like, I’m not using a typewriter anymore. I got a word processor or whatever, so I don’t have to type anymore. I can just talk to it so, but anyways, yeah, and this is, this is great. I appreciate you coming on and thank thanks for joining the databases podcast.

Hyoun Park 37:34 Oh, my pleasure. As always.

Hyoun Park 37:37 See you later. Take care. You.