Kjell Carlsson on why most Gen AI projects fall short, the evolution of AI engineering, practical governance strategies, agentic AI myths, and more.

Watch now on YouTube.

Introduction

I need not say this, but artificial intelligence (AI) is no longer the realm of science fiction. Artificial general intelligence (AGI)? Maybe. But today, it’s simply a business reality that leaders must navigate. In Episode 3 of the Data Faces podcast, Kjell Carlsson, Head of AI Strategy at Domino Data Lab, shares his perspective on the challenges and opportunities AI creates for organizations striving to extract value from the technology.

Kjell brings a wealth of experience to the conversation. At Domino Data Lab, he works closely with leading enterprises, helping them advance their AI capabilities and create practical solutions that deliver business value. Prior to Domino, Kjell was a principal analyst at Forrester Research, where he advised executives on machine learning, data science, and emerging AI trends. His career has centered on the intersection of strategy, governance, and AI innovation.

In this episode, we discuss:

• Why 90% of Gen AI projects fail and what leaders can do differently
• How governance can support innovation instead of stifling it
• The rise of AI engineering and its implications for enterprise strategies
• The realities of agentic AI: what’s real and what’s not
• Actionable advice for business leaders preparing for 2025 and beyond

For organizations embarking on or scaling their AI initiatives, this conversation highlights practical steps to prepare for the demands of tomorrow.

Predictions for AI in 2025: Reality Check for Business Leaders

AI continues to remain front and center in the headlines, but the reality behind the hype can be sobering. One of Kjell Carlsson’s 2025 predictions is the same as 2024:

As many as 90% of generative AI (Gen AI) projects will fail to deliver transformative outcomes.

The reasons for this arise from organizational barriers and not technology impediments.

Carlsson points to a consistent gap that most companies simply lack the internal maturity to effectively implement AI. This includes classic challenges in aligning people, processes, and technology to identify, design, and deploy impactful AI solutions. Many leaders believe that large language models (LLMs) are plug-and-play, underestimating the work required to customize and integrate their capability into enterprise architectures.

Rather than focusing solely on prebuilt tools, organizations must prioritize building their own data science and AI engineering capabilities. Businesses already advanced in these areas, such as biopharma companies using generative AI for drug discovery or ride-sharing firms optimizing customer service, have shown what’s possible when mature infrastructure and skilled teams set their sights on deploying AI.

For Business Leaders:

• Avoid viewing AI as a one-size-fits-all solution. Focus on use cases tailored to your organization’s needs.
• Invest in teams, platforms, and processes to strengthen your AI readiness before deploying new tools.

AI Governance: Balancing Control and Innovation

Governance in AI is a balancing act. When overly rigid, the teeter stifles innovation; when absent, it totters towards risk and inefficiency. Kjell emphasized that effective AI governance isn’t about adding layers of bureaucracy. Instead, it’s about enabling trust and scalability through governance frameworks that adapt to specific business contexts.

Kjell outlined two common governance challenges organizations face. In industries new to regulation, like retail or manufacturing, a lack of governance often creates skepticism about AI’s reliability and scalability. Conversely, in highly regulated industries like finance or healthcare, overly manual and labor-intensive governance processes become a bottleneck to innovation.

Good governance, Kjell argued, is not a question of “more or less” but of quality. Effective governance ensures risks are understood and minimized without slowing progress. Leaders must design governance processes that are adaptable and aligned with the specific risks of AI initiatives.

For Business Leaders:

• Shift from top-down governance frameworks to a bottoms-up approach focused on actionable practices like risk identification, validation, and continuous monitoring.
• Prioritize governance as a way to unlock innovation rather than as a compliance exercise.

AI Engineering: From Trend to Discipline

AI engineering is emerging as an important skill set for businesses aiming to operationalize AI. Kjell highlighted that while AI engineering builds on foundations like MLOps, it represents an evolution driven by the unique demands imposed by generative AI.

Organizations often underestimate the complexity of integrating generative AI models into production systems. Kjell emphasized that deploying LLMs requires significant effort. AI/ML teams must write extensive “glue code” to manage inputs, outputs, guardrails, and integration with business systems. Success often depends on using multiple models in tandem and building pipelines that include both traditional predictive models and generative AI.

The result? AI engineering has become a distinct discipline that combines software development, data science, and system integration. This shift feels like an evolution for organizations with high data science and AI maturity. For those without such foundations, it’s a steep hill to climb.

For Business Leaders:

• Invest in AI engineering capabilities, including hiring or upskilling talent to handle the complexities of generative AI integration.
• Recognize that generative AI is not a standalone solution—it requires robust systems, processes, and ongoing refinement to deliver results.

Agentic AI: Hype vs. Real Opportunity

Agentic AI, portrayed as the panacea of productivity, promises to create fully autonomous systems capable of independent decision-making. Kjell cautioned that while the idea is compelling, the reality is far more complicated.

The early vision of agentic AI, exemplified by tools like Auto-GPT, promised systems that could interpret vague instructions, iteratively test solutions, and execute actions with minimal human intervention. However, these concepts remain impractical for most organizations due to their unpredictability and lack of control. Few businesses would hand over operations to autonomous agents without clear guardrails and accountability mechanisms.

Instead, Kjell emphasized focusing on narrower, high-value use cases. For example, agentic AI is already proving effective in automating repetitive tasks like document processing and customer interactions. By defining clear objectives and boundaries, organizations can take baby steps and safely use agentic AI in a more controlled manner, reducing overall risk exposure.

For Business Leaders:

• Avoid overinvestment in speculative, broad-scope AI agents. Instead, concentrate on narrow applications that address specific business needs.
• Implement guardrails and integration points to ensure AI aligns with operational goals and existing processes.

Conclusion: Advice for Business Leaders

Kjell’s insights highlight a consistent theme: AI success is less about the technology itself and more about the organizational capabilities and maturity that surround it. For leaders, the path forward is clear—prioritize readiness, strategic focus, and disciplined execution.

Rather than chasing the latest hype cycle technologies, leaders must align AI initiatives with their unique business needs. This requires empowering experienced teams, building foundational capabilities like AI engineering, and adopting governance processes that enable rather than inhibit progress. As Kjell noted, the organizations achieving the most with AI today are those with a solid track record of leveraging data science and machine learning.

Leadership also plays an important role. Business leaders don’t need to be AI experts themselves, but they do need to foster expertise within their organizations. By supporting experienced teams and holding them accountable for delivering measurable value, leaders can create a virtuous cycle of innovation and trust.

The lesson is simple, as I mentioned to Kjell: move with purpose. AI has an important role to play, but it requires a measured, thoughtful approach. Leaders who invest in maturity and focus on outcomes, rather than quick fixes, will position their organizations to achieve a competitive advantage.

Listen to the full episode on YouTube.

Summarized Transcript

0:00 – Introduction
David introduces the episode and Kjell Carlsson, Head of AI Strategy at Domino Data Lab. The discussion focuses on AI predictions for 2025, governance challenges, and the realities of agentic AI.

1:16 – Predictions for 2025: Why Most Gen AI Projects Fail
Kjell predicts that 90% of Gen AI projects will fail to deliver transformative value by 2025, not due to technical limitations, but because organizations lack the maturity in processes, infrastructure, and governance. Successful examples, like biopharma and advanced tech companies, show the importance of foundational capabilities.

4:10 – Why AI Adoption is Not Plug-and-Play
Kjell emphasizes that deploying generative AI is more complex than it seems. Organizations need tailored solutions and integration work, rather than relying on off-the-shelf tools to solve all problems.

6:36 – The Impact of AI on Jobs
Kjell discusses the nuanced effects of AI on employment. While AI might not cause significant job loss in developed countries, it could disrupt industries in developing economies where outsourcing and manual processes dominate.

11:56 – AI Engineering: A New Discipline
AI engineering is becoming a critical function, bridging experimentation and deployment. Kjell explains how deploying LLMs often requires extensive custom coding, integration, and ongoing adaptation, making AI engineering a distinct yet evolving discipline.

17:04 – Governance: Balancing Innovation and Risk
Governance doesn’t have to stifle innovation. Kjell outlines how organizations can create governance frameworks that enable scalability and trust without unnecessary bureaucracy.

26:59 – Agentic AI: Separating Myth from Reality
Kjell addresses the hype around autonomous AI agents. While fully agentic AI remains speculative, narrow applications like automating document processes and customer service are showing tangible results.

39:37 – Leadership Advice for AI Success
Kjell advises leaders to focus on empowering experienced teams, aligning AI initiatives with business outcomes, and investing in foundational capabilities like governance and engineering. Leadership, he contends, is key to turning AI potential into business value.

Listen Now on YouTube