Confluent’s mission for enterprise-level, real-time data streaming

Confluent CEO Jay Kreps discusses the changing landscape of data management, and how real-time data streaming facilitates a more holistic approach to data within the modern enterprise.

By Jay Kreps
Published December 18, 2024 | 3 min read

Key Points

  • Confluent is at the forefront of transforming data management, enabling companies to utilize and analyze data in real time
  • The platform supports hybrid and multi-cloud environments, allowing businesses to seamlessly integrate data from various sources
  • Confluent’s strategic partnership with cloud service providers (CSPs) and data system integrators strengthens its position in the market, helping drive innovation
  • AI integration is at the forefront of Confluent’s enterprise product evolution, to further enhance data processing, analysis and decision-making

View audio transcript


As a data management solutions provider, Confluent is transforming how organizations handle information, enabling real-time data streaming to support operational efficiency.

CEO Jay Kreps drives Confluent’s strategic vision by building on the foundation of Apache Kafka, the open-source technology he co-created while at LinkedIn. Originally developed as an internal infrastructure layer to manage data at scale, Kafka was released as an open-source project and quickly gained traction among leading tech companies. “As we started to talk to some of these users, we realized it was actually kind of a big thing—not just in tech, but across industries,” Kreps explains.

Confluent took on the challenge of transforming Kafka into an enterprise-ready platform— adding the essential features needed for success in modern organizations. “The idea behind Confluent was to make real-time data easy to consume, as seamless and accessible as any other enterprise data technology,” he says. This vision has positioned Confluent as a leader in data streaming, powering real-time applications across diverse industries.

Data lags and silos motivate real-time ambitions

Confluent tackles a critical challenge for businesses: the inefficiencies of slow, batch-based data processing. “Confluent’s goal is to shift the world to working with data in real time, dynamically,” says Kreps.  

As data center spending and cloud adoption trends reveal that many companies continue to host data in multiple locations, Confluent is uniquely positioned to excel in these complex hybrid environments. “Data has a gravitational pull that often ties organizations to their data centers or specific environments. What we’re trying to do is help companies fight that gravity, enabling data to flow freely into the environments where it can create value,” Kreps remarks.

“With different layers of databases, applications and platforms, companies are bound to experience fragmentation. These businesses want to be intelligent, relevant and efficient – and ultimately serve customers better. Confluent can connect all these little islands and allow data to flow between them, transparently, quickly and in sync with the business,” explains Kreps.  

The impact of sector-wide transition to real-time data processing has several impacts for Confluent: spurring growth, making day-to-day business easier for customers, and expanding further opportunities.

“Confluent’s goal is to shift the world to working with data in real time, dynamically.”

Jay Kreps, CEO, Confluent

The third wave of growth

On a recent earnings call, Confluent highlighted its entry into the third wave of growth: the Data Streaming Platform (DSP) — a unified DSP that enables organizations to build next-generation, real-time applications.

The DSP represents a significant shift in how businesses approach data infrastructure. By reducing the complexity and investment required to unlock value, Confluent expands the range of use cases organizations can address, driving broader adoption and increasing the platform’s relevance. “The aspiration is to create something as essential as the data warehouse or operational databases—a central nervous system for real-time data flows across the company,” explains CEO Jay Kreps.

This vision positions Confluent as a foundational technology provider, enabling the real-time intelligence and application logic that define modern organizations. While building this platform is a long-term endeavor, it lays the groundwork for transformative opportunities in growth and margins, solidifying Confluent’s role in shaping the future of data infrastructure. Over 40% of F500 companies have already adopted Confluent because of its data streaming capabilities.

Partnerships also help Confluent enhance its value proposition and reach. With CSPs, there’s a reciprocal relationship through go-to-market integration. “We drive consumption for CSPs because, by connecting up the systems our mutual customers use, we send a lot of data their way.”

“It’s important that partners understand our technology and how to work with it. If we talk only to customers and not to the system integrators they’re working with, we’re not really doing our job,” concludes Kreps.

Shaping the intersection between real-time data and AI

Although Gen AI differs from traditional machine learning, the underlying principle remains relevant: “How do we have the right data context to make the output relevant?” Kreps asks. Despite the differences between machine learning and Gen AI, Kreps notes, “The role for Confluent has a fair amount of continuity.”

Confluent’s Retrieval Augmented Generation (RAG) is where real-time data and AI intersect. RAG involves gathering relevant context data and feeding it into AI models to generate improved outputs. “We’re always asking: How can we leverage AI to do it?”

Tools like Apache Kafka and Apache Flink, integral to Confluent’s data streaming platform, empower organizations to build real-time, contextualized, and trustworthy knowledge bases. By streamlining data flow and improving the model’s ability to understand and utilize context, businesses can significantly enhance their operations and decision-making – to the benefit of customers.

“In terms of the difference between machine learning and Gen AI, the role for Confluent has a fair amount of continuity.”

Jay Kreps, CEO, Confluent

Confluent plays a vital role in simplifying the integration of real-time data into AI applications. Whether through vector databases, direct model invocation, or RAG support, the platform enables organizations to harness AI effectively to support agility and innovation.

Experts

Jay Kreps
Jay Kreps
CEO, Confluent

Stay Informed

Get the latest insights and news from RBC Capital Markets delivered to your inbox.