Confluent, similar to Databricks, is a company build on another important open source software for data management – Apache Kafka. If you have streaming data in your company, you will not pass having a look on Kafka. Confluent delivers Kafka from the cloud as a service with an optimized ecosystem.
For data driven companies the speed of collecting and processing data in near-real-time is getting more and more important. If you search the SAP Community you will find, that Kafka is a regular topic here, too.
If you ask yourself now what exactly the difference between Confluent and Kafka is, Confluent itself gives an answer here.
So, Kafka is an important building block for an Event-driven Architecture today. I have seen customers using Kafka a Enterprise Service Bus or as backbone of their data pipelines in microservice-oriented architectures. Originally the technology was build and is still being used by LinkedIn for high throughput of streaming data.
Gartner defines Event-driven Architecture as follows:
Event-driven architecture (EDA) is a design paradigm in which a software component executes in response to receiving one or more event notifications. EDA is more loosely coupled than the client/server paradigm because the component that sends the notification doesn’t know the identity of the receiving components at the time of compiling.
The interesting part will be what features will come to SAP Datasphere to support Kafka in a better way or otherwise. Currently I see nothing on the roadmap. Interesting would also be which functionality from SAP Data Intelligence Cloud goes into SAP Datasphere, as SAP DI already have a good Kafka connectivity.
This is just my opinion and current perspective. I’m happy to hear from you how you see these new partnerships in the context of SAP Datasphere?