Hello everyone, my name is René Kerner and I recently joined Red Hat and the Debezium team.
I was working at trivago since 2011, and in 2016 we started using Debezium at version 0.4/0.5 for capturing clickstreams in the offshore datacenters into Kafka and aggregate them in the central cluster. We really intensified Debezium usage within one year and in 2017 we also used it for trivago’s main data.
In 2014 I did my first OSS contributions to Composer, PHP’s dependency management and gave my first talk on it at the Developer Conference (called code.talks for many years now). Then in 2017 I did my first contributions to Debezium with work on the MySQL snapshot process and fixing a MySQL TIME data type issue.
In 2018 I left trivago and started working at Codecentric as a consultant for software architecture and development (mainly JVM focus) and Apache Kafka, doing many trainings and workshops at German "Fortune 500" companies (insurances, industrial sector, media). I was doing lots of networking at that time, where I learned how awesome the community around Kafka is. I was always quite sad I didn’t have more time to focus on OSS projects.
Let me share a bit more of trivago’s story to Kafka and Debezium. Back in 2015/2016 we introduced Kafka (version 0.9, 0.10 times) at trivago to handle the transport of our clickstream data from offshore datacenters (US + Asia) into our central datacenter (EU).
The first solution that we tried did its job, but getting messages in our desired format, which was Google Protocol Buffers / Protobuf, into Kafka was relatively hard. Furthermore the codebase of that tool wasn’t very clean and extensions were ugly and kind of hard.
With Kafka 0.9.0.0 Kafka Connect was introduced and stabilized over the next months. And in winter 2016/2017 we discovered Debezium. A tool that was based on Kafka Connect, with a much cleaner codebase and an easy, extendible way to apply our requirements regarding Protobuf format and behaviour (with SMTs/Single Message Transforms, released in Kafka 0.10.2)
Since these "old times" Debezium made very big development with many new connectors and since July I’m now part of the Debezium team, and I am proud and excited to work on such a great OSS project because there are still awesome things on Debezium’s Roadmap: the new IBM Db2 connector will leave incubation state, Debezium Server will be easier to operate on Kubernetes, and a PoC for a future Debezium management UI, and more and more to come.
Furthermore, I’ve been dedicating my work to many things around OSS, Kafka and Debezium for years, I always advocated people to use OSS, Kafka and Debezium, and supported them in different jobs and roles to introduce it or extend its usage. That’s why I’m really excited that I am able to focus my work to support Debezium and the Debezium community now!
--René
About Debezium
Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.
Get involved
We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.
