I am excited to share that Debezium 2.0.0.Alpha1 has been released!

This release is the first of several planned pre-releases of Debezium 2.0 over the next five months. Each pre-release plans to focus on strategic changes in the hope that as we move forward, changes can be easily tested and regressions addressed quickly.

In this release, some of the most notable changes include requiring Java 11 to use Debezium or any of its components, the removal of wal2json support for PostgreSQL and the legacy MySQL connector implementation, as well as some notable features such as improved Debezium Server Google Pub/Sub sink support, and a multitude of bugfixes. Let’s take a look at a few of these.

Java 11 required

We have wanted to make the jump to using Java 11 as a build requirement for quite some time now, and with Debezium 2.0 this is now possible. With Java 11, this enables us to take advantage of new language features, such as the new String API and Predicate support changes in the codebase, while also benefiting from many Java performance improvements.

Our very own Vojtech Juranek will be publishing a blog post next week that discusses the switch to Java 11 and 17 in greater detail. I highly recommend giving it a read as it provides a deep dive into the technical background & effort that went into making this possible.

So before migrating to Debezium 2.0, be sure that Java 11 is available.

PostgreSQL wal2json support removed

The PostgreSQL connector has supported several plugins throughout Debezium 1.x, including decoderbufs, wal2json, and pgoutput. PostgreSQL 9.6 recently reached end of life on November 11, 2021. This presented a great opportunity for us to review the supported decoders and to see whether we could streamline those options.

Since pgoutput is a native decoder supported by all non-EOL versions of PostgreSQL (PG10+), it made sense to remove wal2json. Reducing the number of decoders to 2 (down from 3), allows us to streamline the code for PostgreSQL, reduces the overall maintenance cost of the connector, and gives us a much more narrow target for overall support.

If you are still using PostgreSQL 9.6 or were using wal2json previously, you will need to migrate to at least PostgreSQL 10.0 or to decoderbufs or pgougput respectively before upgrading to Debezium 2.0.

Legacy MySQL implementation removed

As some of you may or may not know, we implemented the MySQL connector based on the common-connector framework back in Debezium 1.5 (Feb 2021). As a part of that re-write, we introduced the ability for MySQL users to enable the legacy connector behavior using the configuration option internal.implementation set as legacy. This legacy implementation was deprecated in favor of the new common-connector framework behavior. With Debezium 2.0, this internal.implementation configuration option and the legacy connector implementation have been removed.

If your current connector deployment relies on this legacy implementation, you should be aware that by upgrading to Debezium 2.0, the connector will no longer use that older implementation and will use the common-connector implementation only. Feature-wise, both implementations are on-par with one another with one exception: the legacy implementation had experimental support for changing filter configurations. If you have relied on this legacy behavior, be aware that feature is no longer available.

Other fixes & improvements

There are several bugfixes and stability changes in this release, some noteworthy are:

  • Implement Pub/Sub Lite change consumer DBZ-4450

  • Add Google Pub/Sub emulator support DBZ-4491

  • Making Postgres PSQLException: This connection has been closed. retriable DBZ-4948

  • Should store event header timestamp in HistoryRecord DBZ-4998

  • Getting java.sql.SQLException: ORA-01291: missing logfile while running with archive log only DBZ-4879

  • Debezium uses wrong LCR format for Oracle 12.1 DBZ-4932

  • NPE caused by io.debezium.connector.oracle.antlr.listener.ColumnDefinitionParserListener.resolveColumnDataType DBZ-4976

  • Outbox Transform does not allow expanded payload with additional fields in the envelope DBZ-4989

  • CLOB with single quotes causes parser exception DBZ-4994

  • Cassandra 3 handler does not process partition deletions correctly DBZ-5022

  • SQL Server in multi-partition mode fails if a new database is added to an existing configuration DBZ-5033

  • Upgrade to Quarkus 2.8.2.Final DBZ-5062

Altogether, 55 issues were fixed for this release.

What’s Next?

We have resolved the runtime problem with Debezium Server in the 1.9.1.Final release, so you can expect a 1.9.2.Final later this week which will also address other bugfixes. You can continue to expect updates to 1.9 in the weeks that follow as bugs are reported and fixes are made to address those.

As we continue our efforts on Debezium 2.0, you can expect a second pre-release in the coming weeks, sticking to our regular 3-week cadence. In this next pre-release, we plan to focus on message schema versioning/naming, connector configuration changes with new pass-thru namespaces, removal of deprecated options, as well as unifying default value handling, just to name a few on the roadmap.

And speaking of Debezium’s roadmap, stay tuned as we’ll have more to share about Debezium 2.0, its future releases of 2.x, all on our roadmap soon!

Chris Cranford

Chris is a software engineer at Red Hat. He previously was a member of the Hibernate ORM team and now works on Debezium. He lives in North Carolina just a few hours from Red Hat towers.

   


About Debezium

Debezium is an open source distributed platform that turns your existing databases into event streams, so applications can see and respond almost instantly to each committed row-level change in the databases. Debezium is built on top of Kafka and provides Kafka Connect compatible connectors that monitor specific database management systems. Debezium records the history of data changes in Kafka logs, so your application can be stopped and restarted at any time and can easily consume all of the events it missed while it was not running, ensuring that all events are processed correctly and completely. Debezium is open source under the Apache License, Version 2.0.

Get involved

We hope you find Debezium interesting and useful, and want to give it a try. Follow us on Twitter @debezium, chat with us on Zulip, or join our mailing list to talk with the community. All of the code is open source on GitHub, so build the code locally and help us improve ours existing connectors and add even more connectors. If you find problems or have ideas how we can improve Debezium, please let us know or log an issue.