Debezium Blog

I am happy to announce the release of Debezium 1.2.1.Final!

This release includes several bug fixes to different Debezium connectors, and we highly recommend the upgrade from 1.2.0.Final and earlier versions:

  • The Debezium Postgres connector may have missed events from concurrent transactions when transitioning from snapshotting to streaming events from the WAL (DBZ-2288); this is fixed now when using the exported snapshotting mode; this mode should preferably be used, and for Debezium 1.3 we’re planning for this to be the basis for all the existing snapshotting modes

  • The Postgres JDBC driver got upgraded to 42.2.14 (DBZ-2317), which fixes a CVE in the driver related to processing XML column values sourced from untrusted XML input

  • The Debezium MySQL connector MariaDB’s supports ALTER TABLE statements with IF EXISTS (DBZ-2219); it also handles single dimension DECIMAL columns in CAST expressions (DBZ-2305)

  • The MySQL connector automatically filters out specific DML binlog entries from internal tables when using it with Amazon RDS (DBZ-2275)

  • The Debezium MongoDB connector got more resilient against connection losses (DBZ-2141)

Read More    

It’s my pleasure to announce the release of Debezium 1.2.0.CR1!

This release includes several notable features, enhancements, and fixes:

  • PostgreSQL can restrict the set of tables with a publication while using pgoutput (DBZ-1813).

  • Metrics MBean registration is skipped if a platform MBean server does not exist (DBZ-2089).

  • SQL Server reconnection improved during shutdown and connection resets (DBZ-2106).

  • EventRouter SMT can now pass non-String based keys (DBZ-2152).

  • PostgreSQL include.unknown.datatypes can now return strings rather than hashes (DBZ-1266).

  • Debezium Server now supports Google Cloud PubSub (DBZ-2092).

  • Debezium Server now supports Apache Pulsar sink (DBZ-2112).

You can find the complete list of addressed issues, upgrade procedures, and notes on any backward compatibility changes in the release notes.

Many thanks to all the community members contributing to this release: Andy Teijelo Pérez, Balázs Németh, Bingqin Zhou, Brandon Brown, cobolbaby, Dave Cumberland, Ed Laur, Emmanuel Brard, Fabian Aussems, Ivan Trusov, Justin Hiza, Jeremy Finzel, Kewei Shang, Lukas Krejci, and Robert B. Hanviriyapunt.

Read More    

I’m very happy to share the news that Debezium 1.2.0.Beta2 has been released!

Core feature of this release is Debezium Server, a dedicated stand-alone runtime for Debezium, opening up its open-source change data capture capabilities towards messaging infrastructure like Amazon Kinesis.

Overall, the community has fixed 25 issues since the Beta1 release, some of which we’re going to explore in more depth in the remainder of this post.

Read More    

Setting up change data capture (CDC) pipelines with Debezium typically is a matter of configuration, without any programming being involved. It’s still a very good idea to have automated tests for your CDC set-up, making sure that everything is configured correctly and that your Debezium connectors are set up as intended.

There’s two main components involved whose configuration need consideration:

  • The source database: it must be set up so that Debezium can connect to it and retrieve change events; details depend on the specific database, e.g. for MySQL the binlog must be in "row" mode, for Postgres, one of the supported logical decoding plug-ins must be installed, etc.

  • The Debezium connector: it must be configured using the right database host and credentials, possibly using SSL, applying table and column filters, potentially one or more single message transformations (SMTs), etc.

Read More