1. Releases
  2. 1.3
  3. Release Notes for Debezium 1.3

Release Notes for Debezium 1.3

All notable changes for Debezium releases are documented in this file. Release numbers follow Semantic Versioning.

Release 1.3.1.Final (November 12th, 2020)

See the complete list of issues.

Kafka compatibility

This release has been built against Kafka Connect 2.6.0 and has been tested with version 2.6.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.3.1.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.3.1.Final plugin files, and restart the connector using the same configuration. Upon restart, the 1.3.1.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

There are no breaking changes in this release.

New Features

  • Hide stacktrace when default value for SQL Server cannot be parsed DBZ-2642

Fixes

  • Oracle throw "no snapshot found based on specified time" when running flashback query DBZ-1446

  • SqlExceptions using dbz with Oracle on RDS online logs and logminer DBZ-2624

  • Mining session stopped - task killed/SQL operation cancelled - Oracle LogMiner DBZ-2629

  • Antlr DDL parser fails to interpret BLOB([size]) DBZ-2641

  • WAL logs are not flushed in Postgres Connector DBZ-2653

  • Debezium server Event Hubs plugin support in v1.3 DBZ-2660

  • Should Allow NonAsciiCharacter in SQL DBZ-2670

  • MariaDB nextval function is not supported in grammar DBZ-2671

  • ChangeRecord informations don’t connect with the TableSchema DBZ-2679

  • Sanitize field name do not santize sub struct field DBZ-2680

  • Debezium fails if a non-existing view with the same name as existing table is dropped DBZ-2688

  • No viable alternative at input error on "min" column DBZ-2738

Other changes

  • Upgrade MySQL JDBC driver to version 8.0.19 DBZ-2626

Release 1.3.0.Final (October 1st, 2020)

See the complete list of issues.

Kafka compatibility

This release has been built against Kafka Connect 2.6.0 and has been tested with version 2.6.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.3.0.Final from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.3.0.Final plugin files, and restart the connector using the same configuration. Upon restart, the 1.3.0.Final connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

There are no breaking changes in this release.

New Features

  • Allow configurable CONNECT_LOG4J_LOGGERS in connect images DBZ-2541

  • MySQL connector - ignore statement-based logs DBZ-2583

  • Add a configuration which sanitizes values in mongodb DBZ-2585

Fixes

  • Sqlserver connector block cdc cleanup job DBZ-1285

  • Upgrade Guava library due to GuavaCompatibility errors DBZ-2008

  • mongodb-connector NPE in process of MongoDataConverter DBZ-2316

  • Error with UUID-typed collection column DBZ-2512

  • event.processing.failure.handling.mode doesn’t skip unparseable data events DBZ-2563

  • decoderbufs Segmentation fault on timestamp with infinity DBZ-2565

  • MongoDB ExtractNewDocumentState can not extract array of array DBZ-2569

  • New MySQL 8 ALTER USER password options not supported DBZ-2576

  • MariaDB ANTLR parser issue for grant syntax DBZ-2586

  • Debezium Db2 connector fails with tables using BOOLEAN type DBZ-2587

  • db2 connector doesn’t allow to reprocess messages DBZ-2591

  • Missing links in filter and content-based SMT doc DBZ-2593

  • Format error in doc for topic routing and event flattening SMTs DBZ-2596

  • Debezium refers to database instead of schema in Postgres config DBZ-2605

  • NullPointerException thrown when calling getAllTableIds DBZ-2607

Other changes

  • Coordinate docs work for downstream 1.2 release DBZ-2272

  • Gracefully handle server-side filtered columns DBZ-2495

  • Schema change events fail to be dispatched due to inconsistent case DBZ-2555

  • Use dedicated functional interface for struct generators DBZ-2588

  • Remove obsolete note from docs DBZ-2590

  • Intermittent test failure on CI - ReplicationConnectionIT#shouldResumeFromLastReceivedLSN DBZ-2435

  • Intermittent test failure on CI - PostgresConnectorIT#shouldExecuteOnConnectStatements DBZ-2468

  • Intermittent test failure on CI - AbstractSqlServerDatatypesTest#stringTypes() DBZ-2474

  • Intermittent test failure on CI - PostgresConnectorIT#customSnapshotterSkipsTablesOnRestart() DBZ-2544

  • Intermittent test failure on CI - SQLServerConnectorIT#verifyOffsets DBZ-2599

Release 1.3.0.CR1 (September 24th, 2020)

See the complete list of issues.

Kafka compatibility

This release has been built against Kafka Connect 2.6.0 and has been tested with version 2.6.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.3.0.CR1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.3.0.CR1 plugin files, and restart the connector using the same configuration. Upon restart, the 1.3.0.CR1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

The SMTs for content-based routing and filtering – both using JSR 223 scripting engines for script evaluation – have been moved from the Debezium core module into a separate artifact (DBZ-2549). This must be added to the plug-in directories of those connector(s) for which you wish to use those SMTs. When using the Debezium container image for Kafka Connect, set the environment variable ENABLE_DEBEZIUM_SCRIPTING to true in order to do so. This change was done so to allow for exposing scripting functionality only in environments with an appropriately secured Kafka Connect configuration interface.

New Features

  • Describe configurations options for auto-created change data topics DBZ-78

Fixes

  • Outbox | Heartbeat not working when using ByteBufferConverter DBZ-2396

  • Catch up streaming before snapshot may duplicate messages upon resuming streaming DBZ-2550

  • Fix Quarkus datasource configuration for Quarkus 1.9 DBZ-2558

Other changes

  • Show custom images instead of S2I in docs DBZ-2236

  • Add Db2 tests to OpenShift test-suite and CI DBZ-2383

  • Implement connection retry support for Oracle DBZ-2531

  • Extract scripting SMTs into a separate module with separate installation package DBZ-2549

  • Format updates in doc for topic routing and event flattening SMTs DBZ-2554

  • Coordinate docs work for downstream 1.3 release DBZ-2557

  • Extend connect image build script with ability to add extra libraries DBZ-2560

  • Invalid use of AppProtocol instead of protocol field in OpenShiftUtils service creation method DBZ-2562

  • Doc format updates for better downstream rendering DBZ-2564

  • Prepare revised SMT docs (filter and content-based routing) for downstream DBZ-2567

  • Swap closing square bracket for curly brace in downstream title annotations DBZ-2577

Release 1.3.0.Beta2 (September 16th, 2020)

See the complete list of issues.

Kafka compatibility

This release has been built against Kafka Connect 2.6.0 and has been tested with version 2.6.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading any connector, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.3.0.Beta1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.3.0.Beta2 plugin files, and restart the connector using the same configuration. Upon restart, the 1.3.0.Beta2 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

Certain configuration options (*.whitelist/*.blacklist) could be perceived as promoting racial stereotypes. These options now use a new naming convention: (*.include.list/*.exclude.list) (DBZ-2462). This change should now be implemented for all connectors. It is still possible to use the old names to simplify the migration but a warning is generated upon using the obsolete names.

New Features

  • Ingest change data from Oracle databases using LogMiner DBZ-137

  • Server-side column filtering in SQL Server connector DBZ-1068

  • Introduce column.include.list for MySQL Connector DBZ-2508

Fixes

  • Increase Maven version in enforcer plugin DBZ-2281

  • JSON functions in MySQL grammar unsupported DBZ-2453

  • PostgresStreamingChangeEventSource’s replicationStream flushLsn after closed DBZ-2461

  • Fix link rendering for include.list and exclude.list properties DBZ-2476

  • CVE-2019-10172 - security vulnerability DBZ-2509

  • ArrayIndexOutOfBoundsException with excluded column from CDC table DBZ-2522

  • maven-surefire-plugin versions defined twice in parent pom DBZ-2523

  • Connector Type properties has missing displayName property DBZ-2526

Other changes

  • Allow Postgres snapshotter to set streaming start position DBZ-2094

  • Ability to include Db2 driver in downstream image DBZ-2191

  • Unify representation of events in the documentation DBZ-2226

  • CloudEvents remains TP but has avro support downstream DBZ-2245

  • Document new SMTs: content-based-routing and filtering DBZ-2247

  • Document new Schema Change Topics DBZ-2248

  • Change db2 version in Dockerfile from latest DBZ-2257

  • Prepare DB2 connector doc for TP DBZ-2403

  • Strimzi cluster operator no longer exposes service to access prometheus metrics endpoint DBZ-2407

  • Clarify include/exclude filters for MongoDB are lists of regexps DBZ-2429

  • Mongo SMT dose not support add.fields=patch DBZ-2455

  • Prepare message filtering SMT doc for product release DBZ-2460

  • Avoid divisive language in docs and option names in incubator connectors DBZ-2462

  • Intermittent test failure on CI - FieldRenamesIT DBZ-2464

  • Adjust outbox extension to updated Quarkus semantics DBZ-2465

  • Add a locking mode which doesn’t conflict with DML and existing reads on Percona Server DBZ-2466

  • Ignore SSL issues during release job DBZ-2467

  • [Documentation] Fix Debezium Server documentation for transformations and Google Pub/Sub DBZ-2469

  • Remove unnecessary include/exclude database configuration in order to ensure backwards compatibility in OCP test-suite DBZ-2470

  • Edit the features topic DBZ-2477

  • False negatives by commit message format checker DBZ-2479

  • Document outbox event router SMT DBZ-2480

  • Error when processing commitLogs related to frozen type collections DBZ-2498

  • Doc tweaks required to automatically build Db2 content in downstream user guide DBZ-2500

  • Unify representation of events - part two - update other connector doc DBZ-2501

  • Ability to specify kafka version for OCP ci job DBZ-2502

  • Add ability to configure prefix for the add.fields and add.headers DBZ-2504

  • Upgrade apicurio to 1.3.0.Final DBZ-2507

  • Add more logs to Cassandra Connector DBZ-2510

  • Create Configuration Fields for datatype.propagate.source.type and column.propagate.source.type DBZ-2516

  • Prepare content-based router SMT doc for product release DBZ-2519

  • Add missing ListOfRegex validator to all regex list fields and remove legacy whitelist/blacklist dependents DBZ-2527

  • Add annotations to support splitting files for downstream docs DBZ-2539

Release 1.3.0.Beta1 (August 28th, 2020)

See the complete list of issues.

Kafka compatibility

This release has been built against Kafka Connect 2.6.0 and has been tested with version 2.6.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading the MySQL, MongoDB, PostgreSQL or SQL Server connectors, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.3.0.Beta1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.3.0.Beta1 plugin files, and restart the connector using the same configuration. Upon restart, the 1.3.0.Beta1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

Certain configuration options (*.whitelist/*.blacklist) could be perceived as promoting racial stereotypes. These options now use a new naming convention: (*.include.list/*.exclude.list) (DBZ-2171). This change is now implemented for MySQL, PostgreSQL, MongoDB and SQL Server connectors, and the other connectors will follow in the next release. It is still possible to use the old names to simplify the migration but a warning is generated upon using the obsolete names.

New Features

  • Improve error handling in Cassandra Connector DBZ-2410

  • Add few MySql8 privileges support DBZ-2413

  • Add support for MySql Dynamic Privileges DBZ-2415

  • Support for MySql8 invisible / visible index DBZ-2425

  • Hitting "Unable to unregister the MBean" when stopping an embedded engine DBZ-2427

Fixes

  • Adding new table to cdc causes the sqlconnector to fail DBZ-2303

  • LSNs in replication slots are not monotonically increasing DBZ-2338

  • Transaction data loss when process restarted DBZ-2397

  • java.lang.NullPointerException in ByLogicalTableRouter.java DBZ-2412

  • Snapshot fails if table or schema contain hyphens DBZ-2452

Other changes

  • Upgrade OpenShift guide DBZ-1908

  • Avoid divisive language in docs and option names in core connectors DBZ-2171

  • Refactor: Add domain type for LSN DBZ-2200

  • Entries in metrics tables should be linkable DBZ-2375

  • Update some doc file names DBZ-2402

  • Asciidoc throw warnings while building documentation DBZ-2408

  • Upgrade to Kafka 2.6.0 DBZ-2411

  • Confusing way of reporting incorrect DB credentials DBZ-2418

  • Default value for database port isn’t honoured DBZ-2423

  • Update to Quarkus 1.7.1.Final DBZ-2454

Release 1.3.0.Alpha1 (August 6th, 2020)

See the complete list of issues.

Kafka compatibility

This release has been built against Kafka Connect 2.5.0 and has been tested with version 2.5.0 of the Kafka brokers. See the Kafka documentation for compatibility with other versions of Kafka brokers.

Upgrading

Before upgrading the MySQL, MongoDB, PostgreSQL or SQL Server connectors, be sure to check the backward-incompatible changes that have been made since the release you were using.

When you decide to upgrade one of these connectors to 1.3.0.Alpha1 from any earlier versions, first check the migration notes for the version you’re using. Gracefully stop the running connector, remove the old plugin files, install the 1.3.0.Alpha1 plugin files, and restart the connector using the same configuration. Upon restart, the 1.3.0.Alpha1 connectors will continue where the previous connector left off. As one might expect, all change events previously written to Kafka by the old connector will not be modified.

If you are using our docker images then do not forget to pull them fresh from Docker registry.

Breaking changes

For the sake of consistency with other Debezium configuration options, the option mongodb.poll.interval.sec of the Debezium MongoDB connector has been deprecated; please use the new option mongodb.poll.interval.ms instead. The deprecated option will be removed in a future Debezium release. (DBZ-2400)

New Features

  • Postgres and possibly other DB connections are not properly shutdown when the task encounters thread interrupt DBZ-2133

  • More flexible connection options for MongoDB DBZ-2225

  • Sink adapter for Azure Event Hubs DBZ-2282

  • Implement new snapshot mode - initial_only DBZ-2379

Fixes

  • Ignore non-existing table reported on Aurora via SHOW TABLES DBZ-1939

  • Cassandra connector not getting events DBZ-2086

  • PubSub Sink sends empty records DBZ-2277

  • Skipping LSN is inefficient and does not forward slot position DBZ-2310

  • message size is at least 68x larger for changes with bit varying columns DBZ-2315

  • Change events lost when connnector is restarted while processing transaction with PK update DBZ-2329

  • Error when processing commitLogs related to list-type columns DBZ-2345

  • Fix dependency groupId on Outbox Quarkus Extension documentation DBZ-2367

  • Cannot detect Azure Sql Version DBZ-2373

  • ParallelSnapshotReader sometimes throws NPE DBZ-2387

Other changes

  • Column default values are not extracted while reading table structure DBZ-1491

  • DataException("Struct schemas do not match.") when recording cellData DBZ-2103

  • Provide container image for Debezium Server DBZ-2147

  • Update binlog client DBZ-2173

  • PostgreSQL test matrix runs incorrect test-suite DBZ-2279

  • Use ARG with defaults for Kafka’s versions and sha when building Kafka Docker image DBZ-2323

  • Modularize doc for PostgreSQL component DBZ-2333

  • Featured posts list broken DBZ-2374

  • Deprecate mongodb.poll.interval.sec and add mongodb.poll.interval.ms. DBZ-2400

  • Test failures on Kafka 1.x CI job DBZ-2332

  • Add configurable restart wait time and connection retires DBZ-2362

  • Support data types from other database engines DBZ-2365

  • Add ProtoBuf support for Debezium Server DBZ-2381

  • Intermittent test failure on CI - SqlServerChangeTableSetIT#addDefaultValue DBZ-2389

  • Intermittent test failure on CI - TablesWithoutPrimaryKeyIT#shouldProcessFromStreaming DBZ-2390

  • Include Azure PostgreSQL guidance in the docs DBZ-2394

  • Update JSON Snippet on MongoDB Docs Page DBZ-2395