Debezium Extension (Early Access)
The Debezium extension allows reading from a Debezium source connector and optionally writing from that input out to a JDBC interface.
Setup
In order to be able to use the Debezium extension, first include it as a dependency in your application's pom.xml
file.
Replace {GRAINITE-VERSION}
with the version of Grainite you are also using for libgrainite (the Grainite Client library for Java).
Contents
The Debezium Extension includes:
DebeziumTask
: Task to set up and read from a debezium source connector in a parallel manner.DebeziumCDCProcessor
: Handler that can receive output from a DebeziumTask and convert it into a format that’s suitable for sending to a JDBCWriterHandler (adds the changeType and other fields as needed).
DebeziumTask
has been implemented in a generic fashion that should allow it to work with any Debezium connector but has not been tested with every connector available. If you find a bug when using a particular connector, please share the name of the connector and a description of the issue encountered via email to support@grainite.com
DebeziumTask
The DebeziumTask has some Grainite-only configuration, and additionally supports all of the configuration parameters that Debezium or the source connector may support. For example, here are the Oracle connector properties and the Mongo connector properties.
Usage
To include this task in your application, you must specify the taskClass ext.grainite.tasks.debezium.DebeziumTask
and taskInstanceClass ext.grainite.tasks.debezium.DebeziumInstance
in your application's configuration YAML file.
Below are the configuration options that can be passed in under config
:
Property | Required? | Value | Description |
---|---|---|---|
| REQUIRED | Example: | Class name of the Debezium Source Connector |
| Optional |
| Prints some additional debugging information when true. |
| Optional | Example: | max delay in seconds between successive polls. The task exponentially backs off until this delay value. |
| Optional |
| Will emit a list of all the updates (across tables specified) within the same message, if the updates were part of the same txn |
| Optional | Example: | Property name, specific to the connector, that indicates the tables being queried for CDC. When |
| Optional |
| NOTE: This is only for MongoDB. Emits events with the Document ID in the key. |
... | Additional properties for the specific connector. |
Additionally, all properties provided by Debezium connectors are applicable here.
https://debezium.io/documentation/reference/stable/connectors/index.html
DebeziumCDCProcessor
Processor to take a DebeziumTask input and generate a message compatible with the JDBCWriterHandler
Usage
To include this handler in your application, you must specify the class_name ext.grainite.handlers.debezium.DebeziumCDCProcessor
in your application's configuration YAML file.
Below are the configuration options that can be passed in under config
:
Property | Required? | Value | Description |
---|---|---|---|
| Optional | Example: | Grainite topic to emit output of this task to |
| Optional | Example: | Grainite table to emit output of this task to via Grain message ( |
| Optional | Example: | Given action to invoke via Grain Message to the table specified in |
Debezium is a trademark of Red Hat, Inc.
Last updated