Grainite is often deployed in environments where other services or products are already running, and in many cases, there is need to either consume data from or push data to those other products. As opposed to building integrations for those products into the core Grainite product, we've introduced Extensions. Extensions are individual packages that contain Tasks, Handlers and other common code needed to integrate with those products.
Grainite Extensions enjoy the same platform guarantees as any Grainite application. They are
- No additional infrastructure requirement. Extensions run directly on Grainite platform eliminating additional resource and operational costs.
- No data loss, duplication, or missed records. Once a record is pulled from the source, it is guaranteed to be processed and written to the sink.
- Source or sink data from/to external components at scale with sub-second latencies.
- High availability with built-in fault tolerance.
- Built-in monitoring and observability
Grainite offers two types of Extensions:
Note that Extensions are not the same as Tasks. Extensions are packages that may contain one or more tasks, one or more handlers, and other common library code that we want to share with users.
Extensions are versioned the same way as the main product, so Grainite version 2315 will also have extensions versioned as 2315. However, Extensions are not automatically installed on the server; they are designed to be integrated and packaged along with client applications.
For Java extensions, we publish Maven packages within our Gitlab Maven repository for customer applications to declare dependencies on and package extensions along with their application packages.
An example of how an extension may be used within a client application is below. In this case, the application wants to read data from the CDC tables for SQLServer and pull them into a Grainite topic. From there, further processing can be done by Grainite handlers. Instead of writing custom code to do so, using the Tasks framework, the pre-built
grainite-sqlserverextension can be used.
- task_name: sqlserver_cdc
- topic_name: payments_topic
The pages within this section provide usage details for each individual extension.