ChunJun is a distributed integration framework, and currently is based on Apache Flink. It was initially known as FlinkX and renamed ChunJun on February 22, 2022. It can realize data synchronization and calculation between various heterogeneous data sources. ChunJun has been deployed and running stably in thousands of companies so far. Based on the real-time computing engine--Flink, and supports JSON template and SQL script configuration tasks. The SQL script is compatible with Flink SQL syntax. Supports a variety of heterogeneous data sources, and supports synchronization and calculation of more than 20 data sources such as MySQL, Oracle, SQLServer, Hive, Kudu, etc. Easy to expand, highly flexible, newly expanded data source plugins can integrate with existing data source plugins instantly, plugin developers do not need to care about the code logic of other plugins.
Features
- Not only supports full synchronization, but also supports incremental synchronization and interval training
- Supports dirty data storage, and provide indicator monitoring, etc.
- Cooperate with the flink checkpoint mechanism to achieve breakpoint resuming, task disaster recovery
- Supports Docker one-click deployment, support deploy and run on k8s
- ChunJun abstracts different databases into reader/source plugins
- Supports DDL synchronization, like 'CREATE TABLE', 'ALTER COLUMN', etc.
Project Samples
