The command will start with detached Mode auto start Docker Compose All containers defined in the configuration. Stay docker-compose.yml Execute the following command under the directory to start the components required for this tutorial ： docker-compose up -d MySQL： As the data source of sub database and sub table, Store the of this tutorial user surface. SQL-Client：Flink SQL Client, To submit SQL Query and view SQL The results of the implementation of įlink Cluster： contain Flink JobManager and Flink TaskManager, Used to perform Flink SQL The Docker Compose The containers included in are ： Image: yuxialuo/flink-sql-client:1.13.2.v1 Use the following to create a docker-compose.yml file ： version: '2.1' The next tutorial will begin with docker-compose Prepare the required components in a way. 1.1 Prepare the components needed for the tutorial Prepare one that has been installed Docker Of Linux perhaps MacOS The computer. Next, we will start with data from MySQL Synchronize to Iceberg Show the whole process as an example, The architecture is as follows ：
#Download mongodb 2.7.5 how to
This tutorial will show you how to use Flink CDC Build a real-time data lake to deal with this scenario, The demonstration of this tutorial is based on Docker, Only about SQL, No need for a line Java/Scala Code, There is no need to install IDE, You can easily complete the whole content of the cost course on your computer. īut to facilitate data analysis, It is usually necessary to synchronize the split tables to the data warehouse 、 Data Lake time, Then merge into a large table.
Stay OLTP In the system, In order to solve the problem of large amount of data in a single table, It is usually used to split a single large table by database and table to improve the throughput of the system.
Tips： Click on 「 Read the original writing 」 See more technical dry goods ～ ▼ Focus on 「 Apache Flink」, Get more technical dry goods ▼Ībstract ： This tutorial will show you how to use Flink CDC Build a real-time data lake, And deal with the scene of merging and synchronization of sub database and sub table.
0 kommentar(er)
