Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][cdc] Support Flink CDC pipeline task in FlinkSQL dialect #2712

Closed
3 tasks done
aiwenmo opened this issue Dec 21, 2023 · 7 comments · Fixed by #2713
Closed
3 tasks done

[Feature][cdc] Support Flink CDC pipeline task in FlinkSQL dialect #2712

aiwenmo opened this issue Dec 21, 2023 · 7 comments · Fixed by #2713
Labels
Waiting for reply Waiting for reply

Comments

@aiwenmo
Copy link
Contributor

aiwenmo commented Dec 21, 2023

Search before asking

  • I had searched in the issues and found no similar feature requirement.

Description

source:
type: mysql
hostname: localhost
port: 3306
username: root
password: 123456
tables: app_db..*
server-id: 5400-5404
server-time-zone: UTC

sink:
type: doris
fenodes: 127.0.0.1:8030
username: root
password: ""
table.create.properties.light_schema_change: true
table.create.properties.replication_num: 1

pipeline:
name: Sync MySQL Database to Doris
parallelism: 2

Use case

source:
type: mysql
hostname: localhost
port: 3306
username: root
password: 123456
tables: app_db..*
server-id: 5400-5404
server-time-zone: UTC

sink:
type: doris
fenodes: 127.0.0.1:8030
username: root
password: ""
table.create.properties.light_schema_change: true
table.create.properties.replication_num: 1

pipeline:
name: Sync MySQL Database to Doris
parallelism: 2

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@aiwenmo aiwenmo added the Waiting for reply Waiting for reply label Dec 21, 2023
@aiwenmo
Copy link
Contributor Author

aiwenmo commented Dec 21, 2023

image

@aiwenmo
Copy link
Contributor Author

aiwenmo commented Dec 21, 2023

6c19756eb1f3c4933d1726b9ec6c440

@aiwenmo
Copy link
Contributor Author

aiwenmo commented Dec 21, 2023

a8195b2a553d7719ae84995f65492fc

@leonardBang
Copy link

Thanks @aiwenmo for driving this work, Niubility !

@aiwenmo
Copy link
Contributor Author

aiwenmo commented Dec 23, 2023

image

@aiwenmo
Copy link
Contributor Author

aiwenmo commented Dec 23, 2023

EXECUTE PIPELINE WITHYAML (
source:
  type: mysql
  hostname: localhost
  port: 3306
  username: root
  password: 123456
  tables: app_db.\.*
  server-id: 5400-5404
  server-time-zone: UTC

sink:
  type: doris
  fenodes: 127.0.0.1:8030
  username: root
  password: ""
  table.create.properties.light_schema_change: true
  table.create.properties.replication_num: 1

pipeline:
  name: Sync MySQL Database to Doris
  parallelism: 2
)

@aiwenmo
Copy link
Contributor Author

aiwenmo commented Dec 23, 2023

Thanks @aiwenmo for driving this work, Niubility !

Thank you to the FlinkCDC community for designing and implementing a pipeline that is easy to integrate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Waiting for reply Waiting for reply
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants