flink-cdc在全量阶段使用了SELECT * FROM TABLE读取数据。 #3058
Unanswered
banyue0618
asked this question in
Q&A
Replies: 1 comment
-
看源码Debezium读取时候使用select *, column.include.list参数只是在生成change event时候使用。 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
flink-cdc读取数据第一阶段,快照读,采用了select * 读取数据。
![image](https://private-user-images.githubusercontent.com/40993873/302156266-b412bef3-9d1c-481f-90b9-a04e60f4cea1.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzg5NzUzMjYsIm5iZiI6MTczODk3NTAyNiwicGF0aCI6Ii80MDk5Mzg3My8zMDIxNTYyNjYtYjQxMmJlZjMtOWQxYy00ODFmLTkwYjktYTA0ZTYwZjRjZWExLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjA4VDAwMzcwNlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTY0NTExYzlkMDhkOTgzZTQ4MWYwMTQ2NzE1MDYzNDlkZjU5NjVhMTZiZTQ3ODkyM2Y0NzYzZGJlZDMwYTIxMWEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.T8PhZUJwCH6ukIdN7iBPmNAq3BjSxrE068DdnKTxvlc)
![image](https://private-user-images.githubusercontent.com/40993873/302152971-5827e8ac-7fb4-47c6-9fa2-bb7613240ff6.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzg5NzUzMjYsIm5iZiI6MTczODk3NTAyNiwicGF0aCI6Ii80MDk5Mzg3My8zMDIxNTI5NzEtNTgyN2U4YWMtN2ZiNC00N2M2LTlmYTItYmI3NjEzMjQwZmY2LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjA4VDAwMzcwNlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTJkNjcyMzQzMjU5OGJiZTgxMTlmNWQ1Y2M3NmQ5MGE0ZTAxZjI3MmEzYTE2MzU4ZmJiZjExNDk4NDE3NDVlMDUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.6BQLfnl7KdGBWN7dIP-EIDAAQNEqLJLE9MlC1agudE4)
![image](https://private-user-images.githubusercontent.com/40993873/302152574-eae2db88-2d5a-4751-a4ef-ce3ded85907c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzg5NzUzMjYsIm5iZiI6MTczODk3NTAyNiwicGF0aCI6Ii80MDk5Mzg3My8zMDIxNTI1NzQtZWFlMmRiODgtMmQ1YS00NzUxLWE0ZWYtY2UzZGVkODU5MDdjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAyMDglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMjA4VDAwMzcwNlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTQxM2FiOWQ2MmU4Mzk3MGE5NjI3ZTQ1NzM0ZjIzYjVhMTk4ZGI0Zjg3ZDRkM2MzMzAxZDJmYjc0OWMxZjI5NjcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.pMHaAtlXAlUmfrfyEMxfXB1Qc9ZM2-V5MXxYk_bsz_c)
请问下应该怎么配置,只读取我需要的字段呢。因为:我的一张表有两百个字段,但是实际上我只需要其中的五个字段,完全没有必要采用星号。
看flink-cdc的源码。直接代码中写死了星号。
目前已经进行的尝试是配置debezium属性,
debezium.column.include.list
,但是根据数据库查询日志,还是使用的select *然后我查看debezium的源码,在构建projection的时候校验了
isColumnsFiltered
,根据这个属性来决定select的字段信息为什么flink-cdc没有这样做呢。
感谢您的帮助。
Beta Was this translation helpful? Give feedback.
All reactions