Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kinesis encrypted stream #109

Closed
BreakpointsCA opened this issue Feb 16, 2022 · 1 comment
Closed

Kinesis encrypted stream #109

BreakpointsCA opened this issue Feb 16, 2022 · 1 comment

Comments

@BreakpointsCA
Copy link

I have issues reading an encrypted data stream in AWS. I have no errors in my logs but my spark job is not fetching any records from the stream.
I am using this version of the library: spark-sql-kinesis_2.12-1.2.0_spark-3.0.jar

My stream is configured to use Server-side encryption.
Based on the documenation: "Kinesis Data Streams uses AWS Key Management Service (KMS) to encrypt your data."

When I deactivate this feature, my spark job is able to pull records from the stream without any problems.

Do I need to configure something or it should be transparent?

We have Lambda that reads from that same stream without problems.

@BreakpointsCA
Copy link
Author

The issue is related to #107

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant