Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

400 Bad Request While Using Big Chunks #622

Open
AlmogGal1 opened this issue Feb 12, 2025 · 0 comments
Open

400 Bad Request While Using Big Chunks #622

AlmogGal1 opened this issue Feb 12, 2025 · 0 comments

Comments

@AlmogGal1
Copy link

AlmogGal1 commented Feb 12, 2025

Hi, we are working on the edx Aspects project, where we are using Ralph as our LRS to insert data to Clickhouse. As part of our project we are inserting events from the past, using tracking logs we gathered over the years, and we inserting those events to Clickhouse using Ralph. While doing this we are trying to use the biggest write chunk size of Ralph and biggest batch size of the LMS, to run less inserts to Clickhouse (which is there recommendation). Currently we set the batch size to be 10,000. While doing this, I see in Ralph logs a lot of those:
2025-02-12 09:37:27,540 INFO: 10.35.8.114:11382 - "POST /xAPI/statements HTTP/1.1" 400 Bad Request

Although, I do see past events being inserts to my Clickhouse, as I expect, I don't understand what this message means. I saw in your code, those 400 status codes are meaning some keys were missing from the json events - which makes sense.
But, my question is does it reject all of the batch I have sent (in my case all the 10,000 events) or it just reject the part of the events that were malformed and inserts the rest of the valid events to Clickhouse?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant