Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pagination for submitting responses to central instance #808

Open
davidszkiba opened this issue Jan 28, 2025 · 0 comments
Open

Pagination for submitting responses to central instance #808

davidszkiba opened this issue Jan 28, 2025 · 0 comments
Labels

Comments

@davidszkiba
Copy link
Contributor

davidszkiba commented Jan 28, 2025

Note: this is based on work done in issue #723 and the corresponding branch dev-david-issue-723

The problem: when there are too many response data to be submitted to the central instance, the client sending those data can run out of memory.

At the moment, sending responses works like this:

Solution would be to send the response data in batches of e.g. 1000 responses per request.
This means:

  • Update the catquiz::get_sql_for_model_input() function to take additional parameters to support pagination.
  • Instead of directly sending the data, create a new adhoc task for each batch and send the data in the adhoc task
  • Continue with the next batch
    @georgmaisser knows how this can be done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant