Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

job-manager/history: optimize list insertion #6422

Merged

Conversation

chu11
Copy link
Member

@chu11 chu11 commented Nov 8, 2024

Problem: Internal job history lists are sorted with larger t_submit values at the front of the list. When new jobs are inserted (i.e. jobs with bigger t_submit values), they are inserted with a search starting at the back of the list. This is the opposite of what should be done and leads to a slowdown in job throughput over time.

Insert into the job history list starting at the front.

Fixes #6419

Problem: Internal job history lists are sorted with larger
t_submit values at the front of the list.  When new jobs
are inserted (i.e. jobs with bigger t_submit values), they
are inserted with a search starting at the back of the list.
This is the opposite of what should be done and leads to a
slowdown in job throughput over time.

Insert into the job history list starting at the front.

Fixes flux-framework#6419
Copy link
Contributor

@grondo grondo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Simple change, big improvement 👍

@mergify mergify bot merged commit 3ad7986 into flux-framework:master Nov 8, 2024
34 checks passed
@chu11 chu11 deleted the issue6419_job_manager_history_insert branch November 8, 2024 17:25
Copy link

codecov bot commented Feb 6, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.63%. Comparing base (0e7cecb) to head (722d786).
Report is 488 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6422      +/-   ##
==========================================
- Coverage   83.63%   83.63%   -0.01%     
==========================================
  Files         524      524              
  Lines       87617    87617              
==========================================
- Hits        73277    73276       -1     
- Misses      14340    14341       +1     
Files with missing lines Coverage Δ
src/modules/job-manager/plugins/history.c 81.87% <100.00%> (ø)

... and 12 files with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

throughput of jobs slows down over time - part deux
2 participants