Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attempt to fix logText progressive memory issue in LargeText.java #10236

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

kalindafab
Copy link

See (https://issues.jenkins.io/browse/JENKINS-75081)

Testing done

I tested this change by running a Jenkins pipeline that generates a large amount of log output.

Tested with 7GB logs: The logs streamed correctly without an OutOfMemoryError.
Tested with 10GB logs: The issue still occurs, showing the same large log behavior.
Heap memory usage monitored: Observed heap dump and confirmed the Jetty thread still holds logs in memory.
Manual verification: Accessed /logText/progressiveText API after each build and checked the response.
No automated tests added because this issue requires large-scale log generation that is difficult to replicate in unit tests.

Proposed changelog entries

  • human-readable text

Improve /logText/progressiveText API to handle large logs more efficiently and reduce heap memory usage.
Stream log data in smaller chunks instead of loading large portions into memory at once.
Modify LargeText and AnnotatedLargeText to avoid excessive memory consumption for logs above 7GB.
Partial fix for heap exhaustion in large build logs; further improvements needed for logs exceeding 10GB.

The fix works for logs up to 7GB, but the issue still persists for logs over 10GB. I’d appreciate guidance on further improving memory handling in Jetty and any best practices for handling extremely large logs efficiently.

Looking forward to your feedback—thanks in advance! 🙌

Proposed upgrade guidelines

N/A

Submitter checklist

Preview Give feedback

Desired reviewers

@mention

Before the changes are marked as ready-for-merge:

Maintainer checklist

Preview Give feedback

Copy link

welcome bot commented Feb 4, 2025

Yay, your first pull request towards Jenkins core was created successfully! Thank you so much!

A contributor will provide feedback soon. Meanwhile, you can join the chats and community forums to connect with other Jenkins users, developers, and maintainers.

@MarkEWaite MarkEWaite added the bug For changelog: Minor bug. Will be listed after features label Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug For changelog: Minor bug. Will be listed after features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants