Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added DefaultMemoryHealthChecker #4711

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

lazmond3
Copy link

@lazmond3 lazmond3 commented Feb 28, 2023

Motivation:

Resolves part of #1854
Some users may want to use Memory usage to determine if their instance is healthy.

Modifications:

  • Added DefaultMemoryHealthChecker class that checks memory-usage according to the method shown below.
memory kind how implementation forked from...
TOTAL Runtime.getRuntime().totalMemory()
NON-HEAP ManagementFactory.getPlatformMXBean(BufferPoolMXBean.class).getMemoryUsed() https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/taskmanager/MemoryLogger.java#L183-L193
HEAP ManagementFactory.getMemoryPoolMXBeans() (filtered by its memory type heap)

Result:

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copy link
Member

@trustin trustin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also implement toString() using Guava's ToStringHelper?

@lazmond3
Copy link
Author

lazmond3 commented Mar 6, 2023

@trustin
13f0361
With this commit, the toString() method was overridden using Guava's ToStringHelper.


public class DefaultMemoryHealthChecker implements HealthChecker {

private final double targetHeapMemoryUtilizationRate;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we change Rate to Ratio in this PR? Rate means speed of change which is not what we meant here.

Comment on lines +20 to +22
this.targetHeapMemoryUtilizationRate = targetHeapMemoryUtilizationRate;
this.targetNonHeapMemoryUtilizationRate = targetNonHeapMemoryUtilizationRate;
this.targetTotalMemoryUtilizationRate = targetMemoryTotalUsage;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we could add some checks here, e.g.

this.heapUsageRatioThreshold = validateRatio(heapUsageRatioThreshold, "heapUsageRatioThreshold");
this.nonHeapUsageRatioThreshold = validateRatio(nonHeapUsageRatioThreshold, "nonHeapUsageRatioThreshold");
this.totalUsageRatioThreshold = validateRatio(totalUsageRatioThreshold, "totalUsageRatioThreshold");
...

private static void validateRatio(double ratio, String paramName) {
    checkArgument(ratio >= 0 && ratio <= 1, "%s (expected: >= 0 and <= 1)", paramName);
}

Also, I suggested renaming the parameters and member fields as shown in the above example.

  • Utilization -> Usage (because it's shorter)
  • Memory -> '' (because it's already a part of the enclosing class name)
  • Target -> Threshold (because target means something we aim for, but we don't really want that happen here)

final double totalMemoryUsage = Runtime.getRuntime().totalMemory();
return (heapMemoryUsage / maximumHeapMemory) <= targetHeapMemoryUtilizationRate &&
(nonHeapMemoryUsage == null || (nonHeapMemoryUsage.getMemoryUsed() / nonHeapMemoryUsage.getTotalCapacity()) <= targetNonHeapMemoryUtilizationRate) &&
(runtimeMaxMemory == Long.MAX_VALUE || totalMemoryUsage / runtimeMaxMemory <= targetTotalMemoryUtilizationRate);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we log a warning message if nonHeapMemoryUsage is null or runtimeMaxMemory is Long.MAX_VALUE, so that we tell our users that those properties are not available and thus it will never become unhealthy even if you specified the threshold?

We could keep a boolean flag so make sure that the log message is logged only once, so we don't pollute the log file.

}

private List<MemoryPoolMXBean> getHeapMemories() {
return ManagementFactory.getMemoryPoolMXBeans().stream().filter(e -> MemoryType.HEAP.equals(e.getType())).collect(Collectors.toList());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could call .map(MemoryPoolMXBean::getUsage) before collecting so we don't have to do it twice in isHealthy().

final BufferPoolMXBean nonHeapMemoryUsage = ManagementFactory.getPlatformMXBean(BufferPoolMXBean.class);
final double totalMemoryUsage = Runtime.getRuntime().totalMemory();
return (heapMemoryUsage / maximumHeapMemory) <= targetHeapMemoryUtilizationRate &&
(nonHeapMemoryUsage == null || (nonHeapMemoryUsage.getMemoryUsed() / nonHeapMemoryUsage.getTotalCapacity()) <= targetNonHeapMemoryUtilizationRate) &&
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Convert from long to double before dividing?

@@ -0,0 +1,51 @@
package com.linecorp.armeria.server.healthcheck;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing copyright header


import com.google.common.base.MoreObjects;

public class DefaultMemoryHealthChecker implements HealthChecker {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be package private final:

Suggested change
public class DefaultMemoryHealthChecker implements HealthChecker {
final class DefaultMemoryHealthChecker implements HealthChecker {

@minwoox
Copy link
Contributor

minwoox commented Mar 23, 2023

gentle ping @lazmond3 😄

@github-actions github-actions bot added the Stale label Apr 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants