Refinery accepts two different means to govern how much memory it can use before it takes steps to conserve its use and allow reclaiming garbage that it has abandoned, but both methods require specifying an absolute quantity of bytes: AvailableMemory for the total from which to claim a proportioned amount, or MaxAlloc for the absolute amount it may use. In both cases, those values fall out of step easily with parallel configurations for a container's available memory, such as within a Kubernetes pod's container spec, in which we specify a request for or limit on the amount of memory.
It would help to eliminate the need for specifying the AvailableMemory parameter and allow sole use of the MaxMemoryPercentage parameter with a positive value, emulating the OpenTelemetry Collector's memory_limiter processor. That processor uses its iruntime.TotalMemory function to deduce the amount of memory available to the process, using a variety of techniques such as reading the container's cgroup configuration.
This request comes by way of discussion in the "discuss-refinery" channel of the Honeycomb Pollinators Slack workspace.
Refinery accepts two different means to govern how much memory it can use before it takes steps to conserve its use and allow reclaiming garbage that it has abandoned, but both methods require specifying an absolute quantity of bytes:
AvailableMemoryfor the total from which to claim a proportioned amount, orMaxAllocfor the absolute amount it may use. In both cases, those values fall out of step easily with parallel configurations for a container's available memory, such as within a Kubernetes pod's container spec, in which we specify a request for or limit on the amount of memory.It would help to eliminate the need for specifying the
AvailableMemoryparameter and allow sole use of theMaxMemoryPercentageparameter with a positive value, emulating the OpenTelemetry Collector's memory_limiter processor. That processor uses itsiruntime.TotalMemoryfunction to deduce the amount of memory available to the process, using a variety of techniques such as reading the container's cgroup configuration.This request comes by way of discussion in the "discuss-refinery" channel of the Honeycomb Pollinators Slack workspace.