Ephemeral storage limitations causing POD eviction of web nodes

[dekke046]

Hi,

We run Pega on a shared OCP cluster, this cluster has limited physical storage on the worker nodes available and therefore we are advise to make as much as possible use of PVC's with either ocs-storagecluster-ceph-rbd or ocs-storagecluster-cephfs.

For fileprocessing with agents we use ocs-storagecluster-cephfs without issues.

But what we frequently see is that pega web nodes are being evicted due to using too much ephemeral storage, currently configured to use max. 2GB.
It looks in most cased it is related to tracer activities located at /usr/local/tomcat/work/Catalina/localhost/prweb/Tracer

How should we deal with limited ephemeral storage?
What is the minimal ephemeral storage required to run without being evicted, there is only a specification for the installer (30GB)?

Since the pega web pods are regular deployments (via Helm) and not as a statefullset options to use a PVC with "ReadWriteOnce" is also not an option.

Am I overlooking something to prevent the usage of ephemeral storage, I am open to suggestions.

Maarten

[dekke046]

Anyone?

This is really causing issues with evicted pods.