Pyroscope server crashloop OOM when using backend S3

[maxospiquante]

Describe the bug

Pyroscope server crashloop OOM when using backend S3

On production I use Pyroscope with 6 java agent with heavy loading.
Without backend S3 conf all run perfects.
When backend S3 is in use, Pyroscope pod crash in the minute. I see mem limit of 2000Mi being hit.

Note when I reset S3 content Pyroscope Pod is happy and no crash happens. I try to watch in how many days OOM will come again.

Is it possible that Pyroscope enter in a never loop ending while loading/sync S3 objects ?

Environment

• Infrastructure: k8s AWS
• Deployment tool: helm with last img grafana/pyroscope:1.15.0

[marcsanmi]

Hi @maxospiquante,

To discard any unwanted behaviour introduced in the latest release, is it reproducible with previous versions of the helm chart? Also, could you share your values file?

It might be that blocks are too big. I'd give a try by lowering block duration to something around 10m.