Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON logging not applied for k8s dependencies #1150

Open
der-eismann opened this issue Mar 25, 2025 · 3 comments · May be fixed by #1152
Open

JSON logging not applied for k8s dependencies #1150

der-eismann opened this issue Mar 25, 2025 · 3 comments · May be fixed by #1152

Comments

@der-eismann
Copy link
Contributor

Describe the bug
We enabled JSON logging to consume and process the messages in our logging system (Graylog, like ELK). However it seems that JSON logging is not forwarded to messages produced by k8s dependencies like for client-side throttling.

Steps to reproduce

  1. Run NTH in any mode
  2. Have it do some work?
  3. Wait for Waited for 1.195944684s due to client-side throttling or similar

Expected outcome
k8s client logs should also be in JSON format, ideally with the same message structure and timestamp format

Application Logs

{"time":"2025-03-25T08:46:56Z","message":"evicting pod default/abc-6c6bfb79bb-mhggh"}
{"time":"2025-03-25T08:46:56Z","message":"evicting pod default/def-5b6fc69988-kdfrb"}
I0325 08:47:00.329841       1 request.go:697] Waited for 3.078586713s due to client-side throttling, not priority and fairness, request: GET:https://10.10.0.1:443/api/v1/namespaces/default/pods/ghi-54c7d4fcc9-w69fz
I0325 08:47:10.330255       1 request.go:697] Waited for 1.794203547s due to client-side throttling, not priority and fairness, request: GET:https://10.10.0.1:443/api/v1/namespaces/default/pods/jkl-b9984f4ff-zf5lk

Environment

  • NTH App Version: v1.25.0
  • NTH Mode (IMDS/Queue processor): Queue
  • OS/Arch: Container / Bottlerocket arm64
  • Kubernetes version: 1.32
  • Installation method: Helm
@der-eismann
Copy link
Contributor Author

So there's the SetLogger function in klog, but it seems to be incompatible with zerolog.

@Lu-David
Copy link
Contributor

Lu-David commented Apr 7, 2025

Thanks for raising this issue @der-eismann How come ZeroLogger is incompatible?

@der-eismann
Copy link
Contributor Author

That is a very good question. I assume they just decided to use one relatively popular logging interface.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants