Error polling - EOF

[LeifSec]

up to yesterday evening (~2024-11-27 18:00:00 UT) my configuration

• client (debian12)
  • PushProx
  • several exporter like Prometheus-node-exporter (from Debian repository)
• server (debian12)
  • Prometheus (from Debian repository)
  • PushProx
    works perfectly

Without any changes at the configuration and the firewalls now I got in the log of the PushProxy on the client

pushprox-client[7039]: ts=2024-11-27T11:35:29.979Z caller=main.go:197 level=error msg="Error polling:" err="Post \"http://server:8082/poll\": EOF"

and of curse on the server some time out error due to this:

pushprox-proxy[33701]: ts=2024-11-27T11:40:29.429Z caller=main.go:179 level=error msg="Error scraping:" err="Timeout reached for \"http://client:9101/metrics\": context canceled" url=http://client:9101/metrics

I have check access to the ports and also with curl by

curl -d 'test' http://server:8082/poll

which is received on the server

Nov 27 12:48:04 server pushprox-proxy[33701]: ts=2024-11-27T11:48:04.779Z caller=coordinator.go:136 level=info msg=WaitForScrapeInstruction fqdn=test
Nov 27 12:48:04 server pushprox-proxy[33701]: ts=2024-11-27T11:48:04.779Z caller=main.go:148 level=info msg="Error WaitForScrapeInstruction:" err="request is expired"

I have also pulled the latest version form PushProxy now and checked the system times - same errors.

Sending to the PushProxyon the clientvia curl -d 'test' http://localhost:9101/polls generates a return of

# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 0
go_gc_duration_seconds{quantile="0.25"} 0
go_gc_duration_seconds{quantile="0.5"} 0
....

My configs:

• server
  • pushproxy: --web.listen-address=:8082
  • prometheus: /etc/prometheus/prometheus.yml

 - job_name: push_proxy
   proxy_url: http://localhost:8082/
   scrape_interval: 120s
   scrape_timeout: 120s
   tls_config:
     insecure_skip_verify: true
   static_configs:
     - targets: ['client:9101']  # push proxy (etc, metrics);  Presuming the FQDN of the client is "client".
     - targets: ['client:9732']  # nftables (/home)
     - targets: ['client:9633']  # smartctl (/home)
     - targets: ['client:9902']  # smartctl (/home)

• client: --proxy-url=http:/server:8082/ --log.level=debug --metrics-addr=:9101

[alex-muntean]

hey, i had the same issue, it worked after I've changed the proxy_url, as follows:

in /etc/prometheus/prometheus.yml

from:
proxy_url: http://localhost:8082/

to:
proxy_url: http://<pushprox-master-ip>:8082/

[LeifSec]

This doesn't help for me.

Changing to proxy_url: http://<pushprox-master-ip>:8082/ where pushprox-master-ip is the ip of my PushProx + Prometheus server which is identical to localhost.

I still got Get "http://client:9101/metrics": context deadline exceeded.

[menvol3]

Facing exactly the same issue
@LeifSec have you managed to resolve it?

[LeifSec]

No.