Unhealthy authoritative powerdns docker container

[dpuenteramirez]

Hi! First of all thanks you for your implementation of a local DNS infrastructure. Don't know if this is still maintained, but I have a weird problem, I have updated the local domains to match my current network, but once I am deploying the stack, the authoritative docker fails to check the health check.

Any ideas what so ever about why is this happening to me?

[noxPHX]

Hello ! Thanks for dropping by !

I have not committed for a while but don't worry I am still using Power-Hole as my DNS infrastructure as you called it and I aim to fix any problem that may occur. I might be lucky I haven't got any for about 2 years!

Regarding your problem, we will need more information, it might be useful to check for the logs of the authoritative server for any error or the status of the healthcheck command (with docker inspect powerhole_pdns_authoritative).
Feel free to paste any of these.

Nox

[dpuenteramirez]

Hi Nox!
I wasn't sure you were still around giving some sort of support to the Power-Hole infra/repo.

I tried to deploy it in a Rasp Pi3B+ and in a Debian 11 based OS, both ended up with the same result. The following output is from running the docker insepct command.

❯ docker inspect powerhole_pdns_authoritative
[
    {
        "Id": "b30555013a8f89417b2c2f235e20a09c4092671dfbfa6bdb9119df398439ad4d",
        "Created": "2023-04-26T13:16:33.337480217Z",
        "Path": "pdns_server",
        "Args": [],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 11080,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2023-04-26T13:16:46.028378942Z",
            "FinishedAt": "0001-01-01T00:00:00Z",
            "Health": {
                "Status": "unhealthy",
                "FailingStreak": 3,
                "Log": [
                    {
                        "Start": "2023-04-26T15:17:16.028528256+02:00",
                        "End": "2023-04-26T15:17:21.26795054+02:00",
                        "ExitCode": 1,
                        "Output": ";; connection timed out; no servers could be reached\n\n"
                    },
                    {
                        "Start": "2023-04-26T15:17:51.282263032+02:00",
                        "End": "2023-04-26T15:17:56.429628263+02:00",
                        "ExitCode": 1,
                        "Output": ";; connection timed out; no servers could be reached\n\n"
                    },
                    {
                        "Start": "2023-04-26T15:18:26.442495869+02:00",
                        "End": "2023-04-26T15:18:31.577692651+02:00",
                        "ExitCode": 1,
                        "Output": ";; connection timed out; no servers could be reached\n\n"
                    }
                ]
            }
        },
        "Image": "sha256:2517fe1116ca6edaa8df99b4af4bd2713aa13e07cb7ed905237536461e4981e7",
        "ResolvConfPath": "/var/lib/docker/containers/b30555013a8f89417b2c2f235e20a09c4092671dfbfa6bdb9119df398439ad4d/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/b30555013a8f89417b2c2f235e20a09c4092671dfbfa6bdb9119df398439ad4d/hostname",
        "HostsPath": "/var/lib/docker/containers/b30555013a8f89417b2c2f235e20a09c4092671dfbfa6bdb9119df398439ad4d/hosts",
        "LogPath": "/var/lib/docker/containers/b30555013a8f89417b2c2f235e20a09c4092671dfbfa6bdb9119df398439ad4d/b30555013a8f89417b2c2f235e20a09c4092671dfbfa6bdb9119df398439ad4d-json.log",
        "Name": "/powerhole_pdns_authoritative",
        "RestartCount": 0,
        "Driver": "btrfs",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "docker-default",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": null,
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "powerhole_pdns_admin",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "always",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "ConsoleSize": [
                0,
                0
            ],
            "CapAdd": null,
            "CapDrop": [
                "ALL"
            ],
            "CgroupnsMode": "private",
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": [],
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": [
                "no-new-privileges"
            ],
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": null,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": null,
            "Name": "btrfs"
        },
        "Mounts": [],
        "Config": {
            "Hostname": "powerhole_pdns_authoritative",
            "Domainname": "",
            "User": "pdns",
            "AttachStdin": false,
            "AttachStdout": true,
            "AttachStderr": true,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": null,
            "Healthcheck": {
                "Test": [
                    "CMD-SHELL",
                    "dig +short +norecurse +retry=0 @127.0.0.1 && curl -s --fail http://127.0.0.1:8081 || exit 1"
                ]
            },
            "Image": "powerhole:authoritative",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": [
                "pdns_server"
            ],
            "OnBuild": null,
            "Labels": {
                "com.docker.compose.config-hash": "8aa64afb413916b1d1b73dbf46d9cfc3e81087e200cb335f6c5e949183456788",
                "com.docker.compose.container-number": "1",
                "com.docker.compose.depends_on": "powerhole_pdns_auth_db:service_healthy:false",
                "com.docker.compose.image": "sha256:2517fe1116ca6edaa8df99b4af4bd2713aa13e07cb7ed905237536461e4981e7",
                "com.docker.compose.oneoff": "False",
                "com.docker.compose.project": "power-hole",
                "com.docker.compose.project.config_files": "/home/USER/Tools/Power-Hole/docker-compose.yml",
                "com.docker.compose.project.working_dir": "/home/USER/Tools/Power-Hole",
                "com.docker.compose.service": "powerhole_pdns_authoritative",
                "com.docker.compose.version": "2.17.3"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "7ac60f2920c8ee739e6afae3d90f43be505cb303217a58aa9252921f074a40ee",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/7ac60f2920c8",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "powerhole_pdns_admin": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "powerhole_pdns_authoritative",
                        "powerhole_pdns_authoritative",
                        "b30555013a8f"
                    ],
                    "NetworkID": "db6531d3296911ea4effe6095b9716409e0da16a77fd666189fc68741301b863",
                    "EndpointID": "6798597f83c29a1734d3028612a5fdd333f9720ab4200c39d18e174de4eb5e9c",
                    "Gateway": "172.26.0.105",
                    "IPAddress": "172.26.0.106",
                    "IPPrefixLen": 29,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "REDACTED",
                    "DriverOpts": null
                },
                "powerhole_pdns_auth_db": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "powerhole_pdns_authoritative",
                        "powerhole_pdns_authoritative",
                        "b30555013a8f"
                    ],
                    "NetworkID": "6f3b132130f0aecc25fe72481ecfb0c10030b61a6ed52db0c15f597b6c520118",
                    "EndpointID": "13bc3ad0dfd133a07bfac21ee7bb6a132722f8682a26c2e5fdb9f82cff9bb137",
                    "Gateway": "172.26.0.89",
                    "IPAddress": "172.26.0.91",
                    "IPPrefixLen": 29,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "REDACTED",
                    "DriverOpts": null
                },
                "powerhole_pdns_authoritative": {
                    "IPAMConfig": {
                        "IPv4Address": "172.26.0.122"
                    },
                    "Links": null,
                    "Aliases": [
                        "powerhole_pdns_authoritative",
                        "powerhole_pdns_authoritative",
                        "b30555013a8f"
                    ],
                    "NetworkID": "f343a7bc534614eab7b7a8d3e6bceeedf8cb58081f1601348a63e9c1713b3d8e",
                    "EndpointID": "91fe5aa19fdde38a8a6ad39b209713d95bd49de253cb17546e477f46f067b53c",
                    "Gateway": "172.26.0.121",
                    "IPAddress": "172.26.0.122",
                    "IPPrefixLen": 29,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "REDACTED",
                    "DriverOpts": null
                }
            }
        }
    }
]

[noxPHX]

I tried to do an install from scratch without any problem...
The only difference I can see so far is the docker-compose version, I'm still using the "legacy" (v1) version of docker-compose. I can see you are using docker compose v2.17.3.
It migh be somehow related, I will need to update and test the stack accordingly when I have some spare time.

In the meantime, I can read on your paste that the healthcheck fails when trying to do a DNS lookup on itself, I guess PDNS is not started, it might be useful to have a look at the logs of the container and check for any error.
You can also try to increase the log level (

Power-Hole/docker-compose.yml

Line 36 in a17e20e

LOG_LEVEL: 6 # 3 - 6 (https://doc.powerdns.com/authoritative/settings.html#loglevel)

), rebuild & restart.