API POST requests fail when REMOTE_AUTH_HEADER is enabled

[llamafilm]

Deployment Type

Self-hosted

NetBox Version

v4.2.4

Python Version

3.12

Steps to Reproduce

1. Run Netbox behind a reverse proxy which adds a custom username header
2. Enable REMOTE_AUTH_ENABLED and REMOTE_AUTH_HEADER in Netbox config
3. Make a POST API request like curl http://localhost:8000/api/dcim/sites/ -d '{"name": "site1", "slug": "site1"}'

Expected Behavior

The request should succeed without "logging in". The same as how API requests work when you're not using REMOTE_AUTH_HEADER and you provide an API token in the Authorization header.

Observed Behavior

The request fails with HTTP 403:

{"detail":"CSRF Failed: CSRF cookie not set."}

The problem lies in Netbox customized version of RemoteUserMiddleware. It runs auth.login() which causes Django to enforce CSRF. This of course fails, because a webhook cannot contain a CSRF token.

I can workaround this by writing my own custom REMOTE_AUTH_BACKEND class, and using this logic to skip API requests:

if request.META['PATH_INFO'].startswith('/api'):
    return None

This avoids the login() and the request succeeds, but this causes another minor problem: Every API request triggers a user_login_failed signal which prints this message to the log:

Failed login attempt for username: None from 2607:fb10:7011:1::e82

Furthermore, it should be possible to use REMOTE_AUTH_HEADER without writing a custom class.

[llamafilm]

For a minimal, self-contained example to reproduce this situation, you can clone https://github.com/netbox-community/netbox-docker.git and add these files:

docker-compose.override.yml

services:
  apache:
    image: httpd:2.4
    ports:
      - "8000:80"
    volumes:
      - ./httpd.conf:/usr/local/apache2/conf/httpd.conf

Add to netbox.env:

REMOTE_AUTH_ENABLED=True
REMOTE_AUTH_HEADER = 'HTTP_CUSTOMCUSTOM'
REMOTE_AUTH_AUTO_CREATE_USER=True
REMOTE_AUTH_DEFAULT_GROUPS='group1'
REMOTE_AUTH_DEFAULT_PERMISSIONS = {"dcim.site": None}

httpd.conf

ServerName localhost
Listen 80
LoadModule mpm_event_module modules/mod_mpm_event.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule headers_module modules/mod_headers.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule authz_core_module modules/mod_authz_core.so
User www-data
Group www-data
ErrorLog /proc/self/fd/2
LogLevel warn

<VirtualHost *:80>
    ProxyPreserveHost On
    ProxyRequests Off
    ProxyPass / http://netbox:8080/
    ProxyPassReverse / http://netbox:8080/
    RequestHeader set CUSTOMCUSTOM "some_user"
</VirtualHost>

docker compose up
docker compose exec netbox /opt/netbox/netbox/manage.py nbshell
>>> u = User.objects.first()
>>> u.is_superuser = True
>>> u.save()
>>> t = Token.objects.create(user_id=u.id)
>>> t.save()
>>> t.key

curl http://localhost:8000/api/dcim/sites/ -d '{"name": "site1", "slug": "site1"}'  -H "Authorization: Token $NETBOX_TOKEN"

[llamafilm]

Let me know if you'd like me to submit a patch for this. I think it's as simple as just adding those 2 lines I wrote above to the remote auth middleware.

[arthanson]

@llamafilm I can assign to you if you want to work on a PR for this?

[llamafilm]

Yes please assign to me

[jnovinger]

Hey @llamafilm , I was assigned to review the PR for this and have been looking at the reported issue. I've come to the conclusion that I don't think it is a valid issue.

For the REST API, NetBox supports 2 kinds of authentication:

• token-based (using Netbox's custom netbox.api.authentication.TokenAuthentication)
• cookie-based (using DRF's rest_framework.authentication.SessionAuthentication).

Note that Remote User Auth is not listed as a supported mechanism.

I've gathered from your curl example that this is not attempting to use token-based authentication, so if we expect Remote User Auth (as described above) to work for the example then we must assume that it's meant to use cookie-based authentication. In this case, the DRF docs indicate that session authentication (e.g. cookie-based) needs to include a valid CSRF token for any "unsafe" HTTP method call.

Arthur did point out that you might be trying to use NetBox's ability to read-only from the REST API via the EXEMPT_VIEW_PERMISSIONS setting. However, your curl example includes a -d '{"name": "site1", "slug": "site1"}' payload which implies an unsafe POST operation and must include a valid CSRF token as noted above.

I think the take away here is that your example curl request needs to be using an API token to make the specified POST call.

Please let me know if I've misunderstood the situation.

[llamafilm]

@jnovinger I just forgot to include the API token in my example curl command. I've updated the comment now, and the issue remains.
We do use API tokens to authenticate API requests. But Netbox is additionally trying to validate a CSRF token which doesn't make sense for webhooks.

I just updated the PR for compatibility with Django 5.2 (Netbox 4.3).

[jnovinger]

Please, please make sure to include details like that in future reports as it makes a huge difference when trying to understand the issue.

I assume the remote user authentication is used for other reasons, since you are in fact using token auth for the REST API. In that case, your proxy needs to make sure that the header (CUSTOMCUSTOM in this case) is not set for proxied requests coming from whatever is generating requests to the REST API.

If the remote user header is sent with these requests, then--due to middleware running well before DRF has a chance to look at the API token--DRF will treat this as a session-based authenticated request and enforce CSRF validation as designed.

[llamafilm]

Sorry about that. I'll try to add some more context here, since obviously this is a contrived example; in production, we run a customized version of Envoy which is tied into our SSO / identity system. The proxy validates the caller's identity (whether it be a user or an app) and passes that identity to Netbox as a header.

Envoy does have an option to disable authz for specific paths, but we would not want to do that because then anybody in possession of a Netbox token would be able to access the API. Those tokens are long-lived and possible to share accidentally. Enforcing authz in the proxy improves our security posture.

So in this scenario, the API token is sort of redundant because Netbox already knows the caller's identity and can map it to a Netbox user (and we trust the proxy does its job properly). But that's fine, I'm happy to include the token anyway.