Handbook / Documentation / Version compatibility

[Samoangtagey]

Hey there,
I am a student who has a project working with Netbox. I was asked if it is possible to export DNS-Zonefiles out of Netbox, which seem to be an option with your work (netbox-auto).

First of all, I would like to know how to use "netbox-auto" since I can't see any documentation file or Handbook how to use it. Could you hand a few advises, so I will be able to use "netbox-auto" to export such files?

Second, the actual version of Netbox is 2.5.2 (2018-12-21) and since there have been a lot changes I don't know if it's still compatible.

I hope this is the correct way to contact you since I cannot find any other here, otherwise please let me know how.

Sincerely,

Samoangtagey

[fxkr]

Hi,

sorry for the sparse docs.

          postgres              HTTP(S)             File write            File read
netbox DB  ----  netbox_server.py ---- netbox_update.py --- bind9 zonefiles --- bind9

There are two components:

• netbox_serve.py is the "server" part. It connects to the netbox database (PostgreSQL) and serves a summary via HTTP (TLS is handled outside the app). You configure it using environment variables (12-factor app style - makes it easy to deploy in cloud envs. We run netbox and this in Heroku, thats what Procfile, runtime.txt and requirements.txt are for):

  • BASIC_AUTH_USERNAME, BASIC_AUTH_PASSWORD for HTTP
  • NETBOX_TENANT_SLUG=test // the script ignores all netbox objects not linked to this tenant. Do not forget to set this! The main things that goes wrong is that someone creates records in netbox, and wonders why they dont show up in DNS - its because they forgot to set the tenant on the host or IP or network object.
  • DATABASE_URL=postgres://example:[email protected]:5432/example // for the NetBox DB

  It will look like this (the HTTP path is /devices):

{
  "example-hostname": {
    "cnames": [
      "foo", 
      "bar"
    ], 
    "primary": "10.10.20.30",
    "secondary_ips": [
      "10.21.20.1", 
      "10.21.21.1", 
      "10.21.22.1", 
      "10.21.22.1"
    ]
  }, 
  ...
}

• netbox_update.py is the "client". It runs directly on our DNS servers. This connects to the HTTP server (once, it's up to you to run it repeatedly), generates a zonefile (from template zonefile.j2) in bind9 format. There are some more environment vars that go in it. I can't reach any system it's running on from here right now but the source is pretty short, so.

  • NETBOX_ENDPOINT is the URL where netbox_server.py is (incl user@password:)
  • DNS_ZONE is the name of the zone, e.g. intranet.example.com
  • DNS_SERVERS are the local DNS servers for the zone, as a JSON list of [name, ip] lists, e.g. [["ns0", "192.168.1.1"]]
  • DNS_DIRECTORY is the path where the zonefiles are
  • DNS_CONTACT, DNS_REFRESH_TIME, DNS_EXPIRE_TIME, DNS_NEGATIVE_CACHE_TIME go in the appropriate places in the zonefile, see bind9 manual

We have some special requirements netbox can't represent. So we've abused the comment field. You can add lines that contain a JSON document within backticks, and nothing else!, to generate CNAMEs (for hosts with multiple hostnames), e.g.:

  `{"cnames": ["alternative-hostname"]}`

In case it's not clear, python dependency management crashcourse:

python3 -m venv somedir # or just "venv" for earlier python3 versions, or just "virtualenv" before - python's a mess
source somedir/bin/activate
pip install -r requirements.txt
VAR1=a VAR2=b ... python yourscript.py

This entire scheme has some tradeoffs:

• Advantage: it's reasonably simple to understand (no magic automatic communication, HTTP API can be checked easily with browser for troubleshooting if required. the custom code is really short.)
• Advantage: netbox and DNS server are relatively decoupled
• Advantage: not "incremental", ie no problems with consistency (zonefiles are pretty small so its ok)
• Advantage: no connections into the DNS server -> can be used with firewalled private DNS servers when netbox runs in the cloud
• Advantage: the HTML / JSON API isolates the nameservers from any netbox version specific changes. I felt this was important if we're going directly to the DB.
• Disdavantage: reinvents AXFR
• Disadvantage: polling instead of push notifications -> higher delay and/or constant traffic even when things dont change and/or manual work if you dont run netbox_update automatically
• Disadvantage: requires sole control over zonefiles. e.g. problematic in combination with nsupdate, or DHCP updates

These work very well for our use case, but YMMV.

I briefly considered writing a DNS server that directly reads the netbox database, or possibly using PowerDNS (which lets you use an SQL database to store the zone, instead of zonefiles like in the 80's) and either extending that to read the netbox tables, or maybe using a trigger to crosspopulate the powerdns tables when the netbox database changes, or materialized views. That DNS server would be a hidden master, and the prod DNS servers would be slaves. All of this is certainly a lot more elegant that what I actually built. And also a lot more moving parts, which is why I didn't do the "clean" way.

About versions - err, let's just say I'm not at 2.5.2 yet but it's a good point and I should upgrade it at some point soon. It shouldn't be too hard. After all, netbox_serve.py is literally two SQL queries.

Generally you find email addresses on github by git cloning the repo locally and looking at git log. Mine's up to date there, but anyway having the discussion here in public is fine - then I can link to it if it should come up again.

Let me now if that helps.

Thanks for your interest.
Felix

[Samoangtagey]

Hallo Felix,

leider habe ich Deine eMail-Adresse nicht auffinden können nach dem Cloning, es wird da nur meine aber nicht Deine Adresse angezeigt.

Anyways ... erst einmal vielen Dank für die ganzen Infos. Ich denke, ich konnte das alles nachvollziehen und verstehen, bis auf eine Sache: Der Export bzw. eigentlich ist es ja eher ein Import vom DNS-Server aus ausgeführt. Was ich mich dabei gerade frage ist, ob das unbedingt von einem DNS-Server au sgemacht werden muss, oder ob das nicht ein x-beliebiger Rechner sein kann, der dann am Ende eine Zone-File liegen hat.

Wie anfangs geschrieben, ich bin Student und mache das für das hiesige Rechenzentrum, und ich gehe mal davon aus dass die Mitarbeiter dort nicht sehr begeistert davon sind dass ich da mal eben ein Script auf einem DNS-Server ausprobiere ;-) Theoretisch müsste es ja aber gehen, den Client für die Abfrage einfach zu starten (die Angaben zum DNS-Server können ja trotzdem korrekt sein), und dann einfach eine lokale Zonefile zu erzeugen, die aber nicht gleich in den Wirkbetrieb geht. Erst einmal anschauen ob auch das dabei raus kommt was gewünscht ist (siehe Beispiel mit dem richtigen Setzen der Tenants).

Für uns alle ist Netbox noch neu und ich evaluiere es gerade, so von daher sind wir auch noch nicht mit allen Funktionen vertraut (ich taste mich da stück für stück ran, die ReadTheDocs erklären zwar die Installation sehr gut, aber nicht die Bedienung an sich).

Ich wollte nur schonmal eine Rückmeldung geben (wenn auch verzögert, der Dezember hat ja irgendwie immer andere Prioritäten ;-) ), ich werde mich aber noch weiter mit netbox-auto befassen. Ich werde auch gerne (ob hier oder an anderer Stelle) über meine Versuche und die Ergebnisse berichten, ich kann mir gut vorstellen dass das für den Entwickler ja immer interessant ist, wie gut andere mit dem was man geschaffen hat zurecht kommen.

Ich wünsche einen guten Start ins neue Jahr und lasse wieder von mir hören.

Viele Grüße :-)

Samoangtagey

[fxkr]

Hi,

what did you try to find the email address? Most likely you weren't in the right directory. It's not hard - git clone https://..., cd netbox-auto, git log. Anyway, as I said it's preferable to have technical conversations in the open so that others may benefit too.

I, too, used to be a working student at a university admin team (disclaimer: this project was not for that). We had staging systems. There should be no need to experiment in production. But I think it was also expected that we understand the software we use (especially random scripts found on the internet), both for academic and practical reasons. And the script is really short.

netbox_update.py writes its output to the $DNS_DIRECTORY/$DNS_ZONE/zone.db. There's no requirement for an actual DNS server daemon to run on the same host. However the best way to verify that the generated zonefile is correct is to run one and query it. Setting up a bind9 is pretty straightforward and highly recommended as a learning experience. fyi, bind9 also doesn't reload zone files unless you tell it to (rndc reload), so even if you ran the script in production, the changes wouldn't take effect immediately.

fwiw, I like netbox. It can be a bit rigid (or opinionated - it is clearly tailored to digitalocean's needs), but I find it has generally all the characteristics I want: it's lightweight, easy to maintain, fast, out of the way, looks modern and it's mostly intuitive, which puts it ahead of the competitors I tried.

Best
Felix