feat: environment variable interpolation

[tangowithfoxtrot]

Allow ${VAR_INTERPOLATION} in config file. Addresses #23.

Example

radarr:
  host: "${RADARR_HOST}" # with optional quotes
  port: 7878
  api_token: ${RADARR_API_KEY} # or without optional quotes
  use_ssl: false

[codecov]

Codecov Report

Attention: Patch coverage is 42.85714% with 28 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/app/mod.rs	42.85%	28 Missing ⚠️

Files with missing lines	Coverage Δ
src/app/mod.rs	68.46% <42.85%> (-6.54%)	⬇️

[tangowithfoxtrot]

This is a first attempt at getting variable interpolation working. It does work, but it's breaking optional config values. I keep getting the following error:

Error: radarr: missing field `uri` at line 2 column 3

This occurs with any other optional config value, like ssl_cert_paths as well.

I'm having trouble figuring out where that's coming from or how to make the values actually optional again 😅 @Dark-Alex-17, any ideas? It doesn't seem to be coming from the validate() function, so it seems like something is using the config values before they're validated?

[tangowithfoxtrot]

Aside from breaking optional values, I'm not sure if there's a less verbose way to generalize deserializing the various types of config values (String, Option<String>, and Option<u16>).

For now, I had to make separate deserializers for each, but I wonder if there's a way to generalize them, without doing something icky and just treating them all as String or Option<String> 🤔

[Dark-Alex-17]

I have something working that might address these concerns. Mind if I push to your branch?

[Dark-Alex-17]

In short though:

• To address the concern about security, I created a regex to remove all non-essential characters (including whitespace) from all variable values, not just those interpolated from environment variables. When I tested this, even valid Rust code was rendered unreadable and unusable so I can confirm it does pretty good mitigation of any code injection attacks

• To address the issue of not being able to start with empty optional values, I added #[serde(default, ...)] to the Option fields which allows Serde to construct the default None value when it loads from the YAML file and when there's no value for the corresponding field.

• I tried writing a macro to simplify the need for dedicated deserializers for Option fields but alas, even after all my tinkering and Googling, what you've come up with truly is the best way to achieve this.

• And I also added a new crate, veil, so that debugging is easier; It allows you to derive the Debug trait on a struct but mark which fields you want redacted on output. So, I naturally redacted the api_token field. Then I simply added a new line to main.rs that debugs the config that was loaded to help with debugging issues both here, and in the future. I actually just put this in a separate commit to main to make life better for all.

• I also added a few tests to make sure it was working the way I wanted. The only other tests that would need to be written are for the Option deserializers.

• I also tested this out with YAML fields that have multiple environment variables and it worked perfectly; e.g. radarr.uri: http://${RADARR_HOST}:${RADARR_PORT}/something

What are your thoughts?

[tangowithfoxtrot]

To address the concern about security, I created a regex to remove all non-essential characters (including whitespace) from all variable values, not just those interpolated from environment variables. When I tested this, even valid Rust code was rendered unreadable and unusable so I can confirm it does pretty good mitigation of any code injection attacks

That's a very interesting and defensive approach. I like it!

To address the issue of not being able to start with empty optional values, I added #[serde(default, ...)] ...

🙌 TIL about serde default. Great find! That was way simpler than I thought it might be.

I tried writing a macro to simplify the need for dedicated deserializers for Option fields but alas, even after all my tinkering and Googling, what you've come up with truly is the best way to achieve this.

That's a relief 😌 I had no idea how else to make that nicer and haven't delved into macros yet. lol.

I also tested this out with YAML fields that have multiple environment variables and it worked perfectly; e.g. radarr.uri: http://${RADARR_HOST}:${RADARR_PORT}/something

Good thinking. Combining the vars didn't occur to me. I'm glad it works.

I also added a few tests to make sure it was working the way I wanted. The only other tests that would need to be written are for the Option deserializers.

I'll take a look at these next.

Thanks for your help!

[Dark-Alex-17]

Sorry about the main branch thing. Not entirely sure what's going on right now since it's working locally and that supposed missing } is present in the app_tests.rs file...

[tangowithfoxtrot]

No worries. I'm pretty sure I'm just having git skill issues somewhere. lol.

[Dark-Alex-17]

Hey, I've been there. Even as a pro, there's a reason the initial commit for Git literally calls it the, "information manager from hell", along with this explanation:

GIT - the stupid content tracker

"git" can mean anything, depending on your mood.

• random three-letter combination that is pronounceable, and not actually used by any common UNIX command. The fact that it is a mispronounciation of "get" may or may not be relevant.
• stupid. contemptible and despicable. simple.
  Take your pick from the dictionary of slang.
• "global information tracker": you're in a good mood, and it actually works for you. Angels sing, and a light suddenly fills the room.
• "goddamn idiotic truckload of sh*t": when it breaks

So. You're in good company 😁