run_tower fixes and improvements

[samsrabin]

Description of changes

Resolves some bugs and improves some messaging. Also, two new options:

• --no-inputdata-check option implies --setup-only but skips the check/download of input data. This is used instead of --setup-only in run_tower system testing for a speedup of ~20%.
• --xmlchange option allows user to specify xmlchange settings to apply. E.g., --xmlchange CLM_CO2_TYPE=constant,CCSM_CO2_PPMV=850.

Specific notes

Contributors other than yourself, if any: None

CTSM Issues Fixed:

• Resolves run_neon --overwrite is ignored #2884
• Resolves run_neon --setup-only is ignored #2885
• Resolves Python sys tests much longer with added tower testing... #2946

Are answers expected to change (and if so in what way)? No

Any User Interface Changes (namelist or namelist defaults changes)? No

Does this create a need to change or add documentation? Did you do so? No

Testing performed, if any: Site/regional Python unit and system tests.

• Full Python unit and system testing
• ./run_neon --neon-sites TOOL --run-type ad should complete without issues (see run_neon not working as expected for AD simulations #2739)
• ctsm_pymods suite

Remaining work

• Add system test to check that --setup-only is obeyed
• Add system test to check that --overwrite is obeyed
• Add system test to check that --xmlchange is obeyed

[samsrabin]

In my testing of ctsm5.3.024, test_sys_run_tower.py takes about 871 seconds (n=2). This PR, as of now, speeds that up by >90% (~82 seconds, n=3). I will mark this PR as closing #2946.

[samsrabin]

With the new tests, this now takes about 5 minutes (300 seconds). It would be shorter if I didn't feel like I needed to duplicate tests between both NEON and PLUMBER sites. @ekluzek and @TeaganKing, would it be crazy for me to try and get rid of the separate NeonSite and Plumber2Site classes, and instead just implement those differences within TowerSite? It seems simple to do, but I feel like maybe I'm missing something.

[TeaganKing]

Hi @samsrabin , We were hoping to implement the NeonSite and Plumber2Site classes separately in order to have clearer parts of the code that were shared versus specific to each type of tower. We also wanted to set this up so that we could eventually create more generic tower site classes parallel to those types of classes. Thus, while it's feasible to merge them back together, I think it's preferable to keep these classes as child classes that are separate from the TowerSite class.

[samsrabin]

@TeaganKing Yup, makes sense—that's the context I was missing. Thanks!

[TeaganKing]

Thanks Sam! I'll review this hopefully later this afternoon or maybe Tuesday!

[samsrabin]

(Couldn't stop thinking about simplifying, but I recognize that's a bigger task, so I started a new branch here. It keeps the tower child classes but removes some almost-identical member functions. Will submit a PR for that only after this one is approved, because it branches from this one.)

[TeaganKing]

Requesting a few minor changes that don't actually change the functionality of this PR, but this looks great! Thanks so much for putting this together!