CLUBB active to top in L93

[bstephens82]

@JulioTBacmeister was wondering why we seemed to be unable to run CAM with CLUBB active to the top of the 93-level model, since Peter had tried to run it in this way and got crashes with a "negative moist layer thickness" error. (If these are still available, they are located at /glade/scratch/pel/acom_CLUBB_to_top_ne16pg3_ne16pg3_mg17_1536_long/run.) I worked with Peter to write a script that will run the 93-level model with the latest version of the cam_development branch, and the script I've been using is here: /glade/u/home/stepheba/myscripts/cam_scripts/min_L93.sh. This script specifies a longname compset but I've confirmed that it works with FWscHIST as well (which I think is slightly different). CLUBB active to the top can be activated by uncommenting the line with

echo "trop_cloud_top_press = 0.0007D0"              >>user_nl_cam

I have not encountered any crashes yet using this script. I have two runs (they were both set to run 180 days and both made it almost that far, but they timed out---I had neglected to increase NTASKS from my testing phase), a control run with the above line commented out, and a "test" run with the line uncommented (CLUBB active to top). Here are some plots from those two runs showing T and Q averaged over the deep tropics (-10 to +10). Based on these plots the test case with CLUBB active to the top seems be slightly warmer and wetter (although still very dry in absolute terms) near the top of the model in the deep tropics.

Are there other variables people would like me to plot and post here?

Should I run a longer test with CLUBB active to the top? Several years or more?

[adamrher]

It may be interesting to see the winds up there as that what tends to blow up the model. I think just +/-10 U would be informative. Maybe mid-latitude U as well?

[dan800]

What is the motivation to run CLUBB all the way up? Reasons we have not run cloud microphysics in the mesosphere in the past:

• microphysics is very expensive, so a computational savings for not running it throughout the column
• the pressure, temperature and Q vmr domains up there are untested in the current param
• there aren't really any clouds up there (PMCs and PSCs being the exception)

IIRC, when we did run cloud physics up into the MLT in the past we got some odd polar mesospheric clouds forming globally that were not physical

Also not sure this plays nicely with the PSC parameterization - might want to check with Doug, Simone and Nick.

Are these differences in T and U directly attributable to the additional physics or just chaotic divergence for two runs?

[adamrher]

@dan800 Thanks for filling in the history. That's very helpful for me.

the motivation is as follows -- all pre-CAM6 PBL schemes provided vertical diffusion to the model top, acting to help stabilize the dynamics up there. With CLUBB, it combined the PBL scheme with moist physics and so we opted to not let it go about top_lev~1hPa, which is how all macro/micro were treated pre-CAM6. However, we lost that additional difffusion from the PBL component of CLUBB because we decided to not untangle it from the moist physics. So now that the high-top is harder to stabilize, we are experimenting with turning on (part, all) of clubb to help stabilize the sims.

Will keep an eye out on PSCs or anything else weird --- we def. don't want to move backwards.

[dan800]

The diffusion from the PBL scheme should be dwarfed by the eddy diffusion from the breaking of gravity waves. Have you compared the vertical diffusion coefficient from the PBL scheme to the Kzz from non-orographic gravity wave dissipation? It seems to me it should not be necessary to run the PBL scheme all the way up just to keep the model stable.

[adamrher]

No I haven't compared Kzz's from GW to the PBL scheme up there. But it matters not just how big, but how frequently a KZZ coefficient is applied -- the GW scheme is really punchy right? Whereas I think of the PBL scheme as providing a background mixing all the time up there.

[dan800]

Seems like a numerically expensive way to introduce some background diffusion. If it's necessary for dynamical stability, why not introduce a Kzz min value?

[bstephens82]

@dan800 I think I am not the best person to address your good questions, but maybe @adamrher or @JulioTBacmeister can answer them. In the meantime here are some plots of U and V in the deep tropics, and I will also create some plots of U in the mid-latitudes and post them shortly.

[bstephens82]

@adamrher here are some plots of U averaged over the mid-latitudes, 30-60 south and north.

[bstephens82]

Should I run a longer test with CLUBB active to the top? Several years or more?

As another update at @JulioTBacmeister's request I am running two 10-year experiments with and without CLUBB active to model top. I am also using prognostic momentum fluxes, the taus code, Colin's surface stress update patch, Colin's tuned parameters, no clubbtop limiter and my monotonic flux limiter fixes in these experiments.

[bstephens82]

Should I run a longer test with CLUBB active to the top? Several years or more?

As another update at @JulioTBacmeister's request I am running two 10-year experiments with and without CLUBB active to model top. I am also using prognostic momentum fluxes, the taus code, Colin's surface stress update patch, Colin's tuned parameters, no clubbtop limiter and my monotonic flux limiter fixes in these experiments.

My 10-year control test, with all the things I mentioned except for CLUBB active to the top, is on year 9 and still running. My experiment with CLUBB active to the top failed after about a year, with NaNs coming into SHR_REPROSUM_CALC, an error I see sometimes but which is a bit obscure to me. I did set up a branch run a few days before the crash so I could take a closer look at what's going on, and it does look like there are very high temperatures being calculated in the upper atmosphere, 850 K for example (this is the cam.h0 "T" variable, not potential temperature). So apparently we do have some issues letting CLUBB go to the top of the model. I will try to learn more about what's going on up there.

[dan800]

While 850K is reasonable for the thermosphere, temperatures that high are not present in the middle atmosphere. Something has gone off the rails in a big way. Unless you are confident CLUBB returns physically reasonable tendencies in the low pressure (0.01 hPa), low water vapor (6 ppmv), low temperature (down to 120 K) domain of the mesosphere, I would think it best to not run it above the lower stratosphere. Better to have errors of omission and save a few CPU cycles in the process.

[adamrher]

Wonder if the prognostic equations go hay wire up there for various reason. We could experiment with only letting the down-gradient momentum diffusion operate up to the model top?

[bstephens82]

CLUBB uses the Flatau scheme by default for saturation...I'm not sure offhand over what range of temperatures etc. that scheme is valid but it's conceivable it's not valid in the high-top area. This is something Julio was concerned about.

We could experiment with only letting the down-gradient momentum diffusion operate up to the model top?

You mean turn off prognostic momentum fluxes?

[adamrher]

I thought it was Geoff and Gratch or something like that. I checked what the clubb core was using a while ago and confirmed its the same SVP equation for liquid that we use in CAM.

Oh I thought you were running w/o prognostic momentum. Either way, it's not as easy as I made it seem to have the tendencies for one variable go to the top and one not, all the arrays passed into the clubb dycore need the same number of vertical levels. And so we'd have to use that dumb clubbtop limiter if wanted to start restricting where T/Q/CLDLIQ tendencies are w/o doing so for momentum.