Creating the conda-lock file

[abkfenris]

We are using conda-lock to generate a lock file to get more deterministic and repeatable builds. When the environment.yml is updated, run conda-lock -p linux-64 -f environment.yml to regenerate the lockfile.

[abkfenris]

Conda-lock has evolved. Now to create the lockfile it's a two stage process, first conda-lock -p linux-64 -f environment.yml then conda-lock render -p linux-64.

[emiliom]

FYI it looks like conda-lock fails with a git+https:// pip dependency. See conda/conda-lock#4 (comment)

[emiliom]

I have a pip dependency in my updates to py-base/environment.yml. It looks like conda-lock render doesn't support pip dependencies:

$ conda-lock render -p linux-64
Rendering lockfile(s) for linux-64...
WARNING:conda_lock.conda_lock:WARNING: installation of pip dependencies is only 
supported by the 'conda-lock install' command. Other tools may silently ignore them. 
For portability, we recommend using the newer unified lockfile format (i.e. removing the --kind=explicit argument.
 - Install lock using : conda create --name YOURENV --file conda-linux-64.lock

@abkfenris what do you recommend?

I'll submit a PR with my changes to the environment.yml file, plus the changes resulting from conda-lock -p linux-64 -f environment.yml and conda-lock render -p linux-64, to this point.

[abkfenris]

@emiliom It looks like echopy is a simple enough package to quickly get on Conda-Forge that grayskull probably can auto scaffold the recipe for staged-recipies, so that might be the best route currently, as I don't think conda can directly ingest the non-rendered lockfiles.

@ocefpaf only mamba and micromamba can use the intermediate lockfiles right?

[ocefpaf]

@emiliom It looks like echopy is a simple enough package

🤔 open-ocean-sounding/echopy@1a33be5

@ocefpaf only mamba and micromamba can use the intermediate lockfiles right?

I'm not sure to be honest as I'm stuck with conda-lock 0.13 :-/
However, I agree that, unless there is a good reason to install from git we should not do it. Let me see if I can get echopy in conda-forge.

[abkfenris]

@emiliom It looks like echopy is a simple enough package

🤔 open-ocean-sounding/echopy@1a33be5

Ugg, I missed that.

[ocefpaf]

BTW, not even a pip install from git would get a fully functional package there. I'm working on something and I'd love upstream it if they had tests.

[emiliom]

open-ocean-sounding/echopy@1a33be5

😱

BTW, not even a pip install from git would get a fully functional package there. I'm working on something and I'd love upstream it if they had tests.

Sigh. Lesson to self: I shouldn't have tried to add it to the image before I fully tried out the package myself 😞 (it's a package Wu-Jung and I talk about a fair bit). Anyway, I can reach out to the authors to see if they'd be open to improving the packaging.

Thanks!

[emiliom]

Posting this more general question here, based on @abkfenris 's comment in #63 (comment)

I don't think I'm going to be able convert to a mamba/micromamba based image before this event, and if there are any issues the cross-language debugging might be messy so it's probably better not to try.

I created a local OHW-py conda env to generate the conda lock file using mamba. Should I create it with conda instead, or does it make no difference?

FYI the conda-lock installed in my local conda env is 1.4.0.

[abkfenris]

They should both solve the same, or similar enough environments. Mamba however may be able to solve the lock file before the heat death of the universe.

[emiliom]

Alright, thanks. I'm sticking with mamba and not waiting around.

[ocefpaf]

I created a local OHW-py conda env to generate the conda lock file using mamba. Should I create it with conda instead, or does it make no difference?

If you have the following config options in your conda installation, conda and mamba will have equivalent solutions for the env:

conda install --name base conda-libmamba-solver --yes
conda config --set solver libmamba

if you don't they may differ but most of the time the mamba (libmamba) one is preferable b/c it favors higher build numbers rather than "easier" solve. That makes the desired env more "correct" b/c most of the time a higher build number usually fixes pins.

[emiliom]

Thanks @ocefpaf . I don't have either of those config options set. I'll go ahead with my mamba-generated conda-lock, as is.