permission denied error when attempting to activate an environment

[gzt5142]

Unable to activate an existing environment created in the user space.

An environment listed when I conda env list raises an error when I attempt to activate it:

$ conda env list
# conda environments:
#
nebari-git-dashboard     /home/conda/nebari-git/envs/nebari-git-dashboard
nebari-git-dask          /home/conda/nebari-git/envs/nebari-git-dask
nebari-git-pdash         /home/conda/nebari-git/envs/nebari-git-pdash
users-aws                /home/conda/users/envs/users-aws
users-pangeo             /home/conda/users/envs/users-pangeo
base                  *  /opt/conda
default                  /opt/conda/envs/default

$ conda activate users-pangeo
mkdir: cannot create directory ‘/home/conda/users/envs/users-pangeo/share/gdb/auto-load//home/conda/users/envs’: Permission denied

This error does not seem to impact the environment -- it appears to activate -- but this mkdir error is new behavior suggesting a failure of some kind. I would like to understand the implications of the failure/error.

[iameskild]

Hi @gzt5142, thanks for bringing this to our attention. I haven't run into this issue before and from looking at the mkdir error message, it looks like two paths are being stacked on top of one another. Can you activate other environments?

[gzt5142]

Hi... other environments do not raise this error:

$ conda activate users-aws  #success
$ conda deactivate
$ conda activate users-pangeo
mkdir: cannot create directory ‘/home/conda/users/envs/users-pangeo/share/gdb/auto-load//home/conda/users/envs’: Permission denied
$

[gzt5142]

behavior has evolved:

$ conda activate users-pangeo
mkdir: cannot create directory ‘/home/conda/users/envs/users-pangeo/share/gdb/auto-load//home/conda/users/envs’: Permission denied
ln: failed to create symbolic link '/home/conda/users/envs/users-pangeo/share/gdb/auto-load//home/conda/users/envs/users-pangeo/lib/libarrow.so.1200.0.0-gdb.py': No such file or directory
/home/conda/users/envs/users-pangeo/etc/conda/activate.d/libarrow_activate.sh ERROR: Failed to create symlink from '/home/conda/users/envs/users-pangeo/share/gdb/auto-load/replace_this_section_with_absolute_slashed_path_to_CONDA_PREFIX/lib/libarrow.so.1200.0.0-gdb.py' to '/home/conda/users/envs/users-pangeo/share/gdb/auto-load//home/conda/users/envs/users-pangeo/lib/libarrow.so.1200.0.0-gdb.py'

Is this likely to do with how this particular environment has been created/configured on our host?

[iameskild]

Hey @gzt5142, I'm sorry that this keeps happening. It's quite tricky to determine what the root cause of this issue might be. If it's not too much trouble, I would recommend deleting this environment from the /conda-store/admin (if you're on the latest version of Nebari) and try rebuilding it. Otherwise, if you share your environment, I can try and recreate the issue on my end.

[gzt5142]

admin is not among the namespaces where I can create environments.

Have re-built an environment in a shared namespace, as well as my own private namespace. Same behavior.

conda activate gzt5142-nhgfdev
mkdir: cannot create directory ‘/home/conda/gzt5142/envs/gzt5142-nhgfdev/share/gdb/auto-load//home/conda/gzt5142/envs’: Permission denied
ln: failed to create symbolic link '/home/conda/gzt5142/envs/gzt5142-nhgfdev/share/gdb/auto-load//home/conda/gzt5142/envs/gzt5142-nhgfdev/lib/libarrow.so.1200.1.0-gdb.py': No such file or directory
/home/conda/gzt5142/envs/gzt5142-nhgfdev/etc/conda/activate.d/libarrow_activate.sh ERROR: Failed to create symlink from '/home/conda/gzt5142/envs/gzt5142-nhgfdev/share/gdb/auto-load/replace_this_section_with_absolute_slashed_path_to_CONDA_PREFIX/lib/libarrow.so.1200.1.0-gdb.py' to '/home/conda/gzt5142/envs/gzt5142-nhgfdev/share/gdb/auto-load//home/conda/gzt5142/envs/gzt5142-nhgfdev/lib/libarrow.so.1200.1.0-gdb.py'

The environment def that I am using:

channels:
- conda-forge
dependencies:
- python=3.10
- aiobotocore
- cf_xarray
- cfgrib
- cfunits
- dask-geopandas
- dask-gateway==2022.4.0
- dask-geopandas
- distributed
- fsspec==2023.3.0
- h5netcdf
- h5py
- ipykernel
- ipyleaflet
- ipywidgets
- nco
- netcdf4 == 1.6.0
- numcodecs
- pandas
- param
- pip
- pystac
- pystac-client
- s3fs>=2023.3.0
- xarray
- h5pyd
- zarr
- kerchunk
- nbformat
- rechunker
description: ''
name: nhgfdev
prefix: null

The YML generated in conda-store is attached.
nhgfdev.zip

[iameskild]

Hi @gzt5142, thanks for reporting this! A few of us just stumbled upon this issue yesterday as well. Luckily, this isn't blocking anyone from activating a new environment, but that said, we still need to investigate why this is happening.

cc @kcpevey

[kcpevey]

I'm seeing this same behavior. Its throwing this error even when activating the default environment:

kcpevey:~ 
11:24 $ conda activate nebari-git-dask 
mkdir: cannot create directory ‘/home/conda/nebari-git/envs/nebari-git-dask/share/gdb/auto-load//home/conda/nebari-git/envs’: Permission denied
ln: failed to create symbolic link '/home/conda/nebari-git/envs/nebari-git-dask/share/gdb/auto-load//home/conda/nebari-git/envs/nebari-git-dask/lib/libarrow.so.1000.1.0-gdb.py': No such file or directory
/home/conda/nebari-git/envs/nebari-git-dask/etc/conda/activate.d/libarrow_activate.sh ERROR: Failed to create symlink from '/home/conda/nebari-git/envs/nebari-git-dask/share/gdb/auto-load/replace_this_section_with_absolute_slashed_path_to_CONDA_PREFIX/lib/libarrow.so.1000.1.0-gdb.py' to '/home/conda/nebari-git/envs/nebari-git-dask/share/gdb/auto-load//home/conda/nebari-git/envs/nebari-git-dask/lib/libarrow.so.1000.1.0-gdb.py'
(nebari-git-dask) kcpevey:~ 
11:24 $ 

However, I haven't seen it actually break anything - my env still activates and I can run stuff in python (so far) without issue.

[kcpevey]

I've opened an issue to track this, let's move further discussion over there. @gzt5142 thanks for opening this discussion up! Feel free to add additional info to the issue :)