pak does not see system dependencies under non default path

[skwde]

In an HPC environment, system dependencies are often installed in non default locations and made available e.g. via lmod.

pak does not see such dependencies and lists them as missing.

Even all env variables like LD_LIBRARY_PATH / PATH / ... are adjusted and moreover I manually set variables like GDAL_CONFIG / PROJ_LIBS / ... I get

✖ Missing 8 system packages. You'll probably need to install them manually:
+ abseil-cpp-devel  - s2
+ cmake             - s2
+ gdal              - sf
+ gdal-devel        - sf
+ geos-devel        - sf
+ proj-devel        - sf
+ sqlite-devel      - sf, sf
+ udunits2-devel    - units

The R packages build just fine even though, according to pak the dependencies are missing.

[gaborcsardi]

Indeed, pak only knows about system dependencies installed with the OS package manager.

[skwde]

Thanks for confirming my suspicion @gaborcsardi.

This behavior is very confusing and misleading.

I assume this bug even leads to automatic installation of the "missing" dependencies when root access is available?

[gaborcsardi]

Indeed. In general there is no robust way to correctly detect custom installed system packages, so for these systems it is best to turn off pak's sysreqs support.

[skwde]

You mean turning of automatic installation via

export PKG_SYSREQS=FALSE

?

But even if I switch off automatic installation wrong, misleading information is shown.

I do agree that it is a hard task to do, especially in a reasonable amount of time and platform agnostic...
I would rather not have wrong information shown by default, worse still packages installed even though present.
This could break the system.

I can understand that it is a hard choice to make, switching default behavior.
In the meantime, at least show a fat warning that dependency information might be incomplete / wrong because only certain package sources / locations are checked.

[gaborcsardi]

I am not going to change the default behavior at this point because that breaks a lot of workflows, and because I think the overwhelming majority of people are happy with it.

In a HPC environment a typical R user does not have root access, so no unintended installation happens. We can certainly improve the messages and the documentation:

• do not show system packages at all if sysreqs support is off.
• add a message to listed system packages that says that only the system package manager is used.
• document better that pak only uses the system package manager.

A PR is welcome to do (some of?) these. But no pressure.

[matteodelucchi]

👋 I tried to follow your suggestion from the earlier comment and opened a PR to improve messaging around system requirements checks. Not sure if I covered this exactly as you intended, so any feedback is welcome.

[EricDeveaud]

Indeed. In general there is no robust way to correctly detect custom installed system packages, so for these systems it is best to turn off pak's sysreqs support.

configure does the job nicely
at least you can rely on pkg-config for a large majority of libs,
above mentionned systemes, lmod ,module usualy set up some environnement variable that allows to check for include, libs and so on.
honour LDFLAGS, CPPFLAGS, CFLAGS, etc and try // except to compile versus the required componenent.

regards

Eric