Add <prefix>/lib/rtf to LD_LIBRARY_PATH ?

[traversaro]

Otherwise, I don't think RTF plugins for YARP are working, see robotology/yarp#1375 .

[drdanz]

This is a workaround. I'd rather fix RTF instead...

[traversaro]

In which sense it is a workaround? How do you instruct RTF on where to find its plugins? A manifest file? A RTF-specific env variable?

[drdanz]

Setting LD_LIBRARY_PATH is usually considered a bad practice...
I'd like to standardize some mechanism in shlibpp, but I don't know exactly how yet

[diegoferigo]

I'm interested in this, but I have no idea on how to achieve it without LD_LIBRARY_PATH.

[traversaro]

I checked a few "LD_LIBRARY_PATH is bad" articles:

• http://xahlee.info/UnixResource_dir/_/ldpath.html
• https://stackoverflow.com/questions/25918462/why-ld-library-path-is-bad-and-the-correct-way-to-load-dynamic-libraries
• https://gms.tf/ld_library_path-considered-harmful.html#patch-existing-elf-binaries

but none of them mention dlopen or dynamic plugin loading.

[claudiofantacci]

I'm interested in this, but I have no idea on how to achieve it without LD_LIBRARY_PATH.

Me too! Just came through this "problem" as well while setting up the VM for EPR. I don't really like putting things in LD_LIBRARY_PATH.

[drdanz]

I'm interested in this, but I have no idea on how to achieve it without LD_LIBRARY_PATH.

The idea should be to have some configuration or some manifest file to locate the library, and then open it using the full path.

Unfortunately there are quite a lot of issues with this:

0. Adding further environment variables is probably not such a good idea.
1. If you write a manifest file, it should not contain absolute paths in order to make the package re-locatable
2. multi config generators (i.e. MSVC, Xcode) put the libraries in subfolders
3. on windows, usually, the dll has a postfix d in the name
4. every system uses custom mechanisms and default paths (just to make a couple of examples, ldconfig is used on linux, the PATH variable is searched on windows)

This is what YARP does, just to make an example.

[traversaro]

I think we should add it until we found a better solution.

[traversaro]

This is what YARP documentation suggests after all https://github.com/robotology/yarp/blame/v3.0.0/doc/rtf_plugins.md#L14 .

[diegoferigo]

Waiting a smarter solution, which will take time and brain power, I would add it as well.

[traversaro]

Several projects define their own *_PLUGIN_PATH environmental variable:

• QT_PLUGIN_PATH : http://doc.qt.io/qt-5/deployment-plugins.html
• GAZEBO_PLUGIN_PATH : http://gazebosim.org/tutorials/?tut=plugins_hello_world
• IGN_PLUGIN_PATH : https://ignitionrobotics.org/api/common/3.0/classignition_1_1common_1_1SystemPaths.html#a7b4501e2e593c5e088e33bbbf1e3adbf

It would be nice if there could be a consensus among these kind of projects to support a PLUGIN_PATH environmental variable such that it could be just set once for each "non-global" prefix.

However, sticking with the robotology-superbuild problem, we have this problem for two projects:

• RTF
• BlockFactory

A possible solution would be if these two projects defined their own RTF_PLUGIN_PATH and BLOCKFACTORY_PLUGIN_PATH environmental variables. Even if the projects use different environmental variables, it would probably make sense to add a FindSharedLibrary-like to sharedlibpp to find an arbitrary plugin given a established search path (that could be customized by specifying the <project>_PLUGIN_PATH env variable name.

If instead we want to try to establish a consensus on a shared environmental variable, we could make it easy to use PLUGIN_PATH for sharedlibpp, hoping that it is not conflicting with any existing env variable. : )

[diegoferigo]

I had in mind exactly the approach used in ignition. I found it great also for what concern the tests, where the plugin folder inside the build directory was added in the search path automatically, without the need to manually operate on LD_LIBRARY_PATH.

Despite PLUGIN_PATH is appealing, for the time being, at least for what concern BlockFactory, I would proceed with a BLOCKFACTORY_PLUGIN_PATH env variable, maybe matched with custom cmake commands for installing the plugin library in the right folder.

[traversaro]

Despite PLUGIN_PATH is appealing, for the time being, at least for what concern BlockFactory, I would proceed with a BLOCKFACTORY_PLUGIN_PATH env variable, maybe matched with custom cmake commands for installing the plugin library in the right folder.

I totally agree. Do you think anyhow the logic for finding the library can be added to the sharedlibpp ? This may be a quite useful to easily generalize the approach to RTF.

[diegoferigo]

In robotology/blockfactory#32 I edited the shlibpp version vendored by blockfactory to support the following:

• A generic SHLIBPP_PLUGIN_PATH env variable to specify extra plugin paths
• Extending the plugin search path by calling a new method

[traversaro]

For reference, apparently in ROS's world this "finding the plugin path" is handled by pluginlib ( https://github.com/ros/pluginlib ) while most of the logic that we have in shlibpp.
Interestingly, the directory for searching for plugins are found by append lib (or bin in ROS2 and the latest ROS from Microsoft) to some sort of prefi. For ROS1, the CMAKE_PREFIX_PATH env variable is used, while in ROS2 the get_package_prefix's ament_cpp function. See the relevant code:

• ROS1 : https://github.com/ros/pluginlib/blob/0a3c0de442596b46026eb4c738130bce90421927/pluginlib/include/pluginlib/class_loader_imp.hpp#L302
• ROS2 : https://github.com/ros/pluginlib/blob/c2a604b220803a8fc22168935278e7b0e3d13a10/pluginlib/include/pluginlib/class_loader_imp.hpp#L397

[diegoferigo]

Using CMAKE_PREFIX_PATH is actually a good choice, at least for installed projects. In my opinion tho having a *_PLUGIN_PATH variable can be useful for developers whenever you want to use the build tree, just to make an example.

[traversaro]

Using CMAKE_PREFIX_PATH is actually a good choice, at least for installed projects.

I think it really depends on how the project is installed. If the plugin is installed by the .deb package in the /usr prefix for execution in a system that is not used for development, the need for setting CMAKE_PREFIX_PATH (in a system that may not have even CMake installed) seems a bit odd. I guess it could make sense to always (i.e. hardcode) the search in the relevant sub-directories of the path in CMAKE_SYSTEM_PREFIX_PATH (see https://github.com/Kitware/CMake/blob/0479ae492a6bc6058ae86d3ce01ac26fcc71c744/Modules/Platform/UnixPaths.cmake#L24 for the definition in *nix system).

[diegoferigo]

at least for installed projects

Yes, I meant installed from sources.

If the project is installed from a packaged archive, the plugin might go in something like /usr/lib/x86_64-linux-gnu/ or /usr/lib/x86_64-linux-gnu/<subfolder> as discussed in robotology/blockfactory#34, am I wrong? Also the latter case would not need any env variable to be set, since we could exploit an ld.so.conf-like configuration.

[traversaro]

Great, I did not realized this was already supported by https://github.com/robotology/blockfactory/pull/32/files#diff-ca9447c1aff58642a23f971df7d32a95R138 .

[drdanz]

Can we try to synchronize the versions of shlibpp that we have around? AFAIK there are 5 at the moment:

• robotology/shlibpp
• robotology/yarp
• robotology/robot-testing
• robotology/wbtoolbox
• robotology/block-factory

[diegoferigo]

@drdanz You can strikeout wb-toolbox since from the v5 version (current master) it uses the version vendored by blockfactory. Did you have a look by any chance to the improvements of robotology/blockfactory#32?

[traversaro]

I think the changes in robotology/blockfactory#32 are good to be submitted to https://github.com/robotology-playground/sharedlibpp .

[diegoferigo]

Regardless of the possibility to make shlibpp a dependency of yarp / rtf / etc, merging those features upstream would be a nice addition. Once upstream is aligned, we can add shlibpp to the superbuild, initially as blockfactory dependency.

cc @S-Dafarra is interested in this merge too

[drdanz]

Did you have a look by any chance to the improvements of robotology/blockfactory#32?

Sorry, I didn't have time yet... Maybe you could open a PR on shlibpp and we can discuss the changes there?

[traversaro]

cc @Nicogene as he may be interested in all the shlibpp-related discussion.

[traversaro]

A fix for this issue is provided in #393 . A "proper" fix for RTF will be tracked in robotology/robot-testing-framework#109 .