New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Paraview recipe #39

Open

biddisco wants to merge 53 commits into eth-cscs:main from biddisco:paraview

biddisco commented Aug 23, 2023

This PR build a minimal paraview using EGL on clariden.
It requires stackinator patch PR
eth-cscs/stackinator#129

biddisco force-pushed the paraview branch from 0af747a to 0688c04 Compare

August 23, 2023 09:03

biddisco force-pushed the paraview branch from 96e8c61 to 98362e9 Compare

September 5, 2023 07:51

jfavre commented Feb 7, 2024

installation on ALPS will require two additional items for run-time enabling of the NVIDIA IndeX libraries and license.

The lib package can be found in https://www.paraview.org/files/dependencies/nvidia-index-libs-5.12.0.20231121-linux-aarch64.tar.bz2. untar-ing the file can be done anywhere (presumably under the ParaView install tree), as long as the lib is found in LD_LIBRARY_PATH.
Access and installation of the license file (which is obtained by private request to NVIDIA using my email) is documented at https://confluence.cscs.ch/display/VCUE/NVIDIA+IndeX. Thus, the environment variable NVINDEX_PVPLUGIN_HOME shall be set where the file nvindex_config.xml gets installed.

biddisco force-pushed the paraview branch from 98362e9 to 41e09f6 Compare

February 26, 2024 22:01

RMeli added app paraview labels

biddisco force-pushed the paraview branch from 41e09f6 to 14406cb Compare

August 30, 2024 13:09

Author

biddisco commented Aug 30, 2024

The PR can now be considered for merging. We support

eiger: zen2 : 
  paraview with OSMESA offscreen headless rendering, 
  python, 
  CDI for icon support, 
  raycasting with TBB  
  VTK-m filters using TBB
  + lots lots more

daint: gh200 : 
  paraview with EGL headless accelerated rendering, 
  nvidia index plugin, 
  python, 
  CDI support for icon
  TBB and raycasting
  CUDA enabled VTK-m filters 
  + more

Member

simonpintarelli commented Aug 30, 2024

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.0

Member

simonpintarelli commented Aug 30, 2024

cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.0

1 similar comment

Member

simonpintarelli commented Sep 3, 2024

cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.0

Member

simonpintarelli commented Sep 3, 2024

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.0

4 similar comments

Member

simonpintarelli commented Sep 3, 2024

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.0

Author

biddisco commented Sep 3, 2024

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.0

Author

biddisco commented Sep 3, 2024

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.0

Member

simonpintarelli commented Sep 3, 2024

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.0

Member

simonpintarelli commented Sep 3, 2024

cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.0

Member

simonpintarelli commented Sep 3, 2024

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.0

biddisco force-pushed the paraview branch from f50b5e0 to f4e8ed0 Compare

November 20, 2024 08:51

Author

biddisco commented Nov 20, 2024

Please test this and consider it for deployment on our clusters

Member

simonpintarelli commented Nov 20, 2024

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.0

Member

simonpintarelli commented Nov 20, 2024

cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.0

Author

biddisco commented Nov 20, 2024

cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.1

Author

biddisco commented Nov 20, 2024

please note, paraview release has been bumped from 5.13.0 to 5.13.1

Member

simonpintarelli commented Nov 20, 2024

cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.1

Member

simonpintarelli commented Nov 20, 2024

cscs-ci run alps;system=todi;uarch=gh200;uenv=paraview:5.13.1

Member

simonpintarelli commented Nov 25, 2024

cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.1

Member

simonpintarelli commented Nov 25, 2024

cscs-ci run alps;system=todi;uarch=gh200;uenv=paraview:5.13.1

Member

simonpintarelli commented Nov 25, 2024

I've just tried to build it manually on eiger, there is an error compiling h5hut on zen2 too.

simonpintarelli self-requested a review

November 25, 2024 20:38

biddisco and others added 13 commits

January 10, 2025 08:04


          Add cluster name to RC launch script and cleanup script comments etc

c195036


          delete old/unused c2sm libs

723e1ab


          Rename todi to daint

59f9ef3


          change directory structure

b843e37


          update config.yaml

a8e2ca0


          Fix build script paths after version added to recipe location

457a395


          Set spack commit to recent SHA and get paraview source from jfrog

5d2e0ae


          update paraview to 5.13.1, add adios2, h5py and conduit

e986e1f


          Clean up build-env for plugins, add h5py for pv-python

93c493a


          Remove h5hut from gh200 recipe

f29043e


          Remove ispc customizations and set spack develop tag

b0ede57


          fix for spack@develop

f53c3b5


          paraview: cleanup views in environment setup

7ed8d62

biddisco force-pushed the paraview branch from 9e9107a to a3f261d Compare

January 15, 2025 10:14

Author

biddisco commented Jan 15, 2025 •

edited

Loading

cscs-ci run alps;system=todi;uarch=gh200;uenv=paraview:5.13.2

Author

biddisco commented Jan 15, 2025

Help @simonpintarelli - CI seems to fail with

['#SBATCH --partition=normal', '#SBATCH --reservation=icon', '#SBATCH --time=180', '#SBATCH --account=csstaff']
2025-01-15 16:15:59.256924: Exception while waiting submitting job to queue. Exception=last request: 200 {'tasks': {'4342e4b82769ac9897fce85bc64e74e9': {'created_at': '2025-01-15T15:15:29', 'data': 'sbatch: error: Batch job submission failed: Requested reservation is invalid', 'description': 'Finished with errors

it is using a reservation for icon on machine todi - I don't know where this is set or how to change it.
where is the CI info stored?

Author

biddisco commented Jan 16, 2025

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.2

1 similar comment

Member

simonpintarelli commented Jan 16, 2025

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.2

Author

biddisco commented Jan 21, 2025

Several users have requested a uenv for paraview. What do I need to do to get this accepted as a valid solution for users?

simonpintarelli approved these changes

View reviewed changes

Member

simonpintarelli left a comment

lgtm

recipes/paraview/5.13.0/zen2/config.yaml Outdated Show resolved Hide resolved

recipes/paraview/5.13.1/gh200/environments.yaml Outdated Show resolved Hide resolved

recipes/paraview/5.13.1/gh200/post-install Outdated

+              # remove /dev/shm paths from build-env that will not be available to the final user
+              GCCLOC=$($SPACK_CMD location -i gcc@12)
+              sed -i "s|/dev/shm/biddisco/spack/lib/spack/env/gcc|$GCCLOC/bin|g"  $BUILD_ENV

Member

simonpintarelli Nov 27, 2024

Is this causing the incorrect paths in activate.sh?

Author

biddisco Feb 4, 2025

No. All envs contain stuff like

export SPACK_ENV=/dev/shm/jenkssl/cp2k/2024.3/1502220409/environments/cp2k;

this is there to ensure that when a user wants to build a plugin against the installed paraview in the uenv, the paths are setup correctly for cmake to find things. A build-env generated during stackinator build has /dev/shm paths set if that was the build location, this replaces them to make the uenv usable for custom development

Member

simonpintarelli commented Jan 21, 2025

cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.2

biddisco added 4 commits

February 4, 2025 11:59


          paraview: update 5.13.1 -> 5.13.2

16d6b05


          Merge remote-tracking branch 'origin/main' into paraview

a9ee9c7


          Improve user auto-config scripts for rc-server setup etc

25c5cbf


          Update zen2 recipe with recent changes to install scripts

a5bd3d9

biddisco force-pushed the paraview branch from 26ac004 to a5bd3d9 Compare

February 4, 2025 11:03

Author

biddisco commented Feb 4, 2025 •

edited

Loading

cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.2
cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.2


          Cleanup commits to ensure we always use a fixed version/commit

3dce605

biddisco force-pushed the paraview branch from a9f3865 to 3dce605 Compare

February 4, 2025 13:16

Author

biddisco commented Feb 4, 2025

cscs-ci run alps;system=daint;uarch=gh200;uenv=paraview:5.13.2
cscs-ci run alps;system=eiger;uarch=zen2;uenv=paraview:5.13.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels