Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider Spack Manifest Requirements #195

Open
CodeGat opened this issue Feb 20, 2025 · 3 comments
Open

Consider Spack Manifest Requirements #195

CodeGat opened this issue Feb 20, 2025 · 3 comments
Assignees
Labels
CI 2.0 Relating to the second iteration of build-ci priority:high A high priority issue - has an impact on core functionality type:investigation look into options, do some research!

Comments

@CodeGat
Copy link
Member

CodeGat commented Feb 20, 2025

The spack manifest, or spack.yaml, will have necessarily less requirements than the ones imposed on build-cd (the pre/release deployment pipeline to a HPC), as it is meant to be a highly customizable, multi-spec manifest that is used for testing.

However, there may still be constraints that we need to establish to enable concretization/finding of executables that may have similar paths.

@CodeGat CodeGat added CI 2.0 Relating to the second iteration of build-ci priority:high A high priority issue - has an impact on core functionality type:investigation look into options, do some research! labels Feb 20, 2025
@CodeGat CodeGat self-assigned this Feb 20, 2025
@github-project-automation github-project-automation bot moved this to New Issues 🌅 in Build-ci 2.0 Feb 20, 2025
@CodeGat CodeGat moved this from New Issues 🌅 to In Progress 🏗️ in Build-ci 2.0 Feb 20, 2025
@CodeGat
Copy link
Member Author

CodeGat commented Feb 21, 2025

For example, if we are installing specs under the same environment that have the same name (for example, different installations of mom5 with differing compilers), we need to update both the concretization strategy to at least spack.concretizer.unify:when_possible, and have custom views that get past the following error:

==> Updating view at /opt/environments/mom5-when-possible/.spack-env/view
==> Error: The environment view in /opt/environments/mom5-when-possible/.spack-env/view could not be created, because the following two specs project to the same prefix:
    mom5@=access-om2%intel@=2021.2.0~access-gtracers~deterministic~optimisation_report+restart_repro build_system=makefile type=ACCESS-OM arch=linux-rocky8-x86_64, and
    mom5@=access-om2%intel@=2021.4.0~access-gtracers~deterministic~optimisation_report+restart_repro build_system=makefile type=ACCESS-OM arch=linux-rocky8-x86_64.
    To resolve this issue:
        a. use `concretization:unify:true` to ensure there is only one package per spec in the environment, or
        b. disable views with `view:false`, or
        c. create custom view projections.

The following snippet installs the example correctly:

spack:
  definitions:
    - mom5s:
        - matrix:
          - ['mom5']
          - ['%[email protected] target=x86_64', '%[email protected] target=x86_64']
  specs:
    - $mom5s
  view:
    default:
      # The root may have to be disambiguated by environment name, or maybe removed? (the default is $SPACK_ROOT/../environments/ENV/.spack-env/view
      root: $SPACK_ROOT/../views
      projections:
        mom5: '{name}/{compiler.name}-{compiler.version}'
  concretizer:
    # Due to different compilers under the same source root
    unify: when_possible

@CodeGat
Copy link
Member Author

CodeGat commented Mar 12, 2025

Look into setting the above concretizer and view at a higher configuration scope.
We will need a symbol to refer to the commit hash in the pull request, or alternatively inject the whole thing at pre-runtime time. Leaning towards injection at runtime.

@CodeGat
Copy link
Member Author

CodeGat commented Mar 12, 2025

Tested out using view: false, and I think that we get good enough traction on package paths with spack find --format '{name}@{version}%{compiler.name}@{compiler.version} {prefix}' SPEC rather than having custom ones. Going to test injecting these values in the above PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI 2.0 Relating to the second iteration of build-ci priority:high A high priority issue - has an impact on core functionality type:investigation look into options, do some research!
Projects
Status: In Progress 🏗️
Development

No branches or pull requests

1 participant