Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failure of ufs-weather-model on hercules and orion (probably general) with MAPL/2.40.3 and non spack-stack dependencies. likely build issue of dependencies or MAPL but an issue has been requested #3375

Open
GeorgeVandenberghe-NOAA opened this issue Jan 29, 2025 · 7 comments
Open

Failure of ufs-weather-model on hercules and orion (probably general) with MAPL/2.40.3 and non spack-stack dependencies. likely build issue of dependencies or MAPL but an issue has been requested #3375

GeorgeVandenberghe-NOAA opened this issue Jan 29, 2025 · 7 comments
Assignees
@mathomp4 @tclune @bena-nasa

Comments

@GeorgeVandenberghe-NOAA
Copy link

I believe I am missing something basic and this is not a MAPL defect but user error but I was asked to create an issue when I inquired
about it.

When ufs-weather-model is built with a non spack-stack alternate stack it builds but fails in MAPL at runtime with this trace

this is the error traceback I get

960:
756: forrtl: severe (122): invalid attempt to assign into a pointer that is not associated
756: Image PC Routine Line Source
756: gwv.x 00000000068B54CF Unknown Unknown Unknown
756: gwv.x 0000000002187806 mapl_historygridc 2355 MAPL_HistoryGridComp.F90
756: gwv.x 0000000000B6AF52 Unknown Unknown Unknown
756: gwv.x 0000000000B6EA9F Unknown Unknown Unknown
756: gwv.x 0000000000CF4F3A Unknown Unknown Unknown
756: gwv.x 0000000000CE147B Unknown Unknown Unknown
756: gwv.x 0000000000B6C38A Unknown Unknown Unknown
756: gwv.x 000000000059A2D0 Unknown Unknown Unknown
756: gwv.x 0000000000871381 Unknown Unknown Unknown
756: gwv.x 00000000022B139F mapl_genericmod_m 1896 MAPL_Generic.F90
756: gwv.x 0000000000B6AF52 Unknown Unknown Unknown
756: gwv.x 0000000000B6EA9F Unknown Unknown Unknown
756: gwv.x 0000000000CF4F3A Unknown Unknown Unknown
756: gwv.x 0000000000CE147B Unknown Unknown Unknown
756: gwv.x 0000000000B6C38A Unknown Unknown Unknown
756: gwv.x 000000000059A2D0 Unknown Unknown Unknown
756: gwv.x 0000000000871381 Unknown Unknown Unknown
756: gwv.x 000000000214CF4F mapl_capgridcompm 708 MAPL_CapGridComp.F90
756: gwv.x 0000000002151D2A mapl_capgridcompm 654 MAPL_CapGridComp.F90
756: gwv.x 0000000000B6AF52 Unknown Unknown Unknown
756: gwv.x 0000000000B6EA9F Unknown Unknown Unknown
756: gwv.x 0000000000CF4F3A Unknown Unknown Unknown
756: gwv.x 0000000000CE147B Unknown Unknown Unknown
756: gwv.x 0000000000B6C38A Unknown Unknown Unknown
756: gwv.x 000000000059A2D0 Unknown Unknown Unknown
756: gwv.x 0000000000871381 Unknown Unknown Unknown
756: gwv.x 000000000214A7C0 mapl_capgridcompm 955 MAPL_CapGridComp.F90
756: gwv.x 0000000001E909E7 aerosol_cap_mp_mo 348 Aerosol_Cap.F90
756: gwv.x 0000000000C7D5C8 Unknown Unknown Unknown
756: gwv.x 0000000000C7D533 Unknown Unknown Unknown
756: gwv.x 0000000000C7DBC4 Unknown Unknown Unknown
756: gwv.x 00000000004287FB Unknown Unknown Unknown
756: gwv.x 0000000006784830 Unknown Unknown Unknown
756: gwv.x 0000000000B6AF52 Unknown Unknown Unknown
756: gwv.x 0000000000B6EA9F Unknown Unknown Unknown
756: gwv.x 0000000000CF514A Unknown Unknown Unknown
756: gwv.x 0000000000CE147B Unknown Unknown Unknown
756: gwv.x 0000000000B6C38A Unknown Unknown Unknown
756: gwv.x 000000000059A2D0 Unknown Unknown Unknown
756: gwv.x 0000000000871381 Unknown Unknown Unknown
756: gwv.x 0000000000AF45F0 Unknown Unknown Unknown
756: gwv.x 0000000000B0A262 Unknown Unknown Unknown
756: gwv.x 0000000000B0D2F2 Unknown Unknown Unknown
756: gwv.x 0000000000B25B85 Unknown Unknown Unknown
756: gwv.x 0000000000B6AF52 Unknown Unknown Unknown
756: gwv.x 0000000000B6EA9F Unknown Unknown Unknown
756: gwv.x 0000000000CF4F3A Unknown Unknown Unknown
756: gwv.x 0000000000CE147B Unknown Unknown Unknown
756: gwv.x 0000000000B6C38A Unknown Unknown Unknown
756: gwv.x 000000000059A2D0 Unknown Unknown Unknown
756: gwv.x 0000000000871381 Unknown Unknown Unknown
756: gwv.x 0000000000420F4E MAIN__ 392 UFS.F90
756: gwv.x 000000000041FD7D Unknown Unknown Unknown
756: libc.so.6 000014D6CF6FCEB0 Unknown Unknown Unknown
756: libc.so.6 000014D6CF6FCF60 __libc_start_main Unknown Unknown
756: gwv.x 000000000041FC95 Unknown Unknown Unknown
740: forrtl: severe (122): invalid attempt to assign into a pointer that is not associated
740: Image PC Routine Line Source
740: gwv.x 00000000068B54CF Unknown Unknown Unknown
740: gwv.x 0000000002187806 mapl_historygridc 2355 MAPL_HistoryGridComp.F90
740: gwv.x 0000000000B6AF52 Unknown Unknown Unknown
740: gwv.x 0000000000B6EA9F Unknown Unknown Unknown
740: gwv.x 0000000000CF4F3A Unknown Unknown Unknown
740: gwv.x 0000000000CE147B Unknown Unknown Unknown

When I use the\ /mapl/2.40.3-esmf-8.6.0 (I.E. "Their" mapl) it works.

I build MAPL this way ($PREFIX contains all dependencies in $PREFIX/lib $PREFIX/lib64 and $PREFIX/include and it works for most cases)

cd MAPL-2.40.3
export ZLIB_ROOT=$PREFIX
export PNG_ROOT=$PREFIX
export PIO_ROOT=$PREFIX
export YAFYAML_ROOT=$PREFIX
export jasper_ROOT=$PREFIX

export ESMFMKFILE=$PREFIX/ESMF_8_6_0/lib/esmf.mk
export ESMA_CMAKE_ROOT=$PREFIX
export ECBUILD_ROOT=$PREFIX
export CMAKEMODULES_ROOT=$PREFIX
export NETCDF=$PREFIX
export prefix=$PREFIX
export NetCDF_ROOT=$PREFIX
set -x
export FC=$FC
export CC=$CC
export CXX=$CXX

[[ -d build ]] && rm -rf build
mkdir -p build && cd build

CMAKE_OPTS=""

export PATH=$NETCDF/bin:$PATH
export NTHREADS=4

#source $HOME/envset
cmake ..
-DCMAKE_INSTALL_PREFIX=$NETP/MAPL-2.40.3 -DUSE_F2PY=OFF -DCMAKE_Fortran_COMPILER=$FC
-DCMAKE_MODULE_PATH="${ESMA_CMAKE_ROOT};${CMAKEMODULES_ROOT}/Modules;${ECBUILD_ROOT}/share/ecbuild/cmake"
-DCMAKE_BUILD_TYPE=Release
-DBUILD_WITH_FLAP=OFF
-DBUILD_WITH_PFLOGGER=OFF
-DBUILD_WITH_FARGPARSE=ON
-DESMA_USE_GFE_NAMESPACE=ON
-DBUILD_SHARED_MAPL=OFF
-DUSE_EXTDATA2G=OFF
${CMAKE_OPTS}

gmake -j${NTHREADS:-4} install VERBOSE=1

The compiler is intel

Currently Loaded Modules:

  1. intel-oneapi-compilers/2023.1.0 3) intel-oneapi-mpi/2021.9.0 5) nghttp2/1.57.0 7) curl/8.4.0 9) ubasic
  2. stack-intel/2021.9.0 4) stack-intel-oneapi-mpi/2021.9.0 6) zlib/1.2.13 8) cmake/3.23.1
@tclune
Copy link
Collaborator

tclune commented Jan 30, 2025

@bena-nasa is the real expert so I expect we'll hear something from them by late morning. I took a quick look at the relevant lines, but cannot say much more than there is a pointer involved, so the run-time error might be accurate.

My expectation is that MAPL should have trapped this rather than the compiler, so while @GeorgeVandenberghe-NOAA may indeed have "misused" MAPL/History, there is likely a bug in MAPL that simply has not been detected before.

@bena-nasa
Copy link
Collaborator

bena-nasa commented Jan 31, 2025 •
edited
Loading

I did not see anything obvious when looking at the lines in the code, I do have access to Hercules so I could certainly try to reproduce and debug with enough instruction.

@GeorgeVandenberghe-NOAA
Copy link
Author

@bena-nasa
Copy link
Collaborator

good hear, almost had to be a compiler issue bug, etc ...

@GeorgeVandenberghe-NOAA
Copy link
Author

@GeorgeVandenberghe-NOAA
Copy link
Author

GeorgeVandenberghe-NOAA commented Feb 2, 2025 via email

@GeorgeVandenberghe-NOAA
Copy link
Author

GeorgeVandenberghe-NOAA commented Feb 6, 2025 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mathomp4 mathomp4

@tclune tclune

@bena-nasa bena-nasa

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@mathomp4 @tclune @bena-nasa @GeorgeVandenberghe-NOAA