Skip to content

FillBoundaryAux ignores m_safe_guard_cells #6659

@WeiqunZhang

Description

@WeiqunZhang

Summary

  • WarpX::FillBoundaryAux(int lev, amrex::IntVect) (Parallelization/WarpXComm.cpp:1121-1132) always forwards the caller-supplied ng to ablastr::utils::communication::FillBoundary.
  • Every other FillBoundary helper first checks m_safe_guard_cells and upgrades the requested ng to the full MultiFab::nGrowVect() when safe-guard mode is active, ensuring all allocated guard cells stay synchronized during debugging runs.
  • Because the aux helper skipped that copy-pasted guard, enabling warpx.safe_guard_cells=1 still leaves Efield_aux/Bfield_aux guard regions partially stale, defeating the point of the safety mode and making aux gathers behave differently from the main E/B grids.

Impact

  • Developers rely on m_safe_guard_cells to detect missing guard exchanges; aux grids now silently skip that enforcement, so particle gathers can still read garbage on the aux MultiFabs even though the rest of the code is running in "safe" mode.
  • Diagnostics and solvers that alias Efield_aux see different guard widths refreshed compared to Efield_fp, which is confusing and makes it harder to compare auxiliary grids against their sources when debugging guard overruns.

Suggested patch

--- a/Parallelization/WarpXComm.cpp
+++ b/Parallelization/WarpXComm.cpp
@@
 WarpX::FillBoundaryAux (int lev, IntVect ng)
 {
     ablastr::fields::MultiLevelVectorField Efield_aux = m_fields.get_mr_levels_alldirs(FieldType::Efield_aux, finest_level);
     ablastr::fields::MultiLevelVectorField Bfield_aux = m_fields.get_mr_levels_alldirs(FieldType::Bfield_aux, finest_level);
  
     const amrex::Periodicity& period = Geom(lev).periodicity();
-    ablastr::utils::communication::FillBoundary(*Efield_aux[lev][0], ng, WarpX::do_single_precision_comms, period);
-    ablastr::utils::communication::FillBoundary(*Efield_aux[lev][1], ng, WarpX::do_single_precision_comms, period);
-    ablastr::utils::communication::FillBoundary(*Efield_aux[lev][2], ng, WarpX::do_single_precision_comms, period);
-    ablastr::utils::communication::FillBoundary(*Bfield_aux[lev][0], ng, WarpX::do_single_precision_comms, period);
-    ablastr::utils::communication::FillBoundary(*Bfield_aux[lev][1], ng, WarpX::do_single_precision_comms, period);
-    ablastr::utils::communication::FillBoundary(*Bfield_aux[lev][2], ng, WarpX::do_single_precision_comms, period);
+    auto fill_aux = [this, &period, &ng](amrex::MultiFab* mf)
+    {
+        const amrex::IntVect& nghost = (m_safe_guard_cells) ? mf->nGrowVect() : ng;
+        ablastr::utils::communication::FillBoundary(*mf, nghost, WarpX::do_single_precision_comms, period);
+    };
+    fill_aux(Efield_aux[lev][0]);
+    fill_aux(Efield_aux[lev][1]);
+    fill_aux(Efield_aux[lev][2]);
+    fill_aux(Bfield_aux[lev][0]);
+    fill_aux(Bfield_aux[lev][1]);
+    fill_aux(Bfield_aux[lev][2]);
 }

Prepared by Codex

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions