Saturation speedup

[t-brandt]

Resolves JP-3820

Closes #

This PR improves the performance of the saturation step, reducing its runtime by as much as a factor of 5-10 while leaving results unchanged. It does this primarily with three changes. First, when a pixel saturates, that saturation is propagated into all subsequent reads using a running boolean tally of pixels that have saturated. Second, ndimage.binary_dilation is very slow, an order of magnitude slower than writing the operations out explicitly. Finally, if there is no averaging in groups, there is no point in the diluted saturation check; this check is skipped if the dilution factor is 1.

Tasks

• update or add relevant tests
• update relevant docstrings and / or docs/ page
• Does this PR change any API used downstream? (if not, label with no-changelog-entry-needed)
  • write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see below for change types)
  • run regression tests with this branch installed ("git+https://github.com/<fork>/stcal@<branch>")
    • jwst regression test
    • romancal regression test

news fragment change types...

• changes/<PR#>.apichange.rst: change to public API
• changes/<PR#>.bugfix.rst: fixes an issue
• changes/<PR#>.general.rst: infrastructure or miscellaneous change

[codecov]

Codecov Report

Attention: Patch coverage is 92.68293% with 3 lines in your changes missing coverage. Please review.

Project coverage is 86.15%. Comparing base (68118a2) to head (5cecab4).
Report is 10 commits behind head on main.

Files with missing lines	Patch %	Lines
src/stcal/saturation/saturation.py	92.68%	3 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #331   +/-   ##
=======================================
  Coverage   86.14%   86.15%           
=======================================
  Files          50       50           
  Lines        9313     9332   +19     
=======================================
+ Hits         8023     8040   +17     
- Misses       1290     1292    +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[drlaw1558]

Just tested this empirically as well. Can confirm that the groupdq array is unaffected both for an ungrouped case (jw01864005001_03102_00001_mirimage), a grouped IRS2 case (jw01908004001_02101_00005_nrs1, used for previous testing of spacetelescope/jwst#8593), and a grouped NIRCam case (jw02756003001_03101_00001_nrcalong), with appreciable runtime gains in the long-integration limit (17 seconds to 3 seconds in the 292 group MIRI case here).

Also looks like the slow binary_dilation behavior has been a known issue for the last few years: scipy/scipy#13991

[drlaw1558]

Just checking in on the status of this PR?

[melanieclarke]

It looks like we lost track of this one, sorry! Is there anything more to do from your side, @t-brandt? If not, I can start regression tests.

[t-brandt]

I'm all set with this one, thanks @melanieclarke!

[tapastro]

RTs: https://github.com/spacetelescope/RegressionTests/actions/runs/13268527150

[melanieclarke]

Romancal tests here:
https://github.com/spacetelescope/RegressionTests/actions/runs/13270137990

Tagging @schlafly for review: it looks like this code is used in romancal.

[melanieclarke]

Both regression test runs are clean.

[schlafly]

Thanks!

[melanieclarke]

Tests are passing and we have approval from jwst and romancal - I will go ahead and merge now. Thanks all!