[Draft] Consolidating OpenACC device-host memory transfers #1307
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds an initial port of this routine using OpenACC. More changes are needed for performance and data management.
Ensures the fields which don't change while the model is running are present on the device from model startup to model shutdown.
Ensure that the other, non-invariant fields are available for this routine. Variables that are overwritten during this routine are only created while others are copied in. Any variables overwritten by this routine are copied out at the end. Timing for these transfers are reported in the output log file in the new timer: 'atm_bdy_adjust_scalars [ACC_data_xfer]'. Also add default(present) to parallel directives to ensure data movement is correct and prevent any implicit data movements from the compiler.
…ep_finish This commit provides an interim fix for a potential issue in limited area runs relating to the uninitialized garbage cells in the 1st time level of theta_m.
Keep consistency with explicit loop schedules
Remove newline, keep changes minimal
Add nVertLevels and derefernce integer pointers to loop bounds so they transfer to the GPU correctly. Also make loops in vertical dimension explicit for OpenACC parallel loop directives.
Since this routine is called before mpas_atm_dynamics_init during atm_core_init, these fields must also be transfered within mpas_reconstruct_2d routine. After mpas_atm_dynamics_init, these fields are not transferred during following uses of mpas_reconstruct_2d due to OpenACC present_or_copyin behavior.
This change allows data needed for the mpas_reconstruct_2d routine to be fetched onto the device (GPU) at the beginning and end of the routine. The time for these transfers are captured in a new timer 'mpas_reconstruct_2d [ACC_data_xfer]'.
To this point, results are bit-wise identical.
The changes in this PR haven't been tested, and doing so will require modifying specific namelist options and creating new baselines.
Default configuration gives bit-identical results.
There is some code that hasn't been tested because it requires non-default namelist settings to activate.
Some untested code has been ported.
…bdy_tend' into tmp_dycore_mem_move
|
Replaced by #1315 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Current work in progress of the effort to consolidate OpenACC device <-> host memory transfers to just before and after the call to
atm_srk3in the subroutineatm_timestepinmpas_atm_time_integration.F