Skip to content

Optimize ACC port of atm_advance_scalars_work:4857 - #1521

Draft
abishekg7 wants to merge 6 commits into
MPAS-Dev:developfrom
abishekg7:atmosphere/opt_advance_scalars_4857
Draft

abishekg7 wants to merge 6 commits into
MPAS-Dev:developfrom
abishekg7:atmosphere/opt_advance_scalars_4857

Conversation

@abishekg7

Copy link
Copy Markdown
Collaborator

This PR introduces optimizations for the OpenACC port of atm_advance_scalars_work:4857. It is a redo of #1476 with pre-processor macros so as to not degrade the CPU performance significantly. The table below lists the timings for a real, global 30km experiment on A100 GPU with the nvhpc.

For nvhpc gpu runs, -gpu=math_uniform is introduced as a build flag to ensure optimizations are bit-identical, and the we report the numbers using NV_ACC_TIME=1. The GPU runs are on 1 Derecho GPU node, using 1 A100 via 1 MPI task.

The numbers reported for the CPU runs with gnu and intel compilers use the newly-added timers local to this region, and are averaged across three runs. The CPU runs use a single derecho CPU node each, fully subscribed to 128 MPI tasks.

Version GPU kernel time (ms) CPU timer - gnu CPU timer - intel25
base 36   1.155987   1.0895
Opt 1 + macro swap the seq and vector collapse(2) loops 26.272   1.152207   1.09589
Opt 2 + macro, fuse all scalar_tend_column loops 23.4   1.127923   1.09153
Opt 3 + macro, scalar_tend_column can be replaced by a scalar local to each gang 22.365   1.14839   1.10588
Opt 4 + macro, remove loops to zero scalar_tend_column 17.57        

This work was completed in part at the NCAR/NLR/NOAA Open Hackathon, part of the Open Hackathons program. The authors would like to acknowledge OpenACC-Standard.org for their support.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant