[aliencaocao]

Parallelize the cell loops inside each IB patch's gang

Problem

Currently, s_apply_ib_patches_ib_parallelism had patch_id as its only parallel loop. For cases with few immersed bodies whose bounding boxes cover many cells, they leave the GPU nearly idle, because the code uses one thread per patch and every cell of every box is being processed serially by that thread.

What this does

The bounding-box cell loops (k, j, i in 3D; j, i in 2D) become a collapsed vector loop inside each patch's gang.

grid_cell_parallelism (the default for static patches) already parallelizes the cell loops and is untouched.

Measurements

Benchmark: https://github.com/MFlowCode/MFC/blob/master/examples/2D_mibm_particle_cloud/case.py
(many_ib_patch_parallelism=T), scaled to 7000x7000 on 1xH200.

	master	this PR	Δ
grind (ns/gp/eq/rhs)	1.939	1.443	-25.6%

Validation

On NVHPC 26.9 with OpenACC, taking the saved files on the 20th step and comparing against master:

• conservative field dumps (741M words): bit-identical.
• IB kinematic state (ib_state_20.dat): 74/400 words differ by one ULP due to floating-point ordering, rest bit-identical.

On AMD OpenMP and CPU this code change is a no-op. On NVHPC OpenMP, it just add a !$omp loop bind(thread) which is also a no-op compared to no directive (current).

Contribution Policy

All contributions are expected to demonstrate:

• A clear understanding of the codebase
• Alignment with product direction
• Thoughtful reasoning behind changes
• Evidence of real-world usage or hands-on experience with the problem

If these expectations are not met, we would prefer to implement the changes ourselves rather than spend time reviewing low-effort submissions.

Acknowledgment

This PR is cherry-picked from #2093 and is contributed by Nanyang Technological University Team Supernova for SC26 Student Cluster Competition.

[github-actions]

Lines of Code

File	Lines	Diff
src/simulation/m_ib_patches.fpp	644	+4

Directory	Lines	Diff
simulation	28535	+4
total	47594	+4

[github-actions]

Claude Code Review

Head SHA: 483ce11

Files changed:

• 1
• src/simulation/m_ib_patches.fpp

Findings:

• medium src/simulation/m_ib_patches.fpp (3D and 2D branches of s_apply_ib_patches_ib_parallelism, new GPU_LOOP(collapse=3|2, parallelism='[vector]', private=...)): The change makes the outer patch_id loop gang-only and puts a vector-parallel collapsed nest inside it. Confidence is moderate, because I could not run it on a GPU.
  • On the OpenMP backend, OMP_LOOP in omp_macros.fpp drops collapse, parallelism and private. It emits only !$omp loop bind(thread) on NVHPC/PGI and nothing on Cray/AMD. Under OpenMP, the new inner loop is therefore not collapsed, and the private list is ignored. The per-cell temporaries xyz_local, length, radius, airfoil_id, model_id and eta are then private only through the outer GPU_PARALLEL_LOOP clause, which is per gang or team. If the OpenMP lowering ever runs several threads per team over the inner loop, those scalars and arrays are shared between threads, which is a data race. Please confirm on the OpenMP offload builds (Cray and AMD) that the results are unchanged.
  • I found no other use of GPU_LOOP(collapse=N, parallelism='[vector]', ...) nested in a gang-only GPU_PARALLEL_LOOP under src/. docs/documentation/gpuParallelization.md should be checked for this pattern. For OpenACC, verify that privatizing the same variables in both the parallel loop and the inner loop vector clause is accepted by NVHPC and Cray.
  • The ib_markers%sf writes use an atomic max, so the vector-level parallelism is safe for the markers. The loop bounds il..kr and center are computed at gang level and are only read inside the nest, which is fine.
  • Suggested fix: validate on both OpenACC and OpenMP GPU builds, with the IB tests including multi-patch overlap cases, before merging. If OpenMP is unsupported for this pattern, guard it or pass extraOmpArgs.
• medium src/simulation/m_ib_patches.fpp (line 220, GPU_PARALLEL_LOOP in the 3D branch): The added parallelism='[gang]' pushes the continuation line to about 150 characters. Most of the file wraps at 130. Re-wrap this line, and run ./mfc.sh precheck to catch any formatter or lint complaint.