r.geomorphon: parallelize with OpenMP using memory-bounded banding - #7783
r.geomorphon: parallelize with OpenMP using memory-bounded banding#7783krcoder123 wants to merge 9 commits into
Conversation
github-actions
Bot
added
raster
|
I attached a benchmarking script in this module as well and I will post my benchmark graphs once the full run finishes on my end. I'll make the pytest for this module in a separate PR this week. |
|
Here are the scaling graphs from my machine (8 core Apple M3, 4 performance and 4 efficiency cores). The speedups and the scalings look pretty good. One note, I had to use a Mac specific benchmark wrapper with cooldowns and pmset because my Mac was throttling a lot using the universal benchmark that's in this PR. So the below images of my graphs are with that setup, but if you would like to also see the throttling graphs done with the universal benchmark I can attach those too. 
|
…stead of module globals
The output is processed in bands sized from a new memory option, with the input strip loaded through per thread descriptors and each band's rows computed in parallel. Output is byte identical to the serial module at every thread count across all twelve outputs, both test projections, the memory range, and the option matrix. A new nprocs option matches the other parallel modules.
…mentation Also removes the stale one-off region-size warning; the one-off path now reads only the window rows.
c3a0552 to
2a92e4f
Compare
|
|
||
| /* Compute one cell's pattern and landform, shared by the raster and one-off | ||
| * paths. */ | ||
| static void compute_forms(FCELL **rows, int cur_row, int row, int col, |
There was a problem hiding this comment.
It would be better to move this function below main() and only keep its prototype here.
There was a problem hiding this comment.
Consider restructuring this with one or more structures.
There was a problem hiding this comment.
I moved compute_forms below main now and kept the prototype at the top. I restructured the parameters into a config struct and a result struct.
|
The parallel pytest cases are written and I tested it locally against this branch. But, it needs the conftest.py file which is already in #7785. Committing with a duplicate here doesn't make sense so I'll wait for that PR to merge first. I'll then rebase and add them here, so the test files don't get duplicated across the two PRs. |
|
|
||
| /* Compute one cell's pattern and landform, shared by the raster and one-off | ||
| * paths. */ | ||
| static void compute_forms(FCELL **rows, int cur_row, int row, int col, |
There was a problem hiding this comment.
Consider restructuring this with one or more structures.
| memory = atoi(par_memory->answer); | ||
| nprocs = G_set_omp_num_threads(par_nprocs); | ||
| nprocs = Rast_disable_omp_on_mask(nprocs); | ||
| if (nprocs < 1) |
There was a problem hiding this comment.
This is dead, nprocs won't be < 1 after G_set_omp_num_threads
| strip_rows = strip_hi - strip_lo + 1; | ||
| for (k = 0; k < strip_rows; ++k) | ||
| strip_ptr[k] = strip_block + (size_t)k * ncols; | ||
| load_strip(elevation.fd, elevation.raster_type, tmp_buf, strip_ptr, |
There was a problem hiding this comment.
elevation.fd is opened by open_map and then unused in the raster path, which opens nprocs more descriptors. Reuse it as fd_thread[0].
There was a problem hiding this comment.
Thanks for the suggestion, fd_thread[0] now reuses elevation.fd. This also fixed the descriptor never being closed on the raster path.
3 participants
This PR parallelizes r.geomorphon with OpenMP while keeping memory usage within the amount the user allows.
The actual parallelization method is similar to r.param.scale parallelization in theory. The output is processed in bands, with the band height computed from a new memory option. For each band the input rows are read in parallel with per thread file descriptors into a shared strip, and each band's rows are then computed in parallel. Cells near the map edges see their search window positioned exactly the way the original code positions its row buffer there, which is what allows the output to be exactly the output serial produced.
Output is exactly the same as the current serial module at 1, 2, 4, and 8 threads, across all twelve output maps, on projected and lat-lon data, through the whole memory range, with masks, and on a 100 million cell map. On my Apple M3 machine with 8 cores, I got a speedup of about 1.9x at 2 threads, 3.3 to 3.7x at 4, and 4.2 to 5.1x at 8.