Clamp GetThreadLimit by the live OpenMP thread limit

[bnaras]

Summary

OpenFHEParallelControls::SetNumThreads(n) (and any other runtime
omp_set_num_threads(n)) is silently ignored by the library's parallel hot
regions: the process keeps using every core. This makes it impossible to cap
OpenFHE to fewer threads once the library is loaded.

The cause is a one-line issue in GetThreadLimit, and the fix is one line.

The bug

GetThreadLimit(n) returns min(n, machineThreads), where machineThreads is
latched once from omp_get_max_threads() at static-init (library load) and
never refreshed. Every parallel hot region is emitted as:

#pragma omp parallel for num_threads(OpenFHEParallelControls.GetThreadLimit(size))

Per the OpenMP spec, an explicit num_threads(...) clause overrides the
runtime nthreads ICV. So even after a caller sets the limit at runtime — via
the library's own OpenFHEParallelControls::SetNumThreads(), which calls
omp_set_num_threads() — GetThreadLimit still returns the stale
machineThreads, and the region runs on all cores anyway. There is no
in-process way to cap the library.

The fix

Also clamp by the live omp_get_max_threads():

int lim  = n > machineThreads ? machineThreads : n;
int live = omp_get_max_threads();
return lim > live ? live : lim;

The added call is a cheap ICV read. When the runtime limit is left at the
machine default, GetThreadLimit returns exactly what it did before — no
functional change to an uncapped build.

Reproduction (measured, not inferred)

Both builds below are stock upstream main (ed361af); the only difference is
this one-line patch. Same machine (16 cores), both call
OpenFHEParallelControls.SetNumThreads(2), and the reprex counts the threads
actually spawned by OpenFHE's exact num_threads(GetThreadLimit(size))
clause:

Build	runtime limit (ICV)	GetThreadLimit(32768)	threads actually spawned	50× CKKS EvalMult wall	cpu/wall
Stock	2	16	16 — cap ignored	2.22 s	5.09
Patched	2	2	2 — cap honored	4.87 s	1.89

The capped run is slower (4.87 s vs 2.22 s) precisely because it now uses 2
cores instead of 16 — the cap is real.

Zero-dependency reproduction of the mechanism

No OpenFHE needed — this replicates the exact num_threads(GetThreadLimit(n))
pattern and counts threads before/after the one-line change:

#include <omp.h>
#include <cstdio>
static int machineThreads;                       // latched once at "load"
static int upstream(int n){ return n > machineThreads ? machineThreads : n; }
static int patched (int n){ int l = n>machineThreads?machineThreads:n;
                            int live = omp_get_max_threads();
                            return l > live ? live : l; }
static int threads_used(int limit){ int used = 0;
  #pragma omp parallel num_threads(limit)
  { if (omp_get_thread_num()==0) used = omp_get_num_threads(); }
  return used; }
int main(){
  machineThreads = omp_get_max_threads();         // OpenFHE latches this at load
  omp_set_num_threads(2);                          // ask for 2 at runtime
  printf("ICV after cap(2) = %d\n", omp_get_max_threads());
  printf("UPSTREAM -> %d threads used  (cap ignored)\n", threads_used(upstream(4096)));
  printf("PATCHED  -> %d threads used  (cap honored)\n", threads_used(patched(4096)));
}

On a 16-core machine this prints UPSTREAM -> 16, PATCHED -> 2.

Full library-level reproduction

A self-contained program that drives the real library (calls
OpenFHEParallelControls.SetNumThreads(2), counts the threads OpenFHE's hot
region spawns, and runs a heavy CKKS EvalMult workload) is attached as
thread_cap_repro.cpp. Build it against a stock install and a patched install
and run both; the numbers above are its output.

Compatibility

• No API change; behavior is identical when the runtime limit is left at the
  machine default.
• The extra omp_get_max_threads() is a per-region ICV read. If its cost is a
  concern for very fine-grained regions, it can be cached with an explicit
  refresh hook — happy to adjust to your preference.

[bnaras]

Rebased onto dev — now a single commit against current dev. The change itself is unchanged (parallel.h, +15/−2).

[pascoec]

Thanks for the detailed report — your diagnosis of the mechanism is exactly right: the explicit num_threads(GetThreadLimit(...)) clause overrides the nthreads ICV, so runtime caps never reach the hot regions.

I reviewed your proposed fix, and there's a case it doesn't fully cover: omp_get_max_threads() reads the calling thread's ICV, which is thread-local. If an application sets the cap on its main thread but runs FHE work from a worker/pool thread, the live read on that worker still returns the machine default and the cap is ignored again. You can verify this by calling omp_set_num_threads(2) on the main thread and reading omp_get_max_threads() from a fresh std::thread — it reports the machine default.

I've opened PR #1233 with a different approach: OpenFHEParallelControls now keeps a process-global, atomically-read thread limit that SetNumThreads()/Enable()/Disable() update and every GetThreadLimit() clause consults, regardless of which thread enters the region — at zero per-region overhead. It also adds num_threads clauses to the regions that had none (matrix, matrixstrassen, dgsampling, CKKS scheme switching) and fixes several adjacent problems (Disable() being bypassed the same way, UnitTestStart()/UnitTestStop() clobbering caps, omp_set_num_threads(0) being reachable). Please review and confirm it resolves your use case.

One usage note: a bare omp_set_num_threads() at the application layer is still not sufficient to cap the library — the explicit num_threads clauses override the ICV per the OpenMP spec, and the ICV is per-thread besides. OpenFHEParallelControls.SetNumThreads() is the supported way to cap OpenFHE; it applies process-wide from any thread.

[pascoec]

One workflow request for future contributions: your PR came from a fork of openfhe-development, which meant I couldn't push additional commits or cherry-pick your changes into the PR branch directly. Next time, please create a branch in the openfhe-development repo itself rather than working from a fork — that lets us collaborate on the PR branch and add fixes on top of your commits instead of having to open a separate PR.

[pascoec]

If you don't have write access to the repo, checking 'Allow edits by maintainers' on the PR accomplishes the same thing.