Repository navigation
[RF] Consolidate the unbinned fit backend benchmarks - #315
Merged
Merged
Conversation
The legacy RooFit evaluation backend is deprecated and about to be
removed, so the benchmarks stop running and mentioning it:
* benchRooFitBackends: drop the *_FitLegacy benchmarks and the unused
run configurations. The plot script normalized all timings to the
legacy backend; it now shows the speedup of the CUDA backend relative
to the CPU backend.
* benchRooFitUnbinned: drop the *_FitLegacy benchmarks, including the
NumCPU variants.
* benchHistFactory: drop the BatchMode_OFF benchmarks.
* benchCodeSquashAD: drop the reference minimization with the legacy
NLL. The plot script parses the log by fixed line offsets, which
shift by one without the legacy NLL creation line. While verifying
the new offsets against a log, it turned out that the NLL creation
lines of the two codegen variants were assigned the wrong way round;
this is fixed as well.
* roofitAtlasHiggsBenchmark: drop the legacy NLL slot and the
commented-out benchmarks using it.
* run_benchmarks.sh, compare_benchmarks.py and the ATLAS benchmark
scripts and README no longer run, read or describe a legacy backend.
The benchmark names that still contain "BatchMode" are kept unchanged,
so that the time series of their results stay comparable.
Merge benchRooFitUnbinned into benchRooFitBackends, which now covers all
unbinned models with one protocol: the likelihood is created outside of
the timed loop, and only Migrad with strategy 0 is timed, after one
untimed warm-up fit and with a fresh minimizer for every fit.
* Models from both executables in one file, 100k events by default
(--events=N option), fixed random seed for the datasets, so that
results are reproducible.
* Benchmarks for the single-threaded CPU backend, the multi-threaded
CPU backend with all threads of the implicit MT pool, CUDA, and
codegen where it supports the model and ROOT has clad.
* Wall-clock time instead of CPU time, since the multi-threaded entry
would otherwise be reported as the sum over all threads.
* Counters for the number of likelihood evaluations, the Migrad status
and the minimum NLL, to check that the backends do the same work.
* The run information (CPU, GPU, ROOT version, RooBatchCompute library,
VDT, events, threads) goes into the benchmark context of the JSON
output.
* The plot script shows the speedup of the multi-threaded CPU and the
CUDA backend relative to the single-threaded CPU backend for all
models, as horizontal bars with a fixed axis range, together with
the run information. It uses the median when the benchmarks were run
with --benchmark_repetitions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge benchRooFitUnbinned into benchRooFitBackends, which now covers all unbinned models with one protocol: the likelihood is created outside of the timed loop, and only Migrad with strategy 0 is timed, after one untimed warm-up fit and with a fresh minimizer for every fit.