Skip to content

fix: close streams in LoggerContextAdmin.setConfigLocationUri - #4218

Open
SebTardif wants to merge 6 commits into
apache:2.xfrom
SebTardif:fix/f005-jmx-config-stream-leak
Open

fix: close streams in LoggerContextAdmin.setConfigLocationUri#4218
SebTardif wants to merge 6 commits into
apache:2.xfrom
SebTardif:fix/f005-jmx-config-stream-leak

Conversation

@SebTardif

@SebTardif SebTardif commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

LoggerContextAdmin.setConfigLocationUri opens a FileInputStream or URL stream and builds ConfigurationSource(InputStream, File/URL). That constructor documents that the caller owns the stream.

Built-in factories (XmlConfiguration, JsonConfiguration, PropertiesConfigurationFactory) close getInputStream() when they consume it. That is factory-side cleanup, not a substitute for the caller contract: if a factory path never reads the stream (or returns before consuming it), the descriptor can leak. Same ownership theme as #4127.

Evidence

Leak path (why close is still needed)

Xml/JSON/properties factories close streams they consume. The remaining risk is caller ownership: open stream → hand to ConfigurationSource → factory that never calls getInputStream() (or fails before consume) → stream stays open. Defensive fix is try-with-resources on the caller side while keeping a stream-backed source so resetInputStream() / monitorInterval still re-open the file (buffering into data would short-circuit that and keep stale config; thanks @ramanathan1504).

Fix

try (final InputStream in = new FileInputStream(configFile)) {
  final ConfigurationSource configSource = new ConfigurationSource(in, configFile);
  final Configuration config = ConfigurationFactory.getInstance().getConfiguration(loggerContext, configSource);
  loggerContext.start(config);
}

Same pattern for the URL branch. Double-close after XmlConfiguration's finally is safe.

Red-green

export JAVA_HOME=$(/usr/libexec/java_home -v 17)
./mvnw -pl log4j-core,log4j-core-test -am test \
  -Dtest=LoggerContextAdminSetConfigLocationUriTest \
  -Dsurefire.failIfNoSpecifiedTests=false

Red (production try-with-resources removed, factory that never consumes stream):

AssertionFailedError: expected: <173> but was: <177>  // open FD count

Green (with try-with-resources): 4 tests, 0 failures.

Summary

  • Keep stream-backed ConfigurationSource (no full buffer)
  • try-with-resources around load for file and URL
  • Regression test with non-consuming factory + FD/delete check
  • Changelog entry

ConfigurationSource(InputStream, File/URL) leaves stream ownership with the
caller. Buffer configuration bytes into a Source-backed ConfigurationSource
and close the FileInputStream/URL stream before ConfigurationFactory runs so
failed or successful JMX reconfigure paths do not leak descriptors.

Signed-off-by: Sebastien Tardif <sebtardif@ncf.ca>
Signed-off-by: Sebastien Tardif <sebtardif@ncf.ca>

@ramanathan1504 ramanathan1504 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SebTardif thanks for this one, the missing close() is real, I could follow it in the code.

I pulled it down and ran it locally (./mvnw test -pl log4j-core,log4j-core-test -am -Dtest=LoggerContextAdminSetConfigLocationUriTest), two things came up:

  1. I reverted LoggerContextAdmin.java back to 2.x and kept your tests, all 3 still pass. They only check the config loads, not that anything is closed. Can you make at least one of them fail without the production change?

  2. The buffering looks like it changes more than the close. ConfigurationSource.resetInputStream() returns early when data != null and gives back the same bytes, and that is the method reconfigure() uses. getFile() is still not null so initializeWatchers still registers the file watcher, so a config set over JMX with monitorInterval would keep seeing the file change but keep loading the old config. Same idea as your #4125 — can you keep the stream-backed source and just widen the try-with-resources around the getConfiguration call? Smaller diff and no behaviour change.

One more, XmlConfiguration, JsonConfiguration and PropertiesConfigurationFactory all close getInputStream() in a finally already, so which path is the one that actually leaks? Good to have it written down so the changelog says the right thing.

Let me know when it's updated and I will review it again.

@github-project-automation github-project-automation Bot moved this to Changes requested in Log4j pull request tracker Aug 2, 2026
Address review on apache#4218:
- Drop full buffering so ConfigurationSource remains stream/file backed
  (monitorInterval / resetInputStream still re-read the file).
- Close the caller-owned InputStream with try-with-resources around
  getConfiguration and start.
- Add a regression test that installs a factory which never consumes the
  stream; red fails on open FD count without the try-with-resources, green
  closes cleanly.

Signed-off-by: Sebastien Tardif <sebtardif@ncf.ca>
@SebTardif

Copy link
Copy Markdown
Contributor Author

@ramanathan1504 thanks for the detailed review — updated.

  1. Red-green test: setConfigLocationUri_closesCallerOwnedStreamWhenFactoryDoesNotConsumeIt installs a ConfigurationFactory that returns DefaultConfiguration without reading the stream. Without the try-with-resources, open FD count rises (expected: 173 but was: 177 locally). With the fix, FD count holds and the temp file can be deleted (Windows lock path). Happy-path tests still load real XML.

  2. No full buffering: dropped the toByteArray / Source+byte[] approach. Source stays stream/file-backed so resetInputStream() still re-opens the file for monitorInterval watchers.

  3. Leak note: Xml/JSON/Properties factories already close when they consume the stream. The gap is the caller-owns-stream contract when a factory path never consumes (or fails before consume). try-with-resources on the JMX method covers that without changing reconfigure/watch behavior.

Ready for another look.

ramanathan1504 added a commit to ramanathan1504/log4j2-workout that referenced this pull request Aug 10, 2026
Supersedes `./bench redgreen` from the previous commit. The red-green gates
were half a rebuild of `knowledge-creator/log4j-pr-review.sh`, which owned this
job first and did the build/spotless half better — full-reactor triggers for
build-wide changes, `--offline`, GraphQL feedback that catches resolved and
outdated threads, the port/changelog precedent rule, numbered evidence files.
This merges the two into eleven steps under one command, per the split:
oss-cli knows, workout runs, knowledge-creator remembers.

The pollution check moves ahead of red-green. It asks whether the tests dirtied
the tree, and red-green works by checking files in and out of that same tree,
so in the original order it would report pollution it had created itself.

Step 8 no longer claims a PR is broken when the module is. It runs the whole
module suite — 8729 tests for log4j-core-test, with 6 failures and 9 errors on
this machine in XmlLayoutTest, JsonLayoutTest, MemoryMappedFile*Test and
friends, none of them touched by #4218. The summary now splits failures into
classes the PR touches and classes it does not, and 08-test-failures.md prints
the base-branch command that settles it. Untouched failures are probably
pre-existing; "probably" is not a review finding.

`bench` is now installable on PATH. ROOT was `dirname "$BASH_SOURCE"`, which
through a symlink resolves to ~/.local/bin where configs/ and scripts/ do not
exist — and `bench list --apps` still passes from there, because APPS is a
literal array in the file, so the breakage looks like a working install until a
command touches disk. It now walks the symlink chain by hand, since macOS ships
bash 3.2 with no `readlink -f`. Verified through a single and a chained symlink.

Also fixes a latent bug carried in from the original: `die` was called at line
69 to handle an unresolvable --out but not defined until line 72, so that error
path would have died with "command not found".

Verified end to end against apache/logging-log4j2#4218: build PASS, spotless
clean, tree clean, RED "Tests run: 4, Failures: 1", GREEN PASS. Failure
classification checked under bash against the real 8729-test log — 0 mine / 6
others, and 1/5 when a failing class is spliced into the touched set.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ramanathan1504 added a commit to ramanathan1504/log4j2-workout that referenced this pull request Aug 10, 2026
…ommand (#74)

* Add ./bench redgreen — run a PR's tests without its fix

Automates the hand-check that caught #4218: revert the production change,
keep the tests, and see whether anything actually goes red. That PR's first
revision shipped three tests which all passed without the fix, and only
running them said so.

Four gates in a throwaway git worktree, so the Log4j clone stays on 2.x and
~/.m2 is never overwritten — no `mvn install`, so no later bench run silently
measures the PR:

  BUILD     the PR branch compiles and its own tests pass
  SPOTLESS  formatted the way the project enforces
  RED       base + the PR's test files only  -> must fail
  GREEN     base + tests + the PR's main files -> must pass

RED reports four outcomes, not two. An early version ran under JDK 22, Log4j's
enforcer rejected it with [17,18) in log4j-bom before compiling anything, and
the non-zero exit reported a green tick on RED — the same shape as the
commons-compress repro that passed on four versions having compressed nothing.
So RED now demands a surefire "Tests run: ... Failures:" line or a test-source
COMPILATION ERROR, reports "inconclusive" otherwise, and a failed BUILD aborts
rather than letting later gates report on a broken tree. The JDK is checked by
asking the JVM its own java.specification.version, because java_home -v answers
an unknown version with the newest JDK and exit 0.

Verified against #4218: BUILD pass, SPOTLESS pass, RED "Tests run: 4,
Failures: 1", GREEN pass. That output now backs the paste-ready comment in
docs/pr-reviews/ instead of the author's word for it.

PR-REVIEW.md §3 leads with the gates and spells out how to read each RED
outcome; CLAUDE.md's command count follows the dispatch table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Fold the knowledge-creator harness in: ./bench review

Supersedes `./bench redgreen` from the previous commit. The red-green gates
were half a rebuild of `knowledge-creator/log4j-pr-review.sh`, which owned this
job first and did the build/spotless half better — full-reactor triggers for
build-wide changes, `--offline`, GraphQL feedback that catches resolved and
outdated threads, the port/changelog precedent rule, numbered evidence files.
This merges the two into eleven steps under one command, per the split:
oss-cli knows, workout runs, knowledge-creator remembers.

The pollution check moves ahead of red-green. It asks whether the tests dirtied
the tree, and red-green works by checking files in and out of that same tree,
so in the original order it would report pollution it had created itself.

Step 8 no longer claims a PR is broken when the module is. It runs the whole
module suite — 8729 tests for log4j-core-test, with 6 failures and 9 errors on
this machine in XmlLayoutTest, JsonLayoutTest, MemoryMappedFile*Test and
friends, none of them touched by #4218. The summary now splits failures into
classes the PR touches and classes it does not, and 08-test-failures.md prints
the base-branch command that settles it. Untouched failures are probably
pre-existing; "probably" is not a review finding.

`bench` is now installable on PATH. ROOT was `dirname "$BASH_SOURCE"`, which
through a symlink resolves to ~/.local/bin where configs/ and scripts/ do not
exist — and `bench list --apps` still passes from there, because APPS is a
literal array in the file, so the breakage looks like a working install until a
command touches disk. It now walks the symlink chain by hand, since macOS ships
bash 3.2 with no `readlink -f`. Verified through a single and a chained symlink.

Also fixes a latent bug carried in from the original: `die` was called at line
69 to handle an unresolvable --out but not defined until line 72, so that error
path would have died with "command not found".

Verified end to end against apache/logging-log4j2#4218: build PASS, spotless
clean, tree clean, RED "Tests run: 4, Failures: 1", GREEN PASS. Failure
classification checked under bash against the real 8729-test log — 0 mine / 6
others, and 1/5 when a failing class is spliced into the touched set.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Add `bench hub`, the `--file` hand-off, and the cross-references

Three repos overlapped on PR review and none of them mentioned the others,
which is how `redgreen` came to be written on top of a harness that already
existed. This closes that: each README now states the split — oss-cli knows,
workout runs, knowledge-creator remembers — and the one test that decides where
new work belongs (does it need to execute code against a real app?).

`bench review <n> --file` hands the finished write-up to
knowledge-creator's pr-review-file.py, which files it under
Projects/<topic>/pr-reviews/ and indexes it. That is the only cross-repo call at
runtime, and it goes in the right direction: the evidence under .bench/ is
disposable, the reasoning is not, and a harvester cannot reconstruct the parts
of a review that were deliberately never posted.

`bench hub` serves all three repos as one local page on :8787 — live git state,
the combined PR-review walkthrough, and every doc from all three rendered in
place. Regenerated per request from the working trees, so there is no build
step, no cache and no watcher to drift; a reload is the refresh. It only reads:
never fetches, pulls or writes, and "behind" is measured against the last fetch
you ran rather than the network.

Stdlib only, including a markdown subset renderer, because this machine has no
markdown library and the page has to work with no network. The first version of
that renderer hung instead of failing: every branch guard can be true of the
line the loop is standing on — a `|` row whose table has no separator is the
common case — leaving the paragraph fallback consuming nothing and never
advancing. It now always consumes the current line first. Checked against the
inputs that hung it, plus injection: 23 sections, 58 tables, no unbalanced tags,
`<script>` escaped.

Verified: generation, the renderer, and `bench review --file` dry-run against
the real 4218 write-up. NOT verified: the HTTP serving path — start,
status.json, the reload poll — so treat `bench hub` as unproven until it has
been run once by hand.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* bench hub: open the browser by default, and fix a malformed CSS rule

`bench hub` printed a URL and did nothing else unless you also passed --open,
which reads as broken for a command whose entire job is to show you a page.
Opening is now the default; --no-open serves without a browser, and --open is
kept as a no-op so anything already passing it still works. A browser that
cannot be launched now says so and repeats the URL, rather than leaving a silent
server behind.

Also drops a leftover line from the stylesheet —
`:root:not([data-theme=light]) @media (prefers-color-scheme:dark){}` — which is
not valid CSS. Browsers discarded it through error recovery, so the theme worked
by luck rather than by construction. Braces now balance with no empty rules.

Serving is verified this time, which the previous commit could not claim: HTTP
200 and 343KB through the installed `bench` symlink from an unrelated directory,
status.json returning the three-repo stamp, and a taken port reporting
"cannot bind ... Address already in use" rather than dying silently.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vy vy added bug Incorrect, unexpected, or unintended behavior of existing code jmx labels Aug 13, 2026
@vy vy added this to the 2.27.0 milestone Aug 13, 2026
Address vy review: drop OS FD-count/delete assertions in favor of an
AtomicReference + ConfigurationFactory capture of source.getInputStream(),
then assertThrows(IOException) on read after setConfigLocationUri.

Also switch the file path to Files.newInputStream as requested.
@SebTardif

Copy link
Copy Markdown
Contributor Author

@vy Follow-ups from your review are on the tip (b323bff7):

  1. File branch uses Files.newInputStream(configFile.toPath()).
  2. Closed-stream proof no longer depends on FD counts or deleting the config file. A test ConfigurationFactory captures source.getInputStream() without consuming it; after setConfigLocationUri returns, reading from that stream throws (stream closed).

Could you take another look when you have a moment?

Comment thread src/changelog/.2.x.x/fix_logger_context_admin_config_stream_leak.xml Outdated
@vy
vy enabled auto-merge (squash) August 14, 2026 07:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Incorrect, unexpected, or unintended behavior of existing code jmx

Projects

Status: Changes requested

Development

Successfully merging this pull request may close these issues.

3 participants