Skip to content

feat: resolve relative config paths via server-derivable fallbacks#103

Open
caterryan wants to merge 22 commits into
mainfrom
feat/toml-config-path-fallbacks
Open

feat: resolve relative config paths via server-derivable fallbacks#103
caterryan wants to merge 22 commits into
mainfrom
feat/toml-config-path-fallbacks

Conversation

@caterryan

@caterryan caterryan commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

Closes #99.

All 21 plugins that support TOML configuration resolve a relative config_file_path against the PLUGIN_DIR environment variable — which the InfluxDB 3 server never sets. This breaks the API-only workflow of uploading a config via POST /api/v3/plugins/files and referencing it by the same relative path in trigger_arguments.

This PR extends the resolution in all 21 plugins with two fallbacks consulted only when PLUGIN_DIR is unset:

  1. PLUGIN_DIR — unchanged, keeps highest precedence (existing behavior).
  2. INFLUXDB3_PLUGIN_DIR — the documented env-var form of --plugin-dir; visible to plugin code when the operator configures the server that way (e.g. the official Docker image).
  3. Path(os.environ["VIRTUAL_ENV"]).parent — the processing engine activates a venv at startup whose default location is <plugin-dir>/.venv, so its parent is the plugin directory.

Fallback candidates are accepted only if the resolved file actually exists; otherwise resolution fails with an error listing every location tried.

One commit per plugin (21 commits), identical treatment adapted to each plugin's existing style (_resolve_path helpers in the subscriber/OPC UA plugins, inline blocks elsewhere). Two notes on the recently merged plugins:

  • sagemaker already consulted INFLUXDB3_PLUGIN_DIR alongside PLUGIN_DIR; that precedence is kept and only the VIRTUAL_ENV fallback is appended.
  • valuecounter previously defaulted to the current directory when PLUGIN_DIR was unset; that default is preserved as the final fallback, and the _resolve_config signature used by its unit tests is unchanged (82/82 tests pass).

Why this is non-breaking

  • Absolute paths: unchanged.
  • PLUGIN_DIR set: unchanged result; fallbacks never consulted — verified by pointing PLUGIN_DIR at a different directory than --plugin-dir and observing identical resolution before/after.
  • PLUGIN_DIR unset: previously always an error; the fallbacks only turn some of those errors into successes. The remaining failure case gains a clearer message.
  • system_metrics keeps its original continue-without-config behavior when resolution fails (it never aborted the run, and still doesn't).

Verification

  • python3 -m py_compile clean on all 21 modified files; valuecounter's unit suite passes (82/82).
  • Live end-to-end test against InfluxDB 3 Core 3.9.3 with the modified downsampler, no PLUGIN_DIR exported: uploaded configs/downsampler.toml via POST /api/v3/plugins/files, created a trigger with config_file_path=configs/downsampler.toml, and observed Reading config file / Config file loaded successfully in the server log. The same setup previously failed with Failed to get PLUGIN_DIR env var.
  • Resolution-precedence scenarios (flag-only, PLUGIN_DIR exported, INFLUXDB3_PLUGIN_DIR-configured server) verified live with a side-by-side probe plugin; details and observed output in Support API-only TOML config workflow: resolve relative config_file_path without requiring the operator to export PLUGIN_DIR #99.

🤖 Generated with Claude Code

caterryan and others added 21 commits June 10, 2026 16:14
…tion

When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…resolution

When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…olution

When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tion

When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…solution

When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tion

When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…h resolution

When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… resolution

When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…olution

When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…h resolution

When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…olution

When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv).
Fallback candidates are accepted only if the file exists. Existing
behavior is unchanged when PLUGIN_DIR is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…solution

sagemaker already consults INFLUXDB3_PLUGIN_DIR in addition to PLUGIN_DIR;
keep that precedence and append the remaining server-derivable fallback:
the parent of VIRTUAL_ENV (the processing engine's default venv is
<plugin-dir>/.venv), accepted only if the file exists. Existing behavior
is unchanged when either env var is set or the path is absolute.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When PLUGIN_DIR is unset, fall back to server-derivable locations:
INFLUXDB3_PLUGIN_DIR (env-var form of --plugin-dir) and the parent of
VIRTUAL_ENV (the processing engine's default venv is <plugin-dir>/.venv),
accepted only if the file exists, before the original current-directory
default. Existing behavior is unchanged when PLUGIN_DIR is set, and the
_resolve_config signature used by the unit tests is preserved.

Refs #99

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@caterryan caterryan force-pushed the feat/toml-config-path-fallbacks branch from 22253b1 to 7383610 Compare June 10, 2026 21:16
All 21 plugins gained backward-compatible config path resolution
fallbacks (#99), a semver-minor change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support API-only TOML config workflow: resolve relative config_file_path without requiring the operator to export PLUGIN_DIR

2 participants