diff --git a/CLAUDE.md b/CLAUDE.md index a1a0a9c..ec07c11 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -120,6 +120,38 @@ commit's in-progress run and erase its green check. Pass `--cleanup [dir]` to remove prior scan output non-interactively (reads `output_path` from `config.json` if no directory is given). +## Operator Path Resolution + +Every operator-facing path — `config.json`, `exclusions.txt`, the default +output location, relative `target_file`/`output_path`/`exclusions_file` +values read out of `config.json`, and `--cleanup`'s search for a config to +read `output_path` from — resolves against the current working directory the +command was run from, via `_operator_dir()` (a thin wrapper around +`os.getcwd()`, kept as its own module-level helper so it is testable outside +`main()`'s `# pragma: no cover` region). An operator's config and scan output +belong in the directory they ran the engagement from, not wherever the +program happens to be installed — those are frequently different places, and +this is true independent of how (or whether) the tool is installed. + +This is a behaviour change from resolving against `os.path.dirname(os.path.realpath(__file__))`: +invoking by absolute path from another directory (`cd /tmp && +/opt/spoonmap/spoonmap.py`) previously read `/opt/spoonmap/config.json` and +wrote output there; it now resolves against `/tmp`. The documented invocation +— running `./spoonmap.py` or `uv run spoonmap.py` from inside the checkout — +is unaffected, since CWD and checkout are the same directory there. + +This is a different anchor from `_DIR`/`_NSE_DIR` (`spoonmap.py:2459`), which +stay `__file__`-relative on purpose: the bundled NSE scripts under `nse/` are +program data that ships with SpooNMAP itself, not operator data, and must +resolve identically regardless of the caller's CWD. `_operator_dir()` and +`_DIR` are deliberately two separate anchors — do not collapse them into one. + +The wheel built by `pyproject.toml`'s hatch config is a supported consumption +path (`uv tool install git+https://github.com/trustedsec/spoonmap`; see +README.md for the end-user walkthrough), so a change here must keep +`config.json.sample` and every bundled `nse/` script landing in the wheel — +the `build` CI job asserts this. + ## Architecture ### Host Discovery (Internal) diff --git a/README.md b/README.md index 12b4ec5..b4dbb26 100644 --- a/README.md +++ b/README.md @@ -7,9 +7,53 @@ This script is a wrapper for masscan and nmap. nmap handles host discovery and ( Python 3.8+ is required (`requires-python` in `pyproject.toml`; CI floors at 3.8). +## Installation + +SpooNMAP can be run straight from a checkout — no installation step needed, see +"Usage" below — or installed as a standalone command with +[uv](https://docs.astral.sh/uv/): + +```bash +uv tool install git+https://github.com/trustedsec/spoonmap +``` + +This puts a `spoonmap` executable on your `PATH`, so you can invoke it as +`spoonmap` from any directory instead of cloning the repo and running +`./spoonmap.py`. **There is no PyPI package** — this project has never +published to `pypi.org`, and the command above installs directly from the +git repository instead. Use the full `git+https://...` form above, not a +bare `uv tool install spoonmap`; whatever that name resolves to on PyPI, now +or in the future, is not this project. + +To update to the latest commit: + +```bash +uv tool upgrade spoonmap +``` + +Installing this way does not make SpooNMAP a self-contained scanner: +`masscan` and `nmap` are still separate system tools that must be installed +independently (see "Dependencies" above), exactly as when running from a +checkout — `uv tool install` only packages SpooNMAP's own Python code and its +bundled NSE scripts, not the external binaries it shells out to. + +The one thing worth understanding before installing this way: an installed +`spoonmap`'s Python module lives wherever `uv` put its managed tool +environment — not in a directory you would ever think to look in for a +config file or scan output. That is exactly the scenario the "Where Files +Live" section below is about — read that section for what resolves against +your current directory and why; installing via `uv tool install` doesn't +change the rule, it just makes the rule matter, since there is no checkout +directory left for a config or output path to fall back to by habit. + ## Usage Simply executing the script will prompt you for all required options. +`config.json`, target/exclusion files, and scan output all resolve against +the directory you run the command from — see "Where Files Live" below if +your output isn't where you expect it, especially if you're used to invoking +SpooNMAP by path from outside its own directory. + If you use [uv](https://docs.astral.sh/uv/), you can run without a separate virtual environment: ```bash @@ -145,6 +189,44 @@ uv run spoonmap.py --cleanup ./spoonmap.py --cleanup /path/to/output ``` +## Where Files Live + +Every operator-facing path resolves against the directory you run the command +from — not the directory containing `spoonmap.py`. That covers `config.json`, +`exclusions.txt`, the default output location, any relative `target_file` / +`output_path` / `exclusions_file` value written inside `config.json`, and +`--cleanup`'s search for a config to read `output_path` from. The reasoning is +simple: your config and your scan results belong in the directory where you +ran the engagement from, not wherever the program itself happens to sit on +disk — those are frequently different places, and an operator has no reason to +go looking in the latter for the former. + +**This is a behaviour change.** Previously these paths resolved against the +directory containing `spoonmap.py` itself. If you always run `./spoonmap.py` +or `uv run spoonmap.py` from inside the checkout, nothing changes — your CWD +and the checkout are the same directory. But if you invoke it by an absolute +or relative path from somewhere else — + +```bash +cd /tmp +/opt/spoonmap/spoonmap.py +``` + +— behaviour is different from before: this used to read `/opt/spoonmap/config.json` +and write output under `/opt/spoonmap/`. It now reads `/tmp/config.json` and +writes output under `/tmp/`. If you have a habit of invoking SpooNMAP from +outside its own directory, check where your `config.json` and prior output +actually are before your next run. + +**What does *not* follow this rule:** the bundled NSE scripts (`.nse` files +invoked during `script_scan`) are program data, not operator data — they ship +with SpooNMAP itself and always resolve from the directory containing the +`spoonmap` module, regardless of your CWD. That directory holds a folder +named `nse/` in a checkout and `spoonmap_nse/` in a `uv tool install`; which +one you have on disk depends on how you got SpooNMAP, not on anything you +configure. This is the one exception, and it exists so the script scan works +identically no matter where you happen to run the tool from. + ## Target File (ranges.txt) `ranges.txt` is committed to the repository as an empty placeholder and is marked `skip-worktree`, so git will never stage local edits to it. Fill it with your target ranges freely — they will never be accidentally committed. @@ -171,7 +253,7 @@ git update-index --no-skip-worktree ranges.txt | `target_scan` | `"External"` / `"Internal"` | Selects discovery port lists and NSE script sets; no source-port override is applied | | `max_rate` | Packets/second string | See rate guidance below | | `target_file` | Path | One IP, CIDR, or hostname per line; `ranges.txt` is committed as a blank placeholder (see below) | -| `output_path` | Path | Directory for all output; relative paths resolve to script dir | +| `output_path` | Path | Directory for all output; relative paths resolve to the current working directory (see "Where Files Live" above) | | `exclusions_file` | Path | IPs/CIDRs to exclude; SpooNMAP pre-computes the set intersection with the target file and passes only the net target IPs to masscan (see below) | | `nmap_threads` | Integer | Concurrent nmap processes (default: 5); prompted under "Tune advanced settings" | | `masscan_batch_size` | Integer | Ports per masscan invocation (default: 5); prompted under "Tune advanced settings" | diff --git a/pyproject.toml b/pyproject.toml index 71beda6..6b5c427 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -78,17 +78,18 @@ only-include = ["spoonmap.py"] # `nse/` would be a name collision waiting to happen. # # config.json.sample also force-included, landing next to spoonmap.py at the -# wheel root (same place config.json itself is looked up from, via dir_path = -# os.path.dirname(os.path.realpath(__file__))): spoonmap.py's own error -# messages ("See config.json.sample for the expected keys") name that file by -# bare filename, and it shipped in the sdist already, so a wheel install that -# omitted it left the message pointing at something that doesn't exist. -# exclusions.txt deliberately stays out: its only reference +# wheel root — which is _DIR (os.path.dirname(os.path.realpath(__file__))), +# not the operator's own directory. spoonmap.py's error messages ("See +# {_DIR}/config.json.sample for the expected keys") point there deliberately, +# since that is the one place a sample file is guaranteed to exist regardless +# of the operator's CWD, and it shipped in the sdist already, so a wheel +# install that omitted it left those messages naming a path that doesn't +# exist. exclusions.txt deliberately stays out: its only reference # (`f'{dir_path}/exclusions.txt'`) is an interactive prompt's *suggested -# default path*, not a template any error message tells the operator to open, -# and the checked-in file is empty — shipping it adds no content and the -# prompt's default remains a fine suggestion whether or not a file exists -# there yet. +# default path* built from the operator directory, not a template any error +# message tells the operator to open, and the checked-in file is empty — +# shipping it adds no content and the prompt's default remains a fine +# suggestion whether or not a file exists there yet. [tool.hatch.build.targets.wheel.force-include] "nse" = "spoonmap_nse" "config.json.sample" = "config.json.sample" diff --git a/spoonmap.py b/spoonmap.py index d35073a..f30e475 100755 --- a/spoonmap.py +++ b/spoonmap.py @@ -5357,7 +5357,7 @@ def _config_target_scan(value): return valid print(_COLOR_ERROR + f'ERROR: config.json: target_scan = {value!r} is not ' "'Internal' or 'External'." + _COLOR_RESET) - print('See config.json.sample for the expected keys, or delete ' + print(f'See {_DIR}/config.json.sample for the expected keys, or delete ' 'config.json to be prompted instead.') sys.exit(1) @@ -5365,7 +5365,7 @@ def _config_target_scan(value): def _load_config(config_parser, dir_path, resume=False): """Derive every scan setting from an already-parsed config.json dict. - *config_parser* is the JSON dict, *dir_path* the script directory that the + *config_parser* is the JSON dict, *dir_path* the operator directory that the three relative path settings resolve against, and *resume* whatever the --resume CLI flag already produced: the config's own 'resume' key is ORed into it so the flag can never be turned back off by the file. Returns the @@ -5380,7 +5380,7 @@ def _load_config(config_parser, dir_path, resume=False): print(_COLOR_ERROR + 'ERROR: config.json is missing required ' f'{"key" if len(missing) == 1 else "keys"}: ' + ', '.join(missing) + _COLOR_RESET) - print('See config.json.sample for the expected keys, or delete ' + print(f'See {_DIR}/config.json.sample for the expected keys, or delete ' 'config.json to be prompted instead.') sys.exit(1) @@ -5434,7 +5434,7 @@ def _load_config(config_parser, dir_path, resume=False): resume = resume or _config_bool('resume', config_parser.get('resume'), False) config_generated = bool(config_parser.get(_CONFIG_GENERATED_KEY)) - # Resolve relative paths in config relative to the script directory + # Resolve relative paths in config relative to the operator directory if target_file and not os.path.isabs(target_file): target_file = os.path.join(dir_path, target_file) if output_path and not os.path.isabs(output_path): @@ -5463,12 +5463,25 @@ def _load_config(config_parser, dir_path, resume=False): } +def _operator_dir(): + """Directory operator data (config.json, exclusions, output) resolves against. + + Deliberately the CWD, not _DIR (the module's own location): an installed + `spoonmap`'s module directory lives inside uv's managed tool environment, + which is rebuilt whenever `uv tool upgrade` actually installs a new + version — anything stored there, config or scan results, is lost at that + point. That is the failure mode that makes the CWD the only defensible + choice, not merely a stylistic preference. Pulled out of main() — which + is pragma-no-cover — so this derivation itself stays under test. + """ + return os.getcwd() + + # The Main Guts def main(): # pragma: no cover -- interactive CLI entry point; orchestrates # already-independently-tested functions behind input()-driven prompts, # so there's little signal in mocking every prompt/subprocess in one # giant test versus exercising each called function directly. - global dir_path global output_path # Save initial terminal state @@ -5496,7 +5509,7 @@ def main(): # pragma: no cover -- interactive CLI entry point; orchestrates # Get options from configuration file if it exists - dir_path = os.path.dirname(os.path.realpath(__file__)) + dir_path = _operator_dir() if '--cleanup' in sys.argv: _cleanup_cmd(dir_path) # prints result and exits diff --git a/tests/test_spoonmap.py b/tests/test_spoonmap.py index 881eaa2..6be4834 100644 --- a/tests/test_spoonmap.py +++ b/tests/test_spoonmap.py @@ -89,6 +89,7 @@ _parse_masscan_ping_xml, _parse_nmap_sn_xml, _parse_result_xml, + _operator_dir, _quarantine_failed_output, _resolve_nse_dir, _aggregate_result_dir, @@ -2536,6 +2537,10 @@ def test_every_missing_key_named_in_one_message(self, capsys): for key in ('banner_scan', 'max_rate', 'target_file', 'output_path'): assert key in out assert 'missing required keys' in out + # config.json.sample is program data next to the module (_DIR), not + # the operator's CWD, so the guidance must name a real path rather + # than a bare filename an installed user has no hope of finding. + assert f'{spoonmap._DIR}/config.json.sample' in out # ---- target_scan validation -------------------------------------------- @@ -2558,6 +2563,7 @@ def test_unrecognised_target_scan_exits(self, capsys): out = capsys.readouterr().out assert 'target_scan' in out assert "'Internal' or 'External'" in out + assert f'{spoonmap._DIR}/config.json.sample' in out def test_null_target_scan_exits_rather_than_scanning(self, capsys): with pytest.raises(SystemExit): @@ -11153,3 +11159,37 @@ def test_no_call_site_reverts_to_dir_relative_nse_path(self): 'referenced via _NSE_DIR so they resolve in an installed wheel: ' + str(offenders) ) + + +class TestOperatorDirResolution: + """Operator data (config.json, exclusions, output) must resolve against + the CWD, never the module's own location — the opposite anchor from + _DIR/_NSE_DIR, which stay module-relative for bundled program data.""" + + def test_operator_dir_is_the_cwd(self, tmp_path, monkeypatch): + # Compare realpaths on both sides: os.getcwd() resolves symlinks in + # the path (e.g. macOS's /tmp -> /private/tmp), so comparing directly + # against str(tmp_path) would depend on pytest happening to hand out + # an already-resolved tmp_path rather than on the helper's behaviour. + monkeypatch.chdir(tmp_path) + assert _operator_dir() == os.path.realpath(str(tmp_path)) + + def test_operator_dir_follows_cwd_changes(self, tmp_path, monkeypatch): + # A real behavioural assertion, not a restatement of os.getcwd(): + # confirm the helper tracks a change in CWD rather than caching one + # resolved at import time. + first = tmp_path / 'first' + second = tmp_path / 'second' + first.mkdir() + second.mkdir() + monkeypatch.chdir(first) + assert _operator_dir() == os.path.realpath(str(first)) + monkeypatch.chdir(second) + assert _operator_dir() == os.path.realpath(str(second)) + + def test_operator_dir_is_not_module_relative(self, tmp_path, monkeypatch): + # Regression guard for PR #42: wherever the module is installed must + # never be where operator data resolves, independent of install + # mechanism. + monkeypatch.chdir(tmp_path) + assert _operator_dir() != spoonmap._DIR