From 2d9d896007e8dedca9147a5445c306d4e55968a2 Mon Sep 17 00:00:00 2001
From: Justin Bollinger <justin.bollinger@trustedsec.com>
Date: Tue, 25 Aug 2026 13:23:09 -0400
Subject: [PATCH] fix: disclose Full-scan rate cap instead of clamping silently
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

A Full scan clamps masscan to full_scan_rate (10000 pps External, 1000
Internal) so a single 1-65535 sweep cannot exhaust firewall state tables.
The clamp was silent, while main()'s run summary echoes the operator's
requested max_rate — so requesting 5000 pps on an Internal Full scan
printed "Max Packet Rate (pps): 5000" and then scanned at 1000, with the
cap documented only in the config-help text an interactive run never
shows. That reads as max_rate having been ignored outright.

Print a notice when the clamp actually lowers the rate, naming both rates,
why the cap exists, and that targeted scans use the full rate. Guarded on
the comparison so a rate already under the cap stays quiet, and placed
inside the scan_type == 'Full' branch so batched scans — which are
deliberately uncapped — never mention it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---
 CLAUDE.md              |  2 +-
 spoonmap.py            | 11 +++++++++++
 tests/test_spoonmap.py | 23 +++++++++++++++++++++++
 3 files changed, 35 insertions(+), 1 deletion(-)

diff --git a/CLAUDE.md b/CLAUDE.md
index ec07c11..09525f2 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -175,5 +175,5 @@ Internal discovery runs a single masscan sweep (no source-port override) followe
 - **Hostname support**: hostnames in the target file are resolved once at startup; nmap receives the original hostname (for SNI/vhost), masscan receives the resolved IP
 - **IPv4-only, enforced at the edges**: the tool scans IPv4 exclusively (masscan/nmap invocations, target expansion, and address sorting all assume it). IPv6 is rejected rather than half-supported, in two places. (1) `_build_discovery_target_file()`'s `_parse_ranges()` skips any entry `ipaddress.ip_network()` resolves to a non-v4 network and prints the offending file, line number, and content — previously the v6 bounds were stored silently and only surfaced hundreds of lines later as `AddressValueError: ... (>= 2**32)` from `summarize_address_range()`, and only when an exclusions file happened to be configured. (2) The masscan/discovery XML parsers (`_parse_masscan_ping_xml()`, `_parse_nmap_sn_xml()`, `_run_masscan_batch()`) select `address[@addrtype='ipv4']` instead of the first `<address>` child, matching what the nmap-side parsers already did, so a dual-stacked host's IPv6 or MAC string can't enter `live_ips`/`port_ips` and become a masscan `-iL` target. Address sorting goes through `_ip_sort_key()`, which orders valid IPv4 numerically and sorts anything unparseable last instead of raising — the three former inline `tuple(int(o) for o in x.split('.'))` keys ran *after* a completed sweep, so one odd entry discarded the whole thing.
 - **XML result parsing is per-element defensive**: every `etree.parse()` site guards the *walk* as well as the parse. Attributes are read with `.attrib.get(...)` and the element is skipped when the identifier is missing — never a bare `attrib['addr']` or `findall('address')[0]`, both of which raise `KeyError`/`IndexError` that `except etree.ParseError` does not catch. Those exceptions escaped the guard and discarded the results for *every other host* in the file (or, in `_host_elem_to_dict()`, lost `spoonmap_output.xml`/`.json` for the whole run) over one truncated element. `<script>` elements with no `id=` are filtered out of the comprehensions for the same reason. Where a fallback to the first `<address>` child is wanted after `address[@addrtype='ipv4']` misses (`generate_findings()`, `_scan_extra_sql_ports()`), it is a `None`-checked `find('address')`.
-- **Firewall state table safety**: internal discovery caps masscan at `INTERNAL_DISCOVERY_MAX_RATE = 1000 pps`; at that rate with a 60 s half-open timeout, concurrent state entries peak at ~60 K regardless of target range size; for ranges above `INTERNAL_DISCOVERY_STATE_CEILING = 262_144` hosts the port list is trimmed from 10 to 5 to keep total packet volume bounded
+- **Firewall state table safety**: internal discovery caps masscan at `INTERNAL_DISCOVERY_MAX_RATE = 1000 pps`; at that rate with a 60 s half-open timeout, concurrent state entries peak at ~60 K regardless of target range size; for ranges above `INTERNAL_DISCOVERY_STATE_CEILING = 262_144` hosts the port list is trimmed from 10 to 5 to keep total packet volume bounded. Separately and for the same reason, `mass_scan()` clamps a **Full** scan to `full_scan_rate` — 10000 pps External, 1000 pps Internal — since a single 1-65535 invocation fans out every port across every target at once. This cap applies *only* to `scan_type == 'Full'`; category and custom batched scans scan a handful of ports per invocation and always use the operator's full `max_rate`. The clamp prints a notice when it actually lowers the rate, because `main()`'s run summary echoes the *requested* `max_rate`: clamping silently made the summary contradict what masscan was told to do, and read as the operator's `--max-rate` having been ignored outright.
 - **Honeypot/tarpit detection**: `mass_scan()` flags hosts open on ≥`HONEYPOT_OPEN_PORT_FRACTION` (90%) of scanned TCP ports (min sample `HONEYPOT_MIN_PORTS_SCANNED = 10`) as likely tarpits (LaBrea, portspoof) via `_flag_suspected_tarpits()`/`_report_suspected_tarpits()`, writing `discovery/suspected_tarpits.txt`. Separately, `_count_unmatched_service_ports()` reads `nmap_results/*.xml` and counts open ports whose `-sV` probe captured a `servicefp` (no signature match); `≥HONEYPOT_MIN_UNMATCHED_PORTS = 3` such ports on one host is consistent with decoys (e.g. Artillery) that return random data on full connect. Both signals surface as a single "Likely Honeypot / Decoy Host" MEDIUM finding in `generate_findings()`. `_flag_suspected_tarpits()` counts TCP ports only, skipping any `port_key` that starts with `U:`, so every loop that reconstructs a port key from a `live_hosts/portNN.txt` filename must run the stem through `_fname_port()` (`'U_53'` → `'U:53'`) — the raw stem was counted as TCP, skewing the open-port fraction, and printed as `Hosts Found on Port U_53`.
diff --git a/spoonmap.py b/spoonmap.py
index f30e475..950187c 100755
--- a/spoonmap.py
+++ b/spoonmap.py
@@ -1668,6 +1668,17 @@ def mass_scan(scan_type, dest_ports, source_port, max_rate, target_file, exclusi
             # most exposed to a live_hosts/ directory left over from a wider scope.
             _report_out_of_scope_retained(full_results, scope_ranges, target_file)
             return status_summary
+        # The run summary printed by main() shows the operator's requested
+        # max_rate, not full_scan_rate, so a silent clamp here reads as the rate
+        # having been ignored.  Disclose it at the point of use.
+        if int(full_scan_rate) < int(max_rate):
+            print(_COLOR_ERROR
+                  + f'Notice: Full port scan rate capped at {full_scan_rate} pps '
+                    f'(requested {max_rate}) — a 1-65535 sweep fans out every port '
+                    f'across every target at once, and the {target_scan} cap keeps '
+                    'firewall state tables bounded. Targeted port scans use the '
+                    'full requested rate.'
+                  + _COLOR_RESET)
         print(_COLOR_INFO + 'Full port scan: running masscan 1-65535 (no probe)...' + _COLOR_RESET)
         full_results = _run_masscan_batch(['1-65535'], full_scan_rate, output_file,
                                           target_file, source_port, exclusions_file,
diff --git a/tests/test_spoonmap.py b/tests/test_spoonmap.py
index 6be4834..031fae0 100644
--- a/tests/test_spoonmap.py
+++ b/tests/test_spoonmap.py
@@ -2059,6 +2059,29 @@ def test_full_scan_rate_capped_internal(self, tmp_path):
             wait_secs=2,
         )
 
+    def test_full_scan_warns_when_rate_is_capped(self, tmp_path, capsys):
+        """A clamped rate is disclosed — the run summary shows the requested rate."""
+        spoonmap.output_path = str(tmp_path)
+        with patch('spoonmap._run_masscan_batch', return_value={'22': {'10.0.0.1'}}):
+            mass_scan('Full', ['1-65535'], '88', '5000', '/fake/targets.txt', '')
+        out = capsys.readouterr().out
+        assert 'capped at 1000 pps' in out
+        assert 'requested 5000' in out
+
+    def test_full_scan_no_warning_when_rate_under_cap(self, tmp_path, capsys):
+        """No notice when the operator's rate is already below the cap."""
+        spoonmap.output_path = str(tmp_path)
+        with patch('spoonmap._run_masscan_batch', return_value={'22': {'10.0.0.1'}}):
+            mass_scan('Full', ['1-65535'], '88', '500', '/fake/targets.txt', '')
+        assert 'capped' not in capsys.readouterr().out
+
+    def test_category_scan_does_not_warn_about_full_scan_cap(self, tmp_path, capsys):
+        """The cap applies only to Full — a batched scan uses the full rate silently."""
+        spoonmap.output_path = str(tmp_path)
+        with patch('spoonmap._run_masscan_batch', return_value={'22': {'10.0.0.1'}}):
+            mass_scan('Category', ['22'], '88', '5000', '/fake/targets.txt', '')
+        assert 'capped' not in capsys.readouterr().out
+
     def test_full_scan_writes_live_hosts_files(self, tmp_path):
         spoonmap.output_path = str(tmp_path)
         fake_results = {'22': {'10.0.0.5', '10.0.0.6'}}