feat: discover hostnames from TLS certificate SNI/CN/SAN data

.gitignore

@@ -5,6 +5,10 @@ __pycache__/
 htmlcov/
 .DS_Store
 
+# SDD spec/plan docs carry absolute local paths from the machine they were
+# written on and are not tracked in the public repo (see 9ea2bb3)
+docs/
+
 # generated scan output
 all_live_hosts.txt
 resolved_targets.txt

CLAUDE.md

@@ -436,6 +436,7 @@ Internal discovery runs a single masscan sweep (no source-port override) followe
 - **When the record is written and dropped**: on **success paths only**, exactly as `_EMPTY_RESULT_XML` is — a record on a killed scan would assert coverage that never happened. A `KeyboardInterrupt`, a missing binary and a non-zero exit all leave none; `_nmap_udp_discovery()` needs an explicit `proc.returncode == 0` guard for the last of those, because unlike `_nmap_port_discovery()` it does not treat a non-zero exit as fatal and a failing nmap can still leave parseable partial XML. Each phase also calls `_discard_coverage_record()` **before it scans**, not only on failure: a run killed outright never reaches the stamp, and the previous run's record would otherwise sit beside output that run had already replaced. The hazard is specifically a record that covered *more* than the output now on disk, since a subset test accepts it — a narrower leftover can only cause a redundant re-scan. Both halves live in one file written by a single `_atomic_write` for the same reason: as two sidecars written in sequence, a `KeyboardInterrupt` between them (a `BaseException`, so `except Exception` missed it) left a fresh target list beside a stale exclusion list and the gate accepted the pair as an exclusion-free scan. Neither an unreadable input nor a failed write raises on its own account — the scan already succeeded, so unwinding would discard real results — but every failure path, including an interrupt, deletes the record first. A cache whose record is absent or malformed is rejected, so output from before this change re-scans once on the first resume after upgrading; that is the deliberate direction, since a redundant re-scan is visible and an under-scan is not. The `.coverage` suffix is invisible to every result consumer: `masscan_results/` is aggregated by listing the directory, and `_parse_result_xml()` drops anything not ending in `.xml`, the same guard that hides `portN.xml.failed`. `_delete_previous_results()` removes it with the directory, so `--cleanup` and `[d]elete` need no special handling.
 - **config.json validation**: `_load_config()` refuses to start a scan it cannot run correctly. Missing required keys are reported all at once and exit. `target_scan` goes through `_config_target_scan()`, which accepts any case/whitespace spelling of `Internal`/`External` (normalising to the exact literal the ~25 comparison sites use) and exits on anything else — an unvalidated `"internal"` matched neither literal, so the scan ran and looked completely normal while every `target_scan == 'Internal'` gated check was silently skipped. Every numeric goes through `_config_int(key, value, default, minimum=1)`, mirroring `_prompt_int`'s floor: a non-numeric or null value warns and takes the default, and a value below `minimum` is clamped with a warning. `max_rate` is included (defaulting to the interactive prompt's 20000 external / 2000 internal) and only then re-`str()`-ed for Popen.
 - **Hostname support**: hostnames in the target file are resolved once at startup; nmap receives the original hostname (for SNI/vhost), masscan receives the resolved IP
+- **TLS certificate hostname discovery**: on External scans, `_extract_ssl_cert_hostnames()` parses the `ssl-cert` NSE output SpooNMAP already collects (commonName off the `Subject:` line, each `DNS:` entry off `Subject Alternative Name:`) and reports every name found as a LOW-severity `TLS Certificate Hostname(s) Identified` finding, wildcards included. Separately, `_merge_ssl_cert_hostnames()` runs right after the NSE script pass (gated on `script_scan`, since that's the only time `nse_results/` exists) and fills gaps in `ip_to_hostname` with the first non-wildcard name per host — never overwriting an operator-supplied hostname from the target file — then rewrites `discovery/ip_hostname_map.json`. It runs before `_aggregate_result_dir()` and `generate_findings()` (which re-reads that file fresh) so `spoonmap_output.*` and the findings report both reflect the merged map for the current run. It does not retroactively change what *this* run's nmap invocations targeted — hostname-based targeting via `create_hostname_target_file()` already happened earlier in the same run using whatever `ip_to_hostname` looked like at that point; the benefit is to this run's reporting. A `--resume` run does not inherit the merged file directly — `main()` calls `preprocess_targets()` unconditionally, including on resume, and it rewrites `discovery/ip_hostname_map.json` from the target file alone with no merge of the existing file's contents — but it re-derives the same cert hostnames from the still-cached `nse_results/*.xml`, since a persisted cert-derived hostname reaching `create_hostname_target_file()` could send nmap after a name-resolved address different from the one actually in scope (a commonName lifted off a shared/CDN certificate can resolve elsewhere entirely). The clobber-on-every-run behavior of `preprocess_targets()` is what accidentally prevents that, and is deliberately left as-is. `'TLS Certificate Hostname(s) Identified'` is also added to `_PER_HOST_DETAIL_TITLES`, since its detail (the actual discovered names) differs per host — without that, `findings.txt` would collapse the group to a single shared description that doesn't exist for this finding.
 - **IPv4-only, enforced at the edges**: the tool scans IPv4 exclusively (masscan/nmap invocations, target expansion, and address sorting all assume it). IPv6 is rejected rather than half-supported, in two places. (1) `_build_discovery_target_file()`'s `_parse_ranges()` skips any entry `ipaddress.ip_network()` resolves to a non-v4 network and prints the offending file, line number, and content — previously the v6 bounds were stored silently and only surfaced hundreds of lines later as `AddressValueError: ... (>= 2**32)` from `summarize_address_range()`, and only when an exclusions file happened to be configured. (2) The masscan/discovery XML parsers (`_parse_masscan_ping_xml()`, `_parse_nmap_sn_xml()`, `_run_masscan_batch()`) select `address[@addrtype='ipv4']` instead of the first `<address>` child, matching what the nmap-side parsers already did, so a dual-stacked host's IPv6 or MAC string can't enter `live_ips`/`port_ips` and become a masscan `-iL` target. Address sorting goes through `_ip_sort_key()`, which orders valid IPv4 numerically and sorts anything unparseable last instead of raising — the three former inline `tuple(int(o) for o in x.split('.'))` keys ran *after* a completed sweep, so one odd entry discarded the whole thing.
 - **XML result parsing is per-element defensive**: every `etree.parse()` site guards the *walk* as well as the parse. Attributes are read with `.attrib.get(...)` and the element is skipped when the identifier is missing — never a bare `attrib['addr']` or `findall('address')[0]`, both of which raise `KeyError`/`IndexError` that `except etree.ParseError` does not catch. Those exceptions escaped the guard and discarded the results for *every other host* in the file (or, in `_host_elem_to_dict()`, lost `spoonmap_output.xml`/`.json` for the whole run) over one truncated element. `<script>` elements with no `id=` are filtered out of the comprehensions for the same reason. Where a fallback to the first `<address>` child is wanted after `address[@addrtype='ipv4']` misses (`generate_findings()`, `_scan_extra_sql_ports()`), it is a `None`-checked `find('address')`.
 - **Firewall state table safety**: internal discovery caps masscan at `INTERNAL_DISCOVERY_MAX_RATE = 1000 pps`; at that rate with a 60 s half-open timeout, concurrent state entries peak at ~60 K regardless of target range size; for ranges above `INTERNAL_DISCOVERY_STATE_CEILING = 262_144` hosts the port list is trimmed from 10 to 5 to keep total packet volume bounded. Separately and for the same reason, `mass_scan()` clamps a **Full** scan to `full_scan_rate` — 10000 pps External, 1000 pps Internal — since a single 1-65535 invocation fans out every port across every target at once. This cap applies *only* to `scan_type == 'Full'`; category and custom batched scans scan a handful of ports per invocation and always use the operator's full `max_rate`. The clamp prints a notice when it actually lowers the rate, because `main()`'s run summary echoes the *requested* `max_rate`: clamping silently made the summary contradict what masscan was told to do, and read as the operator's `--max-rate` having been ignored outright.

spoonmap.py

@@ -583,6 +583,39 @@ def resolve_hostname(hostname):
         print(_COLOR_ERROR + f'Warning: Could not resolve hostname {hostname}: {e}' + _COLOR_RESET)
         return None
 
+def _extract_ssl_cert_hostnames(ssl_cert_output):
+    """Extract hostnames from ssl-cert NSE script output (CN + SAN).
+
+    Returns a deduped, order-preserved list: the certificate's commonName
+    first, then each Subject Alternative Name DNS: entry in the order nmap
+    printed them.  Wildcard names (e.g. '*.example.com') are returned like
+    any other name -- callers that feed a scan target must filter those out
+    themselves.  Anchored to the 'Subject:' line specifically (not
+    'Issuer:'), since ssl-cert output carries a commonName for both and only
+    the subject's identifies the host being scanned.
+    """
+    hostnames = []
+    seen = set()
+
+    cn_match = re.search(r'^Subject:.*?commonName=([^\s/,]+)', ssl_cert_output, re.MULTILINE)
+    if cn_match:
+        cn = cn_match.group(1).strip()
+        if cn and cn not in seen:
+            hostnames.append(cn)
+            seen.add(cn)
+
+    san_match = re.search(r'^Subject Alternative Name:\s*(.+)$', ssl_cert_output, re.MULTILINE)
+    if san_match:
+        for entry in san_match.group(1).split(','):
+            entry = entry.strip()
+            if entry.startswith('DNS:'):
+                name = entry[len('DNS:'):].strip()
+                if name and name not in seen:
+                    hostnames.append(name)
+                    seen.add(name)
+
+    return hostnames
+
 def _write_if_changed(path, content):
     """Write *content* to *path* only if it differs from the current contents.
 
@@ -947,6 +980,56 @@ def preprocess_targets(target_file, output_path):
 
     return masscan_file, ip_to_hostname
 
+
+def _merge_ssl_cert_hostnames(output_path, ip_to_hostname):
+    """Fill gaps in ip_to_hostname from ssl-cert CN/SAN data in nse_results/.
+
+    Never overwrites an existing (operator-supplied) entry -- a name typed
+    into the target file always wins over a cert-derived guess.  For an IP
+    with no prior entry, prefers the certificate's commonName; falls back to
+    the first non-wildcard Subject Alternative Name.  A host whose only
+    names are wildcards gets no entry, since a wildcard is not a usable
+    scan target.  Returns a new dict; persists it to
+    <output_path>/discovery/ip_hostname_map.json via _write_if_changed().
+    """
+    nse_dir = f'{output_path}/nse_results'
+    merged = dict(ip_to_hostname)
+
+    if os.path.isdir(nse_dir):
+        for fname in sorted(os.listdir(nse_dir)):
+            if not fname.endswith('.xml'):
+                continue
+            try:
+                root = etree.parse(f'{nse_dir}/{fname}')
+            except Exception:
+                continue
+
+            for host in root.findall('host'):
+                addr_elem = host.find("address[@addrtype='ipv4']")
+                ip = addr_elem.attrib.get('addr') if addr_elem is not None else None
+                if not ip or ip in merged:
+                    continue
+
+                for port_elem in host.iter('port'):
+                    scripts = {s.attrib.get('id'): s.attrib.get('output', '')
+                               for s in port_elem.findall('script') if s.attrib.get('id')}
+                    ssl_out = scripts.get('ssl-cert')
+                    if not ssl_out:
+                        continue
+                    for name in _extract_ssl_cert_hostnames(ssl_out):
+                        if not name.startswith('*.'):
+                            merged[ip] = name
+                            break
+                    if ip in merged:
+                        break
+
+    mapping_file = os.path.join(_disc(output_path), 'ip_hostname_map.json')
+    os.makedirs(_disc(output_path), exist_ok=True)
+    _write_if_changed(mapping_file, json.dumps(merged, indent=2))
+
+    return merged
+
+
 def _get_scripts_for_port(dest_port, target_scan):
     """Return comma-separated NSE script list for dest_port, or None.
 
@@ -3907,6 +3990,13 @@ def port_str_from_fname(fname):
                             add('MEDIUM', ip, port_str, 'Expired TLS Certificate',
                                 f'Certificate expired on {expiry}.')
 
+                # ── ssl-cert — hostnames from CN/SAN (External only) ─────
+                if 'ssl-cert' in scripts and target_scan == 'External':
+                    cert_hostnames = _extract_ssl_cert_hostnames(scripts['ssl-cert'])
+                    if cert_hostnames:
+                        add('LOW', ip, port_str, 'TLS Certificate Hostname(s) Identified',
+                            f'Certificate presents: {", ".join(cert_hostnames)}.')
+
                 # ── ldap-signing-check (ports 389 / 3268) ────────────────────
                 if 'ldap-signing-check' in scripts and target_scan == 'Internal':
                     if 'NOT REQUIRED' in scripts['ldap-signing-check'].upper():
@@ -4536,6 +4626,15 @@ def _signing_not_req(key):
             '|_Not valid after:  2022-01-01T00:00:00'
         ),
     },
+    'TLS Certificate Hostname(s) Identified': {
+        'flags': '--script ssl-cert',
+        'sample': (
+            'PORT    STATE SERVICE\n'
+            '443/tcp open  https\n'
+            '| ssl-cert: Subject: commonName=example.corp\n'
+            '|_Subject Alternative Name: DNS:example.corp, DNS:www.example.corp'
+        ),
+    },
     'SQL Server Instance Discovered': {
         'flags': '--script ms-sql-info',
         'sample': (
@@ -4855,6 +4954,7 @@ def _write_artifact(path, content):
 _PER_HOST_DETAIL_TITLES = frozenset({
     'Service Exposed Externally',
     'VNC Desktop Name Disclosed',
+    'TLS Certificate Hostname(s) Identified',
 })
 
 
@@ -4897,8 +4997,9 @@ def _write_findings_txt(output_path, target_scan, findings):
             lines.append(f'  Affected hosts ({len(hosts)}):')
             if title in _PER_HOST_DETAIL_TITLES:
                 # Detail varies per host (exposure label + embedded vuln check;
-                # the VNC desktop name), so render it inline rather than
-                # collapsing the group to one shared description.
+                # the VNC desktop name; the discovered TLS certificate
+                # hostname list), so render it inline rather than collapsing
+                # the group to one shared description.
                 for h, d in sorted(grp['host_details']):
                     lines.append(f'{h}  —  {d}')
             else:
@@ -7365,6 +7466,7 @@ def main():  # pragma: no cover -- interactive CLI entry point; orchestrates
             if script_scan:
                 snmp_any_validated = _validate_snmp_any_community(
                     output_path, target_scan, extra_script_args)
+                ip_to_hostname = _merge_ssl_cert_hostnames(output_path, ip_to_hostname)
 
         # Combine all live hosts into one file
         disc = _disc(output_path)