Skip to content

ENG-8848 fix(network-ops): surface structured findings in Agent Output; fix report_item models - #182

Open
thomasqbrady wants to merge 1 commit into
mainfrom
eng-8848-network-ops-structured-outputs
Open

thomasqbrady wants to merge 1 commit into
mainfrom
eng-8848-network-ops-structured-outputs

Conversation

@thomasqbrady

Copy link
Copy Markdown
Contributor

Fixes ENG-8848 and ENG-8832. Found during pre-public QA of network-ops.

Problem

  • ENG-8848 — after a full enumeration, Agent Output showed zero structured records. network-ops shipped its own local report_item that called dn.log_output(item.__class__.__name__, item) — a generic span ref under ad-hoc names, never a registered item. So nothing surfaced and the agent fell back to a freeform markdown report.
  • ENG-8832 — the record models required fields an operator can't know from enumeration (Share.owner/group/size/last_modified, User.description), and report_item(item: DomainController | MemberServer | ...) was a non-discriminated union, so host role was decided by which optional fields happened to be populated.

Change

  • capability.yaml declares an outputs: contract — built-in finding/asset plus AD types host/user/credential/hash/share/weakness — so the runtime builds the contract-backed report_item and records reach Agent Output (mirrors web-security).
  • items.py (new) holds the record models: a single Host with an explicit role discriminator (domain_controller/member_server); unknowable Share/User fields optional; Weakness.severity aligned to the built-in finding enum (the runtime flattens all types into one report_item schema first-wins, and finding owns severity, so a narrower set here would offer the agent a value the model rejects).
  • Removed the local report_item (tools/reporting.py); the runtime tool of the same name takes over. The coordinator's finding extraction already handles the flat call shape and keys on preserved field names.
  • Updated the discovery/methodology/agent markdown to the new host+role contract.

Tests

  • test_items_outputs.py: the outputs: block resolves against items.py (guards against a typo'd items:ClassName silently dropping a type), unknowable fields optional, host role required/explicit, no cross-type field-shape collisions.
  • test_worker_coordinator.py: binds the coordinator's extracted field names (text/hash_value/severity) to the models so a rename breaks the suite instead of silently dropping findings.
  • dreadnode capability validate clean (warnings are only missing local pentest binaries).

Security note — accepted, by design

Enabling credential/hash as output types moves harvested plaintext passwords (Credential.text) and NTLM/Kerberos hashes (Hash.hash_value) from trace-only logging into the durable, project-scoped item corpus, where they are readable by anyone with project read access and appear unmasked in search_items previews. This is the intended destination — operators need harvested credentials recorded, and web-security already stores sensitive data the same way — and it is accepted here. It is called out because it is a real change in where secrets live: if a project's read-access is ever broader than the engagement team, that scope is the control that matters. No masking/redaction is applied at emit time; if the platform should redact credential-bearing item types at rest or in search, that is a separate platform-side change.

Breaking change

report_item is now the runtime item_type-based tool; the local DomainController/MemberServer union and the old field names are removed. Version bumped 2.2.1 → 3.0.0.

…port_item models (ENG-8848, ENG-8832)

network-ops shipped a local report_item that logged items under ad-hoc class
names via log_output, so nothing ever reached Agent Output; the agent fell back
to a freeform markdown report. The finding models also required fields an
operator cannot know from enumeration and inferred host role from which optional
fields happened to be populated.

- capability.yaml: declare an outputs: contract (built-in finding/asset plus
  host/user/credential/hash/share/weakness) so the runtime builds the
  contract-backed report_item and records surface in Agent Output.
- items.py: move the record models here; single Host with an explicit
  role discriminator (domain_controller/member_server); make Share/User
  fields an operator can't know optional; align Weakness severity to the
  built-in finding enum so the flattened report_item schema has no collision.
- Remove the local report_item tool (tools/reporting.py).
- Update agent/skill guidance to the host+role report_item contract.
- Tests: assert the outputs: block resolves against items.py, and bind the
  coordinator's extracted field names to the models so a rename fails loudly.

BREAKING CHANGE: report_item is now the runtime item_type-based tool; the local
DomainController/MemberServer union and the old field names are removed.

@mkultraWasHere mkultraWasHere left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gold

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants