Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
.git
.gitignore
__pycache__/
*.pyc
*.pyo
*.pyd
.pytest_cache/
.mypy_cache/
.ruff_cache/
.venv/
venv/
data/
logs/
build/
dist/
*.log
node_modules/
tmp/
tmp-*/
*.sqlite3
*.db
*.DS_Store
coverage.xml
htmlcov/
44 changes: 44 additions & 0 deletions .github/workflows/nightly-product-eval.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Nightly Product Evaluation

on:
schedule:
- cron: "0 9 * * *"
workflow_dispatch:
inputs:
mode:
description: "Select dry-run for a no-side-effects preview or run for the full evaluation."
required: false
default: "run"
type: choice
options:
- run
- dry-run

jobs:
eval:
runs-on: macos-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: "20"

- name: Install dependencies
run: |
npm install

- name: Run nightly product evaluation
run: npm run nightly-eval:${{ github.event.inputs.mode || 'run' }}

- name: Upload evaluation artifacts
uses: actions/upload-artifact@v4
with:
name: nightly-product-eval
path: |
docs/notes/eval-logs/nightly-product-eval.log
docs/notes/eval-logs/nightly-product-eval-server.log
docs/notes/eval-logs/nightly-product-eval.json
docs/notes/eval-logs/nightly-product-eval-socketio-smoke.log
84 changes: 75 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# TM-NMapUI

Network Scanner GUI - A web-based network scanning and monitoring tool powered by Nmap.
macOS-first network scanning and monitoring app powered by Nmap.

## Quick Start

Expand All @@ -11,7 +11,7 @@ cd TM-NmapUI
sudo npm start
```

Then open http://localhost:9000 in your browser.
The app will open its local UI automatically. If you need to access it directly, use the loopback URL shown by the launcher, usually `http://127.0.0.1:9000`.

`npm start` also checks for missing Node packages and installs them automatically, so a clean checkout will recover if `node_modules/` has not been created yet.

Expand All @@ -20,36 +20,102 @@ Then open http://localhost:9000 in your browser.
| Layer | Technology |
|-------|------------|
| Runtime | Node.js |
| Desktop Shell | Swift menu bar app |
| Web Framework | Express |
| Real-time | Socket.IO |
| Scanner | Nmap + NSE (Nmap Scripting Engine) |
| PDF Generation | wkhtmltopdf / Chromium |
| XML Processing | xml2js |
| Scheduling | node-cron |
| HTTP Client | axios |
| Cloud Sync | Google Drive API (Python) |
| Cloud Sync | Google Drive helper |

## Requirements

- **macOS** (tested on macOS)
- **macOS** (primary target)
- **Homebrew** (for package management)
- **Node.js** (via Homebrew)
- **Nmap** (with script database updated)
- **Python 3** (for Google Drive integration)
- **wkhtmltopdf** or **Chromium** (for PDF generation)
- **xsltproc** (for HTML report styling)

All dependencies are installed automatically by `install.sh`.

## Installation & Quick Start (macOS)

1. Clone the repository:
```bash
git clone https://github.com/techmore/NmapUI.git
cd NmapUI
```

2. Build and launch the menu bar app:
```bash
open NmapUIMenuBar.app
```

After launch, look for the network icon in your macOS menu bar. The app serves NmapUI on a local loopback URL, defaulting to `http://127.0.0.1:9000` and falling back to the next available local port if needed.
The menu bar app now exposes a `Launch at Login` toggle and an `Uninstall NmapUI` menu action. Uninstall removes the login item registration first and then moves the app bundle to the Trash.

> **Prerequisites**: Xcode Command Line Tools (`xcode-select --install`) and [Homebrew](https://brew.sh) are needed by `install.sh` to pull in `nmap`, `arp-scan`, etc.

## Repository Layout

- Root: stable entrypoints and runtime files such as `server.js`, `install.sh`, and `deploy.sh`
- `NmapUI.app/` and `NmapUIMenuBar.app/`: macOS app bundles produced by the current packaging flow
- `packaging/macos/`: Swift/AppKit shell scaffold for the macOS-native direction
- `docs/guides/`: user and maintainer guides
- `docs/notes/`: internal implementation notes and working analysis
- `docs/audits/`: deeper audit writeups that are not part of the main setup flow

Runtime-only files such as `auto_scan_config.json`, generated scan outputs, local wrapper binaries, and ad hoc scratch directories should stay untracked.

## Admin Commands

Export the runtime database from the Settings tab, or download it directly:

```bash
curl -OJ http://127.0.0.1:9000/api/runtime/export
```

If you are migrating an existing runtime database into the menu bar app bundle, copy the runtime data directory into the bundle resources before launch:

```bash
cp /path/to/runtime.sqlite3 NmapUIMenuBar.app/Contents/Resources/data/runtime.sqlite3
```

The current repository does not include the old installer flow referenced in earlier notes.

## Usage

### Start the server
### Start the app

```bash
sudo npm start
```

The server runs on port 9000 by default. Access at http://localhost:9000
The launcher starts the local runtime on port 9000 by default and opens the app shell around it. On the macOS wrapper, the menu bar icon is the primary way back into the app.

### Nightly Eval

```bash
npm run nightly-eval
```

This runs the nightly product evaluation loop, which boots the app, checks key runtime endpoints and assets, and records an evaluation artifact under `docs/notes/eval-logs/`.

For a no-side-effects preview of the loop shape, run:

```bash
npm run nightly-eval:dry-run
```

To install or remove the macOS scheduler from the repo root, use:

```bash
npm run nightly-eval:launchd-install
npm run nightly-eval:launchd-uninstall
```

### Scans

Expand All @@ -73,10 +139,10 @@ HTML and PDF reports are generated after each scan. Reports include:

```
.
├── server.js # Main application
├── server.js # Main local runtime
├── install.sh # Dependency installer
├── package.json # Node dependencies
├── google_drive.py # Google Drive sync helper
├── google_drive_bridge.js # Google Drive helper dispatch layer
├── nmap-modern.xsl # Report stylesheet
├── config.json # App configuration
├── history.json # Scan history
Expand Down
51 changes: 51 additions & 0 deletions docs/notes/NMAPUI_NIGHTLY_PRODUCT_EVALUATION_LOOP.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# NmapUI Nightly Product Evaluation Loop

This loop is the best fit from the loop library for NmapUI because it tests the
real product surfaces the repo already automates: scan execution, report
generation, Socket.IO replay, and update flows.

## Loop Shape

Trigger the loop nightly or before a release candidate, then run the same small
scenario set every time:

1. Quick scan against a safe target.
2. Deep scan with CVE output enabled.
3. HTML and PDF report generation.
4. Auto-scan schedule validation.
5. Auto-monitor rule validation.
6. Job reconnect and replay handling.
7. Update banner and idle-state flow.

## What To Record

- Scenario results in order.
- The target or config used.
- Generated artifact paths.
- Any blocker or warning.
- The git revision the run came from.

## Stop Condition

Stop only when the same scenario set passes under the same conditions and the
current runtime contract still matches the expected Socket.IO and report
behavior.

## Runner

Use [`scripts/nightly_product_eval.sh`](/Users/seandolbec/Projects/NmapUI/scripts/nightly_product_eval.sh)
for dry runs, recording, or the socket smoke verification pass.

For macOS scheduling, install
[`packaging/macos/com.nmapui.nightly-product-eval.plist`](/Users/seandolbec/Projects/NmapUI/packaging/macos/com.nmapui.nightly-product-eval.plist)
into `~/Library/LaunchAgents/` and load it with `launchctl`.

For GitHub Actions, the workflow supports both nightly scheduled runs and
manual dispatches with either `run` or `dry-run` mode.

## Why This Loop Matters

- It complements the existing PR loop instead of duplicating it.
- It turns the app's existing automation surface into a repeatable validation
loop.
- It gives a consistent artifact trail for regressions and release confidence.
34 changes: 34 additions & 0 deletions docs/notes/native-shell-strategy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Native Shell Strategy

## Decision

This project is now targeting macOS first, so a Swift-based desktop app is on the table.

## Recommended Direction

Keep Nmap as the external scanning engine and move the app toward a macOS-native desktop shell:

- Use Swift/AppKit or SwiftUI for the native shell.
- Keep the scan orchestration and real-time UI behavior, but move the runtime into a packageable desktop app.
- Preserve the current Nmap subprocess model rather than trying to reimplement scanner logic in the app layer.

For a macOS-only target, Swift is the more natural choice than a web-shell wrapper.

## Why Swift Now Makes Sense

- Native macOS look and behavior.
- Better fit for menu bar apps and system integration.
- No need to carry cross-platform abstraction when macOS is the only target.
- Nmap still runs externally, so the app layer can stay focused on orchestration and UI.

## Migration Shape

1. Keep the existing Nmap execution, reporting, and settings logic intact.
2. Move the UI into a desktop shell that can launch a local runtime.
3. Treat Nmap as an installed dependency, not an embedded library.
4. Keep scan state, logs, and report generation accessible through the desktop shell.
5. Replace platform-specific packaging last, after the app behavior is stable.

## Current State

The repository already contains desktop-oriented packaging work and a web-first runtime. The new `packaging/macos/` scaffold is the starting point for consolidating that runtime into a macOS-native app.
33 changes: 33 additions & 0 deletions docs/notes/python-migration-audit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Python Migration Audit

This is the first pass at separating Python that is useful as deployment glue from Python that is part of the product surface.

## Likely glue

- `scripts/nightly_product_eval.sh`
- Mostly orchestration around checks, logs, and automated validation.
- Its timestamp and JSON report generation are now handled without Python.
- The socket smoke path now uses a native Node helper instead of Python.

## Still required for parity

- `google_drive_bridge.js` now dispatches directly to the native helper.
- The legacy `google_drive.py` helper has been removed.
- Any remaining Python usage should be treated as legacy or test-only until proven otherwise.

## Likely product behavior

- Any Python that participates directly in scan/report generation or user-visible export behavior should be treated as product logic, not deployment glue.
- Those paths should be migrated only after a Swift replacement proves parity on the same inputs and outputs.

## Migration order

1. Keep the web frontend and backend behavior unchanged.
2. Replace packaging and launch glue first.
3. Move user-visible helpers next, starting with Google Drive sync plumbing and continue until any remaining native-helper rough edges are closed.
4. Leave core scan/report semantics alone until the native replacements match them closely.

## Current risk

- The deployment story feels brittle when Python is used for startup or packaging glue.
- That makes the shell and launcher the best first targets for SwiftUI migration.
65 changes: 65 additions & 0 deletions docs/notes/swiftui-migration-plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# SwiftUI Migration Plan

Goal: move the macOS experience toward a SwiftUI-first shell while keeping the existing web frontend and backend behavior intact until parity is proven.

## Current State

- The menu bar app shell already exists in Swift under `packaging/macos/Sources/NmapUIApp/`.
- The Google Drive helper path is now native Swift-first through `GoogleDriveHelper`.
- The nightly validation loop runs without Python in the active path.
- The old Docker packaging path and the old Python helper file have been removed from the active runtime path.

## Non-Negotiables

- Keep the menu bar icon and native macOS app feel.
- Keep the web frontend available during the migration.
- Keep the backend behavior stable while implementation details move.
- Keep the fixed loopback port behavior stable for the launcher.

## Migration Phases

1. Audit Python usage
- Complete for the active runtime path.
- Any remaining Python artifacts should be treated as legacy, archived, or test-only until proven otherwise.

2. SwiftUI app shell
- Build a native SwiftUI shell for the menu bar app, preferences, and onboarding.
- Preserve the current open-on-launch and auto-open browser behavior.
- Keep the web frontend as the primary scan/report surface for now.

3. Swift launcher and preferences
- Move startup, restart, port checks, and settings persistence into Swift.
- Keep the runtime contract identical until the UI is fully stable.

4. Replace Python glue
- Convert any remaining deployment helpers and maintenance scripts one by one.
- Keep the backend observable behavior unchanged while swapping implementations.

5. Parity validation
- Verify scan start, scan completion, reports, and local UI behavior.
- Verify launch-at-login, restart, timeout handling, and browser auto-open.
- Verify the nightly eval loop after each migration slice.

## Parity Checks

- App launches from the macOS bundle.
- Menu bar icon appears.
- Browser opens automatically after readiness.
- `http://127.0.0.1:9000` stays the default UI entrypoint.
- First scan completes successfully.
- Reports and settings still persist correctly.

## Recommended First Implementation Slice

1. Finish the Swift launcher so it owns startup, readiness, and browser auto-open.
2. Move preferences and menu bar state into the Swift shell without changing scan behavior.
3. Replace any remaining legacy helper scripts only after the shell and startup flow are stable.
4. Keep the web frontend untouched until the Swift shell can launch, persist settings, and reopen reliably.

## Verification Gates

- `./packaging/macos/bundle.sh` builds successfully.
- `./scripts/nightly_product_eval.sh --run` completes successfully.
- `curl http://127.0.0.1:9000/api/app-identity` returns the expected app identity.
- The menu bar app opens the browser automatically when ready.
- A first scan can still be started and completed from the existing UI.
Loading