Skip to content

wvcollenburg/sizer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

225 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SC// Infrastructure Sizer

A web application that sizes hyperconverged (HCI) clusters from a customer's existing VMware environment and produces branded, ready-to-send hardware proposals.

Point it at a Live Optics or RVTools export (or enter workload numbers by hand), and it recommends how many SC// nodes are needed — CPU, RAM, storage, and IOPS — then generates a PowerPoint deck, Word document, and PDF for the customer, in any of 15 languages.


What it does

  • Imports real workload data — parses Live Optics and RVTools .xlsx exports to derive total vCPUs, RAM, storage, utilized GHz, IOPS, and VM counts (including deduplication of shared datastores reported per-host).
  • Sizes a cluster (app/recommend.py) — picks node counts and hardware to meet compute, memory, storage-capacity, and IOPS demand with N+1 headroom. Compute is sized to an active floor that blends utilized GHz with CPU benchmark scores (SPECrate), not nameplate capacity, so recommendations track what a workload actually consumes.
    • Two sizing modesCertified (fixed SC// appliance configurations) and Validated software-only (same models, disk counts trimmed to need within the supported flash band and per-cluster disk limits).
    • Storage-only nodes — for storage-heavy workloads, extra nodes join the storage cluster with no VMs (low CPU, minimal RAM) to add capacity and IOPS while keeping compute licensing down.
  • Generates deliverables — branded PowerPoint proposal (app/export_pptx.py), Word proposal (app/export_docx.py), and PDF conversions (via headless LibreOffice), including per-recommendation cluster network diagrams (app/cluster_diagram.py) and utilization/benchmark charts.
  • Speaks 15 languages — the UI and every generated export are localized (EN, DE, FR, NL, ES, IT, PT, JA, SV, DA, NO, FI, ET, LV, LT), with CJK-capable fonts bundled for Japanese output.
  • Multi-tenant with accounts — optional login, email-domain-based tenancy, roles, and saved/shared sizings, with a super-admin panel for editing the hardware catalog, CPU data, and sizing tunables live (no redeploy). Includes rate limiting, email verification, account lockout, and a daily GDPR retention/anonymization job.

Features & functionalities

A full inventory of what the sizer does today.

Data import & workload capture

  • Live Optics import — parses the .xlsx assessment export; derives total vCPUs, RAM, storage (provisioned/used), utilized GHz, IOPS, and VM counts.
  • RVTools import — the same, from an RVTools .xlsx export.
  • Shared-datastore deduplication — datastores reported per-host under local names are de-duplicated by a capacity/used/free/VM-count signature, so shared storage isn't counted multiple times.
  • Manual entry — key in workload numbers by hand when no assessment file is available.
  • Guided import wizard — a stepped, walk-through import flow as the default experience, with the original single-page "classic" view kept as an advanced mode.
  • Upload hardening — magic-byte sniffing rejects non-.xlsx files, and row/column caps stop decompression-bomb spreadsheets.

Sizing engine

  • Full-stack sizing — recommends node counts to satisfy compute, memory, storage-capacity, and IOPS demand simultaneously, with N+1 failure headroom.
  • Active compute floor — compute is sized to what the workload actually consumes, blending utilized GHz with CPU benchmark scores (SPECrate) rather than nameplate core counts.
  • Two sizing modesCertified (fixed SC// appliance configurations) and Validated software-only (same models, disks trimmed to need within the supported flash band and per-cluster disk limits).
  • Storage-only nodes — no-VM nodes that join the storage cluster to add capacity/IOPS without adding compute licensing.
  • Day-one consumption caps — bound how full the cluster may be on day one (storage vs. full capacity, RAM vs. N-1), leaving planned growth headroom.
  • IOPS-aware — sizes on per-drive IOPS, with replication-factor write-amplification treated as real demand.
  • Multi-site — size several clusters in one session and roll them into a single combined proposal.
  • Determinant transparency — each recommendation reports which resource (compute / RAM / storage / IOPS) drove the node count.

CPU catalog & performance intelligence

  • 625-CPU catalog — make, generation, and model, with base / all-core / max clocks and P-/E-core counts.
  • Benchmark scores — SPECrate and PassMark per CPU.
  • Benchmark autofill — look up a CPU's performance data on demand (/api/cpu-perf) to feed the perf-based compute floor.

Deliverables & exports

  • Branded PowerPoint proposal — leads with the recommendation, fully branded.
  • Word (DOCX) proposal — the same content as an editable document.
  • PDF — proposal and presentation PDFs rendered via headless LibreOffice.
  • Config slide — a standalone configuration slide (PPTX + PDF).
  • Multi-site exports — one combined document with per-cluster sections, in any format.
  • Cluster network diagrams — a per-recommendation C4-style network diagram embedded in the exports.
  • Replication topology — a cluster-to-cluster replication diagram for multi-site designs.
  • Utilization & benchmark charts — utilization bars and benchmark visuals in the deck and document.
  • Editable vs. read-only — editable source files (PPTX / Word) for Scale users; read-only PDF for everyone else.

Localization

  • 15 languages, UI and documents — EN, DE, FR, NL, ES, IT, PT, JA, SV, DA, NO, FI, ET, LV, LT. Every generated export is localized, with CJK-capable fonts bundled for Japanese.

Accounts, multi-tenancy & collaboration

  • Optional accounts — sign-up, login, logout, email verification, and password reset.
  • Email-domain tenancy — users are grouped into tenants by their email domain automatically.
  • Roles — super-admin, tenant-admin, and user.
  • Saved sizings — save, reload, update, and delete sizings against your account.
  • Share by code — hand a colleague a short code to open a shared sizing.

Administration (live, no redeploy)

  • Hardware catalog — create / edit / delete models, CPUs, NICs, and drives, plus their per-model compatibility.
  • Drive IOPS & sizing tunables — edit the per-drive IOPS table and the sizing constants live, with reset-to-defaults.
  • Catalog import/export — bulk-manage the catalog and models via Excel, with a downloadable template.
  • User & tenant management — list / disable / restore / delete users, change roles, reset passwords, find stale accounts, purge; assign tenant admins and block tenants.
  • Config oversight — list and purge any saved sizing.
  • Email/SMTP settings — configure and test outbound mail.
  • Audit log — a super-admin audit trail of sensitive actions.

Security, privacy & compliance

  • Rate limiting — per-client-IP limits (Redis-backed for exact cross-worker enforcement) on auth, export, and enumeration-prone endpoints.
  • Session & auth hardening — session rotation on login/signup, full clear on logout, constant-time login (no account enumeration), password-length caps, and a CSRF same-origin guard.
  • Email verification — mandatory once SMTP is configured, with a time-boxed temporary-suspend grace.
  • Input hardening — upload content sniffing, HTML-escaped warnings, and an SMTP SSRF guard.
  • GDPR retention — soft-delete plus hard-delete after a retention window via a daily job, plus on-demand purge.
  • Safe-by-default config — refuses to boot in production without SECRET_KEY; debug mode is env-gated off.

Performance & resilience

  • Concurrency-guarded exports — the CPU-heavy document/PDF generation is admission-controlled: a bounded number build at once and excess requests shed gracefully (HTTP 503), so an export burst can't starve interactive traffic.
  • Threaded app tier — gunicorn gthread workers keep light traffic (page loads, sizing calls) responsive while slow exports run.
  • Stateless app tier — signed-cookie sessions and Redis-shared rate limits make the app horizontally scalable behind a load balancer.

Architecture

Layer Technology
Web framework Flask 3 + gunicorn (3 workers × 6 threads, gthread)
Database PostgreSQL 16 (SQLAlchemy ORM)
Rate-limit / shared state Redis 7
Exports python-pptx, python-docx, cairosvg, headless LibreOffice
Frontend Server-rendered Jinja templates + vanilla JS (app/static/)
Packaging Docker + Docker Compose

Key modules under app/: app.py (routes/wiring), recommend.py (sizing engine), liveoptics.py / rvtools.py (importers), tunables.py (admin-editable sizing constants), auth.py (accounts/tenancy), and i18n.py (localization).

Quick start (Docker Compose)

The Compose stack brings up the app, PostgreSQL, and Redis together. The database is seeded automatically on first boot (entrypoint.sh).

# 1. Configure environment
cp .env.template .env
#    then edit .env — at minimum set SECRET_KEY, and SUPER_ADMIN_EMAIL /
#    SUPER_ADMIN_PASSWORD to bootstrap the admin account.
#    Generate a secret with:  python -c "import secrets; print(secrets.token_hex(32))"

# 2. Build and run
docker compose up --build

# 3. Open the app
open http://localhost:5000

The super-admin account is created on boot from .env; sign in there to reach the admin panel and edit the hardware catalog and sizing tunables.

Note: JavaScript and template changes require a rebuild/recreate of the sizer image — there is no host bind-mount. Rebuild with docker compose up --build.

Configuration

All configuration is via environment variables — see .env.template for the annotated list. The essentials:

Variable Purpose
SECRET_KEY Required in production. Signs session cookies; must be shared across gunicorn workers or logins won't persist.
DATABASE_URL PostgreSQL connection string (defaults to the Compose db service).
RATELIMIT_STORAGE_URI Rate-limit backend; point at Redis for exact cross-worker limits.
SUPER_ADMIN_EMAIL / SUPER_ADMIN_PASSWORD Bootstraps the super-admin account on boot (never overwrites an existing password).
APP_BASE_URL Public base URL for links in verification/reset emails (avoids Host-header poisoning).
SESSION_COOKIE_SECURE Set true when served over HTTPS.
ENABLE_SCHEDULER Set 0 to disable the in-app daily retention/GDPR job (e.g. for CLI/one-off processes).

Deployment

The app is designed to run behind an nginx TLS-terminating reverse proxy (one trusted proxy hop; ProxyFix reads X-Forwarded-For/X-Forwarded-Proto). In production set SECRET_KEY, APP_BASE_URL, and SESSION_COOKIE_SECURE=true, and point RATELIMIT_STORAGE_URI at Redis.

Tests

python -m pytest tests/

The suite covers the compute-floor sizing model and the end-to-end perf-based sizing and export paths (tests/). Auth/DB tests run on SQLite via create_all; note that the full seed.py requires PostgreSQL.

Adding a language

A new language means adding both the UI catalog (app/static/js/lang/<code>.js) and the export catalog (app/locales/<code>.json) in the same change, plus registering the code in app/i18n.py. Never ship one without the other, or the UI and the generated documents fall out of sync.

About

experimental scale computing resources sizer

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors