Synthetic Vision System (SVS): GL terrain renderer, nav-data backends, and touch updater#274
Open
billmallard wants to merge 214 commits into
Open
Synthetic Vision System (SVS): GL terrain renderer, nav-data backends, and touch updater#274billmallard wants to merge 214 commits into
billmallard wants to merge 214 commits into
Conversation
…e format Adds EFIS-SVS-011 through -015: three rendering tiers (cpu_sparse/cpu_dense/opengl) with per-tier performance targets, SRTM3 HGT as the standard tile format, full configuration schema, FPM coordinate frame reuse, and display size guidance. Updates SVS-004/008/010 with specifics from terrain data research. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…I integration svs.py: SRTM3 HGT tile reader with bilinear interpolation, LRU tile cache, SVSRenderer that projects a configurable terrain grid (48×48 cpu_sparse, 128×128 cpu_dense) into the AI viewport using the same pixelsPerDeg/roll coordinate frame as the FPM. Clearance colours: green ≥1000 ft, amber 500–999 ft, red 0–499 ft, magenta at/below terrain. Disabled by default. AI: subscribes to LAT/LONG/ALT for SVS position; set_svs_config() wires renderer from screenbuilder. SVS draws before FPM so terrain is behind the flight path marker. Screen YAML enables SVS via options.svs dict. 25 SVS tests: tile naming, HGT loading, void handling, bilinear interpolation, LRU eviction, renderer config, clearance colours, AI paint integration. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…d z-order fix - Add filled terrain quad rendering (QPainterPath-batched by shade bucket, ~32 Qt draw calls regardless of grid size) giving a solid surface instead of dot points - Add Lambertian slope shading: surface normals via np.gradient, fixed NW sun direction, 8 intensity levels × 4 clearance colours = 32 precomputed shades - Add configurable grid line overlay (grid_lines: true, default on) drawn on top of fill and colour-coded by worst-clearance of adjacent vertices - Fix z-order: SVS now renders before the overlay image so yellow wing bars, horizon reference line, bank angle marks, and FPM all appear in front of terrain - Add terrain_fill config key (default true); dots retained as fallback when grid_lines is also false - Add antialiasing hint within SVS draw pass - Add tests/visual_svs_test.py standalone smoke-test script (no fix-gateway needed) Known: animated pitch ladder is a QGraphicsScene item rendered by super() before SVS; moving it in front requires refactoring SVS as a low-Z scene item. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the nested Python loop (n² calls to cache.elevation) with a tile-grouped NumPy pass. Grid points are bucketed by their 1°×1° tile using np.floor; bilinear interpolation runs in NumPy per tile subset. A 20 NM view touches 1–4 tiles regardless of grid resolution, reducing cpu_dense sampling from 16,384 Python calls to 1–4 NumPy operations. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ssian smooth - AMBIENT 0.35→0.10, DIFFUSE 0.65→0.90: shadow faces drop to 10% brightness - N_SHADE 8→32: smoother gradation across the wider contrast range - SLOPE_EXAG=4.0: amplify surface normals so gentle terrain shades dramatically (without this, step_m ~700m >> dz ~30m so all normals point nearly vertical) - 3×3 Gaussian smooth on intensity array: softens hard colour steps at quad boundaries where ridgelines bisect grid cells Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Runway rendering: - _AIRPORT_DB: embedded airport/runway database (KASE seed entry) - _project_point(): reusable lat/lon/alt → AI viewport screen position - _draw_runways(): projects runway rectangles (grey fill, white outline) and flag markers (yellow pole + flag + identifier label) from airport DB Rendering tier: - cpu_ultra: 192×192 grid (~2.3× more quads than cpu_dense, same 128 Qt draw calls; SRTM3 data supports it with margin to spare) Auto-range scaling: - Samples terrain under aircraft each frame (tile cached after frame 1) - range = min(config_max, max(min_range_nm, 0.1×√AGL, 0.001×MSL)) - AGL term reduces range on low approaches; MSL term keeps range long when flying high over a plateau (prevents terrain cutoff in valleys) - auto_range config key (default true) disables scaling for testing - min_range_nm config key (default 8 NM) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds COLOR_WATER (dark blue) as a 5th clearance category so open ocean
renders distinctly from terrain rather than as sea-level green.
Detection handles two SRTM3 product variants:
- Unfilled (NASA original): void pixels are -32768, converted to
_WATER_SENTINEL (-9999) in load_tile; detected as elev < -4999.5
- Void-filled (our downloaded product): ocean pixels are exactly 0.0;
detected as elev == 0.0 after bilinear interpolation
load_tile now returns float32 to preserve the sentinel value.
fill_paths and segs arrays sized to len(_BASE_COLS)*N_SHADE = 160
to accommodate the 5th water category (was 128, causing silent
IndexError that left ocean quads unrendered).
Closes #30 (California coast test — ocean/mountain rendering confirmed).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Conflict in docs/requirements.md resolved by keeping our version: - SVS-004 through SVS-015 (detailed SRTM spec, rendering tiers, YAML schema) - FPM requirements (EFIS-FPM-001 through EFIS-FPM-011) - ALTSEL requirements (EFIS-ALTSEL-001 through EFIS-ALTSEL-010) Upstream had simpler SVS-004 through SVS-010 and no FPM/ALTSEL sections. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
True Airspeed: - Solid white box pinned to bottom of airspeed tape, full width - Small 'TAS' label above the value, both in dark text on white - Sourced from FIX key TAS; shows '---' on fail/bad/old - Controlled by show_tas option (default: true) Airspeed trend arrow: - Cyan vertical line + filled arrowhead overlaid on tape at pointer edge - Points up when accelerating, down when decelerating - Magnitude represents projected airspeed in trend_lookahead seconds (default: 6s) - Smoothed over trend_window seconds (default: 3s) to suppress noise - Suppressed below trend_min_change knots noise floor (default: 0.5kt) - Controlled by show_trend option (default: true) All parameters configurable per-tape instance via YAML options. Fixed duplicate signal connection bug in resizeEvent (UniqueConnection). Also removes SCREEN_ANDROID from default screen list; the Weston/xwininfo dependency crashes on Windows and the screen is not functional there. Adds docs/requirements.md with EFIS-TAS, EFIS-TREND, and EFIS-SVS requirement sets as a baseline for ongoing PFD feature development. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Captures the rendering tier table (cpu_sparse / cpu_dense / cpu_ultra / opengl), cell-size math at typical ranges, the auto_range formula, the grid_lines wireframe overlay, native SRTM3 resolution numbers and what that would cost on the CPU path, and the OpenGL tier as the path forward. Notes practical guidance for screenshot work (cpu_dense at 30 NM range is the sweet spot for visible depth without flattening). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Lets the screenshot-iteration loop toggle the terrain wireframe overlay without editing the script. Defaults to true to match the SVSRenderer default. Useful for comparing clean shaded fills against the gridded view at long range, where the wireframe densifies into a mesh from perspective foreshortening. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Captures the analysis we worked through on: - Why Virtual VFR's visual polish isn't a fidelity ceiling (it's curated decorations, not expensive geometry — SVS terrain is the costly path). - The two projection models and their concrete divergence (horizon dip 1.36° at 6,000 ft, growing with altitude; the COURSE vs HEAD heading reference issue in VirtualVfr.py:72). - Four alignment options, with replicating VVfr features inside SVS as the recommended path; option 1 (view-declination correction) is cheap enough to do alongside. - Hardware budget — current SVS tier targets are estimates, not Pi measurements. Profile first, buy Pi 5 only if needed. - Issue #28 (NASR runway/airport database) scoped at ~250-300 LOC across downloader / importer / loader / SVS integration / tests / docs. Recommended ordering: profile, view-declination fix, #28, file a separate issue for the COURSE/HEAD question. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Companion to svs_planning.md section 4. Covers the lab/desk vs aircraft-installable split, the three aircraft tiers (DIY Pi 5 + TracoPower converter, Compulab Fitlet 4 mid-tier, Onlogic high-end), and the practical knock-on issues that aren't on the spec sheets — vibration mounting, display selection, engine-start brownouts, heat in a cowled installation. Concludes with a recommended purchase order: Beelink Mini S13 for the desk first, then re-evaluate based on profiling results. Lists four open questions (12V vs 24V system, display choice, performance ceiling target, certification path) for the author to resolve while researching. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A large session of related SVS improvements. Bundled into one commit for
the push; happy to break into focused commits via interactive rebase
before opening the upstream PR.
Terrain rendering
-----------------
- Add polar (range, azimuth) renderer tier with quadratic radial warp:
~90 m cells near the aircraft (matches SRTM3 native sample size),
coarsening toward the horizon. No samples wasted behind the aircraft.
Configurable via radial_warp / n_range / n_az / fov_deg / r_min_nm.
New POLAR_DEFAULTS — 80 x 120 fan, warp 1.5, 140 deg FOV — tuned
for both perf and visual quality on Pi-class hardware.
- Vectorise the per-cell fill loop and per-edge grid-lines loop with
whole-array NumPy ops + index sort/group by shade key. ~1.5x faster
across all tiers, bit-identical output to the previous loop.
- Slope shading for the polar grid: rotate the per-index gradients
into geographic (E, N) via per-column bearing so the same hill shades
consistently regardless of viewing heading.
AI scene-graph integration
--------------------------
- Move SVS rendering into the AI's QGraphicsScene as a low-Z item so
it participates in scene z-order. Pitch ladder now renders in front
of the terrain. Z value configurable via svs.z_value (default 0.5 —
above sky/land background at z=0, below pitch ladder at z=1).
- Graceful FIX-key initialisation in AI.__init__: missing keys (TRACK
etc.) log a warning and leave the Flight Path Marker disabled rather
than raising KeyError. Same treatment for SVS LAT/LONG/ALT.
- Add TRACK to tests/conftest.py so the existing AI integration tests
pass (was missing since FPM-GPS was added in an earlier commit).
Airport / runway database (FAA NASR + CIFP)
-------------------------------------------
- New src/pyefis/instruments/ai/airport_db.py with two backends:
* NASRAirportDB (preferred) reads airports.sqlite built by the new
tools/build_airport_db.py from the FAA NASR APT_BASE / APT_RWY /
APT_RWY_END CSVs.
* CIFPAirportDB (fallback) wraps pyavtools.CIFPObjects.
Both expose the same airports_in_range(lat, lon, range_nm) API.
Hand-coded _AIRPORT_DB (KASE only) remains as a last-ditch fallback.
- New tools/build_airport_db.py joins the three NASR CSVs into a
5.6 MB sqlite carrying widths, displaced thresholds, marking types,
TDZ elevations, and approach lighting. ~1 second to rebuild.
- .gitignore now excludes cifp/ and nasr/ — they're user-supplied data
drops, not source.
Tier C runway surface markings (FAA AC 150/5340-1L)
----------------------------------------------------
- New SVSRenderer._draw_runway_markings paints (when within
detail_distance_nm, default 3 NM) on top of the runway quad:
threshold bars, centerline stripes, designators, aiming point, TDZ
markers (PIR runways only), side stripes (PIR only), and displaced-
threshold chevrons. Marking categories driven by RWY_MARKING_TYPE_CODE
from NASR (PIR / NPI / BSC).
- Asphalt fill darkened to (55, 55, 55) so white markings read crisply.
- Designators use QTransform.quadToQuad: each character occupies its
FAA-spec slot (5.33 ft "1", 24 ft L/C/R, 20 ft other digits) and the
font is warped to fit, so markings foreshorten with the runway just
like the threshold bars. "1" is hand-drawn (no font) as a plain
vertical stroke — real runway "1"s have no flag/foot. L/C/R letters
render on their own row below the numbers per the AC.
Tests
-----
- Six new TestPolarTier cases (default config, overrides, warp math,
paint at varied headings).
- New test_ai_construction_tolerates_missing_fpm_key.
- All 31 SVS tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
On a normal landing approach the screen was filling up red — every cell of terrain below the aircraft but within 1000 ft of clearance fell into the warning/caution bands, which is the natural state of affairs in the pattern and not a useful signal. When the aircraft is within ``airport_proximity_nm`` (default 5 NM) of any airport in the SVS airport database, terrain colouring now uses a simpler 2-colour scheme: * SAFE (green) — terrain below the aircraft * CONFLICT (magenta) — terrain above the aircraft Water remains its own colour. Off-airport behaviour is unchanged: the full 500/1000 ft warning/caution thresholds apply as before. Set ``airport_proximity_nm: 0`` to disable the rule (revert to legacy behaviour everywhere). A more nuanced "elevation-aware" alternative is captured for follow-up in #32. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Brings the FAA Digital Obstacle File (DOF) into the SVS view. Towers,
antennas, tall buildings, stacks, water tanks, oil rigs and wind
turbines now render as vertical poles from base elevation to tip
elevation, with a small filled circle at the tip.
Pole colour follows the same convention as terrain shading:
* CONFLICT magenta when the tip is at or above the aircraft altitude
* lit-red / lit-white / lit-dual when the obstacle has obstruction
lights (LIGHTING code R / W / D)
* grey when unlit or unknown
Implementation
--------------
* New tools/build_obstacle_db.py — DOF.CSV → obstacles.sqlite (71 MB,
~636k records nationwide, builds in ~9 s). FAA's CSV is Latin-1; we
decode that explicitly. No height filtering at import — that's a
runtime config so users can tune without rebuilding.
* New src/pyefis/instruments/ai/obstacle_db.py — ObstacleDB with
obstacles_in_range(lat, lon, range_nm, min_agl_ft) generator,
mirroring the AirportDB API. Construction never raises; if the
sqlite is missing the renderer no-ops.
* SVS config keys:
- dof_db_path path to obstacles.sqlite
- obstacle_min_agl_ft default 200 (matches FAA's charting threshold)
* SVSRenderer._draw_obstacles called after _draw_runways inside the
polar/rect draw block, using the same _project_point routine the
airport flag pole uses.
* visual_svs_test.py picks up dof/obstacles.sqlite automatically when
present; the SVS_DOF_PATH env var overrides.
* .gitignore: dof/ joins cifp/ and nasr/ as user-supplied data drops.
Refs
----
* Closes-when-shipped: #34
* Companion roads work tracked in #35
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Single-page overview: project context, where the working tree lives, fork/upstream policy, how to run tests and the visual harness, the data-dependency paths, an architecture map of the SVS modules, the code conventions visible in this code (graceful FIX-key init, construct-never-raises loaders, _WATER_SENTINEL, _project_point as the projection chokepoint), pointers to existing docs, and a one-line summary of each open issue. Lets a fresh session pick up the SVS work without re-deriving the data-source plumbing or the fork/branch context. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Operational doc for the NASR / CIFP / DOF refresh cadence. Captures the URLs, the expected on-disk layout under nasr/, cifp/, dof/, the build commands and expected output, per-source verification queries, and the troubleshooting cases we've actually hit during this branch (Latin-1 decode in DOF, partial downloads, stale sqlite after refresh). Companion to CLAUDE.md and docs/svs_planning.md: this is the how-to-keep-it-current ops doc. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The screenbuilder's 'ai' branch already forwards `svs:` options to SVSRenderer.set_svs_config(); the 'virtual_vfr' branch did not. Since VirtualVfr subclasses AI it inherits set_svs_config — wiring this up takes a few lines and lets the PFD screen enable SVS via screen YAML without changing widget types. Without this change, `svs:` in a screen YAML under a virtual_vfr instrument is silently ignored — exactly what was happening when SVS was first tried on real hardware. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A focused PFD layout for hardware that's tight on CPU budget for SVS,
or for SVS development/demos. Uses the same includes/ahrs/virtual_vfr.yaml
bundle as the standard PFD (attitude indicator + airspeed and altitude
tapes, VSI, HSI, wind display) but stretched across the full 200-column
width, dropping the engine-monitoring arcs/bars and the radio panel.
To make it the default screen, set in main.yaml:
defaultScreen: PFD_AI_ONLY
Or leave it in the screen list and switch to it via the button bar.
Note: the new screen file needed -f to add through the (overly broad)
config/ gitignore rule; existing tracked files in src/pyefis/config/
modify normally. .gitignore left untouched in this commit.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Conflicts: * src/pyefis/screens/screenbuilder.py — upstream refactored the monolithic screenbuilder into per-concern modules (screenbuilder_config, _factory, _options, _layout, _display, _encoder, _preferences, _overlay). Our yesterday's `virtual_vfr` SVS-config wiring no longer applies to the old setup_instruments body. Resolution: keep the upstream modular init_screen / setup_instruments. Re-apply the SVS wiring in screenbuilder_factory by adding build_atitude_indicator and build_virtual_vfr functions; both check for `svs:` in options and call widget.set_svs_config() after construction. Both factory dict entries switched from one-line lambdas to those builders. Notable upstream picks: * 9edf401 Make Xlib import optional so weston instrument loads on non-X11 — this is the fix for the xwininfo crash we worked around yesterday on the Pi. Once this lands on the Pi we shouldn't need to keep ANDROID out of the screen list to avoid weston crashes. Removed-upstream files we never authored (kept as deletions): * src/pyefis/screens/{pfd,pfd_sm,r582_sm,epfd,ems_sm}.py * src/pyefis/instruments/gauges/egt.py * src/pyefis/config/examples/{rotax_ems,ems,beagle}.yaml Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Captures the design and step-by-step path for moving terrain rasterisation off the CPU and onto the Pi 5's V3D GPU, building out the `opengl` renderer tier that's been a stub in GRID_SIZES since the original SVS commit. Key design decisions baked into the doc: * FBO render + blit into the existing SVSGraphicsItem.paint() rather than changing the QGraphicsView viewport — keeps the scene-graph z-order and CPU overlays (runways, markings, obstacles) intact. * Polar (range, az) mesh topology reused as-is from the CPU tier; config keys (n_range, n_az, fov_deg, radial_warp) stay the same. * Elevation lookup via heightmap texture so the mesh stays static; the texture re-uploads only on integer-degree boundary crossings. * Lambertian shading + clearance bucket colouring + 2-colour airport-proximity rule all move into fragment shaders. * Hard fallback path: any GL init failure logs a warning and downgrades self.renderer to "polar"; no hard crashes. Eight numbered implementation steps from scaffolding through tile paging to perf measurement, plus a Stage-2 roadmap for moving the vector overlays onto the GPU later. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
First step of docs/svs_opengl_plan.md. Lands the safety net that every subsequent GL step depends on: a new svs_gl module containing the SVSGLRenderer skeleton, plus lazy-init and per-frame fallback machinery in SVSRenderer.draw() that downgrades to the polar CPU tier on ANY exception from the GL path — both at construction (missing/incompatible Qt OpenGL bindings) and during draw (NotImplementedError today, shader compile errors / runtime GL failures later). Once the fallback is hit, self.renderer is set to "polar" and we never re-attempt GL in the process. The user-facing behaviour of ``renderer: opengl`` is therefore "use GPU if possible, else polar silently and log a warning." Behavioural deltas for existing tiers: zero. The dispatch only fires when self.renderer == "opengl". Tests: * TestSVSGLFallback.test_renderer_initially_opengl — config-time choice preserved, GL not yet probed * test_first_draw_falls_back_to_polar — NotImplementedError downgrade * test_subsequent_draws_use_polar_without_retrying_gl — no re-init thrash after fallback * test_init_failure_also_falls_back — constructor exceptions handled with a monkeypatched broken SVSGLRenderer All 35 SVS tests pass; 133 screenbuilder tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Smallest possible "GPU pixels through the existing pipeline" milestone.
Per docs/svs_opengl_plan.md step 2:
* SVSGLRenderer owns a QOpenGLContext bound to a hidden QOffscreenSurface
* Per frame: makeCurrent, render the test quad into a size-matched
QOpenGLFramebufferObject, fbo.toImage(), doneCurrent (restoring
whatever context Qt had current)
* The parent QPainter then drawImage()s the result at viewport (0, 0)
* The SVSGraphicsItem.paint() pipeline is unaware anything changed —
scene z-order, CPU overlays, and the airport-proximity rule all
continue to behave as today
Implementation notes:
* Pre-3.0 GLSL (attribute / gl_FragColor / precision mediump float)
for portability across Pi 5 V3D and desktop drivers. Will bump to
GLSL ES 3.00 in Step 4 when we need texture sampling for heightmaps.
* Qt-native QOpenGLContext / QOpenGLShaderProgram /
QOpenGLFramebufferObject / QOpenGLVertexArrayObject for the
high-level resource management. PyOpenGL ('from OpenGL import GL')
for the few raw glClear / glDrawArrays / glViewport calls Qt doesn't
wrap with constants.
* Core-profile desktop drivers require a bound VAO for any
glVertexAttribPointer call — created and bound around the draw.
* Distinctive teal (0.18, 0.55, 0.55 → rgb 46,140,140) chosen so a
successful Step 2 frame is visually unmistakable in screenshots.
Tests:
* test_step2_paints_teal_via_gpu — positive verification when a GL
context is available; skips gracefully on headless / offscreen
test environments. Verified manually on Windows native: produces
rgb(46,140,140) at the centre as expected.
Verified manually on Windows native (rgb(46,140,140) at centre); Pi 5
verification queued.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…teless calls
PyQt6 doesn't expose QOpenGLFunctions, so we can't use Qt's wrapped GL
function table for the raw glClear/glDrawArrays. PyOpenGL works fine
for those but its glVertexAttribPointer wrapper tries to record the
pointer in per-context storage — which fails under Qt's EGL/eglfs
context on the Pi ("Attempt to retrieve context when no valid context").
Resolution: split the GL surface area along the boundary where context
tracking matters.
* Vertex setup (VBO upload + attribute → buffer binding) goes through
Qt-native wrappers: QOpenGLBuffer for the static fullscreen-quad VBO,
QOpenGLShaderProgram.setAttributeBuffer to wire the attribute index.
These never touch raw glVertexAttribPointer.
* Stateless raw calls (glViewport / glClearColor / glClear /
glDrawArrays) go through PyOpenGL. Setting
OpenGL.STORE_POINTERS = False + OpenGL.ERROR_ON_COPY = True at
import time disables the per-context pointer cache that was tripping
on the Pi for the few stateless calls we do make.
Verified manually on Windows: rgb(46,140,140) at centre, fallback
tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per docs/svs_opengl_plan.md step 3, the polar (range, azimuth) fan moves to the GPU as an indexed triangle mesh: * Per-vertex attribute is (t, az_deg) where t parameterises the radial axis and az is degrees from the nose. The mesh and index buffer are built once at first draw from the parent SVSRenderer's polar config (n_range, n_az, fov_deg) and uploaded to QOpenGLBuffer-backed VBOs. * The vertex shader carries the polar -> aircraft-frame -> screen projection in GLSL ES 1.00, mirroring the CPU polar tier's math: radial warp -> r_nm -> r_deg, elevation angle to the z=0 plane (terrain still ignored; that arrives in Step 4 as a heightmap texture), pitch / roll, NDC. Step 4 swaps the elev_angle calc for a texture-sampled height. * The fragment shader colours each cell by its t band (eight concentric arcs, red near -> blue far) so the polar topology is visible during bring-up. Verified manually on Windows native: * Forward, no roll: bands project as horizontal stripes in the lower half of the viewport — correct geometry for z=0 plane viewed level at altitude. * Roll = +30 deg: bands tilt 30 deg, vertex shader rotation matches. Tests: * TestSVSGLFallback.test_step3_polar_mesh_renders replaces the old teal-quad assertion: confirms the renderer stays in opengl mode (no fallback) and that the polar fan paints something below the horizon scan row. Skips cleanly on offscreen / headless environments. All 35 SVS tests pass on Windows. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per docs/svs_opengl_plan.md step 4, terrain elevation moves from "z=0 plane" to a real heightmap texture built from SRTM3 tiles. Per vertex, the vertex shader samples the elevation at that vertex's world (lat, lon) and uses it for the perspective projection. Implementation notes: * Heightmap is a 2x2 SRTM3 tile patch (2402 x 2402 R32F samples, ~23 MB texture) centred on the aircraft. Built lazily on the first draw and re-uploaded only when the aircraft crosses an integer-degree boundary, so per-frame cost is just a glActiveTexture + glBindTexture. * Tile rows arrive row-0-at-north from svs.py's TileCache; we flip each tile vertically when assembling the patch so the texture u/v coordinate runs west->east / south->north (matches GL convention, shader UV (0,0) = SW corner). * Shaders bump to GLSL ES 3.00 / GLSL 1.30 to use texture() and user-defined fragment outputs. The #version header is selected at shader-compile time based on the actual context's renderable type (ES on Pi 5 eglfs, desktop on Windows native) so the same source body works on both. * Vertex shader is now the full polar-tier projection from svs.py: warp -> r_nm -> r_deg -> world (lat, lon) -> texture sample for elev_m -> elevation angle -> aircraft frame -> pitch/roll/viewport -> NDC. * Fragment shader outputs greyscale by elevation (0..15000 ft -> black..white) so a real terrain silhouette is visible for the bring-up. Lambertian shading + clearance buckets arrive in Step 5. Verified manually on Windows native: * Flat 500 ft test tile renders as a dim ground (~8/255 grey) below the horizon with clear sky above. Renderer stays in opengl mode. * Gradient test tile (0 ft north -> 13000 ft south) produces a geometrically-correct mountain silhouette in the upper screen. Pi 5 verification queued. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per docs/svs_opengl_plan.md step 5, the fragment shader now produces the SVS look: shaded terrain coloured by the same clearance buckets (safe/caution/warning/conflict/water) the CPU polar tier uses. Vertex shader changes: * Computes the world normal via finite differences against the two neighbour heightmap texels (one east, one north). The texel size in metres is derived from the patch span and lat_cos, matching the CPU code's per-cell-step adjustment. * Applies the SLOPE_EXAG = 4.0 amplification that the polar CPU shading uses so the Lambertian response is visually comparable. * Sun direction (-1, 1, 2) in geographic (E, N, Up) frame, normalised - identical to svs.py. * Outputs interpolation values to fragment: v_intensity (AMBIENT + DIFFUSE * diffuse), v_clearance_ft (= u_ac_alt_ft - elev_ft), v_is_water (1 for cells below the sentinel threshold). Fragment shader changes: * Five-way bucket on v_clearance_ft: water -> conflict -> warning -> caution -> safe. Thresholds (u_green_ft, u_yellow_ft) come from the parent SVSRenderer's config so the GL and CPU paths use identical breakpoints. * base colour multiplied by v_intensity for the shaded output. * No per-cell Gaussian smoothing — fragment-shader interpolation already gives smooth intensity across triangles, vs the CPU's per-cell flat shading that needed it to soften ridge boundaries. Verified manually on Windows native: * Heading-toward-high-terrain test (gradient tile, aircraft below the peaks): renders CONFLICT magenta at the expected (155, 0, 155), matching COLOR_CONFLICT * 0.86 intensity from the upper-NW sun. Pi 5 verification with real Aspen SRTM queued. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Wires the GL fragment shader to the same near-airport rule the polar CPU tier uses in svs.py:595. When the configured airport_db reports any airport within airport_proximity_nm (default 5), the warning and caution clearance bands are suppressed so the screen shows only: - water blue - SAFE green for terrain at or below the aircraft - CONFLICT magenta for terrain above the aircraft Previously a normal stabilised approach toward a runway lit half the screen WARNING red / CAUTION amber even though the pilot was intentionally low. The per-frame proximity check (one airport_db query) reuses SVSRenderer's existing airport_db and airport_proximity_nm, so any config tuning applied to the polar tier also applies to the GL tier.
The 2x2-tile patch around the aircraft used floor(ac_lat - 1.0) for its origin, which centred the patch when ac_lat sat exactly on an integer degree but pushed the aircraft right to the north/east patch edge as it approached the next integer. At that point the heightmap sampler's CLAMP_TO_EDGE would replicate the boundary elevation for the entire forward fan, producing fake-flat terrain ahead of the nose just before each crossing. Switch to floor(x - 0.5): the aircraft is now always >= 0.5 deg (~30 NM) from every patch edge, and the rebuild fires at half- integer-degree crossings instead of integer crossings. Same rebuild frequency, but always with usable forward terrain in the texture. Adds three boundary-detection tests that don't need a live GL context: sweep verifies the >= 0.5 deg buffer holds across a representative lat/lon grid, crossing test pins the rebuild trigger to half-integer degrees, and negative-longitude test confirms floor() correctly handles the western hemisphere.
A long transfer (a terrain or the 3.9 GB water pack) previously trapped the user on the install screen with no way out. Add a Cancel button in the busy/progress mode: it terminates the updater (SIGTERM), which unwinds the download cleanly and leaves the installed data untouched (verify-then-atomic-swap), then drops back to the pack picker with the selection preserved so they can adjust, retry, or Cancel out to the status screen. The updater's exit is routed through the existing finish handler (and the errorOccurred path, since terminate() can surface as a crash); a race where the install actually completes (exit 0) before SIGTERM lands is treated as success, not a cancel. A safety-net kill fires if the updater ignores SIGTERM so the user is never stuck on a spinner. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an SVS section covering the GL terrain/water/obstacle/airport rendering on the attitude widget, the makerplane-data navigation-data integration (OTA/USB updater, Data Status screen, DATA flag), and how to enable it (nested svs: block). The README otherwise described only the base EFIS, omitting this branch's headline feature. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document minimum vs suggested hardware grounded in the reference Pi 5 unit (8 GB, Debian 13, Python 3.13, VideoCore VII). Key points the user asked for: a supported GPU is required only when Synthetic Vision is enabled (SVS is GL-rendered; the core instruments are CPU/QPainter-drawn, so no GPU is needed with SVS off — the recent draw-path work made SVS faster but didn't remove the GL requirement); and storage guidance — a large A2 microSD or, preferably, an M.2 HAT + NVMe SSD for the data store, since terrain packs run tens of GB. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
If SVS is enabled on a machine with no usable GPU, the GL renderer disables itself (gl_failed) and the AI correctly falls back to the normal sky/ground attitude indicator. But the "SVS UNAVAIL" annunciation read self.svs directly, which the screenbuilder clobbers to the YAML config DICT on a real device — getattr(dict, "gl_failed") is always False, so the warning never showed and the pilot got a silent downgrade. Factor the clobber-safe renderer resolution _svs_drawing already used into a shared _live_svs() helper and route the annunciation through it, so the pilot is told when synthetic vision they configured isn't running. Add a test that reproduces the clobber + gl_failed locally (no GL), and document the graceful-degradation behaviour in the README. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Capture the plan to port the legacy VirtualVfr PAPI glidepath guidance into the GL SVS: examination of the existing code, the data finding (NASR VGSI/glide-path-angle not yet captured), and a two-tier plan (Tier 1 synthetic parity in GL, Tier 2 real data-driven VGSI). Tracked as a GitHub issue. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The boxed heading readout showed a bare integer ("30", "3"). Always
zero-pad to three digits and append a degree sign via a small testable
_fmt_heading helper; 360 wraps to 000°.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The font was sized to a "999" mask; the trailing degree sign made the rendered "030°" wider than that and pushed the centred text against the box edges. Include the ° in the mask so the font scales to the real content and the side margins return. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…red) build_altimeter_tape forwarded only dbkey, and Altimeter_Tape hardcoded majorDiv/minorDiv/total_decimals/font_mask — so YAML options on a tape (e.g. a VS tape's minorDiv/majorDiv) were silently dropped and the tape always rendered at its defaults (minorDiv=100). Accept the options as constructor params and forward those present in the config (defaults unchanged for anything unset, so existing tapes are unaffected). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A VS tape's numeric readout scrolled its odometer digits frantically because VS is jittery. Add a round_to option: the value shown in the numeric box snaps to that step (e.g. 100 fpm), so the trailing scroll digits settle and the box stops rolling. The tape scroll itself stays on the raw value. Off by default (round_to=0); forwarded by the factory. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
With build_altimeter_tape forwarding the tape options (d26231a), set sensible defaults in the AHRS include: the altitude tape labels every 100 ft; the VS tape uses coarser 200 fpm ticks and rounds its numeric box to 50 fpm so the odometer no longer scrolls frantically on jittery VS. Previously these options were silently ignored, so this just makes the intended behavior real for a fresh install. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The numeric readout is a NumericalDisplay baked inside Altimeter_Tape, so there was no way to hide it. Add numeric_box (default True); when False the embedded box is omitted and only the scrolling tape + its fixed read pointer render. Guards the old/bad/fail setters and redraw against the absent box. Forwarded by the factory. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a font_scale multiplier (default 1.0) applied to the tape's tick label font, so a tape's scale numbers can be sized up without touching tick spacing. Forwarded by the factory. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fold the dialed-in VSI/altitude tape config into the AHRS include so a fresh install matches: altitude tape labels every 100 with font_scale 1.1; the VS tape is tape-only (numeric_box: false, no readout box), ticks every 200, rounds to 50 fpm, font_scale 1.45; and the stray "VSI" static_text that labeled the removed box is dropped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…cks) Airport data is now provider-based: the US FAA NASR pack stays the primary, and supplemental provider packs (e.g. airports-canada from OurAirports, which NASR lacks) are auto-discovered under airport_provider_dir (<dir>/*/current/airports.sqlite) and merged at query time, de-duped by ICAO. A new provider needs no config change. make_airport_db returns the single backend when there's only one, a _MultiAirportDB when several. Documented the new svs option in the config example. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
So supplemental airport packs (airports-canada, …) appear in the on-device pack picker alongside the other navdata. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ized The --ocean-max-vertices help advised setting it low (32) on the premise that "coastline gets no benefit" from a high cap. That was wrong: a low cap smooths coastal bays/inlets into land — at cap 32 the Baie de Gaspe (by CYGP) rendered as land. Measured the data: only ~1 in 100 ocean polygons are complex enough to use the cap; open ocean simplifies to a few verts regardless. So ocean should match the inland cap (the arg's default behavior) — coastline detail for ~+10% size. Corrected the help. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rrect HSI.resizeEvent rebuilt the scene and re-applied self.rotate(-self._heading) without first clearing the view transform. The rotation is a view transform that survives the scene rebuild, so every resize after the first (Qt fires it several times during layout, and again on any geometry change) stacked another -heading rotation -- baking in a permanent offset equal to the heading at that moment. The card then showed heading+offset (observed: 290 deg when actually 052) while the numeric readout (which reads HEAD independently) stayed correct. Add resetTransform() before the rotate so each resize re-applies the absolute -heading from identity; setHeading continues to track incrementally from there (this is what DG_Tape.redraw already does). Regression test asserts repeated resizes are idempotent and the transform is exactly the heading rotation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three touchscreen fixes to the data-pack picker: - Tap vs. scroll. Rows toggled their checkbox on mouse *press*, which fired before a gesture could be recognised as a scroll, so two-finger scrolling kept flipping whatever row a finger first landed on. Toggle on *release* instead, and only when the pointer moved <= 12px since press; a scroll drags past that and no longer selects, while a clean tap still toggles. - Scroll hint. Added a muted "Tap a row to select - use two fingers to scroll" next to the title so the gesture is discoverable. - Opaque overlay. The picker (and the drive-chooser overlay) is raised over the live data-status screen, but as a plain QWidget it ignored its stylesheet background, so the screen behind bled through the title/footer margins around the scroll list. Set WA_StyledBackground so the dark background fully blocks what's underneath. Adds no-network unit tests for the tap/scroll/stray-release row behaviour. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
|
This is a large PR — it's the full SVS body of work we've been discussing over email — so if it's easier to take in without touching If you create an integration branch (e.g. If you'd rather pull it straight onto a branch yourself instead of merging: Either path works — whatever fits your review flow. And as noted in the description, a merge commit (not squash) keeps the per-commit history intact, which is the reviewable unit for a change this size. |
The runner's pre-baked apt index can point at a libegl1 dependency .deb (e.g. libegl-mesa0) that Ubuntu has rotated off the mirror, so a bare `apt install` 404s and fails the build before lint/pytest run. Refresh the index first so the package URL re-resolves. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two things kept CI red after the apt-index fix: - numpy and PyOpenGL are real SVS dependencies (svs.py / svs_gl.py import them at module level) but were never declared, so pytest hit ModuleNotFoundError collecting the AI/SVS tests. They were present on dev and Pi installs, which masked it. Add an optional 'svs' extra (base pyEfis imports the SVS/GL modules lazily and runs without them) and install .[qt,svs] in CI. - test_void_values_replaced_with_zero asserted the old void->0 behaviour, but void SRTM cells are intentionally replaced with _WATER_SENTINEL (coastal interpolation needs the magnitude to detect water). Update + rename the test to assert the sentinel. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tests/visual_svs_test.py is a manual visual harness, not an automated test: it builds a QApplication and calls app.exec() at module level (no __main__ guard, no test functions). pytest imports modules during collection, so collecting it enters the Qt event loop and hangs the run indefinitely. (The earlier numpy ImportError masked this by aborting the import first.) Exclude it via addopts --ignore, matching the existing tests/end_to_end/repo ignore. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- test_main FakeApp: add a no-op class-level setAttribute. main() now sets
AA_ShareOpenGLContexts before constructing the app (required pre-construction
for the SVS GL share group); without the stub every patched_runtime test
raised AttributeError mid-main(). That failure, hitting while the suite's
logging.getLogger mock was active, also corrupted the pytest logging plugin
at session teardown ('Mock' object is not iterable), which aborted the
FAILURES summary and masked other errors.
- test_data_status: the row now toggles on release (with a movement slop) so
two-finger scrolling doesn't select; update the tap test to send a full
press+release instead of press-only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_build_patch clamps the heightmap to min(config cap, GL_MAX_TEXTURE_SIZE). Without a current GL context, glGetIntegerv raises on some platforms but RETURNS 0 on others (PyOpenGL on headless Linux) — and min(cap, 0) drove the decimation loop to shrink the patch toward 1px. This is why the test_glo30 mixed-resolution patch tests passed on Windows (query raised, config cap kept) but failed on the Linux CI (query returned 0). Only apply the driver limit when it's a positive value; otherwise keep the config cap. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
test_main_sets_requested_log_level mocks logging.getLogger to return a bare Mock. Newer pytest's logging plugin iterates logging.getLogger().manager.loggerDict around each phase, so the bare Mock raised "'Mock' object is not iterable" in teardown, which prevented the monkeypatch from being restored and cascaded ERRORs into every later test (test_main tail + test_version) and aborted the FAILURES summary. Give the mock an empty real loggerDict so the plugin can iterate it. Pre-existing isolation issue in this upstream test, exposed by the CI pytest version (older local pytest doesn't iterate loggerDict). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
It holds dev-environment specifics (ssh alias, local paths, Pi IP) that don't belong in the public upstream contribution. Untrack it and gitignore it so it stays on disk as a local orientation doc only.
Collaborator
|
@billmallard How large are the data sets for this? |
Contributor
Author
|
It's ready for review and early adopters. I would consider this a strong alpha or early beta level. I've been in email exchanges with John and Jack about it. Maybe connect with them for their opinion. If you can make an "svs" branch for this I can rebase the pull request to that branch. Might make an easier and safer review process. There is a complete data management back end currently hosted at https://navdata.aerocommons.org/. Data sets are pretty large (10's of gigs) and constantly being updated (monthly) by the FAA and other sources. CI builds poll for fresh data daily. Data and automatically make it available at https://navdata.aerocommons.org/. The big weaknesses are around water rendering. I'm still covering all the edge cases on that, but it's ready for others to see and try it. I'll be adding more features over time as well. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Synthetic Vision System (SVS) + nav-data pipeline + touch updater
This is the comprehensive SVS contribution discussed over email and tracked
around #266 (which builds on this work). It brings a GPU terrain renderer to the
AI/attitude widget, the data backends it needs, an on-device updater UI for
keeping that data current, and a handful of supporting instrument fixes.
It's a large branch by necessity (the renderer, its data layer, and the updater
are interdependent), but it's organized into focused commits and is current
with
master(0 behind). Reviewing by area (below) is the easiest path.What's included
SVS terrain renderer —
src/pyefis/instruments/ai/svs.py/svs_gl.py: a GL synthetic-vision renderer that drapes SRTM terrainunder the attitude indicator, with a forward "polar" fan + radial-warp LOD,
vectorised per-cell shading, and cached per-layer VBOs. Participates in the
scene z-order via a
QGraphicsItemso the pitch ladder/horizon stay on top.obstacles as colour-coded poles, and near-airport terrain-colour collapse so a
normal approach doesn't paint the screen red.
affected layer rather than raising; SVS annunciates UNAVAIL when it can't run.
Nav-data backends —
airport_db.py,water_db.py,obstacle_db.py+tools/ICAO), water-polygon, and obstacle (FAA DOF) databases, all construct-never-
raises. Build tools for terrain/water/airport/obstacle DBs under
tools/.Touch data-pack updater —
src/pyefis/instruments/data_status/makerplane-data pipeline). Touch-tuned: tap-to-select with scroll
discrimination, opaque overlay, drive chooser.
Instrument polish
rotation accumulating on resize. Altimeter/VSI: config-honored tape options
(tape-only mode, round_to, font scaling). Airspeed tweaks.
Hardware / runtime notes
disabled, pyEfis runs as before on non-GPU hardware. See the hardware section
in the README and
docs/svs_*.Data dependencies
tools/builders generate the SQLite/tile artifacts. No bundled data.Testing
tests/instruments/ai/test_svs.py),the updater UI, and the instrument changes. (One pre-existing terrain-void test
is unrelated to this work.)
Merge request
Please use a merge commit rather than squash — GitHub's squash merge has had
defects during this period, and the per-commit history is the reviewable unit
here.
Related: #266 (photorealistic imagery texturing — a future additive layer on top
of this renderer).