Skip to content

Replace Space-Track with IAU CPS SatChecker for orbit records - #44

Open
chrisfinlay wants to merge 1 commit into
mainfrom
satchecker-orbits
Open

Replace Space-Track with IAU CPS SatChecker for orbit records#44
chrisfinlay wants to merge 1 commit into
mainfrom
satchecker-orbits

Conversation

@chrisfinlay

Copy link
Copy Markdown
Owner

Summary

Replaces tabsim's Space-Track TLE retrieval with the credential-free
IAU CPS SatChecker service, by vendoring
tabascal/satchecker/
from epfl-radio-astro/tabascal#92
and rebuilding tabsim's satellite selection on top of it.

That PR explicitly left this out of scope — "Generating a simulation with sim-vis
from the separately installed tabsim still uses Space-Track; that is out of scope
here."
This is that follow-up.

Both orbital-data formats SatChecker serves are supported: TLEs for epochs before its
2026-07-12 handover, and OMM (Orbit Mean-Elements Message) records after it. No account
or credentials are required, and the spacetrack dependency is removed.

The vendored package

tabsim/satchecker/ is a direct copy of tabascal/satchecker/ at commit
bd51a11, not a reimplementation, so the two stay diffable and a fix on either side
transplants without a merge. The deviations are listed in full in its __init__.py:

  1. the package name in cross-references, and the USER_AGENT contact URL;
  2. read_legacy_tle_records passes precise_float=True to pandas.read_json.

(2) is a real bug, present upstream too, and worth sending back. pandas' default JSON
float parser is not correctly rounded: it reads 0.0066635 back as
0.006663499999999999, a different double. For an OMM record that is the
eccentricity, so a run replayed from its own used_orbits.json propagated to a
measurably different trajectory than the run that wrote it. Found by asserting exact
equality on a real end-to-end replay rather than approx; see Verification.

tabsim-specific additions live outside the vendored package, so it stays a clean copy.
Today that is one module, tabsim/satchecker_names.py.

What tabsim needed that TABASCAL did not

TABASCAL resolves a fixed list of NORAD IDs at one epoch. tabsim additionally
searches: it takes a candidate set (IDs and/or names), propagates each one, and
keeps the satellites that actually pass near the target. Two consequences shaped this PR.

Names

tabsim's sat_names has no TABASCAL counterpart. SatChecker offers two name endpoints
and only one of them is right here:

Endpoint Matching navstar
norad-ids-from-name exact, case-insensitive 0 results
search-satellites substring, case-sensitive 80 results (as NAVSTAR)

Space-Track's op.like(name) wraps the pattern in wildcards, so it was a substring
search, and tabsim always passed name.upper(). search-satellites plus the same
upper-casing reproduces that exactly. The exact-name index would silently have reduced
sat_names: [navstar] — which every shipped example config uses — to nothing at all,
and the simulation would have run to completion with no satellites and no error. This
was caught end-to-end rather than by unit test, and the reasoning is written into the
module so it does not recur.

Decayed objects are dropped: they cannot be observed, have no record near any
present-day epoch, and would otherwise turn every re-entered namesake into a coverage
failure.

Coverage: numbered vs named satellites

TABASCAL's rule is that every configured satellite must resolve or the run stops.
Applied unchanged here it would be wrong for names, so the two are separated:

  • Numbered satellites (norad_ids, norad_ids_path) keep the strict rule. The user
    named these individually; one dropped for want of a record is indistinguishable from
    one that simply never passed the target, which is exactly the silent failure the rule
    exists to prevent. OrbitError names each failure, how close the best available
    record was, and the remedies.
  • Named satellites are a catalogue query. A name matching nothing means there is
    no satellite for a record to be missing for — reported, not fatal. A named satellite
    whose record cannot be obtained is reported and excluded.

Retrieval behaviour

Unchanged from #92, since it is the same code. Each NORAD ID resolves independently:
extra_orbit_dir → managed per-NORAD cache → SatChecker, with endpoint selection
against the handover date, failover to the other archive when the first yields nothing
acceptable, bounded concurrency, outage handling, and validated caching.

tabsim/orbit.py is ported from tabascal/orbit.py less the multi-process broadcast
and the Measurement Set preflight — tabsim's simulation is single-process and builds its
own time grid — plus resolve_names. tabsim/orbit_config.py is ported from
tabascal/orbit_config.py less the MS epoch derivation and model-component
introspection, which tabsim has no equivalent of.

Propagation

FixedOrbit-style propagation builds a Skyfield EarthSatellite from two TLE lines, so
an OMM record could not be propagated at all — post-handover simulations would fail on
the default satellite component. An OMM's elements now go straight into an sgp4.Satrec
via sgp4init, wrapped by EarthSatellite.from_satrec. The TLE path is untouched.
ndot/nddot are passed as zero — SGP4 models drag through bstar alone and never
reads them while propagating.

Output schema

rfi_tle_sat_orbit is a fixed-width string array of the two TLE lines. An OMM record
has none — its elements are the record — so those rows are written empty rather than
filled with something that looks like a TLE and is not one. The zarr/MS schema is
otherwise unchanged.

The complete record goes to input_data/used_orbits.json instead, in
extra_orbit_dir format. Pointing a later run's extra_orbit_dir at that directory
reproduces the run's satellite trajectories exactly, independently of the shared cache,
of the age ceiling, and of what SatChecker serves by then. It is written by hand rather
than with DataFrame.to_json, which formats floats to a fixed number of decimal places
— the default 10 rounds an OMM element outright, and even the maximum 15 writes
0.0066635 as 0.006663499999999999. json.dump writes a float through repr, the
shortest representation that reads back identically.

Configuration

Added under rfi_sources.tle_satellite:

Key Default Meaning
extra_orbit_dir null Directory of local orbit files (TLE or OMM), searched before the cache and SatChecker
extra_orbit_max_age_days null Max abs(record epoch − observation epoch) to accept a local file; null = unlimited, keeping exact replay working
remote_max_age_days 3 Hard ceiling on a record accepted from SatChecker or the cache
cache_reuse_max_age_days 1 A cached record this close to the observation avoids a network request

Also: ORBIT_CACHE_DIR relocates the managed cache, and sim-vis --extra_orbit_dir
(-eod) sets extra_orbit_dir at runtime. A path given on the command line resolves
against the working directory; one in the config resolves against the config, like every
other path there.

Removed: spacetrack_path, tle_dir, the -st/--spacetrack flags on sim-vis and
tle-region, the tabsim-setup-spacetrack entry point and its script, the shipped
Space-Track TLE cache under tabsim/data/rfi/tles/, the Space-Track credentials step in
CI, and the spacetrack dependency. sgp4 and platformdirs are added; pandas is
promoted from transitive to declared, since it is imported directly.

tabsim-import-tles now writes to ./orbits by default rather than into the installed
package, and its output is documented as an extra_orbit_dir — which is what it always
produced, but there was no longer anything reading the old location.

norad_ids_path is read by read_norad_ids_file rather than np.loadtxt, which reports
the offending file and line number on a typo. It still takes the first column, so a
file pairing IDs with a name or note reads as before.

Verification

  • Full suite: 57 passed, including a new 53-test tests/test_orbit.py covering
    source precedence, the age policy, endpoint failover, outage handling, coverage
    errors, replay, configuration validation, name lookup, and both propagation paths.
    Everything in it runs offline.
  • TLE and OMM propagate identically. The OMM fixture derives its elements from the
    same TLE it is compared against, so a degrees-for-radians or rev/day-for-rad/min slip
    would show up as kilometres. Measured max separation over 12 h on a 6800 km orbit:
    0.11 m, and a field-by-field Satrec comparison shows every orbital element
    transfers exactly — the residual is entirely the OMM EPOCH being an ISO 8601
    string, so the epoch survives at microsecond resolution (~15 µs here, ~0.1 m of
    along-track motion at 7.7 km/s).
  • Live end-to-end, pre-handover (2023-02-21 epoch, sat_names: [navstar]): 80
    NAVSTAR satellites resolved via nearest-TLE, 3 visible, RFI added, zarr + MS written,
    rfi_tle_sat_orbit carrying the expected TLE lines.
  • Live end-to-end, post-handover (2026-08-13 epoch): the same three satellites
    resolved via nearest-OMM, propagated, and written with empty line columns and full
    OMM records in used_orbits.json.
  • Replay is bit-exact. A second run of the OMM simulation with an empty cache and
    --extra_orbit_dir pointed at the first run's input_data produces
    byte-identical rfi_tle_sat_xyz and vis_obs (np.array_equal, not approx).
    This is the assertion that surfaced the precise_float bug; before the fix it was off
    by 11 nm and nothing would ever have said so.
  • Two identical runs were confirmed bit-identical first, so the replay comparison is
    measuring what it claims to.

Known limitations

  • A broad name search is now many requests. SatChecker serves one record per request,
    so sat_names: [starlink] (>21000 objects) is that many requests on a cold cache,
    where Space-Track answered in one bulk query. Inherent to the service's API, and
    equally true of #92. A warning fires above 500 matches; navstar (80) is the realistic
    case, and every response is cached.
  • remote_max_age_days: 3 is a backstop against obviously unsuitable records, not a
    claim of three-day positional accuracy.
  • OMM records cannot be checksum-validated and their epoch cannot be independently
    re-derived; range checks and an absolute epoch plausibility window are what remain.
    Documented in the vendored records.py.
  • tests/test_sim-vis.py::test_simulation_runs_with_config reaches the network. It
    previously required Space-Track credentials, so this is a reduction in what CI needs,
    not a new requirement.

Vendors tabascal/satchecker/ from epfl-radio-astro/tabascal#92 and rebuilds
tabsim's satellite retrieval on top of it. No account or credentials are
required and the spacetrack dependency is removed.

Both orbital-data formats SatChecker serves are handled: TLEs for epochs before
its 2026-07-12 handover, and OMM records after it. An OMM record has no lines to
parse, so its elements go straight into an sgp4.Satrec via sgp4init; both kinds
end up as the same propagator over the same model.

Retrieval gains per-satellite source precedence (extra_orbit_dir, managed cache,
SatChecker), a validated per-NORAD cache, an age policy, and a reproducible
used_orbits.json written into each simulation's input_data.

Satellite names resolve through SatChecker's substring search rather than
Space-Track's LIKE query, preserving the old semantics including the upper-casing
tabsim always applied.

Removes spacetrack_path, tle_dir, the tabsim-setup-spacetrack entry point, the
shipped Space-Track TLE cache, and the Space-Track credentials from CI.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant