Problem
tick_writer.py runs DROP TABLE IF EXISTS on the tick tables at the first write_tick() of a run. The rows already carry id_scenario and id_run — the schema is shaped for keeping many runs, and the code deletes them.
The #38 branch narrowed this further: the drop list is now derived from the loaded PDL roster instead of a fixed list, so tables written by a previous run under a different PDL survive with stale rows while Result_Simulator_Environment is always dropped. A query joining them returns plausible but wrong numbers.
Nothing calls run_stepwise() today — this path is pre-built for palaestrAI and other RL agents. Fixing the drop list is not worth doing, because this ticket removes dropping entirely.
Decision — append-only
- Never drop a table
run_id is a UUID generated once in TickWriter.__init__, shared with the per-run CSV directory from the CSV output ticket
- A
runs registry table mirrors runs.json for ordering and metadata
- Index on
(id_scenario, id_run, period)
- Deleting a run becomes
DELETE FROM ... WHERE run_id = ..., scoped rather than destructive
Schema drift
Columns come from the agent's role (_PROPS_BY_ROLE in data_collector.py). If an entity id appears under a different role in another PDL, the table name matches but the column set does not, and df.to_sql(..., if_exists="append") fails with column ... does not exist. write_tick() catches that and sets self.enabled = False, so the whole run would write nothing after one log line.
Before appending to an existing table, read its columns and ALTER TABLE ... ADD COLUMN for anything missing.
Open question for the team: can an entity id legitimately change role between PDLs?
Scale
palaestrAI training means many episodes. A demonstrator run worth keeping and a throwaway training episode should not be written identically — flag the difference in the registry, or skip tick writes during training.
Scope
tick_writer.py, model.py (id_run=0 is hardcoded at the call site).
Acceptance
- Two runs against different PDLs both keep their rows
- Every row is attributable to exactly one run
- The
runs table lists both runs with their PDL and start time
- A role change on an existing entity id extends the table instead of disabling the writer
Problem
tick_writer.pyrunsDROP TABLE IF EXISTSon the tick tables at the firstwrite_tick()of a run. The rows already carryid_scenarioandid_run— the schema is shaped for keeping many runs, and the code deletes them.The #38 branch narrowed this further: the drop list is now derived from the loaded PDL roster instead of a fixed list, so tables written by a previous run under a different PDL survive with stale rows while
Result_Simulator_Environmentis always dropped. A query joining them returns plausible but wrong numbers.Nothing calls
run_stepwise()today — this path is pre-built for palaestrAI and other RL agents. Fixing the drop list is not worth doing, because this ticket removes dropping entirely.Decision — append-only
run_idis a UUID generated once inTickWriter.__init__, shared with the per-run CSV directory from the CSV output ticketrunsregistry table mirrorsruns.jsonfor ordering and metadata(id_scenario, id_run, period)DELETE FROM ... WHERE run_id = ..., scoped rather than destructiveSchema drift
Columns come from the agent's role (
_PROPS_BY_ROLEindata_collector.py). If an entity id appears under a different role in another PDL, the table name matches but the column set does not, anddf.to_sql(..., if_exists="append")fails withcolumn ... does not exist.write_tick()catches that and setsself.enabled = False, so the whole run would write nothing after one log line.Before appending to an existing table, read its columns and
ALTER TABLE ... ADD COLUMNfor anything missing.Open question for the team: can an entity id legitimately change role between PDLs?
Scale
palaestrAI training means many episodes. A demonstrator run worth keeping and a throwaway training episode should not be written identically — flag the difference in the registry, or skip tick writes during training.
Scope
tick_writer.py,model.py(id_run=0is hardcoded at the call site).Acceptance
runstable lists both runs with their PDL and start time