Skip to content

ENH: configurable dataset map for ZebraSaveIOC#28

Open
Max Rakitin (mrakitin) wants to merge 12 commits into
mainfrom
configurable-zebra-dataset-map
Open

ENH: configurable dataset map for ZebraSaveIOC#28
Max Rakitin (mrakitin) wants to merge 12 commits into
mainfrom
configurable-zebra-dataset-map

Conversation

@mrakitin

@mrakitin Max Rakitin (mrakitin) commented May 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

Adds a --dataset-map startup argument to ZebraSaveIOC so the mapping between IOC PV attribute names and HDF5 dataset names can be configured at launch without code changes.

Changes

src/srx_caproto_iocs/zebra/caproto_ioc.py

  • New --dataset-map CLI argument (JSON string): e.g. --dataset-map '{"enc1": "x_pos", "enc2": "y_pos"}'
  • ZebraSaveIOC.__init__(dataset_map=None): stores the user-supplied mapping as self._dataset_map
  • _DEFAULT_DATASET_MAPS class attribute contains the two built-in SRX sets (zebra / scaler), preserving backward compatibility with commit 135d93c
  • _get_current_dataset: uses self._dataset_map when set; falls back to the dev_type-based default otherwise
  • Bug fix: saver now uses mode="a" (append/create) instead of mode="x" (fail-if-exists)

src/srx_caproto_iocs/zebra/ophyd.py

  • Extends ZebraWithCaprotoIOC with all data PV components: enc1–enc3, zebra_time, i0, im, it, sis_time, dev_type

tests/test_zebra_ioc.py (new)

Three integration tests (cloud_friendly):

  1. Default zebra dev_type → HDF5 keys are enc1/enc2/enc3/zebra_time
  2. Scaler dev_type → HDF5 keys are i0/im/it/sis_time
  3. Custom --dataset-map → HDF5 keys match the user-supplied names, original PV attr names absent

tests/conftest.py

  • Added session fixtures zebra_caproto_ioc and zebra_caproto_ioc_custom_map
  • Added per-test fixtures zebra_ophyd_device and zebra_ophyd_device_custom_map
  • Refactored start_ioc_subprocess to accept extra_args for passing additional CLI flags

pyproject.toml / pixi.lock

  • Added pixi workspace config with correctly structured dev and test environments so pixi run --environment test pytest works out of the box

Closes #27

Max Rakitin (mrakitin) and others added 12 commits May 12, 2026 00:10
@mrakitin
ENH: configurable dataset map for ZebraSaveIOC
0a57015 
- Add `--dataset-map` CLI argument (JSON string) to ZebraSaveIOC that lets
  users specify a custom {pv_attr_name -> hdf5_dataset_name} mapping at
  startup.  When omitted, the mapping falls back to the built-in SRX defaults
  chosen by the existing `dev_type` PV (zebra or scaler), preserving full
  backward compatibility with the behaviour deployed at commit 135d93c.
- Add `dataset_map` parameter to `ZebraSaveIOC.__init__` and
  `_DEFAULT_DATASET_MAPS` class attribute for the two SRX default sets.
- Refactor `_get_current_dataset` to iterate the active mapping, producing
  an HDF5 dataset keyed by the user-supplied names rather than PV attr names.
- Fix `saver`: change `mode="x"` (fail-if-exists) to `mode="a"`
  (append/create) so repeated acquisitions work correctly.
- Extend `ZebraWithCaprotoIOC` ophyd device with all data PV components
  (enc1-3, zebra_time, i0, im, it, sis_time, dev_type).
- Add `tests/test_zebra_ioc.py` with three cloud_friendly integration tests:
  default zebra mapping, default scaler mapping, and custom --dataset-map.
- Add pixi workspace configuration with fixed test/dev environments so that
  `pixi run --environment test pytest` works out of the box.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mrakitin
Fix pre-commit/ruff formatting and codespell issues
8245c94 
- Remove unused `ChannelType` import from zebra/ophyd.py
- Replace `Optional[dict[str, str]]` with `dict[str, str] | None` (UP007)
- Remove useless expression and unused variable in test_zebra_ioc.py (B018, F841)
- Apply ruff-format line-wrapping to conftest.py and test_zebra_ioc.py
- Remove `@pytest.fixture()` parens where not needed (PT001) in conftest.py
- Add `astroid` to codespell ignore list (false positive in pixi.lock)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mrakitin
Autoupdate pre-commit hooks and fix new ruff violations
1104cf3 
- Run pre-commit autoupdate (ruff v0.4.5→v0.15.12, codespell v2.3.0→v2.4.2,
  blacken-docs, pre-commit-hooks, shellcheck, validate-pyproject, check-jsonschema)
- Fix SIM103: inline boolean return in base.py callback
- Fix PT028: remove default args from test_base_ophyd_templates signature,
  move num_frames/remove to local variables inside the function
- Fix PT023: add () to @pytest.mark decorators in test_base_ophyd.py and test_string_ioc.py
- Apply ruff-format reformatting

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mrakitin
Fix pylint W0621: rename dataset_map local var in __main__ to dataset…
c5033e1 
…_map_arg

Avoids redefined-outer-name warning — the __init__ parameter 'dataset_map'
was shadowing the module-level variable of the same name on line 262.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mrakitin
Update minimum Python requirement to 3.10+
c3fa415 
- requires-python: >=3.8 -> >=3.10
- Remove Python 3.8 and 3.9 from classifiers
- Update mypy python_version: 3.8 -> 3.10
- Update CI matrix: [3.8, 3.9, 3.10, 3.11] -> [3.10, 3.11, 3.12]

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mrakitin
Fix pixi test task: remove unsupported {args} template from pytest cmd
ca4279f 
The {args} placeholder syntax is not supported by this version of pixi's
shell parser. Use plain 'pytest' instead; extra args can be passed via
'pixi run test -- <extra-args>' if needed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mrakitin
Fix CI: always run checks job on PRs, limit push trigger to main
c7504df 
The checks job had an 'if' condition that skipped it for same-repo PRs
(to avoid duplicate runs on branch push). This meant tests never appeared
in the PR view — only the Format job was visible.

Fix:
- Remove the 'if' condition from checks so it always runs on PRs
- Restrict push trigger to main branch only (avoids duplicate runs
  on feature branch pushes while keeping tests visible on PRs)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mrakitin
Fix hanging CI tests: add timeouts and EPICS_CA_ADDR_LIST for client
51ba829 
Root causes of hanging:
1. put()/get() calls had no timeout, blocking forever if EPICS CA
   could not connect in the CI network environment
2. Only server-side EPICS_CAS_* env vars were set; the ophyd/pyepics
   client also needs EPICS_CA_ADDR_LIST to find the IOC
3. No per-test timeout, so a single hang blocks the job for 6 hours

Fixes:
- conftest.py: add EPICS_CA_ADDR_LIST + EPICS_CA_AUTO_ADDR_LIST=no
  to get_epics_env() so the client knows where to search
- conftest.py: add wait_for_connection(timeout=30) in zebra ophyd
  fixtures so connection failures surface immediately with clear errors
- conftest.py: add timeout=10 to teardown put() calls
- test_zebra_ioc.py: add timeout=10 to all put()/get() calls
- base.py: add timeout=10 to obj.put(cmd) inside set()
- pyproject.toml: add pytest-timeout>=2 to test deps; set global
  --timeout=120 so any hung test is killed after 2 minutes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mrakitin
Fix EPICS_CA regression: keep auto-discovery, only add explicit broad…
7787ef8 
…cast

Setting EPICS_CA_AUTO_ADDR_LIST=no broke the pre-existing base tests by
disabling pyepics auto-discovery of local IOCs. Revert that; keep only
EPICS_CA_ADDR_LIST so the computed broadcast is added ON TOP of the
auto-discovered addresses (default EPICS_CA_AUTO_ADDR_LIST=yes behavior).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mrakitin
CI: fix issues with tests
9ef776b
@mrakitin
STY: fix linter issues
2dd8d03
@mrakitin
FIX: ignore upstream opentelemetry DeprecationWarning in pytest
8f66674 
The ophyd package imports opentelemetry, which uses the deprecated
importlib.metadata dict interface (SelectableGroups), causing a
DeprecationWarning that is promoted to an error by our filterwarnings=error
pytest configuration.

Add a specific ignore filter for this upstream warning to unblock CI
for py310 and py311.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mrakitin Max Rakitin (mrakitin) marked this pull request as ready for review June 9, 2026 22:31
This was referenced Jun 9, 2026
Closed
Draft
skarakuzu
seher karakuzu (skarakuzu) reviewed Jun 10, 2026
View reviewed changes
Comment thread tests/test_base_ophyd.py

@skarakuzu seher karakuzu (skarakuzu) Jun 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The num_frames=50 was sometimes failing to collect all before timeout time to time if I remember correctly from our tests earlier this week. I remember we had reduced the frame number last time, was there anything different previously in the test?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dihenriksen Dan Henriksen (dihenriksen) dihenriksen approved these changes

@skarakuzu seher karakuzu (skarakuzu) skarakuzu approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

FXI implementation

3 participants

@mrakitin @dihenriksen @skarakuzu