Skip to content

Read a blank dat_path as "no raw data file" - #61

Merged
rossant merged 1 commit into
cortex-lab:masterfrom
adityasingh2400:fix-57-empty-dat-path
Aug 9, 2026
Merged

Read a blank dat_path as "no raw data file"#61
rossant merged 1 commit into
cortex-lab:masterfrom
adityasingh2400:fix-57-empty-dat-path

Conversation

@adityasingh2400

Copy link
Copy Markdown
Contributor

Fixes #57.

A dataset with no raw data file on disk carries a blank dat_path in params.py. SpikeInterface writes one when the Recording object has an unknown source, a simulated recording being the usual case, and this issue links SpikeInterface/spikeinterface#4569 for the downstream report.

The assert ext the issue points at is already gone, #58 turned it into a warning. What is left is quieter and, I think, the actual problem. Path('') is Path('.'), so _make_abs_path() resolves a blank entry against dir_path and it becomes the dataset directory. On current master, loading such a dataset gives you a model whose dat_path is the directory it was loaded from, describe() reports that directory as the raw data file, and get_ephys_reader() logs "Unknown file extension" twice on a directory before returning None. Downstream code that tests model.dat_path to decide whether raw data exists gets the wrong answer.

I read the surrounding code as already having an answer for this case. _load_traces() returns early on an empty dat_path, merge.py sets dat_path = [] on purpose, and TemplateModel(dat_path='') already lands in that path because '' is falsy. So a blank dat_path is best read as "this dataset has no raw data file", which is a state phylib supports, rather than as an error. That also matters practically: raising here would stop phy from opening SpikeInterface outputs that it can open today, which is the "lots of sorting outputs which currently can't be read" concern in the issue.

So this drops blank entries instead of resolving them. _clean_dat_path() is shared between get_template_params() and TemplateModel.__init__() so the params.py route and the direct kwargs route agree, and it handles blank entries inside a list too, for example dat_path = ['', 'recording.dat']. The original non list wrapping semantics are kept exactly, this only adds the blank filter.

Tests cover both the scalar and the list form, at the get_template_params() level and end to end through load_model(). Reverting phylib/io/model.py to master makes both fail, the first asserting the dataset directory against an expected empty list. pytest phylib is green at 275, which is the 273 baseline plus these two. flake8 phylib reports 16 errors against a baseline of 17, so zero new. The one that disappeared is a pre-existing E501 on the old self.dat_path one liner, which had to be rewritten anyway.

Disclosure: this change was prepared with AI assistance. I have reviewed and tested it.

A dataset with no raw data file on disk has a blank dat_path in params.py.
SpikeInterface writes one for recordings with an unknown source, such as
simulated recordings.

Path('') is Path('.'), so a blank entry was resolved against dir_path and
silently became the dataset directory itself. The model then reported the
directory as its raw data file, and get_ephys_reader() warned twice about an
unknown file extension before giving up.

Drop blank entries instead, so dat_path ends up empty, which is the path the
model already supports for datasets with no raw data.

@rossant rossant left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the blank-path normalization and focused regressions. The implementation preserves existing non-empty path handling while preventing Path('') from resolving to the dataset directory.

@rossant
rossant merged commit 97118ca into cortex-lab:master Aug 9, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot read params.py file with empty string dat_path

2 participants