Dataloader output: dict instead of 4-tuple - #21
Merged
Conversation
__getitem__ and neural_collate now return a dict with keys 'stims', 'responses', 'valid_mask', 'stim_meta' instead of a positional 4-tuple. This lets datasets add extra per-trial keys later (e.g. 'behav' for behavioural covariates, à la Sinz-lab neuralpredictors / Sensorium) without changing the unpacking contract — neural_collate passes any unknown per-item key through as a length-B list. Breaking change to positional unpacking; done pre-0.1.0 while there are no published users. Updated the Fitter (both train/eval loops) + all test consumers. Fast loop (pytest -m 'not slow') 440 passed; edited slow integration files (ns1/downer/ wingert) 87 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Updated all dataloader-unpacking examples to the dict API (batch['stims'], batch['responses'], …): README quickstart, fitter.md, data_paradigm.md, metrics_paradigm.md, and the 4 notebooks that unpack a batch manually (crcns_aa_tutorial, dataset_concatenation, explore_nat4, alice_eeg_tutorial). Also rewrote the stale README_datasets.md 'Conventions' page — it documented the long-gone (spectrogram, responses, ccmax, ttrc) 4-tuple and select_neuron; now describes the dict batch, (B,N,R,T) shapes, NaN sentinels, and the current selection API, pointing to data_paradigm.md for the full contract. Notebooks: source cells updated; outputs are data-identical (access pattern only) but a re-exec pass is folded into the pre-release notebook smoke test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
NeuralDataset.__getitem__andneural_collatenow return a dict(
{'stims', 'responses', 'valid_mask', 'stim_meta'}) instead of a positional4-tuple.
Why
So future per-trial variables (behaviour, pupil size, running speed — as in the
Sinz-lab
neuralpredictors/ Sensorium datasets) can be added as extra keyswithout changing the unpacking contract.
neural_collatepasses any unknownper-item key straight through as a length-
Blist, so a dataset only has to addthe key to its
__getitem__dict:This is a hard break to positional unpacking, done pre-0.1.0 while there are no
published users.
Changed
__getitem__+neural_collate(library);Fitter(both loops)._ToyDataset.__getitem__.fitter.md,data_paradigm.md,metrics_paradigm.md; full rewrite of the staleREADME_datasets.mdconventions page (it documented the long-gone
(spectrogram, responses, ccmax, ttrc)tuple).dataset_concatenation, explore_nat4, alice_eeg_tutorial). Source updated;
a re-exec pass is folded into the pre-release notebook smoke test.
Verification
pytest -m "not slow"-> 440 passed (~17 s)baseline — zero regressions)
🤖 Generated with Claude Code