From 4bdded61d889bd7ce06620d2d2ee66739c09059c Mon Sep 17 00:00:00 2001 From: Kabilar Gunalan Date: Fri, 10 Jul 2026 00:08:29 -0500 Subject: [PATCH] Add docs for downloading and uploading a partial Zarr --- docs/user-guide-sharing/uploading-data.md | 13 +++++++++++++ .../accessing-data/downloading.md | 18 ++++++++++++++++++ 2 files changed, 31 insertions(+) diff --git a/docs/user-guide-sharing/uploading-data.md b/docs/user-guide-sharing/uploading-data.md index 25ef3af8..30b97541 100644 --- a/docs/user-guide-sharing/uploading-data.md +++ b/docs/user-guide-sharing/uploading-data.md @@ -57,6 +57,19 @@ For command-line users or those with larger datasets, the DANDI CLI provides a p dandi upload -i dandi-sandbox ``` +#### Uploading part of a Zarr + +By default, `dandi upload` synchronizes a [Zarr](https://zarr.dev/) asset in full, whereby files that exist on the server but not locally are deleted so the remote copy matches your local copy exactly. +When you only want to add or update a subset of a Zarr's files, use `--zarr-mode patch`: + +```bash +dandi upload --zarr-mode patch +``` + +In patch mode, only modified or new local files are uploaded, and remote files that are absent locally are **not** deleted. +This is useful for editing metadata or adding to a large Zarr without needing a complete local copy. +The default, `--zarr-mode full`, performs the usual full synchronization (including deletions). + ## Storing Access Credentials There are two options for storing your DANDI access credentials: diff --git a/docs/user-guide-using/accessing-data/downloading.md b/docs/user-guide-using/accessing-data/downloading.md index 68736258..a5291a03 100644 --- a/docs/user-guide-using/accessing-data/downloading.md +++ b/docs/user-guide-using/accessing-data/downloading.md @@ -78,6 +78,24 @@ For example: dandi download --preserve-tree dandi://dandi/000026@draft/sub-I58/ses-Hip-CT/micr/sub-I58_sample-01_chunk-01_hipCT.json +### Download part of a Zarr +[Zarr](https://zarr.dev/) assets are directory trees that can contain many files, and you may need only a subset of them. +The `--zarr` option lets you filter which entries within a Zarr asset are downloaded, so you can avoid transferring the full tree. + +The most common use is the `--zarr metadata` alias, which downloads only the Zarr metadata files (e.g. `zarr.json`, `.zarray`, `.zgroup`, `.zattrs`, `.zmetadata`) without any of the chunk data: + + dandi download --zarr metadata dandi://dandi/001278@draft/sourcedata/raw/sub-Hb1/micr/sub-Hb1_sample-lefthemi_acq-overview_XPCT.ome.zarr + +You can also filter using an explicit `--zarr TYPE:PATTERN` expression, where `TYPE` is one of `glob`, `path`, or `regex`: + +| Type | Syntax | Description | +|------|--------|-------------| +| `glob:PATTERN` | `--zarr glob:'**/.z*'` | Download entries whose zarr-internal path matches the glob pattern, using `fnmatch` semantics per path component. `*` matches within a component, `**` matches across directory levels. | +| `path:PREFIX` | `--zarr path:0/1/2` | Download entries whose zarr-internal path starts with the prefix. | +| `regex:PATTERN` | `--zarr regex:'\.z(array|group|attrs)$'` | Download entries whose zarr-internal path matches the regular expression. | + +The `--zarr` option may be given multiple times, in which case an entry is downloaded if it matches **any** of the filters. + ## Using DataLad All dandisets are regularly mirrored to DataLad datasets which are made available at the GitHub organization https://github.com/dandisets.