Repository navigation
0.2: metadata and placed creation - #3
Merged
Merged
Conversation
worldarray.write: ome_multiscales (OME 0.4 and 0.5 documents with one explicit placement per level), level_placement and level_shape (Convention 1 inverted, origin included), chunk_key and parse_chunk_key for zarr v2, zarr v3 (both encodings, shards), N5 and precomputed, codec dialects, group-document merge that rewrites only this level's dataset entry, create_array (group document first, then the array; the format follows the group; chunks required; exists error or adopt; a lost creation race adopted after backoff; nothing ever deleted or overwritten). WorldArray: mode="r+", writable, write_shape, chunks_covering(write=), chunk_key/parse_chunk_key (list_chunks shares the parser), write_index with whole-unit alignment and an explicit partial mode, write by world box, store_fill, pickling by arguments. Pyramid: mode, add_level (metadata plus an empty array, placement and shape from the level below, dtype/chunks/shards/ codecs/fill copied). wa.create wraps create_array with the adopt check. ArrayMeta.shard_shape and write_shape; read_meta takes the shard from tensorstore's write chunk. Tests: 37 behavioural cases for Behaviours 10 to 16, including a two-process disjoint-unit write and the shared fixture tests/fixtures/write.json. README gains the writing section and the current N5 placement row. Claude-Session: https://claude.ai/code/session_0141wgF8mkGbjGfBFKfKhp64
Merged
cellmap-analyze and cellmap-flow: create writes a bare group document for every missing ancestor down from the .zarr root and follows the nearest existing ancestor's zarr version, so a level made in a v2 container is v2 with OME 0.4 and the hierarchy browses. registration-station: adopting an identical array writes no group document (an entry that already says the same is kept), and a write is re-read, a lost entry written again, then RuntimeError. mesh-n-bone: ensure_group_entry is public, and a pickled handle carries its context's spec so a capped context stays capped. chunkmirage: units written in OME's names (units.ome_name), ome_multiscales(type=), level_shape snaps float-noise factors, the fixture carries resolved per-axis units and types with seven new cases. write.codecs_of takes a path. Spec Behaviours 10 to 12, open questions 7 and 9 resolved, 11 to 13 added. Claude-Session: https://claude.ai/code/session_0141wgF8mkGbjGfBFKfKhp64
Claude-Session: https://claude.ai/code/session_0141wgF8mkGbjGfBFKfKhp64
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
worldarray 0.2: metadata and placed creation, built to spec draft 3 (Behaviours 10 to 16). Nothing on the read side changes; 136 tests pass (99 before, 37 new).
wa.ome_multiscales(axes, units, levels, version)builds the complete OME-NGFF 0.4 or 0.5 document with one explicit placement per level;wa.level_placement/wa.level_shapederive a level from factors and an origin. Expected documents are pinned intests/fixtures/write.jsonfor chunkmirage's TypeScript builder.wa.create(path, shape, dtype, voxel_size, translation, chunks=..., ...): the group document first (made when absent, else this level's entry added or replaced with everything else kept byte for byte), then the array, as zarr v2 + OME 0.4 or zarr v3 + OME 0.5 following the group's format.chunksrequired, codecs explicit (zstd 3 default), shards optional on v3.exists="error"raisesFileExistsError;exists="adopt"opens an identical array and raises naming every difference otherwise; a creation lost to another process is adopted after backoff. The package never deletes, truncates or overwrites. Fill-only units are not stored unlessstore_fill=True.wa.open(path, mode="r+"),vol.writable,vol.write_index(begin, end, data, partial=False)with whole-unit alignment (the clipped last unit allowed;partial=Trueis tensorstore's read-modify-write),vol.write(box, data)by the centre rule; no cast, no padding, no resampling on write. AWorldArraypickles by its arguments.wa.open_pyramid(path, mode="r+").add_level(name, factors=, origin=): the entry and an empty array, placed and sized from the level below, dtype/chunks/shards/codecs/fill copied; downsampling stays with the caller.chunk_key/parse_chunk_keyfor zarr v2, zarr v3 (both encodings, shards), N5, precomputed, free and onWorldArray;list_chunksshares the parser;write_shape,ArrayMeta.shard_shape,chunks_covering(write=True).Verified end to end: create, aligned writes from two processes, read back through 0.1's reader with
placed_by == "ome",add_levelplacing exactly where Behaviour 4's block mean lands (compared against a direct numpy block mean).0.3 (grids and blocks) follows in its own PR.
https://claude.ai/code/session_0141wgF8mkGbjGfBFKfKhp64