This note records the current Chunk convolution design.
Convolution is Chunk-native. The preferred path for separable filters is:
Chunk slice stream
-> z-window/parallel collect where a halo is needed
-> native convolveX / convolveY / convolveZ
-> emitted Chunk slice stream
Single-axis native convolution uses float32 kernels and supports:
uint8int8uint16int32float32
Input and output keep the same pixel type. Integer outputs are converted back with the same clamp/round policy used by the Chunk convolution wrappers.
Separable filters are stage compositions over the single-axis primitives:
- Box smoothing
- Gaussian smoothing
- finite differences
- Sobel-axis responses
- gradient vectors
- Hessian upper matrices
- Laplacian
- gradient magnitude
- Sobel magnitude
- structure tensor component smoothing
Box and Gaussian stages accept separate width/radius parameters per axis. This keeps anisotropic volumes and pipeline experiments straightforward.
Vector Chunk payloads store components in the chunk data layout. Structure tensor and related derivative stages need to convolve each component without unpacking every component into independent scalar chunks.
The active fast path is a native Float32 component-wise convolution helper. It walks the byte-backed chunk storage directly, performs the requested axis pass for each vector component, and returns a Float32 vector Chunk. This is the right shape for smoothing the six structure-tensor outer-product components.
Convolution stages should expose their halo needs through bounded windows, not by materializing full volumes. A stage may read a bounded z-window, emit only valid center slices, and release consumed chunks according to the window resource rules.
For separable filters:
- X and Y passes are slice-local.
- Z passes require a bounded z-halo.
- Multi-pass filters compose ordinary stages so the plan graph and cost model can see the work.
The peak memory of a linear streaming pipeline is not the simple sum of every stage's standalone peak. In a pull-driven pipeline, a batch normally travels through earlier stages and is then retained only at the first downstream stage that cannot emit yet. For a linear chain, a better model is:
linearPeak ~= max(stage-frontier-peaks) + boundary/batch slack
rather than:
linearPeak != sum(all stage peaks)
The distinction matters for separable convolution:
read/cast -> convolveX -> convolveY -> convolveZ -> cast/write
For X and Y convolution, each z-stream element is a singleton window. The
stage may process a parallelCollect batch of workers singleton windows,
but it does not need to hold a z-neighbourhood. The batch is therefore a
travelling frontier rather than a permanent reservoir.
For Z convolution, each output window overlaps its neighbours. With kernel
size k, stride 1, and workers parallel windows, the real input slices
needed by one batch have approximate union size:
zWindowUnion = k + workers - 1
At stream boundaries, padding chunks add a radius-sized boundary term:
boundaryPad = (k - 1) / 2
However, the upstream stages also deliver in parallelCollect batches. If a
downstream Z batch needs a slice that belongs to the next upstream singleton
batch, the whole upstream batch may be produced before Z can run. This makes
the first batch and some boundary-aligned batches larger than the simple
zWindowUnion.
For the observed smoothWGauss example with k = 9 and workers = 3:
read/cast:
max 2 chunks, leaves 1 Float32 chunk per input slice
convolveX:
waits for 3 singleton windows
creates 3 outputs
releases 3 inputs
convolveY:
same singleton-batch shape
convolveZ startup:
creates 4 leading padding chunks
needs real Y slices 0..6 for output windows 0,1,2
upstream supplies Y in batches of 3, so Y slices 0..8 are produced
creates 3 Z outputs before all old inputs are released
The resulting worst observed peak is:
4 boundary padding chunks
+ 9 real Y chunks pulled in upstream batches
+ 3 Z output chunks created by the parallel batch
= 16 live chunks
After the first Z batch, the cast/write tail consumes three outputs and the live count drops. Later steady-state batches are governed by the same rules, but the exact peak can vary by one or more chunks depending on whether a create happens before the corresponding release in a parallel batch.
parallelCollect contributes explicit hold memory. Its current shape is:
collect batch of windows
process windows with Parallel.For
store all batch outputs
yield outputs downstream in order
This means that, at a batch boundary, the model should include:
retained input/window frontier
+ all outputs created for the current batch
For overlapping windows, retained input includes the union of the overlapping window items plus any suffix retained to prevent one worker/window releasing a shared chunk while another worker still needs it. For singleton windows, there is no overlap retention, but the output batch may still be created before the input batch has been fully released.
A conservative stage-level rule for a parallelCollect stage is:
parallelCollectPeak ~= inputWindowUnion + outputBatch + paddingBoundary
where:
inputWindowUnion =
windowSize + (batchSize - 1) * stride for overlapping stride windows
batchSize for singleton windows
outputBatch =
batchSize * outputElementMemory
paddingBoundary =
up to pad * elementMemory near stream boundaries
The model should treat paddingBoundary as a boundary/startup term, not a
steady-state term. The exact observed peak also depends on allocation/release
ordering, so the calibrated model should reserve a small batch-boundary slack
term.
For fanout, the memory model should be additive over branch peaks, because the shared input frontier must be retained until all branches have consumed their view of it:
fanoutPeak ~= sharedInputFrontier + sum(branchLinearPeaks)
If the branches are synchronized streamers, each branch peak should be the
linear peak of that branch under the same pull position. If the branches are
reducers or otherwise consume at different rates, the model should either
reject that synchronization shape or add an explicit buffering term. In the
current DSL, >=>> should be reserved for branches with compatible streaming
profiles; reducer/reducer fanout is safe only when both consume the full
upstream stream and emit once.
For ordinary linear composition, do not sum all stages. For fanout, sum the
branch maxima plus shared-frontier retention. For parallelCollect, add the
current batch's output hold and overlap/padding retention.
- Prefer native single-axis convolution for primitive numeric chunks.
- Prefer stage composition for separable filters; it keeps the implementation small and the graph/cost model visible.
- Avoid per-pixel generic callbacks in hot convolution loops.
- Use F# span loops for simple maps and reducers where native code would only add call overhead.
- Keep output chunks owned by the stage and release input chunks at the usual ownership boundary.
FFT convolution is not the default path for local finite kernels. The current FFT work is its own chunked complex64 pipeline. Local filters should stay on the single-axis/native convolution path unless a benchmark shows the FFT route wins for a specific large kernel and memory shape.