diff --git a/docs/_toc.yml b/docs/_toc.yml index d8b139b32..c1a1a3de3 100644 --- a/docs/_toc.yml +++ b/docs/_toc.yml @@ -66,6 +66,7 @@ subtrees: - file: guides/event_loop - file: guides/threading - file: guides/events_reference + - file: guides/progressive_loading - file: further-resources/glossary - file: further-resources/napari-workshops - file: further-resources/sample_data diff --git a/docs/guides/progressive_loading.md b/docs/guides/progressive_loading.md new file mode 100644 index 000000000..317c80c93 --- /dev/null +++ b/docs/guides/progressive_loading.md @@ -0,0 +1,712 @@ +# Progressive image loading +The progressive loader in napari is experimental. It provides viewport aware, progressive chunk-wise +loading for very large multiscale images in napari. It uses the already existing multiscale layers in +napari + +The progressive loader ensures that an entire image pyramid level is not fully materialized before +displaying it. For this, it does the following: +1. It determines which resolution and spatial region are currently useful. +2. Ensures that something renderable is available immediately, e.g. (part of the) lowest resolution of the pyramid +3. Fetches missing chunks of current resolution level based on a priority order (distance from center chunk). +4. Progressively replaces the coarse, low resolution fallbak data with higher-resolution data. +5. Updates the displayed texture while limiting work that could interrupt interaction. + +As a result, the napari viewer can work with datasets substantially larger than CPU or GPU memory +while still keeping navigation responsive. + +## Using progressive loading + +### When it applies + +Progressive loading applies to lazy, chunked, multiscale Image and Labels data, such as Zarr or Dask pyramids. Single-scale data and in-memory pyramids are not converted automatically. Spatially chunked data with reasonably sized chunks gives the best results. + +### Enable and use it + +Enable **Preferences > Experimental > Progressive loading of multiscale data** before adding the layer. Then open the data normally using File > Open, drag-and-drop, a reader plugin, or `viewer.add_image(..., multiscale=True)`. The setting only affects layers added after it is enabled. + +It can also be enabled from the command line: + + NAPARI_PROGRESSIVE_LOADING=1 napari + +### Advanced Python usage + +The experimental Python helper provides access to tuning options: + + from napari.experimental._progressive_loading import ( + add_progressive_loading_image, + ) + + layer = add_progressive_loading_image( + pyramid, + viewer=viewer, + interval_max_bytes=512 * 1024**2, + max_bytes_per_second=100 * 1024**2, + ) + +Use `add_progressive_loading_labels` for Labels data. These underscored APIs are experimental and may change. + +### Practical tuning + +| Symptom | Adjustment | +| --- | --- | +| Loading interrupts interaction | Lower `max_bytes_per_second` | +| Resident data uses too much memory | Lower `interval_max_bytes` | +| 3D texture updates are expensive | Lower `tile_max_bytes_3d` | +| 3D selects expensive levels too early | Increase `max_pixel_size_3d` | +| Fine detail takes too long to begin loading | Try `coarse_first=False` | + +Chunk layout often matters more than these settings. Very large chunks, or chunks spanning most of a spatial axis, limit how progressively napari can load the data. + +### Troubleshooting + +- **Nothing changed:** enable the setting before adding the layer and confirm that the data is both chunked and multiscale. +- **Loading is blocky or slicewise:** inspect the source chunk shape; smaller spatial chunks may work better. +- **The unexpected 3D level is shown:** return the resolution selector to **Auto**, or adjust `max_pixel_size_3d` when using the Python helper. +- **More diagnostics are needed:** pass `debug_overlay=True` or set `NAPARI_PROGRESSIVE_DEBUG=1`. + +### Current user-facing limitations + +- The behavior and Python API are experimental and may change. +- Automatic conversion supports only eligible chunked multiscale Image and Labels layers. +- Changing the setting does not convert existing layers; attached loaders continue until their layer is removed. +- Enabling progressive loading also enables asynchronous rendering. +- Multiscale Labels layers are read-only. +- Unsuitable source chunking may still cause pauses or uneven refinement. +- Moving a dims slider may require a subsequent pan or zoom before the display refreshes. + +## Core concepts +In order to understand the progressive loader we will first explain some of the concepts needed to +understand it. + +### Multiscale data and `VirtualData` +The source data is a multiscale pyramid ordered from highest to lowest resolution. + +Rather than passing the source arrays directly to napari, each pyramid level is wrapped by +`VirtualData`. These arrays still look like arrays to napari, with the full shape of their +corresponding resolution level, but internally they keep only a bounded resident interval +in CPU memory (RAM). + +The purpose of `VirtualData` is to separate the actual logical size of an image from the amount +of data that must actually be materialized at any given time. + +```{mermaid} +flowchart TD + A["Chunked source"] + B["VirtualData / multiscale
pyramid"] + C["Resident interval
in RAM"] + D["vispy texture"] + E["Canvas"] + + A --> B + B -->|"fetch missing chunks"| C + C -->|"slice / patch"| D + D --> E +``` +Napari's rendering is not replaced. Rather the progressive loader adds coordination between +the different components of the existing rendering pipeline. We will now go more in detail. + +### Resident interval +Only the region needed for the current view is kept resident at a fine-resolution level. The interval +follows the viewport as the user pans, zooms, or changes dimensions. For displayed dimensions, it is +derived from the layer's corner_pixels; non-displayed dimensions are restricted to the current +dimension step. So for 2D data it would conceptually look like this: + +```{mermaid} +flowchart TB + subgraph P["Full pyramid level"] + direction TB + R["Resident interval
currently held in
CPU memory"] + end +``` +This resident interval is also bounded by memory and GPU texture limits. Moving the viewport moves the +resident interval instead of the camera on a fully materialized resolution level. + +### Resident coarsest level +This level is the lowest resolution level of the pyramid and it is treated specially. When it fits +within `resident_max_bytes`, it is kept fully in RAM. This to provide: +- immediate low-resolution context +- data for layer thumbnails +- a backdrop source for finer levels + +It gives the user something to immediately show, while higher resolution chunks are still being fetched. + +### Backdrop data +When zooming and panning, a fine-resolution interval does not have to start empty. +hen the new interval overlaps the previous one, such as during a pan, already loaded +target-resolution data in the overlapping region is retained. Only the newly exposed or +otherwise unloaded regions need temporary coverage. +For those unloaded regions, the loader uses already available data from another pyramid +level as a backdrop. It prefers the closest-resolution level whose resident data covers +the requested region. If no closer level provides sufficient coverage, the resident coarsest +level can serve as the fallback. Backdrop data is upsampled to the target resolution before +being written into the target resident interval. +The backdrop therefore provides temporary image content. It does not mark those regions as +loaded at the target level. They still need to be replaced by actual target-resolution chunks. +When `coarse_first=True`, the backdrop can be progressively improved before the target level is +fetched. Missing chunks from intermediate pyramid levels are fetched from coarse to fine. As +those chunks arrive, their data is upsampled and folded into regions of the target interval +that do not yet contain loaded target chunks. Intermediate levels are not rendered directly. +Finally, missing target-level chunks are fetched and written into the target level's resident +interval in `VirtualData`, which is held in RAM. These chunks replace any backdrop or +intermediate-resolution data in the corresponding regions. Once all required target-level +chunks for the requested view have been loaded, those regions of the resident interval +contain actual target-rresolution data. + +The flow for populating and progressively refining a resident interval is therefore like this: +```{mermaid} +flowchart TD + A["View changes
pan or zoom"] --> B["Establish target
resident interval"] + + B --> C["Carry over already loaded
target-level data"] + + C --> D["Identify unresolved regions"] + + D --> E["Fill unresolved regions
from available backdrop data"] + + E --> F["Upsample backdrop data
to target resolution"] + + F --> G["Resident interval has
useful image coverage"] + + G --> H{"coarse_first?"} + + H -- Yes --> I["Fetch missing chunks from
intermediate pyramid levels
coarse → fine"] + + I --> J["Upsample intermediate data into
regions without target-level chunks"] + + J --> K["Progressively sharper
temporary coverage"] + + K --> L["Fetch missing
target-level chunks"] + + H -- No --> L + + L --> M["Write target chunks into
VirtualData in CPU memory"] + + M --> N["Replace backdrop or intermediate
data in corresponding regions"] + + N --> O{"Missing target
chunks remain?"} + + O -- Yes --> L + O -- No --> P["Requested view is fully resolved
at the target resolution"] +``` +The key take away here is that backdrop data, intermediate data and target-level data can coexist +within the same resident interval while loading is in progress. The resident coarsest level, backdrop +and intermediate data provide immediate visual coverage, while the set of loaded target chunks +determines which regions have actually been resolved at the requested resolution. With `coarse_first=True`, +intermediate levels progressively sharpen unresolved regions and with `coarse_first=False`, +the loader proceeds directly from the initial backdrop to target-level chunks. + +## How progressive loading works in the experimental implementation +In the previous section we described some of the nomenclature, how data is represented and how it +is progressively refined. Here, we will describe how the loader decides what to load as the user +interacts with the viewer. +`ProgressiveLoader` responds to changes in the viewer and layer that can affect what data is needed. +Examples of this are camera movement, dimension changes, display mode, layer visibility and changes +to the selected resolution level. +When there is such an event, the loader does not always immediately start a new fetch. Events such +as camera interactions often generate many events in quick succession. Therefore, the loader waits +for the view to settle before determining the next region to load. While the user is actively interacting, +expensive streaming and rendering work can also be temporarily suspended to keep navigation responsive. + +Once the view is settled, the loader determines a couple of things: +1. Which pyramid level should be used. +2. Which region of that level is needed for the view. +3. What part of the region is already resident or currently being fetched. + +If the required data is not fully resident / in RAM, a new data fetch is started. +We present this here schematically: + +```{mermaid} +flowchart TD + A["Viewer or layer
state changes"] --> B["Interaction detected"] + + B --> C["Temporarily limit
streaming and rendering work"] + + C --> D["Wait for view
to settle"] + + D --> E["Determine target
pyramid level"] + + E --> F["Determine required
resident interval"] + + F --> G{"Required region
already fully covered?"} + + G -- Yes --> H["No new fetch
pass required"] + + G -- No --> I["Start new
fetch pass"] + + I --> J["Populate and progressively
refine resident interval"] +``` +We will now describe these 3 steps in more detail + +### 1. Selecting the resolution level +The way the resolution level is determined varies between 2D and 3D data in napari. +This because loading 3D data at a given resolution level could prove to be too +expensive. We will describe the selection of the resolution level in both 2D and 3D. + +#### Selection the resolution level for 2D data +For 2D data, napari's existing multiscale API selects the resolution level. The +progressive loader uses the elvel selected by this API rather than introducing +a separate mechanism for selecting the target resolution. +As such, `locked_data_level` (TODO: explain this) is respected. Once the level is known, +the loader determines the region required for the current view from the layer's +`corner-pixels`. + +#### Selecting the resolutoin level for 3D data +For 3D data, the resolution level can either be selected automatically by the progressive loader +or controlled through napari's normal level selection. This has different trade offs. + +With a setting `auto_level_3d=True`, the progressive loader automatically selects the resolution +level based on the camera zoom (TODO check whether this is napari selected). It then aims to select +the coarsest level of which the voxels project to no more than `max_pixel_size_3d` screen pixels +(the unit here is pixels per voxel). Lower values cause finer, more expensive levels to be selected +sooner while zooming in. For example, with `max_pixel_size_3d=2.0`, a finer level is preferred once a +voxel would occupy more than two screen pixels. +The visually appropriate level must also be practical to load and render. If the desired level would +exceed constraints such as the resident-interval memory budget or the maximum number of chunks +allowed for a 3D fetch pass, the loader selects a coarser level instead. +When auto_level_3d=False, the progressive loader does not automatically change the pyramid level in +response to camera zoom. Level selection is left to napari or to the user. The loader still performs +progressive loading for whichever level is currently selected: it determines the required resident +interval, provides backdrop coverage, and fetches the missing chunks for that level. +If the user explicitly selects a resolution level while automatic 3D level selection is enabled, +that selection takes precedence and automatic level changes are suspended. Returning the resolution +selector to Auto gives level selection back to the progressive loader. + +The whole mechanism of 3D resolution level selection can therefore be summarized schematically as: +```{mermaid} +flowchart TD + A["3D view"] --> B{"auto_level_3d?"} + + B -- No --> C["Use level selected by
napari or the user"] + + B -- Yes --> D{"User explicitly
selected a level?"} + + D -- Yes --> C + + D -- No --> E["Determine coarsest level that
satisfies max_pixel_size_3d"] + + E --> F{"Level fits memory and
chunk-count constraints?"} + + F -- Yes --> G["Use selected level"] + + F -- No --> H["Try a coarser level"] + + H --> F + + C --> I["Progressively load
selected level"] + G --> I +``` +With this design, 2 concerns are separated: automatic level selection determines which resolution +to use, while progressive loading determines how the selected level is populated and displayed. + +### 2. Determining the region to load +ONce a resolution level has been selected, the loader determines which part of that elvel is relevant +to the current view. +For displayed dimensions, this region is based on the layer's `corner_pixels`. For dimensions that are +not currently displayed, only the current dimension step needs to be represented (TODO: check with Kyle +if he sees value here for different mode, e.g. with tracking data). +The requested region is then converted into the resident interval used by `VirtualData`. + +The resident interval can be larger than the exact rendered viewport. Specifically, the GPU texture +contains the crop described by `corner_pixels`, while the resident interval on the CPU follows underlying +chunk boundaries and can therefore extend beyond the rendered region as shown below: + +```{mermaid} +flowchart TB + subgraph R["Resident interval in RAM"] + direction TB + + V["Rendered viewport
represented by GPU texture"] + end +``` + +### 3. Fetching chunks +After the resident interval and backdrop have been established, the loader determines which chunks +still need to be loaded. +The source arrays are chunked, meaning that the data is divided in predefined blocks, rather than +arbitrary pixel slices. Only chunks intersercting the requested region are considered with chunks +already recorded as loaded excluded from the fetch queue. +As stated before, the resident interval follows the underlying data source chunk boundaries, +while the viewport contains a crop of it. While this may cause the resident interval to contain +data outside the visible viewport, it avoids repeatedly partially fetching chunks as the view moves. +Thus, an overview of fetching chunks looks like this: + +```{mermaid} +flowchart TD + A["Visible viewport"] --> B["Determine intersecting
source chunks"] + + B --> C["Chunk-aligned
resident region"] + + C --> D["Exclude chunks that
are already loaded"] + + D --> E["Prioritize remaining
missing chunks"] + + E --> F["Fetch chunks"] +``` +We will now discuss chunk priority and fetch workers. + +#### Chunk priority +Missing chunks are not fetched with storage order in mind, rather they are prioritized based on +their relevance to the current view. + +In 2D, chunks are prioritized based on their distance to the center of the resident interval. Distance is +not calculated for all chunks of the underlying data source. Instead, only distance is only calculated for +chunks intersecting with the resident interval. + +For 3D, the camera is also taken into account. Chunks are primarily fetched based on their depth +along the view direction, with those closest to the viewer loading first. The distance from the camera +center line provides a second, but lower weighted, prioritization for fetching chunks. This ensures +chunks on the camera center line will be prioritized when multiple chunks are present at equal depth. + +For both 2D and 3D data, the aim is not to resolve every chunk intersecting with the resident interval simultaneously, +but to spend the available loading capacity first on those chunks likely to visually matter the most. + +#### Fetch workers +Fetching chunks happens on background worker threads, rather than on the main thread. The number of concurrent workers +is controlled by fetch_workers. By default, the implementation uses up to four workers while deliberately leaving +CPU capacity (2 cores by default) available for the GUI. + +In summary, a fetch thus happens in a way shown in this diagram: +```{mermaid} +flowchart TD + A["Target resident interval"] --> B["Find intersecting chunks"] + + B --> C["Remove already
loaded chunks"] + + C --> D["Prioritize missing chunks"] + + D --> E{"coarse_first?"} + + E -- Yes --> F["Fetch intermediate-level
chunk stages first"] + + F --> G["Fill unresolved target regions with
intermediate-level data"] + + G --> H["Fetch target-level chunks"] + + E -- No --> H + + H --> I["Write fetched chunks into
target VirtualData in RAM"] + + I --> J["Update displayed data"] + + J --> K{"More target chunks
to fetch?"} + + K -- Yes --> H + K -- No --> L["Fetch pass complete"] +``` + +### 4. Keeping interaction responsive +Progressive loading runs while the user is navigating the dataset, so solely maximizing chunk +throughput is not the only goal. Rather, we try to maximize the chunk throughput with the viewer +still responding smoothly to user input. For this, the `ProgressiveLoader` has a few mechanisms +in place. + +#### Interaction hold +The `ProgressiveLoader` has a setting called `interaction_hold`. If set to `True`, the loader +temporarily reduces background loading and display-update work while the user is actively +interacting with the viewer. This prevents progressive loading from competing with interactions +such as panning, zooming, rotating, or moving through dimensions. During this period: + +- Fetch workers are paused +- Chunks that finish loading are kept for later processing rather than immediately updating the display +- Non-essential display updates are deferred +- In 3D, rendering quality can be temporarily reduced to make interactive frames cheaper + +Once the interaction settles, the loader resumes the suspended work, processes any chunks that +completed in the meantime, and reevaluates the current view. In short, upon user interaction +this diagram is followed: + +```{mermaid} +flowchart TD + A["User starts interacting"] --> B["Enter interaction hold"] + + B --> C["Pause chunk fetching"] + B --> D["Defer completed chunk
display updates"] + B --> E["Reduce 3D rendering cost"] + + C --> F["Interaction settles"] + D --> F + E --> F + + F --> G["Resume loading and
display updates"] + + G --> H["Reevaluate current view"] +``` + +#### Rate limiting the loading of chunks +Even when chunk fetching happens on background threads, loading data as quickly as possible can +still affect viewer responsiveness. Fetching a chunk can involve more than waiting for I/O: each +chunk leads to additional CPU and display-update work. +The `ProgressiveLoader` provides a setting, `max_bytes_per_second`, that limits the rate at which +fetch workers start loading chunks. +This pacing happens on the worker threads, before each fetch. However, it indirectly limits the pace +of work that has to happen downstream: + +```{mermaid} +flowchart TD + A["Rate-limit chunk fetching"] --> B["Reduce source loading /
computation pressure"] + + B --> C["Reduce frequency of
completed chunk batches"] + + C --> D["Reduce display-update work"] + + D --> E["Reduce GPU upload pressure"] +``` + +The purpose of the rate limit is therefore not necessarily to restrict network bandwidth. It +provides a way to control the amount of loading-related work entering the pipeline so that +background loading does not overwhelm interactive rendering. + +Together, rate limiting and interaction hold serve different purposes. Rate limiting controls +how aggressively loading proceeds during normal operation, while interaction hold temporarily +gives priority to active user interaction. + +### 5. Updating the displayed data +So far, we have mainly discussed how chunks are fetched and written into the target level's +`VirtualData` in RAM. Once those chunks arrive, the corresponding changes also need to become +visible on screen. +The CPU-side `VirtualData` remains the source of the progressively loaded image data. The displayed +representation is a texture held in GPU memory by the napari/vispy rendering pipeline: + +```{mermaid} +flowchart TD + A["Fetched target chunk"] --> B["Write chunk into target
VirtualData in RAM"] + + B --> C["Update corresponding
displayed data"] + + C --> D["GPU texture"] + + D --> E["Canvas"] +``` + +There are two main ways in which the progressive loader can make newly loaded data visible: +through napari's normal refresh (`layer.refresh()`) path or, when possible, by directly updating +only the affected part of the GPU texture. + +#### Normal refresh +A normal refresh by `layer.refresh()` causes the currently required data to be sliced again and +updates the displayed texture: + +```{mermaid} +flowchart TD + A["VirtualData changed"] --> B["napari refresh"] + + B --> C["Slice current view"] + + C --> D["Upload texture data"] + + D --> E["Render updated view"] +``` +However, repeatedly performing the complete path for every arriving chunk batch can become expensive, particularly +for large 3D textures. However, performing a complete refresh every time a batch of chunks arrives can be expensive. +In particular, a refresh may involve re-slicing data and uploading an entire texture even when only a small region +has changed. The progressive loader therefore uses several mechanisms to reduce this work. + +#### Refresh throttling +When a `layer.refresh` is required, the loader avoids triggering one for every arriving chunk batch. +The `refresh_interval_s` setting of the `ProgressiveLoader` specifies the minimum time between refreshes +while chunks are being loaded. The effective interval can increase when refreshes themselves are +expensive (such as with 3D volumes), spacing subsequent refreshes farther apart. +This allows multiple chunk updates to accumulate between refreshes: + +```{mermaid} +flowchart TD + A["Chunks arrive"] --> B["Write chunks into
VirtualData"] + + B --> C{"Refresh interval
has elapsed?"} + + C -- No --> D["Defer refresh"] + D --> A + + C -- Yes --> E["Refresh current view"] + + E --> F["Measure refresh cost"] + + F --> G["Adjust effective
refresh interval"] + + G --> A +``` +Refresh throttling therefore trades immediately displaying each update, for fewer expensive +slicing and texture-uploads. + +#### Texture patching +A full refresh is not always necessary. When the setting `texture_patching=True`, the loader +first attempts to update only the region of the existing GPU texture affected by the newly +loaded chunks. +The implementation supports partial texture updates for both 2D and 3D displays. This is +particularly useful in 3D, where a normal refresh can require re-slicing and re-uploading an +entire volume tile. +For a batch or arriving chunks, the loader combines their affected area into a bounding region +and attempts to upload that region as a single partial texture update. Data within that region +that has not yet reached target resolution simply retains or re-uploads its current backdrop +content. + +```{mermaid} +flowchart TD + A["Target chunks written into
VirtualData in RAM"] --> B{"Texture can be
patched safely?"} + + B -- Yes --> C["Determine affected
texture region"] + + C --> D["Upload changed region
to GPU texture"] + + D --> E["Update displayed view"] + + B -- No --> F["Fall back to
throttled refresh"] + + F --> G["Slice current view"] + + G --> H["Upload displayed data
to GPU texture"] + + H --> E +``` +Texture patching is only performed when the loader can establish that the current GPU texture +corresponds to the current rendered region. For example, its shape and position must match the +crop represented by layer's current `corner_pixels`. If the texture cannot be matched safely +to the current view, the loader falls back to a normal refresh. +Texture patching only changes how newly loaded data is transferred from the CPU-side `VirtualData` +to the GPU texture. It does not change how chunks are fetched or stored: `VirtualData` in RAM +remains the authoritative representation of the resident image data. + +During a fetch pass, the loader therefore generally follows this strategy: +1. Write arriving target chunks into `VirtualData` +2. Try patch the corresponding GPU texture region directly +3. Fall back to a throttled `layer.refresh()` if patching is not possible +4. After a fully fetched pass successfully texture-patched, defer the normal refresh and later use it to reconcile +the slice state and layer thumbnail in napari, while avoiding a redundant full texture upload. + +#### Double buffering +When the view changes, the texture needed for the new view may not be ready immediately. Replacing +the currently displayed texture too early could expose an incomplete tile while backdrop data is +being prepared or chunks are still arriving. + +When `double_buffer=True` for the `ProgressiveLoader`, it uses double buffering to separate the +texture currently being displayed from the texture being prepared for the new view. Think of it +as one texture being the front buffer which remains visible, while another texture acts as a back +buffer and receives data for the new view. + +```{mermaid} +flowchart TD + A["View changes"] --> B["Keep current front texture
visible"] + + B --> C["Prepare new view in
back texture"] + + C --> D["Apply backdrop and
chunk updates"] + + D --> E{"Back texture ready
to present?"} + + E -- No --> D + E -- Yes --> F["Present back texture"] + + F --> G["Prepared texture becomes
the displayed texture"] +``` + +At the start of a fetch pass, the double buffer is attached before the initial backdrop refresh. +This allows a full texture upload or texture reallocation for the new view to be staged without +immediately replacing the texture that is currently being rendered. +This is especially useful for 3D volumes. When the view changes, the data needed for the new +view may not be ready immediately. The previous texture can remain visible while the new texture +is populated with backdrop data and newly fetched chunks, preventing an incomplete or empty view +from being shown. + +Once the back buffer / texture is ready to be displayed, it can be presented and becomes the new +front texture. It is important to note that this whole mechanism does not introduce another copy +of the image pyramid in GPU memory. The two texture buffers are maintained for the currently +rendered region, image region in 2D or volume tile in 3D, so that one can remain visible while the +other is beign prepared. + +#### GPU upload metering +Even with texture patching and double buffering, transferring a large texture from CPU memory to GPU +memory can take enough time to interrupt interaction. +The loader therefore meters larger texture uploads. Instead of allowing a large pendign upload +to monopolize a rendeirng frame, the transfer can be divided into smaller pieces and spread +across multiple frames: + +```{mermaid} +flowchart TD + A["Texture data waiting
for GPU upload"] --> B["Upload a bounded
amount"] + + B --> C["Return control to
rendering"] + + C --> D{"More texture data
pending?"} + + D -- Yes --> B + D -- No --> E["Upload complete"] +``` +This mechanism addresses a different cost from refresh throttling and texture patching: +- refresh throttling limits how often the normal `layer.refresh` is used. +- Texture patching avoids a full refresh when an existing texture can be updated directly. +- Double buffering keeps a valid texture visible while a replacement is prepared. +- GPU upload metering limits how much texture-transfer to the GPU is performed at once. + +All these mechanisms also connect back to the interaction hold explained previously. During +active interaction, metered texture uploads can be held along with chunk fetching and other +non-essential display updates. Once interaction settles, the pending operations are resumed. + +### 6. Changing views while loading +A fetch pass may still be running when the user pans, zooms, changes dimensions or selects +another resolution level. In that case, continuing to treat the old request as current would +be wasteful and could cause the now stale result to affect the new view. +Therefore, the loader associates loading work with the current view. The old stale request +is replaced when a new fetch pass is required: + +```{mermaid} +flowchart TD + A["Fetch pass in progress"] --> B["View changes"] + + B --> C["Determine new level
and resident interval"] + + C --> D{"Same active
view?"} + + D -- Yes --> E["Keep current
fetch pass"] + + D -- No --> F["Cancel active pass"] + + F --> G["Establish new
resident interval"] + + G --> H["Start fetch pass
for new view"] +``` +When a new pass starts, the active worker is canceled and a new generation is created +for the new request. The previous workers are ignored by callbacks. This is possible +by keeping track of the number of the generation of workers. +This is important because canceling background work does not necessarily mean that every +operation in progress stops instantaneously. Generation checking is an ultimate defensive +measure for preventing stale results from being treated as the results for the current view. + +### 7. Fetch pass completion +A fetch pass is complete once all required stges for the current view have finished. +With `coarse_first=True`, these stages include the intermediate pyramid levels followed +by the target level. Otherwise, the target level is the main fetch stage. +By this point, the requested regions have been populated with the target-resolution data and +the loader can release any presentation holds associated with the pass. If texture patching +successfully handled the entire target pass, the displayed texture already contains the fetched +daata. A normal refresh can therefore be deferred until it is needed to update the slice state +and layer thumbnail, as explaiend previously. A final schematic summary of a progressive loading +request thus becomes: + +```{mermaid} +flowchart TD + A["View changes"] --> B["Select target level"] + + B --> C["Determine resident interval"] + + C --> D["Carry over existing data
and establish backdrop"] + + D --> E["Build prioritized
chunk fetch stages"] + + E --> F["Fetch chunks in
background workers"] + + F --> G["Write chunks into
VirtualData in RAM"] + + G --> H["Patch GPU texture or
use throttled refresh"] + + H --> I{"More chunks or
stages remain?"} + + I -- Yes --> F + I -- No --> J["Complete fetch pass"] + + J --> K["Present final staged
content if necessary"] + + K --> L["View remains available
for further interaction"] + + L --> M["Next view change"] + M --> B +```