Skip to content

Add append method to VaryingDimension for explicit chunk extension #3870

@maxrjones

Description

@maxrjones

Context:

From review discussion in #3802 (comment), @dcherian suggested adding an append(self, sizes: Iterable[int]) method to VaryingDimension to allow explicitly extending chunk edges.

Current behavior:

VaryingDimension.resize(new_extent) handles growth by appending a single chunk covering the gap:

# [10, 10, 10] resized from 30 to 45 produces [10, 10, 10, 15]

This works but gives the caller no control over how the new region is partitioned.

Proposed behavior:

# Explicit chunk extension
dim = VaryingDimension([10, 10, 10], extent=30)
dim = dim.append([10, 5])  # -> edges=(10, 10, 10, 10, 5), extent=45

# Would raise on FixedDimension (regular grids don't have explicit edges)

Open questions:

  • Should this live on VaryingDimension, ChunkGrid, or Array?
  • Should FixedDimension.append raise, or convert to VaryingDimension?
  • How does this interact with metadata serialization? Appending chunks to a regular grid would change the serialization format from "regular" to "rectilinear".
  • Should Array.resize() accept an optional chunks parameter that delegates to this? (Also noted in the design doc's resize section)

Related:

shall we also add append(self, sizes: Iterable[int]) to allow a different mode of extension (would fail for the FixedDimension).

Originally posted by @dcherian in #3802 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions