Skip to content

layer-stack.md

Latest commit

 

History

History
112 lines (91 loc) · 4.79 KB

layer-stack.md

File metadata and controls

112 lines (91 loc) · 4.79 KB

Layer stack

blockstor mirrors LINSTOR's layer_list model: a ResourceDefinition declares an ordered chain of layers, each adding capabilities on top of the layer below. The satellite walks the chain bottom-up when provisioning and top-down when tearing down.

Layers

Layer What it does
STORAGE Allocates the raw block device (LVM-thin LV, ZFS volume, loopfile, …)
LUKS cryptsetup luksFormat + luksOpen over the storage device
DRBD Renders .res and runs drbdadm to replicate over the network

The first entry in layerStack is the topmost layer the consumer Pod mounts. The last entry is always STORAGE (the storage layer is required — every replica needs a backing device, even diskless ones where it's the network leg).

Common compositions

Stack Use case
["DRBD","STORAGE"] Default. Replicated PVC, the cozystack production case.
["LUKS","STORAGE"] Single-replica encrypted PVC. No DRBD overhead.
["DRBD","LUKS","STORAGE"] Encrypted at-rest + replicated. Per-volume cipher.
["STORAGE"] Single-replica local mode. Ephemeral cache, scratch, ZFS dataset.

The default is ["DRBD","STORAGE"] — when RD.Spec.LayerStack is empty the controller inherits from the parent ResourceGroup.Spec.SelectFilter.LayerStack, and when both are empty the dispatcher falls through to the satellite's default-DRBD path.

Setting the stack

Per-RD via REST

curl -XPOST http://controller:3370/v1/resource-definitions \
  -H 'Content-Type: application/json' \
  -d '{"resource_definition":{"name":"pvc-1","layer_stack":["LUKS","STORAGE"]}}'

Per-RG via spawn template

linstor rg create encrypted-rg
linstor rg sp encrypted-rg DrbdOptions/Encryption/passphrase '<32-byte-secret>'
linstor rg c --place-count 2 encrypted-rg
linstor rg spawn encrypted-rg pvc-2 1G --layer-list LUKS DRBD

(blockstor accepts linstor's --layer-list flag verbatim because the REST shape mirrors upstream LINSTOR.)

Per-RD via kubectl

apiVersion: blockstor.cozystack.io/v1alpha1
kind: ResourceDefinition
metadata: {name: pvc-encrypted}
spec:
  layerStack: ["LUKS", "STORAGE"]
  props:
    DrbdOptions/Encryption/passphrase: "32-byte-secret"
  volumeDefinitions:
    - {volumeNumber: 0, sizeKib: 1048576}

LUKS specifics

  • Passphrase is per-RD via DrbdOptions/Encryption/passphrase. The upstream linstor rd set-property key.
  • The dispatcher folds the resolver-resolved passphrase onto the wire as DesiredResource.Props["LuksPassphrase"]; the satellite reads it from there.
  • Empty passphrase with LUKS in the stack fails the apply rather than silently producing an unencrypted volume.
  • Mapper name: <rd>-<vol>-luks → /dev/mapper/<rd>-<vol>-luks. Stable across reconciles so reopen on satellite restart re-uses it.
  • Volume grow: the satellite runs cryptsetup resize on the mapper after the storage layer has resized the underlying LV, before DRBD resizes the replicated device.

What blockstor doesn't yet support

  • Pluggable layer ordering: the stack must be one of the four rows in the table above. Arbitrary orderings (e.g. ["LUKS","DRBD","STORAGE"] with LUKS-over-DRBD instead of DRBD-over-LUKS) aren't yet validated or rendered.
  • Per-volume LUKS keys: every volume on a 2-volume RD currently uses the same RD-level passphrase. Per-volume keys (with master-key wrapping in the controller's KV store) is a follow-up.
  • Mid-stack changes: editing layerStack after a Resource is active doesn't re-encrypt or unwrap existing data. Treat it as set-once-per-RD until migration support lands.
  • Cluster-passphrase rotation: the cluster passphrase (POST /v1/encryption/passphrase) wraps RD passphrases in the controller's KV store; rotating it doesn't re-encrypt LUKS headers. Rotating the per-RD passphrase requires cryptsetup luksChangeKey which isn't wired yet — operators should drop the RD and recreate.

Testing

  • pkg/api/v1/layer_stack_test.go — RD → RG → default resolution.
  • pkg/satellite/reconciler_drbd_test.go::TestApplySkipsDRBDWhenLayerStackOmits — ["STORAGE"] produces no .res and no drbdadm.
  • pkg/satellite/reconciler_drbd_test.go::TestApplyLayersLUKS — pin cryptsetup luksFormat + luksOpen run on first activation.
  • pkg/satellite/reconciler_drbd_test.go::TestApplyLUKSFailsWithoutPassphrase — pin the explicit-error path.