Module tests.test_csi_snapshotter

Functions

def create_pod_from_bi_type_volumesnapshot_pvc_and_check_data(core_api, csivolsnap_name, pod_make, pvc, request)
def delete_volumesnapshot(name, namespace)
def get_volumesnapshotcontent(volumesnapshot_uid)
def prepare_bi_type_test(bi_checksum, bi_url, volumesnapshotclass, volumesnapshotcontent, volumesnapshot)
def prepare_test_csi_snapshot(apps_api, client, make_deployment_with_pvc, volumesnapshotclass, core_api, csi_snapshot_type='snap')

Context:

After deploy the CSI snapshot CRDs, Controller at version>/snapshots-and-backups/ csi-snapshot-support/enable-csi-snapshot-support/

Create VolumeSnapshotClass with type=snap - longhorn-snapshot (type=snap) Create Longhorn volume test-vol - Size 5GB - Create PV/PVC/Workload for the Longhorn volume - Write data into volume - Setup backup store

def restore_csi_volume_snapshot(core_api, client, csivolsnap, pvc_name, pvc_request_storage_size, wait_for_restore=True)
def test_csi_snapshot_create_csi_snapshot(set_random_backupstore, apps_api, client, make_deployment_with_pvc, volume_name, volumesnapshotclass, volumesnapshot, csi_pv, pvc, core_api, pod_make, csi_snapshot_type)

Context:

After deploy the CSI snapshot CRDs, Controller at https://longhorn.io/docs/1.4.2/snapshots-and-backups/ csi-snapshot-support/enable-csi-snapshot-support/

Create VolumeSnapshotClass with type=snap|bak - longhorn-snapshot (type=snap|bak)

Test the extend CSI snapshot type=snap|bak support to Longhorn snapshot|backup

Steps:

  1. Create Longhorn volume test-vol
    • Size 5GB
    • Create PV/PVC/Workload for the Longhorn volume
    • Write data into volume
    • Setup backup store
  2. Test create CSI snapshot
    • Volume is in detached state
      • Scale down the workload
      • Create VolumeSnapshot with class longhorn-snap|bak
      • Verify that the volumesnapshot object is ready
    • Volume is in attached state
      • Scale up the workload
      • Verify the Longhorn snapshot generated
def test_csi_snapshot_delete_csi_snapshot_volume_detached(set_random_backupstore, apps_api, client, make_deployment_with_pvc, volumesnapshotclass, volumesnapshot, core_api, csi_snapshot_type)
  1. Create volumesnapshotclass with type=snap|bak
  2. Create Longhorn volume test-vol
    • Size 5GB
    • Create PV/PVC/Workload for the Longhorn volume
    • Write data into volume
    • Setup backup store
    • Create volumeSnapshot by volumesnapshotclass in step 1
  3. Test delete CSI snapshot : Type is snap|bak
    • volume is detached
      • Delete the VolumeSnapshot
      • Verify that VolumeSnapshot is not stuck in deleting
def test_csi_snapshot_snap_create_volume_from_snapshot(apps_api, client, make_deployment_with_pvc, volume_name, volumesnapshotclass, volumesnapshot, csi_pv, pvc, core_api, pod_make)

Context:

After deploy the CSI snapshot CRDs, Controller at https://longhorn.io/docs/1.4.2/snapshots-and-backups/ csi-snapshot-support/enable-csi-snapshot-support/

Create VolumeSnapshotClass with type=snap - longhorn-snapshot (type=snap)

Test the extend CSI snapshot type=snap support to Longhorn snapshot

Steps:

  1. Create Longhorn volume test-vol

    • Size 5GB
    • Create PV/PVC/Workload for the Longhorn volume
    • Write data into volume
    • Setup backup store

  2. Test create new volume from CSI snapshot

    • Create VolumeSnapshot with class longhorn-snap
    • Create volume from longhorn-snapshot
      • Source volume is attached && Longhorn snapshot exist
        • Create PVC from snapshot generated from step 1
        • Attach the PVC and verify data
      • Source volume is detached
        • Scale down the workload
        • Create PVC from VolumeSnapshot generated from step beginning
        • Verify PVC provision failed
        • Scale up the workload
        • Wait for PVC to finish provisioning and be bounded
        • Attach the PVC test-restore-pvc and verify the data
      • Source volume is attached && Longhorn snapshot doesn’t exist
        • Use VolumeSnapshotContent.snapshotHandle to specify Longhorn snapshot generated in step beginning
        • Delete the Longhorn snapshot
        • Create PVC from VolumeSnapshot generated from step beginning
        • PVC should be stuck in provisioning state
def test_csi_snapshot_snap_delete_csi_snapshot_snapshot_exist(apps_api, client, make_deployment_with_pvc, volumesnapshotclass, volumesnapshot, core_api)
  1. Create volumesnapshotclass with type=snap
  2. Create Longhorn volume test-vol
    • Size 5GB
    • Create PV/PVC/Workload for the Longhorn volume
    • Write data into volume
    • Setup backup store
    • Create volumeSnapshot by volumesnapshotclass in step 1
  3. Test delete CSI snapshot : Type is snap
    • volume is attached && snapshot exist
      • Verify the creation of Longhorn snapshot with the name in the field VolumeSnapshotContent.snapshotHandle
      • Delete the VolumeSnapshot
      • Verify that Longhorn snapshot is removed or marked as removed
      • Verify that the VolumeSnapshot is deleted.
def test_csi_snapshot_snap_delete_csi_snapshot_snapshot_not_exist(apps_api, client, make_deployment_with_pvc, volumesnapshotclass, volumesnapshot, core_api)
  1. Create volumesnapshotclass with type=snap
  2. Create Longhorn volume test-vol
    • Size 5GB
    • Create PV/PVC/Workload for the Longhorn volume
    • Write data into volume
    • Setup backup store
    • Create volumeSnapshot by volumesnapshotclass in step 1
  3. Test delete CSI snapshot : Type is snap
    • volume is attached && snapshot doesn’t exist
      • Delete the VolumeSnapshot
      • VolumeSnapshot is deleted
def test_csi_snapshot_with_bak_param(set_random_backupstore, volumesnapshotclass, volumesnapshot, client, core_api, volume_name, csi_pv, pvc, pod_make, volsnapshotclass_delete_policy, backup_is_deleted)

Context:

After deploy the CSI snapshot CRDs, Controller at https://longhorn.io/docs/1.2.3/snapshots-and-backups/ csi-snapshot-support/enable-csi-snapshot-support/

Create VolumeSnapshotClass with type=bak - longhorn-backup (type=bak)

Test the extend CSI snapshot type=bak support to Longhorn snapshot

Steps:

  1. Create Longhorn volume test-vol
    • Size 5GB
    • Create PV/PVC/Workload for the Longhorn volume
    • Write data into volume
    • Setup backup store
  2. Test create CSI snapshot
    • Create VolumeSnapshot with class longhorn-backup
    • Should have backup generated
  3. Test create new volume from CSI snapshot
    • Create PVC from VolumeSnapshot generated in step 1
    • Attach PVC and verify data
  4. Test delete CSI snapshot
    • Delete VolumeSnapshot
    • The backup should deleted as well
def test_csi_snapshot_with_invalid_param(volumesnapshotclass, volumesnapshot, client, core_api, volume_name, csi_pv, pvc, pod_make, request)

Context:

After deploy the CSI snapshot CRDs, Controller at https://longhorn.io/docs/1.2.4/snapshots-and-backups/ csi-snapshot-support/enable-csi-snapshot-support/

Create VolumeSnapshotClass with type=invalid - invalid (type=invalid)

Test the extend CSI snapshot type=invalid behavior to Longhorn snapshot

Steps:

  1. Create Longhorn volume test-vol
    • Size 5GB
    • Create PV/PVC for the Longhorn volume
    • Write data into volume
    • Setup backup store
  2. Test create CSI snapshot
    • Create VolumeSnapshot with class invalid
    • Verify that the volumesnapshot object is not ready
def test_csi_volumesnapshot_backing_image_basic(client, core_api, csi_pv, pod_make, pvc, request, volume_name, volumesnapshotclass, volumesnapshot)

Test Create/Delete BackingImage using VolumeSnapshot with a given Volume

Setup - Create a VolumeSnapshotClass with type bi kind: VolumeSnapshotClass apiVersion: snapshot.storage.k8s.io/v1 metadata: name: longhorn-snapshot-vsc driver: driver.longhorn.io deletionPolicy: Delete parameters: type: bi

Given - The Volume attached to a workload, having data and computed md5sum.

When - Creating the VolumeSnapshot apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshot metadata: name: test-snapshot-backing spec: volumeSnapshotClassName: longhorn-snapshot-vsc source: persistentVolumeClaimName: test-vol

Then - A BackingImage is created with the following properties apiVersion: longhorn.io/v1beta2 kind: BackingImage metadata: name: `snapshot-${VolumeSnapshot.uuid}` namespace: longhorn-system spec: sourceType: export-from-volume sourceParameters: volume-name: test-vol export-type: raw

When - Creating a PVC with dataSource pointing to the VolumeSnapshot apiVersion: v1 kind: PersistentVolumeClaim metadata: name: test-restore-pvc spec: storageClassName: longhorn dataSource: name: test-snapshot-backing kind: VolumeSnapshot apiGroup: snapshot.storage.k8s.io accessModes: - ReadWriteOnce resources: requests: storage: 5Gi

Then - A Volume is created using BackingImage snapshot-${VolumeSnapshot.uuid} - Verifying the data and md5sum in the new Volume

When - Delete the new Volume from the the VolumeSnapshot - Delete the VolumeSnapshot > kubectl delete vs/test-snapshot-backing

Then - The BackingImage is deleted as well

def test_csi_volumesnapshot_backing_image_with_selectors(client, core_api, csi_pv, pod_make, pvc, request, volume_name, volumesnapshotclass, volumesnapshot)

Test Create BackingImage with nodeSelector and diskSelector using VolumeSnapshot with a given Volume

This test is to test users can set the nodeSelector and diskSelector through csi volumeSnapshot. For the scheduling, we already test it in backing image e2e tests. For the content verification, we already test it in test_csi_volumesnapshot_backing_image_basic.

Setup - Create a VolumeSnapshotClass with type bi kind: VolumeSnapshotClass apiVersion: snapshot.storage.k8s.io/v1 metadata: name: longhorn-snapshot-vsc driver: driver.longhorn.io deletionPolicy: Delete parameters: type: bi nodeSelector: "node1" diskSelector: "ssd"

Given - The Volume test-vol attached to a workload, having data and computed md5sum. - Create the PV and PVC from the Volume

When - Creating the VolumeSnapshot apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshot metadata: name: test-snapshot-backing spec: volumeSnapshotClassName: longhorn-snapshot-vsc source: persistentVolumeClaimName: test-vol

Then - A BackingImage is created with the correct selectors apiVersion: longhorn.io/v1beta2 kind: BackingImage metadata: name: `snapshot-${VolumeSnapshot.uuid}` namespace: longhorn-system spec: sourceType: export-from-volume sourceParameters: volume-name: test-vol export-type: raw nodeSelector: - node1 diskSelector: - node1

When - Delete the new Volume from the the VolumeSnapshot - Delete the VolumeSnapshot > kubectl delete vs/test-snapshot-backing

Then - The BackingImage is deleted as well

def test_csi_volumesnapshot_basic(set_random_backupstore, volumesnapshotclass, volumesnapshot, client, core_api, volume_name, csi_pv, pvc, pod_make, volsnapshotclass_delete_policy, backup_is_deleted, csi_snapshot_type=None)

Test creation / restoration / deletion of a backup via the csi snapshotter

Context:

We want to allow the user to programmatically create/restore/delete longhorn backups via the csi snapshot mechanism ref: https://kubernetes.io/docs/concepts/storage/volume-snapshots/

Setup:

  1. Make sure your cluster contains the below crds https://github.com/kubernetes-csi/external-snapshotter /tree/master/client/config/crd
  2. Make sure your cluster contains the snapshot controller https://github.com/kubernetes-csi/external-snapshotter /tree/master/deploy/kubernetes/snapshot-controller

Steps:

def csi_volumesnapshot_creation_test(snapshotClass=longhorn|custom): 1. create volume(1) 2. write data to volume(1) 3. create a kubernetes VolumeSnapshot object the VolumeSnapshot.uuid will be used to identify a longhorn snapshot and the associated VolumeSnapshotContent object 4. check creation of a new longhorn snapshot named snapshot-uuid 5. check for VolumeSnapshotContent named snapcontent-uuid 6. wait for VolumeSnapshotContent.readyToUse flag to be set to true 7. check for backup existence on the backupstore

the csi snapshot restore sets the fromBackup field same as

the StorageClass based restore approach.

def csi_volumesnapshot_restore_test(): 8. create a PersistentVolumeClaim object where the dataSource field references the VolumeSnapshot object by name 9. verify creation of a new volume(2) bound to the pvc created in step(8) 10. verify data of new volume(2) equals data from backup (ie old data above)

default longhorn snapshot class is set to Delete

add a second test with a custom snapshot class with deletionPolicy

set to Retain you can reuse these methods for that and other tests

def csi_volumesnapshot_deletion_test(deletionPolicy='Delete|Retain'): 11. delete VolumeSnapshot object 12. if deletionPolicy == Delete: 13. verify deletion of VolumeSnapshot and VolumeSnapshotContent objects 14. verify deletion of backup from backupstore 12. if deletionPolicy == Retain: 13. verify deletion of VolumeSnapshot 14. verify retention of VolumeSnapshotContent and backup on backupstore

  1. cleanup
def test_csi_volumesnapshot_restore_existing_backup(set_random_backupstore, client, core_api, volume_name, csi_pv, pvc, pod_make, volumesnapshotclass, volumesnapshotcontent, volumesnapshot, volsnapshotclass_delete_policy, backup_is_deleted)

Test retention of a backup while deleting the associated VolumeSnapshot via the csi snapshotter

Context:

We want to allow the user to programmatically create/restore/delete longhorn backups via the csi snapshot mechanism ref: https://kubernetes.io/docs/concepts/storage/volume-snapshots/

Setup:

  1. Make sure your cluster contains the below crds https://github.com/kubernetes-csi/external-snapshotter /tree/master/client/config/crd
  2. Make sure your cluster contains the snapshot controller https://github.com/kubernetes-csi/external-snapshotter /tree/master/deploy/kubernetes/snapshot-controller

Steps:

  1. create new snapshotClass with deletionPolicy set to Retain
  2. call csi_volumesnapshot_creation_test(snapshotClass=custom)
  3. call csi_volumesnapshot_restore_test()
  4. call csi_volumesnapshot_deletion_test(deletionPolicy='Retain'):
  5. cleanup
def test_csi_volumesnapshot_restore_on_demand_backing_image(bi_url, bi_checksum, client, core_api, pod_make, pvc, request, volumesnapshotclass, volumesnapshotcontent, volumesnapshot)

Test Restore Volume from CSI VolumeSnapshot with on-demand BackingImage

Setup - Create a VolumeSnapshotClass with type bi kind: VolumeSnapshotClass apiVersion: snapshot.storage.k8s.io/v1 metadata: name: longhorn-snapshot-vsc driver: driver.longhorn.io deletionPolicy: Delete parameters: type: bi

Given - Creating VolumeSnapshotContent and VolumeSnapshot to associate with the BackingImage # NOQA - (snapshotHandle was dynamic with 2 different of backing images in test script) apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshotContent metadata: name: test-on-demand-backing spec: volumeSnapshotClassName: longhorn-snapshot-vsc driver: driver.longhorn.io deletionPolicy: Delete source: snapshotHandle: bi://backing?backingImageDataSourceType=download&backingImage=test-bi&url=https%3A%2F%2Flonghorn-backing-image.s3-us-west-1.amazonaws.com%2Fparrot.qcow2&backingImageChecksum=bd79ab9e6d45abf4f3f0adf552a868074dd235c4698ce7258d521160e0ad79ffe555b94e7d4007add6e1a25f4526885eb25c53ce38f7d344dd4925b9f2cb5d3b # NOQA volumeSnapshotRef: name: test-snapshot-on-demand-backing namespace: default

```
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
    name: test-snapshot-on-demand-backing
spec:
    volumeSnapshotClassName: longhorn-snapshot-vsc
    source:
        volumeSnapshotContentName: test-on-demand-backing
```

When - Creating the PVC apiVersion: v1 kind: PersistentVolumeClaim metadata: name: test-restore-on-demand-backing spec: storageClassName: longhorn dataSource: name: test-snapshot-on-demand-backing kind: VolumeSnapshot apiGroup: snapshot.storage.k8s.io accessModes: - ReadWriteOnce resources: requests: storage: 5Gi Then - A BackingImage is created (sourceParameters was dynamic in test script) apiVersion: longhorn.io/v1beta2 kind: BackingImage metadata: name: test-bi namespace: longhorn-system spec: sourceType: download sourceParameters: url: <https://longhorn-backing-image.s3-us-west-1.amazonaws.com/parrot.qcow2> # NOQA checksum: bd79ab9e6d45abf4f3f0adf552a868074dd235c4698ce7258d521160e0ad79ffe555b94e7d4007add6e1a25f4526885eb25c53ce38f7d344dd4925b9f2cb5d3b # NOQA - A Volume is created using the BackingImage test-bi - Verify the data (Directories of the backing images) exists in the mount point.

def test_csi_volumesnapshot_restore_pre_provision_backing_image(bi_url, bi_checksum, client, core_api, pod_make, pvc, request, volumesnapshotclass, volumesnapshotcontent, volumesnapshot)

Test Restore Volume from CSI VolumeSnapshot with existing BackingImage

Setup - Create a VolumeSnapshotClass with type bi kind: VolumeSnapshotClass apiVersion: snapshot.storage.k8s.io/v1 metadata: name: longhorn-snapshot-vsc driver: driver.longhorn.io deletionPolicy: Delete parameters: type: bi

Given - Creating a BackingImage apiVersion: longhorn.io/v1beta2 kind: BackingImage metadata: name: test-bi namespace: longhorn-system spec: sourceType: download sourceParameters: url: <https://longhorn-backing-image.s3-us-west-1.amazonaws.com/parrot.qcow2> # NOQA checksum: bd79ab9e6d45abf4f3f0adf552a868074dd235c4698ce7258d521160e0ad79ffe555b94e7d4007add6e1a25f4526885eb25c53ce38f7d344dd4925b9f2cb5d3b # NOQA - Creating VolumeSnapshotContent and VolumeSnapshot to associate with the BackingImage - (snapshotHandle was dynamic with 2 different of backing images in test script) apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshotContent metadata: name: test-existing-backing spec: volumeSnapshotClassName: longhorn-snapshot-vsc driver: driver.longhorn.io deletionPolicy: Delete source: snapshotHandle: bi://backing?backingImageDataSourceType=download&backingImage=test-bi&url=https%3A%2F%2Flonghorn-backing-image.s3-us-west-1.amazonaws.com%2Fparrot.qcow2&backingImageChecksum=bd79ab9e6d45abf4f3f0adf552a868074dd235c4698ce7258d521160e0ad79ffe555b94e7d4007add6e1a25f4526885eb25c53ce38f7d344dd4925b9f2cb5d3b # NOQA volumeSnapshotRef: name: test-snapshot-existing-backing namespace: default

```
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
    name: test-snapshot-existing-backing
spec:
    volumeSnapshotClassName: longhorn-snapshot-vsc
    source:
        volumeSnapshotContentName: test-existing-backing
```

When - Creating the PVC apiVersion: v1 kind: PersistentVolumeClaim metadata: name: test-restore-existing-backing spec: storageClassName: longhorn dataSource: name: test-snapshot-existing-backing kind: VolumeSnapshot apiGroup: snapshot.storage.k8s.io accessModes: - ReadWriteOnce resources: requests: storage: 5Gi

Then - A Volume is created using the BackingImage test-bi - Verify the data (Directories of the backing images) exists in the mount point.

def volumesnapshot(request)
def volumesnapshotclass(request)
def volumesnapshotcontent(request)
def wait_for_volumesnapshot_ready(volumesnapshot_name, namespace, ready_to_use=True)
def wait_volumesnapshot_deleted(name, namespace, retry_counts=150, can_be_deleted=True)