Backing Up VM Images to Wasabi (or S3, B2, etc.)
May 20, 2020
My current setup is a RAIDz2 with VM images stored in the raw
img format. I backup important individual files using
rsync, but sometimes I wish I had a copy of each VM so I can easily restore it if something goes terribly wrong. Wasabi is probably the best option for this because everyone else charges you to download your data which can get very expensive if you have large VM images.
- ZFS pool name -
- VM path -
- Cloned snapshot path -
You will need to install these packages:
zstd mbuffer s3cmd
Each time this script is run it will destroy the previous snapshot for the day of week it was taken on. The snapshot is then cloned so each file can be compressed and uploaded using
s3cmd. The clone is then destroyed so there are no hanging snapshots.
#!/bin/bash ZFS_BIN=$(which zfs || echo /sbin/zfs) BUCKET="YOUR-BUCKET" DOW=$(date +"%a") SNAPSHOT="[email protected]$DOW" VM="local/[email protected]$DOW" $ZFS_BIN destroy local/clone $ZFS_BIN destroy -r $SNAPSHOT $ZFS_BIN snapshot -r $SNAPSHOT $ZFS_BIN clone $VM local/clone s3cmd put --recursive /etc/libvirt/qemu/ s3://$BUCKET/ for filename in /mnt/local/clone/*.*; do pzstd -c -1 $filename | mbuffer -q -m 100M | s3cmd put --multipart-chunk-size-mb=100 - s3://$BUCKET/$(basename "$filename").zst done $ZFS_BIN destroy local/clone
zstd -d <path>