Problem:
You are trying to run a Veeam backup on your EC2 machine (e.g. to a remote SMB or NFS service), but the backup fails immediately after the backup job is being started. The log looks like this:
21:52:29 Job BackupJob1 started at 2020-12-23 21:52:29 GMT 21:52:30 Preparing to backup 21:52:40 [error] Failed to create volume snapshot 21:52:41 [error] Failed to take volume snapshot 21:52:41 [error] Failed to perform backup 21:52:41 [error] Cannot find a compatible file system for storing snapshot data 21:52:41 [error] Processing finished with errors at 2020-12-23 21:52:41 GMT
The detailed log in /var/log/veeam/veeamsvc.log
shows these errors:
[23.12.2020 21:52:41.069] <140589764957952> lpbcore| Searching for the best candidate to store snapshot data. [23.12.2020 21:52:41.069] <140589764957952> lpbcore| Logical block size limit 512 bytes [23.12.2020 21:52:41.071] <140589764957952> lpbcore| Searching for the best candidate to store snapshot data. Failed. [23.12.2020 21:52:41.071] <140589764957952> lpbcore| ERR |No suitable devices for snapshot data storage were found. [23.12.2020 21:52:41.071] <140589764957952> lpbcore| >> |An exception was thrown from thread [140589764957952]. [23.12.2020 21:52:41.071] <140589764957952> lpbcore| Detecting snapshot storage parameters. Failed. [23.12.2020 21:52:41.071] <140589764957952> lpbcore| Creating snapshot storage. Storage type: stretch file Failed. [23.12.2020 21:52:41.071] <140589764957952> lpbcore| Creating machine snapshot using VeeamSnap kernel module Failed. [23.12.2020 21:52:41.071] <140589764957952> lpbcore| ERR |Snapshot creation operation has failed. [23.12.2020 21:52:41.071] <140589764957952> lpbcore| >> |Cannot find a compatible file system for storing snapshot data [23.12.2020 21:52:41.071] <140589764957952> lpbcore| >> |--tr:Failed to create machine snapshot [23.12.2020 21:52:41.071] <140589764957952> lpbcore| >> |An exception was thrown from thread [140589764957952]. [23.12.2020 21:52:41.071] <140589764957952> | Thread finished. Role: 'snapshot operation'. [23.12.2020 21:52:41.102] <140590156179200> lpbcore| ERR |Cannot find a compatible file system for storing snapshot data [23.12.2020 21:52:41.102] <140590156179200> lpbcore| >> |--tr:Failed to create machine snapshot [23.12.2020 21:52:41.102] <140590156179200> lpbcore| >> |--tr:Failed to finish snapshot creation process.
Solution:
Veeam currently fails to detect the EC2 EBS boot device /dev/xvda as a proper device to store snapshot data on.
- You need to create a separate EBS block volume and attach it to the VM. The recommended size is computed as follows: 15% * total size of the disks that will be backed up. In case you’re backing up files or directories, use 0.25 * the maximum anticipated size of the files or directories to be backed up.
- Attach that new EBS block device to the VM (e.g. as
/dev/xvdb
– uselsblk
to find the correct drive !) - Create a new partition table and a new partition on the EBS device using something like
sudo fdisk /dev/xvdb
then enter these commands into fdisk:
g
to create a new GUID partition table, thenn
to create a new partition. You can use the default parameters everywhere. Then, runw
to write the changes to disk and exitfdisk
usingCtrl+D
) - Create a new filesystem on the new partition e.g. using
sudo mkfs.ext4 /dev/xvdb1
- Mount the partition somewhere (e.g. on
/mnt/
) using a command likemount /dev/xvda1 /mnt
. In order to auto-mount on boot, use a line like/dev/xvdb1 /mnt ext4 defaults,auto 0 0
in
/etc/fstab
.lsblk
should now tell you that the partition is mounted, e.g.xvdb 202:16 0 2G 0 disk └─xvdb1 202:17 0 2G 0 part /mnt
- Re-run
veeam
. The backup should now work properly.
I don’t know why exactly this issue occurs but EC2’s /dev/xvda
doesn’t seem to be a normal block device from Veeam’s viewpoint.
Note that veeam computes the minimum space for a snapshot store for entire machine backups as 10% * free space + 5% * used space. I don’t know if these factors are constant or determined dynamically, hence my recommendation of 15% * total space is much more conservative.