I’m moving large-ish (half-TB or larger) files between hosts. It’s important to avoid extra copies in this workflow, since each pass over one of the larger files to read the whole thing (and there are several of them) takes hours. I managed to decrypt VM disk images, transform them from one disk image format to another, copy them from one host to another, calculate SHA-256 hashes on both sides to verify data integrity, compress and encrypt them on the destination, and to display a progress bar, all without any additional copies. One big block-device read on the source end and one big block-device write on the destination end is all of the disk I/O that’s happening.
See below for how I did this.
Continue reading “Minimizing I/O when migrating from Virtualbox on Veracrypt to Proxmox on ZFS”