-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Description
Overview of the Issue
I am testing out incremental backups and when trying to restore via vtctldclient RestoreBackup, it only ever seems to find the full backup and then catch up via. MySQL replication. However, the documentation seems to support that only explicit PITR recovery will use incrementals.
I then tried explicitly using --restore-to-timestamp flag to restore a backup, but I am just getting Can't restore backup: no path found that leads to timestamp ....
To verify this, I created a full backup from a running tablet, then waited a few minutes and created another backup, this time using --incremental-from-pos="auto". Both backups were successfully created.
vtctldclient RestoreFromBackup --restore-to-timestamp 2025-12-12T14:11:25Z zone1-1980950187
tenant04/- (zone1-1980950187): time:{seconds:1765549206 nanoseconds:943911917} file:"restore.go" line:255 value:"Restore: original tablet type=REPLICA"
tenant04/- (zone1-1980950187): time:{seconds:1765549206 nanoseconds:953920689} file:"backup.go" line:376 value:"Restore: looking for a suitable backup to restore"
tenant04/- (zone1-1980950187): time:{seconds:1765549214 nanoseconds:97286302} file:"restore.go" line:283 value:"Restore: got a restore manifest: <nil>, err=Code: FAILED_PRECONDITION\nno path found that leads to timestamp 2025-12-12T14:11:25Z\n, waitForBackupInterval=0s"
E1212 14:20:14.517903 21606 main.go:56] rpc error: code = Unknown desc = TabletManager.RestoreFromBackup on zone1-1980950187: Can't restore backup: no path found that leads to timestamp 2025-12-12T14:11:25Z
I'm using the Vitess Operator and my backups are configured to use engine: xtrabckup to S3.
My VTTablet xtrabackup flags:
xbstream_restore_flags: --parallel=4 --decompress
xtrabackup_backup_flags: --parallel=4 --compress --compress-threads=2
Is the issue that I'm using xtrabackup for my fulls and it has to use builtin for the incrementals?
Reproduction Steps
-
Create a full backup using
xtrabackupenginevtctldclient BackupShard tenant04/- # wait a while until a new binlog is created vtctldclient BackupShard --incremental-from-pos="auto" tenant04/-
-
Try to restore the incremental
vtctldclient GetBackups tenant04/- # last 2 backups are 2025-12-12.141013.zone1-4261562132 # full 2025-12-12.141355.zone1-4261562132 # incremental vtctldclient RestoreBackup --restore-to-timestamp 2025-12-12T14:13:55Z <tablet-alias>
I have also tried earlier timestamps just to make sure, and I have also checked the backup MANIFEST to ensure the times are correct.
Binary Version
vttablet version Version: 21.0.3 (Git revision 94fdc736eae8928a8fdde44e9ec9c3bee1868d6f branch 'HEAD') built on Wed Feb 12 16:52:22 UTC 2025 by vitess@buildkitsandbox using go1.23.5 linux/amd64Operating System and Environment details
Linux