Skip to content

Bug Report: PITR recovery not detecting full backup created with xtrabackup #19019

@james-at-upmind

Description

@james-at-upmind

Overview of the Issue

I am testing out incremental backups and when trying to restore via vtctldclient RestoreBackup, it only ever seems to find the full backup and then catch up via. MySQL replication. However, the documentation seems to support that only explicit PITR recovery will use incrementals.

I then tried explicitly using --restore-to-timestamp flag to restore a backup, but I am just getting Can't restore backup: no path found that leads to timestamp ....

To verify this, I created a full backup from a running tablet, then waited a few minutes and created another backup, this time using --incremental-from-pos="auto". Both backups were successfully created.

vtctldclient RestoreFromBackup --restore-to-timestamp 2025-12-12T14:11:25Z zone1-1980950187
tenant04/- (zone1-1980950187): time:{seconds:1765549206 nanoseconds:943911917} file:"restore.go" line:255 value:"Restore: original tablet type=REPLICA"
tenant04/- (zone1-1980950187): time:{seconds:1765549206 nanoseconds:953920689} file:"backup.go" line:376 value:"Restore: looking for a suitable backup to restore"
tenant04/- (zone1-1980950187): time:{seconds:1765549214 nanoseconds:97286302} file:"restore.go" line:283 value:"Restore: got a restore manifest: <nil>, err=Code: FAILED_PRECONDITION\nno path found that leads to timestamp 2025-12-12T14:11:25Z\n, waitForBackupInterval=0s"
E1212 14:20:14.517903   21606 main.go:56] rpc error: code = Unknown desc = TabletManager.RestoreFromBackup on zone1-1980950187: Can't restore backup: no path found that leads to timestamp 2025-12-12T14:11:25Z

I'm using the Vitess Operator and my backups are configured to use engine: xtrabckup to S3.
My VTTablet xtrabackup flags:

                xbstream_restore_flags: --parallel=4 --decompress
                xtrabackup_backup_flags: --parallel=4 --compress --compress-threads=2

Is the issue that I'm using xtrabackup for my fulls and it has to use builtin for the incrementals?

Reproduction Steps

  • Create a full backup using xtrabackup engine

    vtctldclient BackupShard tenant04/-
    # wait a while until a new binlog is created
    vtctldclient BackupShard --incremental-from-pos="auto" tenant04/- 
  • Try to restore the incremental

    vtctldclient GetBackups tenant04/-
    
    # last 2 backups are
    2025-12-12.141013.zone1-4261562132 # full
    2025-12-12.141355.zone1-4261562132 # incremental
    
    vtctldclient RestoreBackup --restore-to-timestamp 2025-12-12T14:13:55Z <tablet-alias>

I have also tried earlier timestamps just to make sure, and I have also checked the backup MANIFEST to ensure the times are correct.

Binary Version

vttablet version Version: 21.0.3 (Git revision 94fdc736eae8928a8fdde44e9ec9c3bee1868d6f branch 'HEAD') built on Wed Feb 12 16:52:22 UTC 2025 by vitess@buildkitsandbox using go1.23.5 linux/amd64

Operating System and Environment details

Linux

Log Fragments

Metadata

Metadata

Assignees

No one assigned

    Labels

    Needs TriageThis issue needs to be correctly labelled and triagedType: Bug

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions