dgraph-io/dgraph · critical

complete backup failed

Error message

complete backup failed

What it means

CompleteBackup is the final stage of a backup: after all data is written, it appends the final manifest entry and calls CreateManifest to persist the manifest that marks the backup complete. If writing that manifest fails (storage error, permissions, network), the backup cannot be considered complete and the error is wrapped with this message.

Source

Thrown at worker/backup.go:729

		return err
	}
	uri, err := url.Parse(pr.Request.Destination)
	if err != nil {
		return err
	}
	handler, err := NewUriHandler(uri, GetCredentialsFromRequest(pr.Request))
	if err != nil {
		return err
	}

	manifest, err := GetManifestNoUpgrade(handler, uri)
	if err != nil {
		return err
	}
	manifest.Manifests = append(manifest.Manifests, m)

	if err := CreateManifest(handler, uri, manifest); err != nil {
		return errors.Wrap(err, "complete backup failed")
	}
	// Best-effort: write summary manifest. Failure does not abort the backup.
	if err := CreateManifestSummary(handler, manifest); err != nil {
		glog.Warningf("Failed to write backup summary manifest (non-fatal): %v", err)
	}
	glog.Infof("Backup completed OK.")
	return nil
}

// GoString implements the GoStringer interface for Manifest.
func (m *Manifest) GoString() string {
	return fmt.Sprintf(`Manifest{Since: %d, ReadTs: %d, Groups: %v, Encrypted: %v}`,
		m.SinceTsDeprecated, m.ReadTs, m.Groups, m.Encrypted)
}

func (tl *threadLocal) toBackupList(key []byte, itr *badger.Iterator) (
	*bpb.KVList, *pb.DropOperation, error) {
	list := &bpb.KVList{}

View on GitHub (pinned to 759e242be6)

Solutions

1. Check the wrapped inner error and re-run the backup (or resume with forceFull if state is inconsistent)
2. Verify credentials won't expire mid-backup (use long-lived creds or refresh tokens)
3. Ensure the destination has sufficient space and is reachable for the whole backup duration
4. Retry the backup after confirming object storage health

Example fix

// before
if err := CreateManifest(handler, uri, manifest); err != nil {
    return errors.Wrap(err, "complete backup failed")
}
// after
if err := CreateManifest(handler, uri, manifest); err != nil {
    glog.Errorf("manifest upload failed for %s: %v", uri, err) // log root cause
    return errors.Wrap(err, "complete backup failed")
}

// ensure the destination remains writable for the full backup duration
if err := handler.WriteProbe("./"); err != nil {
    return fmt.Errorf("destination not writable, aborting backup: %w", err)
}

err := CompleteBackup(ctx, req)
if err != nil && strings.Contains(err.Error(), "complete backup failed") {
    // check storage availability/credentials, then re-run the backup
    time.Sleep(backoff)
    err = CompleteBackup(ctx, req)
}

When it happens

Trigger: Running a full backup where CreateManifest(handler, uri, manifest) fails after the data upload — storage backend became unreachable, credentials expired mid-backup, bucket write failed, or network blip at the final manifest upload.

Common situations: Long backups whose session token expires before completion; S3/MinIO outage at the end of the backup; disk full at destination; flaky network to object storage.

Related errors

• cannot read manifests at location %s
• failed to read master manifest:
• while Getting latest manifest:
• readManifest failed to read the file:
• while creating backup directory

AI-assisted analysis of dgraph-io/dgraph@759e242be6 (2026-09-01). Data as JSON: /api/errors/8977e1554038a7f7. Report an issue: GitHub.