puppetlabs/puppet · error · Puppet::FileBucket::BucketError

Existing backup and new file have different content but same

Error message

Existing backup and new file have different content but same checksum, %{value}. Verify existing backup and remove if incorrect.

What it means

During a filebucket save, if the destination contents file already exists and is not byte-identical to the incoming file, yet its bytes still hash to the requested checksum, Puppet concludes that two different contents share one digest: a suspected hash collision. It logs Puppet.err 'Unable to verify existing FileBucket backup at ...' and raises Puppet::FileBucket::BucketError instead of overwriting, because the stored copy may be the valid one and the path is deliberately omitted from the message to avoid disclosing server layout.

Source

Thrown at lib/puppet/indirector/file_bucket_file/file.rb:183

          if Puppet::FileSystem.exist?(contents_file)
            if verify_identical_file(contents_file, bucket_file)
              # TRANSLATORS "FileBucket" should not be translated
              Puppet.info _("FileBucket got a duplicate file %{file_checksum}") % { file_checksum: bucket_file.checksum }
              # Don't touch the contents file on Windows, since we can't update the
              # mtime of read-only files there.
              unless Puppet::Util::Platform.windows?
                Puppet::FileSystem.touch(contents_file)
              end
            elsif contents_file_matches_checksum?(contents_file, bucket_file.checksum_data, bucket_file.checksum_type)
              # If the contents or sizes don't match, but the checksum does,
              # then we've found a conflict (potential hash collision).
              # Unlikely, but quite bad. Don't remove the file in case it's
              # needed, but ask the user to validate.
              # Note: Don't print the full path to the bucket file in the
              # exception to avoid disclosing file system layout on server.
              # TRANSLATORS "FileBucket" should not be translated
              Puppet.err(_("Unable to verify existing FileBucket backup at '%{path}'.") % { path: contents_file.to_path })
              raise Puppet::FileBucket::BucketError, _("Existing backup and new file have different content but same checksum, %{value}. Verify existing backup and remove if incorrect.") %
                                                     { value: bucket_file.checksum }
            else
              # PUP-1334 If the contents_file exists but does not match its
              # checksum, our backup has been corrupted. Warn about overwriting
              # it, and proceed with new backup.
              Puppet.warning(_("Existing backup does not match its expected sum, %{sum}. Overwriting corrupted backup.") % { sum: bucket_file.checksum })
              copy_bucket_file_to_contents_file(contents_file, bucket_file)
            end
          else
            copy_bucket_file_to_contents_file(contents_file, bucket_file)
          end

          unless path_match(f, files_original_path)
            f.seek(0, IO::SEEK_END)
            f.puts(files_original_path)
          end
        end
      end

View on GitHub (pinned to e227c27540)

Solutions

Locate the stored backup under bucketdir using the checksum (bucketed path) and compare its bytes with the new file to decide which is correct.
If the stored copy is wrong, remove that checksum directory so the next backup recreates it.
Verify the digest you send is computed from exactly the bytes you send (Puppet::FileBucket::File computes checksum_data from content; never override it).
If this recurs with real data on one host, suspect faulty RAM/disk or a code path that rewrites content behind a cached digest.

Example fix

# before
fb = Puppet::FileBucket::File.new(File.binread('/etc/app.conf'))
fb.checksum_data = stale_sum # digest no longer matches these bytes -> collision error on save

# after: never override the digest; it is derived from the same content
fb = Puppet::FileBucket::File.new(File.binread('/etc/app.conf'))
# fb.checksum_data stays computed-from-content

Defensive patterns

Strategy: try-catch

Validate before calling

# verify the digest really belongs to the bytes you are backing up
digest = Puppet::Util::Checksums.send(Puppet[:digest_algorithm].to_sym, content)
raise ArgumentError, 'digest/content mismatch' unless digest == desired_sum

Try / catch

begin
  Puppet::FileBucket::File.indirection.save(fb_file, request_key)
rescue Puppet::FileBucket::BucketError => e
  Puppet.err(e.message) # quarantine and alert an operator; never overwrite silently
  raise
end

Prevention

Never override checksum_data on a FileBucket::File; let it hash the content.
Compute digests with the algorithm configured in digest_algorithm.
Alert on 'Unable to verify existing FileBucket backup' in server logs - it always precedes the raise.

When it happens

Trigger: A REST or local backup whose md5/sha256 digest matches different bytes already stored under the same bucket checksum directory. In practice almost always a caller that sets checksum_data to a digest that does not belong to the content it sends, memory corruption of the digest, or deliberate md5 collision test data.

Common situations: Test suites that hand-craft FileBucket::File objects without recomputing digests; code that reuses a file object after mutating its content; crafted md5 chosen-prefix collision experiments; rare disk corruption.

Related errors

AI-assisted analysis of puppetlabs/puppet@e227c27540 (2026-08-21). Data as JSON: /api/errors/befa4d995734aa69. Report an issue: GitHub.