{"record":{"id":"a814de9cb4448427","repo":"juicedata/juicefs","slug":"read-wrong-data-expected-x-got-x","errorCode":null,"errorMessage":"read wrong data: expected %x, got %x","messagePattern":"read wrong data: expected %x, got %x","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"cmd/format.go","lineNumber":366,"sourceCode":"\t\tif err := store.Put(ctx, key, bytes.NewReader(data)); err != nil {\n\t\t\treturn fmt.Errorf(\"Failed to put: %s\", err)\n\t\t}\n\t}\n\t// GLACIER storage class doesn't allow read after write\n\tif _, ok := ctx.Value(object.TierKey{}).(uint8); ok {\n\t\treturn nil\n\t}\n\tp, err := store.Get(ctx, key, 0, -1)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"Failed to get: %s\", err)\n\t}\n\tdata2, err := io.ReadAll(p)\n\t_ = p.Close()\n\tif err != nil {\n\t\treturn err\n\t}\n\tif !bytes.Equal(data, data2) {\n\t\treturn fmt.Errorf(\"read wrong data: expected %x, got %x\", data, data2)\n\t}\n\terr = store.Delete(ctx, key)\n\tif err != nil {\n\t\t// it's OK to don't have delete permission, but we should warn user explicitly\n\t\tlogger.Warnf(\"Failed to delete, err: %s\", err)\n\t}\n\treturn nil\n}\n\nfunc test(ctx context.Context, store object.ObjectStorage) error {\n\tkey := \"testing/\" + randSeq(10)\n\tdata := make([]byte, 100)\n\tutils.RandRead(data)\n\tnRetry := 3\n\tvar err error\n\tfor i := 0; i < nRetry; i++ {\n\t\terr = doTesting(ctx, store, key, data)\n\t\tif err == nil {","sourceCodeStart":348,"sourceCodeEnd":384,"githubUrl":"https://github.com/juicedata/juicefs/blob/c9a67b23e8e08ec23ec331aa6f1675e2319e921c/cmd/format.go#L348-L384","documentation":"The data read back from the object store does not match the data written. JuiceFS compares the test object byte-for-byte; a mismatch means the backend is corrupting, truncating, or returning different content — possibly a broken proxy or an incompatible S3 implementation.","triggerScenarios":"`juicefs format` connectivity test where Get succeeds but the returned bytes differ from the written pattern — middleboxes/caches rewriting content, a misbehaving S3-compatible server returning another object, or key collisions from a shared/prefixed bucket used concurrently.","commonSituations":"CDN or caching proxy in front of the store serving stale content; non-conformant S3 clones (some NAS gateways) mangling binary data; two concurrent formats to the same test key racing.","solutions":["Bypass any proxy/CDN and format directly against the storage endpoint","Verify backend integrity by round-tripping a file manually with the provider CLI (put then get, then checksum)","Upgrade or check the S3-compatible server for known data-corruption issues","Re-run format — a concurrent collision on the test key resolves itself on retry"],"exampleFix":"// before\nendpoint via caching proxy: https://cdn.example.com\n// after\nendpoint direct: https://s3.us-east-1.amazonaws.com (or disable cache for PUT/GET)","handlingStrategy":"validation","validationCode":"// manual round-trip check\n// aws s3 cp testfile s3://bucket/t && aws s3 cp s3://bucket/t back && sha256sum testfile back","typeGuard":null,"tryCatchPattern":"err := doTesting(ctx, store, key, data)\nif err != nil && strings.Contains(err.Error(), \"read wrong data\") {\n    // backend corruption or proxy interference; compare checksums manually\n}","preventionTips":["Use the storage endpoint directly, not through caching proxies/CDNs","Round-trip a test file with checksums before formatting","Keep S3-compatible servers up to date; avoid known-buggy gateways","Avoid concurrent formats sharing the same test key path"],"tags":["object-storage","data-corruption","s3"],"backgroundTag":"checksum-mismatch","analyzedSha":"c9a67b23e8e08ec23ec331aa6f1675e2319e921c","analyzedAt":"2026-09-06T17:55:48.476Z","contentChangedAt":"2026-09-06T17:55:48.476Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}