{"record":{"id":"72f423adea3da1c6","repo":"juicedata/juicefs","slug":"read-s-fully-v-d-d-after-s","errorCode":null,"errorMessage":"read %s fully: %v (%d < %d) after %s","messagePattern":"read (.+?) fully: (.+?) \\((.+?) < (.+?)\\) after (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"pkg/chunk/cached_store.go","lineNumber":813,"sourceCode":"\tres := getResult{sc: object.DefaultStorageClass}\n\tif err == nil {\n\t\tres = tmp\n\t}\n\tlogRequest(\"GET\", key, \"\", res.reqID, err, used)\n\tif store.downLimit != nil && compressed {\n\t\tstore.downLimit.Wait(int64(res.n))\n\t}\n\tstore.objectDataBytes.WithLabelValues(\"GET\", res.sc).Add(float64(res.n))\n\tstore.objectReqsHistogram.WithLabelValues(\"GET\", res.sc).Observe(used.Seconds())\n\tif err != nil {\n\t\tstore.objectReqErrors.Add(1)\n\t\treturn fmt.Errorf(\"get %s: %s\", key, err)\n\t}\n\tif compressed {\n\t\tres.n, err = store.compressor.Decompress(page.Data, p.Data[:res.n])\n\t}\n\tif err != nil || res.n < len(page.Data) {\n\t\treturn fmt.Errorf(\"read %s fully: %v (%d < %d) after %s\", key, err, res.n, len(page.Data), used)\n\t}\n\tif cache {\n\t\tstore.bcache.cache(key, page, forceCache, !store.conf.OSCache)\n\t}\n\treturn nil\n}\n\n// NewCachedStore create a cached store.\nfunc NewCachedStore(storage object.ObjectStorage, config Config, reg prometheus.Registerer) ChunkStore {\n\tcompressor := compress.NewCompressor(config.Compress)\n\tif compressor == nil {\n\t\tlogger.Fatalf(\"unknown compress algorithm: %s\", config.Compress)\n\t}\n\tif config.MaxRetries == 0 {\n\t\tconfig.MaxRetries = 10\n\t}\n\tif config.GetTimeout == 0 {\n\t\tconfig.GetTimeout = time.Second * 60","sourceCodeStart":795,"sourceCodeEnd":831,"githubUrl":"https://github.com/juicedata/juicefs/blob/c9a67b23e8e08ec23ec331aa6f1675e2319e921c/pkg/chunk/cached_store.go#L795-L831","documentation":"After downloading a block, load checks that the (optionally decompressed) payload is at least as long as the requested page: if the compressor reports an error or res.n < len(page.Data), it returns \"read <key> fully: <err> (<got> < <want>) after <duration>\". This indicates the object storage returned truncated or corrupt data — the block could not be read in full, so the caller must not use the page.","triggerScenarios":"The stored object is shorter than expected (truncated upload, partial multipart, externally-managed bucket content changed), or Decompress fails on malformed compressed data — surfacing when load reads a page whose backing object yields too few bytes.","commonSituations":"Buckets written by other tools/humans with colliding keys; a previous upload interrupted leaving a truncated object; bit-rot or wrong compressor setting on the volume (data written with one compression read as another); flaky network causing short reads from an S3-compatible store.","solutions":["Delete the offending object (or remove local cache entries) and re-write/re-upload the data — a truncated stored object cannot be repaired in place.","Verify the volume's compression setting matches what was used at format time; mismatched compressor causes Decompress errors.","Check who else writes to the bucket — conflicting external writers or lifecycle rules corrupting keys must be removed.","Re-run the failing upload (e.g. fsck/gc or re-write the file) to repopulate the block from a good copy.","If the network is dropping data mid-read, stabilize connectivity or increase retry budget, then retry the read."],"exampleFix":"// before: object truncated by interrupted upload\n//   read 0/0/1_0 fully: <nil> (1048576 < 4194304) after 120ms\n// after: re-upload the block, then invalidate cache\nstore.bcache.remove(key)\n_ = store.client.Delete(key)\nerr := slice.upload(...) // rewrites the block from staging","handlingStrategy":"fallback","validationCode":"if got, want := len(page.Data), expectedSize; got < want {\n    return fmt.Errorf(\"object %s shorter than expected: %d < %d\", key, got, want)\n}","typeGuard":null,"tryCatchPattern":"if err != nil && strings.HasPrefix(err.Error(), \"read \") && strings.Contains(err.Error(), \"fully:\") {\n    // truncated/corrupt object: drop cache entry, delete object, re-read after re-upload\n    store.bcache.remove(key)\n    return readAfterInvalidation(ctx, key, page)\n}","preventionTips":["Never share the JuiceFS bucket with external writers or lifecycle policies that can mutate/truncate keys.","Keep the volume's compression setting unchanged after format; changing it corrupts decompression of existing blocks.","Verify uploads completed (fsck/gc) after crashes so truncated objects are detected and rewritten.","Enable integrity checking on the object store (checksums) and monitor for short reads in objectReqErrors."],"tags":["short-read","corruption","decompression","object-storage","data-integrity"],"backgroundTag":"checksum-mismatch","analyzedSha":"c9a67b23e8e08ec23ec331aa6f1675e2319e921c","analyzedAt":"2026-09-06T17:55:48.476Z","contentChangedAt":"2026-09-06T17:55:48.476Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}