{"record":{"id":"86dd7bb61d294c66","repo":"apache/beam","slug":"content-length-for-object-s-was-nil","errorCode":null,"errorMessage":"content length for object %s was nil","messagePattern":"content length for object (.+?) was nil","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/go/pkg/beam/io/filesystem/s3/s3.go","lineNumber":168,"sourceCode":"\tbucket, key, err := parseURI(filename)\n\tif err != nil {\n\t\treturn -1, fmt.Errorf(\"error parsing S3 uri %s: %w\", filename, err)\n\t}\n\n\tparams := &s3.HeadObjectInput{\n\t\tBucket: aws.String(bucket),\n\t\tKey:    aws.String(key),\n\t}\n\toutput, err := f.client.HeadObject(ctx, params)\n\tif err != nil {\n\t\treturn -1, fmt.Errorf(\"error getting metadata for object %s: %w\", filename, err)\n\t}\n\n\tif output.ContentLength != nil {\n\t\treturn *output.ContentLength, nil\n\t}\n\n\treturn -1, fmt.Errorf(\"content length for object %s was nil\", filename)\n}\n\n// LastModified returns the time at which the file was last modified.\nfunc (f *fs) LastModified(ctx context.Context, filename string) (time.Time, error) {\n\tbucket, key, err := parseURI(filename)\n\tif err != nil {\n\t\treturn time.Time{}, fmt.Errorf(\"error parsing S3 uri %s: %v\", filename, err)\n\t}\n\n\tparams := &s3.HeadObjectInput{\n\t\tBucket: aws.String(bucket),\n\t\tKey:    aws.String(key),\n\t}\n\toutput, err := f.client.HeadObject(ctx, params)\n\tif err != nil {\n\t\treturn time.Time{}, fmt.Errorf(\"error getting metadata for object %s: %v\", filename, err)\n\t}\n","sourceCodeStart":150,"sourceCodeEnd":186,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/io/filesystem/s3/s3.go#L150-L186","documentation":"Size returns this error when HeadObject succeeds but output.ContentLength is unexpectedly nil, so no size can be reported. This is a defensive check against an anomalous S3 response and is rare with real S3. Size returns -1 in this case.","triggerScenarios":"HeadObject returns a 200 response whose ContentLength pointer is nil — essentially only when a stubbed/proxied S3-compatible endpoint (mocks, MinIO misbehavior, custom middleware) omits Content-Length.","commonSituations":"Testing against fake S3 implementations that don't populate all HeadObjectOutput fields; using third-party S3-compatible storage with non-conformant HEAD responses.","solutions":["Retry or query via GetObject and read the body length if HEAD metadata is missing.","Check the S3-compatible endpoint implementation (MinIO, mock) for correct Content-Length in HEAD responses.","Update the AWS SDK for Go v2 in case of a deserialization bug in older versions.","Treat -1/error as 'size unknown' and continue if size is only advisory."],"exampleFix":"// before\nn, err := fs.Size(ctx, uri)\n// after\nn, err := fs.Size(ctx, uri)\nif err != nil || n < 0 {\n  n = -1 // fall back to streaming length via io.Copy / GetObject\n}","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"n, err := fs.Size(ctx, uri)\nif err != nil {\n  log.Warnf(\"size unavailable for %s: %v; falling back to streaming count\", uri, err)\n  n = -1 // stream via OpenRead + io.Copy counter if exact size is required\n}","preventionTips":["Prefer the real AWS S3 endpoint; verify S3-compatible services return Content-Length on HEAD.","Keep aws-sdk-go-v2 updated to pick up deserialization fixes.","Treat size as advisory where possible and handle -1 gracefully.","Add unit tests with realistic HeadObjectOutput including ContentLength when mocking."],"tags":["aws","s3","metadata","unexpected-response"],"backgroundTag":"unexpected-response-shape","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}