{"record":{"id":"5a4a3cbcaf85135f","repo":"apache/beam","slug":"textio-restriction-lies-outside-the-file-being-read","errorCode":null,"errorMessage":"TextIO restriction lies outside the file being read. Restriction begins at %v bytes, but file is only %v bytes.","messagePattern":"TextIO restriction lies outside the file being read\\. Restriction begins at (.+?) bytes, but file is only (.+?) bytes\\.","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"sdks/go/pkg/beam/io/textio/textio.go","lineNumber":247,"sourceCode":"\tif err != nil {\n\t\treturn err\n\t}\n\tdefer fd.Close()\n\n\trd := bufio.NewReader(fd)\n\n\ti := rt.GetRestriction().(offsetrange.Restriction).Start\n\tif i > 0 {\n\t\t// If restriction's starts after 0, we cannot assume a new line starts\n\t\t// at the beginning of the restriction, so we must search for the first\n\t\t// line beginning at or after restriction.Start. This is done by\n\t\t// scanning to the byte just before the restriction and then reading\n\t\t// until the next newline, leaving the reader at the start of a new\n\t\t// line past restriction.Start.\n\t\ti--\n\t\tn, err := rd.Discard(int(i)) // Scan to just before restriction.\n\t\tif err == io.EOF {\n\t\t\treturn errors.Errorf(\"TextIO restriction lies outside the file being read. \"+\n\t\t\t\t\"Restriction begins at %v bytes, but file is only %v bytes.\", i+1, n)\n\t\t}\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tline, err := rd.ReadString('\\n') // Read until the first line within the restriction.\n\t\tif err == io.EOF {\n\t\t\t// No lines start in the restriction but it's still valid, so\n\t\t\t// finish claiming before returning to avoid errors.\n\t\t\trt.TryClaim(rt.GetRestriction().(offsetrange.Restriction).End)\n\t\t\treturn nil\n\t\t}\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\ti += int64(len(line))\n\t}\n","sourceCodeStart":229,"sourceCodeEnd":265,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/io/textio/textio.go#L229-L265","documentation":"In textio's splitter (process on restriction-split readers), the reader seeks to the start of a restriction by discarding bytes up to restriction.Start and then advancing to the next newline. If Discard hits io.EOF before reaching that offset, the restriction points past the end of the file — the bundle/restriction is inconsistent with the file actually being read, so this error is returned.","triggerScenarios":"The file shrinks (or is replaced by a smaller file) after restrictions were computed, so restriction.Start exceeds the file size; or a restriction/offset pair from a different (stale or mismatched) file/metadata is applied to the current reader.","commonSituations":"A pipeline rereads a file that was truncated or overwritten with less content between bundle scheduling and execution; resuming/failover with cached restrictions after the input changed on GCS/S3; manually constructed restrictions from stale file sizes.","solutions":["Make input files immutable for the duration of the pipeline — never truncate/overwrite files being read","If files must change, write new versioned files (e.g. with timestamps) and point the pipeline at the new paths","Recompute restrictions/split the source again instead of reusing restrictions from a previous run","Verify the correct file version is being read (object generation/versioning on GCS, etags on S3)","Update the Beam SDK — newer versions have hardening around restriction/file-size mismatches"],"exampleFix":"// before (pipeline reuses stale restriction on a changed file)\nrd.Discard(int(i)) // io.EOF: restriction beyond EOF\n// after: guard by checking size before processing\ninfo, err := fs.Stat(path)\nif err != nil || info.Size() < restriction.End {\n    return fmt.Errorf(\"file %v changed (%v bytes); re-splitting\", path, info.Size())\n}","handlingStrategy":"validation","validationCode":"// before processing, confirm restriction fits the current file\ninfo, err := fs.Stat(path)\nif err != nil { return err }\nif restriction.End > info.Size() {\n    return fmt.Errorf(\"restriction [%v,%v) exceeds file size %v; re-split required\",\n        restriction.Start, restriction.End, info.Size())\n}","typeGuard":null,"tryCatchPattern":"// recover at the DoFn level and mark the bundle failed instead of crashing\nfunc (fn *splitterFn) ProcessElement(...) (emit func(string), err error) {\n    defer func() {\n        if r := recover(); r != nil { err = fmt.Errorf(\"textio process failed: %v\", r) }\n    }()\n    return fn.process(...)\n}","preventionTips":["Treat input files as immutable while the pipeline reads them","Use object versioning/generations so readers see a stable file","Re-split sources after any input mutation instead of reusing restrictions","Upgrade Beam; recent versions improved restriction/size consistency checks"],"tags":["go","beam","textio","splittable-dofn","file-size"],"backgroundTag":"value-out-of-range","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}