{"record":{"id":"5bf0363c7c492736","repo":"apache/beam","slug":"compression-must-be-resolved-into-a-concrete-type-before","errorCode":null,"errorMessage":"compression must be resolved into a concrete type before obtaining a reader","messagePattern":"compression must be resolved into a concrete type before obtaining a reader","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"sdks/go/pkg/beam/io/fileio/file.go","lineNumber":106,"sourceCode":"func compressionFromExt(path string) compressionType {\n\tswitch filepath.Ext(path) {\n\tcase \".gz\":\n\t\treturn compressionGzip\n\tdefault:\n\t\treturn compressionUncompressed\n\t}\n}\n\n// newDecompressionReader returns an io.ReadCloser that can be used to read uncompressed data from\n// reader, based on the specified compression. If the compression is compressionAuto, a non-nil\n// error is returned. It is the caller's responsibility to close the returned reader.\nfunc newDecompressionReader(\n\treader io.ReadCloser,\n\tcompression compressionType,\n) (io.ReadCloser, error) {\n\tswitch compression {\n\tcase compressionAuto:\n\t\treturn nil, errors.New(\n\t\t\t\"compression must be resolved into a concrete type before obtaining a reader\",\n\t\t)\n\tcase compressionGzip:\n\t\treturn newGzipReader(reader)\n\tdefault:\n\t\treturn reader, nil\n\t}\n}\n\n// Read reads the entire file into memory and returns the contents.\nfunc (f ReadableFile) Read(ctx context.Context) (data []byte, err error) {\n\trc, err := f.Open(ctx)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tdefer func() {\n\t\tcloseErr := rc.Close()","sourceCodeStart":88,"sourceCodeEnd":124,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/go/pkg/beam/io/fileio/file.go#L88-L124","documentation":"fileio's newDecompressionReader converts a raw file reader into a decompressing one based on a compressionType. compressionAuto is only a hint to be resolved during match/manifest processing; if it is still unresolved when a reader is requested, the library refuses with this error because it cannot know which decompressor to apply.","triggerScenarios":"Using fileio.Match* with a fileio.MatchConfiguration{Compression: compression.Auto} but obtaining readers directly from the match results (or calling newDecompressionReader/Open paths) without first resolving the auto-detected compression per file, e.g. reading results.Metadata[i].Compression instead of letting the match result populate it.","commonSituations":"Manually constructing fileio.Read(s, col, compression.Auto) / building readers from fileio.MatchAllAll output while bypassing the compression resolution step; copying match configuration code that sets Auto but never calls the resolution that replaces it with Gzip or None.","solutions":["Use fileio.MatchAll (or MatchAllFilepattern) with compression.Auto and then read via the returned match results so the library resolves Auto to a concrete type (Gzip/Uncompressed) per file.","If specifying compression manually, use a concrete type: compression.Gzip or compression.Uncompressed instead of compression.Auto.","If you manage readers yourself, resolve the compression from the file metadata (e.g. .gz extension / magic bytes) before requesting the reader."],"exampleFix":"// before\nfiles, _ := fileio.MatchAll(s, \"gs://bucket/*.gz\", fileio.MatchConfiguration{Compression: compression.Auto})\n// reader built directly with compression still Auto -> error\n\n// after\nfiles := fileio.MatchAll(s, \"gs://bucket/*.gz\", fileio.MatchConfiguration{Compression: compression.Auto})\nfileio.Read(s, files, func(ctx context.Context, src *fileio.ReadableFile) error {\n\t// src.Compression is resolved (Gzip/None) before the reader is opened\n})","handlingStrategy":"validation","validationCode":"if compression == compression.Auto {\n\t// resolve first: use fileio.MatchAll results, whose metadata carries the\n\t// concrete per-file compression, or specify Gzip/Uncompressed explicitly\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never request readers while compression is still compression.Auto — let fileio.MatchAll resolve it.","Pass a concrete compression type (Gzip, Uncompressed) when you already know the file format.","Use the fileio.Read transform rather than hand-building decompression readers."],"tags":["go","apache-beam","fileio","compression","invalid-state"],"backgroundTag":"invalid-state-transition","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}