{"record":{"id":"b037cd9908866438","repo":"apache/hadoop","slug":"cannot-read-gzip-encoded-files-content-encoding","errorCode":null,"errorMessage":"Cannot read GZIP encoded files - content encoding support is disabled.","messagePattern":"Cannot read GZIP encoded files - content encoding support is disabled\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorageClientReadChannel.java","lineNumber":90,"sourceCode":"  GoogleCloudStorageClientReadChannel(\n      Storage storage,\n      GoogleCloudStorageItemInfo itemInfo,\n      GoogleHadoopFileSystemConfiguration config)\n      throws IOException {\n    validate(itemInfo);\n    this.storage = storage;\n    this.resourceId =\n        new StorageResourceId(\n            itemInfo.getBucketName(), itemInfo.getObjectName(), itemInfo.getContentGeneration());\n    this.contentReadChannel = new ContentReadChannel(config, resourceId);\n    initMetadata(itemInfo.getContentEncoding(), itemInfo.getSize());\n    this.config = config;\n  }\n\n  protected void initMetadata(@Nullable String encoding, long sizeFromMetadata) throws IOException {\n    gzipEncoded = nullToEmpty(encoding).contains(GZIP_ENCODING);\n    if (gzipEncoded && !config.isGzipEncodingSupportEnabled()) {\n      throw new IOException(\n          \"Cannot read GZIP encoded files - content encoding support is disabled.\");\n    }\n    objectSize = gzipEncoded ? Long.MAX_VALUE : sizeFromMetadata;\n  }\n\n  @Override\n  public int read(ByteBuffer dst) throws IOException {\n    throwIfNotOpen();\n\n    // Don't try to read if the buffer has no space.\n    if (dst.remaining() == 0) {\n      return 0;\n    }\n    LOG.trace(\n        \"Reading {} bytes at {} position from '{}'\", dst.remaining(), currentPosition, resourceId);\n    if (currentPosition == objectSize) {\n      return -1;\n    }","sourceCodeStart":72,"sourceCodeEnd":108,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorageClientReadChannel.java#L72-L108","documentation":"GoogleCloudStorageClientReadChannel.initMetadata throws when the object's metadata Content-Encoding contains \"gzip\" but the read configuration has gzip encoding support disabled. Support is opt-in because a gzip-encoded object's decompressed size is unknown: when enabled the channel treats objectSize as Long.MAX_VALUE and disables range reads; when disabled (the default) reads are refused outright.","triggerScenarios":"Opening a read channel on an object uploaded with Content-Encoding: gzip (e.g. `gcloud storage cp -j`, gsutil -j, or setContentEncoding(\"gzip\") in the storage client) while fs.gs.inputstream.support.gzip.encoding.enable is false (the default) or GoogleCloudStorageOptions was built without enableGzipEncodingSupport(true).","commonSituations":"Data pipelines that gzip objects transparently on upload; migrating from gsutil-based ingest to the Hadoop connector; older connector versions used a different key name (fs.gs.gzip.encoding.support.enabled) so configs silently lose the setting after upgrade.","solutions":["Set fs.gs.inputstream.support.gzip.encoding.enable=true in core-site.xml (or enableGzipEncodingSupport(true) on GoogleCloudStorageOptions) on every reader node.","Re-upload the data as plain bytes in a .gz file (no Content-Encoding header) and read it through Hadoop compression codecs, which also restores random access.","Remove the gzip Content-Encoding from the object metadata if the payload is not actually gzip."],"exampleFix":"# before (core-site.xml)\n<property><name>fs.gs.inputstream.support.gzip.encoding.enable</name><value>false</value></property>\n\n# after\n<property><name>fs.gs.inputstream.support.gzip.encoding.enable</name><value>true</value></property>","handlingStrategy":"validation","validationCode":"GoogleCloudStorageItemInfo info = gcs.getItemInfo(resourceId);\nboolean gzipEncoded = info.getContentEncoding() != null\n    && info.getContentEncoding().contains(\"gzip\");\nif (gzipEncoded && !options.isGzipEncodingSupportEnabled()) {\n  throw new IllegalStateException(\n      \"Enable fs.gs.inputstream.support.gzip.encoding.enable or re-upload without Content-Encoding: gzip\");\n}\ntry (SeekableByteChannel ch = gcs.open(info)) { /* ... */ }","typeGuard":null,"tryCatchPattern":"try (SeekableByteChannel ch = gcs.open(info)) { /* read */ } catch (IOException e) { if (e.getMessage().contains(\"GZIP\")) { /* enable config or switch to codec-based .gz read */ } else throw e; }","preventionTips":["Set fs.gs.inputstream.support.gzip.encoding.enable=true on reader nodes when ingesting gzip-encoded objects.","Prefer plain .gz objects + Hadoop compression codecs when random access is needed.","After connector upgrades, re-check config key names (fs.gs.inputstream.support.gzip.encoding.enable)."],"tags":["gcs","gzip","content-encoding","configuration","read-channel"],"backgroundTag":"gzip-content-encoding-unsupported","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}