{"record":{"id":"ce9808a29ab453c9","repo":"apache/iceberg","slug":"failed-to-create-shredded-variant-writer","errorCode":null,"errorMessage":"Failed to create shredded variant writer","messagePattern":"Failed to create shredded variant writer","errorType":"exception","errorClass":"UncheckedIOException","httpStatus":null,"severity":"error","filePath":"parquet/src/main/java/org/apache/iceberg/parquet/ParquetFormatModel.java","lineNumber":318,"sourceCode":"\n      return new BufferedFileAppender<>(\n          bufferSize,\n          bufferedRows -> {\n            Map<Integer, Type> shreddedTypes =\n                variantAnalyzer.analyzeVariantColumns(bufferedRows, schema, engineSchema);\n            LOG.debug(\n                \"Variant inference: rows={}, shredded fields={}\",\n                bufferedRows.size(),\n                shreddedTypes.size());\n\n            if (!shreddedTypes.isEmpty()) {\n              internal.variantShreddingFunc((fieldId, name) -> shreddedTypes.get(fieldId));\n            }\n\n            try {\n              return internal.build();\n            } catch (IOException e) {\n              throw new UncheckedIOException(\"Failed to create shredded variant writer\", e);\n            }\n          },\n          copyFunc);\n    }\n\n    private static boolean hasVariantColumns(Schema schema) {\n      return schema != null\n          && schema.columns().stream().anyMatch(field -> field.type().isVariantType());\n    }\n  }\n\n  private static class ReadBuilderWrapper<R, D, S> implements ReadBuilder<D, S> {\n    private final Parquet.ReadBuilder internal;\n    private final ReaderFunction<R, S, MessageType> readerFunction;\n    private final boolean isBatchReader;\n    private S engineSchema;\n    private Map<Integer, ?> idToConstant = ImmutableMap.of();\n","sourceCodeStart":300,"sourceCodeEnd":336,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/parquet/src/main/java/org/apache/iceberg/parquet/ParquetFormatModel.java#L300-L336","documentation":"When the schema contains variant columns and shredding is enabled, buildShreddedAppender() constructs a writer whose build() may throw IOException. That exception is wrapped in an UncheckedIOException with this message, indicating the shredded-variant Parquet writer could not be created (e.g. failure opening/initializing the output stream).","triggerScenarios":"Writing a Parquet file with variant columns and shredVariants enabled when internal.build() throws IOException — typically due to the underlying output file/stream failing to open or initialize.","commonSituations":"Unwritable destination path, full disk, closed or failed output stream, Hadoop FileSystem errors during file creation while writing shredded variant data.","solutions":["Check the wrapped IOException cause for the underlying file/IO error","Verify the output file location is writable and the filesystem is healthy","Ensure the output stream/FileIO is properly opened and not already closed","Retry the write after fixing storage issues"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// pre-check output target writability before building the writer\nVerify that the output file location exists and is writable before calling build()","typeGuard":null,"tryCatchPattern":"try {\n  return buildShreddedAppender();\n} catch (UncheckedIOException e) {\n  logger.error(\"Shredded variant writer creation failed\", e.getCause());\n  throw e;\n}","preventionTips":["Ensure output paths and directories are writable","Watch disk space on write nodes","Keep the output stream/FileIO open until after build()"],"tags":["parquet","io","writer","unchecked-io"],"backgroundTag":"file-write-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}