{"record":{"id":"b2ab09907781a107","repo":"apache/iceberg","slug":"provided-schema-problems-illegalargumentexcep","errorCode":null,"errorMessage":"Provided schema:...Problems: (IllegalArgumentException with schema and error list)","messagePattern":"Provided schema:\\.\\.\\.Problems: \\(IllegalArgumentException with schema and error list\\)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"api/src/main/java/org/apache/iceberg/types/TypeUtil.java","lineNumber":562,"sourceCode":"    } else {\n      errors = CheckCompatibility.typeCompatibilityErrors(schema, providedSchema, checkOrdering);\n    }\n\n    if (!errors.isEmpty()) {\n      StringBuilder sb = new StringBuilder();\n      sb.append(errMsg)\n          .append(\"\\n\")\n          .append(schema)\n          .append(\"\\n\")\n          .append(\"Provided schema:\")\n          .append(\"\\n\")\n          .append(providedSchema)\n          .append(\"\\n\")\n          .append(\"Problems:\");\n      for (String error : errors) {\n        sb.append(\"\\n* \").append(error);\n      }\n      throw new IllegalArgumentException(sb.toString());\n    }\n  }\n\n  /**\n   * Estimates the number of bytes a value for a given field may occupy in memory.\n   *\n   * <p>This method approximates the memory size based on heuristics and the internal Java\n   * representation defined by {@link Type.TypeID}. It is important to note that the actual size\n   * might differ from this estimation. The method is designed to handle a variety of data types,\n   * including primitive types, strings, and nested types such as structs, maps, and lists.\n   *\n   * @param field a field for which to estimate the size\n   * @return the estimated size in bytes of the field's value in memory\n   */\n  public static int estimateSize(Types.NestedField field) {\n    return estimateSize(field.type());\n  }\n","sourceCodeStart":544,"sourceCodeEnd":580,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/api/src/main/java/org/apache/iceberg/types/TypeUtil.java#L544-L580","documentation":"TypeUtil.checkSchemaCompatibility verifies a provided schema is compatible with an expected schema and collects all mismatches. If any errors are found it throws an IllegalArgumentException whose message embeds the full provided schema and a bulleted list of problems. It is the gate behind validateWriteSchema and validateSchema.","triggerScenarios":"Calling Schema.validateWriteSchema(expected, provided, checkNullability, checkCompatibility) or TypeUtil.validateSchema when the provided schema is missing fields, has incompatible types, or (with checkNullability) fields that are optional where required.","commonSituations":"Writing a DataFrame/dataset whose columns don't match the Iceberg table schema (missing column, wrong type like string vs long, nullability differences); evolving writer code after the table schema changed; case-sensitivity mismatches in field names.","solutions":["Read the 'Problems:' list in the message and fix each listed field/type/nullability mismatch in the provided schema.","Cast or convert the incoming data to match the expected schema before writing.","If extra columns are expected to be tolerated, project the provided schema to the expected one before validation.","Relax checks only deliberately: set checkNullability=false if optional-vs-required differences are acceptable."],"exampleFix":"// before\ntable.validateWriteSchema(expected, provided, true, true); // fails on int vs long\n// after\nTypes.StructType cast = TypeUtil.join(provided.asStruct(), expected.asStruct());\nDataFrame fixed = data.select(col(\"id\").cast(\"long\"), col(\"data\"));\ntable.validateWriteSchema(expected, fixed.schema(), true, true);","handlingStrategy":"validation","validationCode":"List<String> mismatches = new ArrayList<>();\nfor (Types.NestedField expectedField : expected.asStruct().fields()) {\n  Types.NestedField providedField = provided.asStruct().field(expectedField.name());\n  if (providedField == null) mismatches.add(\"missing field: \" + expectedField.name());\n  else if (!providedField.type().equals(expectedField.type())) mismatches.add(\"type mismatch: \" + expectedField.name());\n}\nif (!mismatches.isEmpty()) { /* reconcile schema before writing */ }","typeGuard":"boolean compatible = expected.asStruct().fields().stream()\n    .allMatch(f -> {\n      Types.NestedField pf = provided.asStruct().field(f.name());\n      return pf != null && pf.type().equals(f.type());\n    });","tryCatchPattern":"try {\n  Schema.validateWriteSchema(expected, provided, true, true);\n} catch (IllegalArgumentException e) {\n  // log e.getMessage(); reconcile/cast data schema before retry\n}","preventionTips":["Read the 'Problems:' list in the message — it enumerates every mismatch at once.","Cast or project incoming dataframes to the table schema before writes.","Track table schema evolution and regenerate writer schemas.","Watch nullability differences between required and optional fields."],"tags":["schema","validation","iceberg"],"backgroundTag":"schema-validation-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}