{"record":{"id":"e4ecfd43b28049a8","repo":"apache/iceberg","slug":"failed-to-remove-dropped-properties-from-replac","errorCode":null,"errorMessage":"Failed to remove dropped properties {} from replaced view {}","messagePattern":"Failed to remove dropped properties (.+?) from replaced view (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java","lineNumber":757,"sourceCode":"  private void removeDroppedViewProperties(\n      org.apache.iceberg.view.View view,\n      Set<String> previousProperties,\n      Set<String> replacementProperties) {\n    // Iceberg replacements merge properties, while Spark replacement views carry the complete\n    // property set. ViewBuilder does not support property removals, so this is a separate retrying\n    // metadata commit and can fail after the view version has already been replaced.\n    Set<String> removals = Sets.newHashSet(previousProperties);\n    removals.removeAll(replacementProperties);\n    removals.removeAll(ICEBERG_MANAGED_VIEW_PROPERTIES);\n    removals.removeAll(catalogManagedViewProperties);\n    if (!removals.isEmpty()) {\n      UpdateViewProperties update = view.updateProperties();\n      removals.forEach(update::remove);\n      try {\n        update.commit();\n      } catch (RuntimeException e) {\n        // TODO: Remove this best-effort follow-up when ViewBuilder supports atomic removals.\n        LOG.warn(\n            \"Failed to remove dropped properties {} from replaced view {}\",\n            removals,\n            view.name(),\n            e);\n      }\n    }\n  }\n\n  @Override\n  public boolean dropView(Identifier ident) {\n    if (null != asViewCatalog) {\n      return asViewCatalog.dropView(buildIdentifier(ident));\n    }\n\n    return false;\n  }\n\n  @Override","sourceCodeStart":739,"sourceCodeEnd":775,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java#L739-L775","documentation":"A logged warning emitted when removing legacy/stale properties from a replaced Iceberg view fails to commit. During replaceView, dropped properties are removed via UpdateViewProperties; if that commit throws a RuntimeException, it is logged and swallowed as a best-effort cleanup because view property updates are not atomic with the replace operation yet.","triggerScenarios":"Calling SparkCatalog.replaceView where the view carries properties that must be dropped (e.g., leftover Spark-specific keys) and the follow-up UpdateViewProperties.commit() fails — typically due to a concurrent modification, catalog commit conflict, or catalog backend error.","commonSituations":"Concurrent SQL statements modifying the same view; REST/Hive catalog rejecting the property update; legacy views carrying obsolete Spark catalog properties after connector upgrades; commit conflicts from another engine updating the view simultaneously.","solutions":["Re-run the REPLACE VIEW statement once the conflicting operation completes so property removal succeeds.","Check the chained exception for the catalog-side commit error (conflict, permission, rate limit).","Manually remove the stale properties with ALTER VIEW ... SET/UNSET TBLPROPERTIES if the follow-up keeps failing.","Retry with fewer concurrent writers to the same view; view property updates are not atomic with replace.","Upgrade to a version where ViewBuilder supports atomic property removal, eliminating this best-effort path."],"exampleFix":"// before\nALTER VIEW db.v UNSET TBLPROPERTIES ('spark.some.legacy.key');  // manual cleanup needed after warn\n// after\n-- ensure no concurrent writers, then re-run:\nREPLACE VIEW db.v AS SELECT ...;  // property removal commits atomically with the new builder API","handlingStrategy":"retry","validationCode":"// check for concurrent writers before replacing a view\nView view = catalog.loadView(TableIdentifier.of(\"db\", \"v\"));\n// ensure no other jobs alter the view concurrently","typeGuard":null,"tryCatchPattern":"try { update.commit(); } catch (RuntimeException e) { LOG.warn(\"retry or clean properties manually\", e); }","preventionTips":["Serialize view DDL across jobs","Remove legacy view properties before connector upgrades","Retry REPLACE VIEW on commit conflicts","Upgrade to versions with atomic view property removal"],"tags":["spark","view","properties","commit"],"backgroundTag":"http-error-response","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}