apache/druid · error · SegmentLoadingException

Failed to create factory.json for tombstone in dir

Error message

Failed to create factory.json for tombstone in dir [%s]

What it means

TombstoneLoadSpec.loadSegment writes a factory.json marker file into the destination directory so the segment loader recognizes the tombstone; if any IOException occurs during that write it fails with SegmentLoadingException naming the directory. It indicates the tombstone could not be materialized locally.

Solutions

  1. Ensure destDir exists and the Druid process has write permission to it
  2. Free disk space if the volume is full
  3. Recreate the segment cache directory and reload
  4. Check the wrapped IOException/cause for the underlying filesystem error

Example fix

// before
// writing into a read-only dir
File destDir = new File("/var/druid/segment-cache/seg");
// after
if (!destDir.canWrite()) { logger.error("Dir not writable: %s", destDir); throw ... }
result = tombstoneLoadSpec.loadSegment(destDir);
Defensive patterns

Strategy: try-catch

Validate before calling

if (!destDir.exists() && !destDir.mkdirs()) throw new IOException("cannot create " + destDir); if (!destDir.canWrite()) throw new IOException("not writable: " + destDir);

Type guard

boolean writableDir(File d) { return d != null && d.isDirectory() && d.canWrite(); }

Try / catch

try { return loadSpec.loadSegment(destDir); } catch (SegmentLoadingException e) { log.error(e, "Tombstone load failed in %s", destDir); FileUtils.deleteQuietly(destDir); throw e; }

Prevention

When it happens

Trigger: loadSegment(destDir) called when the destination dir is not writable, does not exist, is full, or writeFactoryFile fails with an IOException.

Common situations: Segment cache directory owned by another user; read-only filesystem or full disk; race where the target directory was removed while loading tombstones after segment deletion.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors


AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/22fcbe4e4fdaed66. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/segment/loading/TombstoneLoadSpec.java:41

import com.google.common.annotations.VisibleForTesting;
import com.google.common.io.Files;
import org.apache.druid.initialization.TombstoneDataStorageModule;

import java.io.File;
import java.io.IOException;
import java.nio.charset.StandardCharsets;

@JsonTypeName(TombstoneDataStorageModule.SCHEME)
public class TombstoneLoadSpec implements LoadSpec
{
  @Override
  public LoadSpecResult loadSegment(File destDir) throws SegmentLoadingException
  {
    try {
      return new LoadSpecResult(writeFactoryFile(destDir));
    }
    catch (IOException e) {
      throw new SegmentLoadingException(
          "Failed to create factory.json for tombstone in dir [%s]",
          destDir.getAbsolutePath()
      );

    }
  }

  @VisibleForTesting
  public static int writeFactoryFile(File destDir) throws IOException
  {
    final String factoryJSONString = "{\"type\":\"tombstoneSegmentFactory\"}";
    final File factoryJson = new File(destDir, "factory.json");
    factoryJson.createNewFile();
    Files.write(factoryJSONString.getBytes(StandardCharsets.UTF_8), factoryJson);
    return factoryJSONString.length();
  }
}

View on GitHub (pinned to 9b90983fd2)