apache/hadoop · error · MetricsException

Error creating {}

Error message

Error creating {}

What it means

FileSink is the metrics2 file sink. On init() it opens an auto-flush UTF-8 PrintStream on the file named by the sink's filename property (falls back to System.out when no filename is configured). If Files.newOutputStream(Paths.get(filename)) throws — parent directory missing, permission denied, path is a directory, read-only filesystem — the sink wraps the cause in MetricsException("Error creating " + filename) and sink initialization fails.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/FileSink.java:53

/**
 * A metrics sink that writes to a file
 */
@InterfaceAudience.Public
@InterfaceStability.Evolving
public class FileSink implements MetricsSink, Closeable {
  private static final String FILENAME_KEY = "filename";
  private PrintStream writer;

  @Override
  public void init(SubsetConfiguration conf) {
    String filename = conf.getString(FILENAME_KEY);
    try {
      writer = filename == null ? System.out
          : new PrintStream(Files.newOutputStream(Paths.get(filename)),
                            true, "UTF-8");
    } catch (Exception e) {
      throw new MetricsException("Error creating "+ filename, e);
    }
  }

  @Override
  public void putMetrics(MetricsRecord record) {
    writer.print(record.timestamp());
    writer.print(" ");
    writer.print(record.context());
    writer.print(".");
    writer.print(record.name());
    String separator = ": ";
    for (MetricsTag tag : record.tags()) {
      writer.print(separator);
      separator = ", ";
      writer.print(tag.name());
      writer.print("=");
      writer.print(tag.value());
    }

View on GitHub (pinned to 2add963021)

Solutions

1. Create the parent directory and give the daemon user ownership: mkdir -p /var/log/app && chown <daemonuser> /var/log/app
2. Verify writability as the daemon user: sudo -u <daemonuser> touch <filename>
3. Point filename at a known-writable location such as the daemon's log directory
4. Omit the filename property entirely to send metrics to stdout

Example fix

# before (hadoop-metrics2.properties)
datanode.sink.file.class=org.apache.hadoop.metrics2.sink.FileSink
datanode.sink.file.filename=/var/log/hadoop/metrics.log  # /var/log/hadoop missing

# after
datanode.sink.file.class=org.apache.hadoop.metrics2.sink.FileSink
datanode.sink.file.filename=/var/log/hadoop-yarn/containers/metrics.log  # pre-created, owned by 'yarn'

Path p = Paths.get(filename);
Path parent = p.getParent() != null ? p.getParent() : Paths.get(".");
if (!Files.isDirectory(parent) || !Files.isWritable(parent)) {
  throw new IllegalStateException("FileSink target not writable: " + parent
      + " (run as user " + System.getProperty("user.name") + ")");
}

try {
  sink.init(subsetConf);
} catch (MetricsException e) {
  // e.getCause() holds the original IOException from Files.newOutputStream
  LOG.error("FileSink init failed for {}: {}", filename, e.getCause(), e);
}

When it happens

Trigger: hadoop-metrics2.properties sets <prefix>.sink.<instance>.class=org.apache.hadoop.metrics2.sink.FileSink and <prefix>.sink.<instance>.filename to a path whose parent directory does not exist or is not writable by the daemon user, or that names a directory, or lives on a read-only mount.

Common situations: Hadoop daemons (running as hdfs/yarn/nobody) pointed at /var/log/... directories owned by root; containers with read-only filesystems; SELinux denials; relative filenames resolving against an unexpected working directory.

Related errors

• Failed to create ${basePath}[source=${source}, allow-append=
• Illegal tag pattern: {}
• Error creating plugin: {}
• Error stringify config
• Error creating connection, {}:{}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/6ea211bda719c2ae. Report an issue: GitHub.