apache/hadoop · critical · PathIOException

Attempted to write to file without lease

Error message

Attempted to write to file without lease

What it means

AbfsOutputStream.write throws PathIOException with ERR_WRITE_WITHOUT_LEASE ("Attempted to write to file without lease...") when the stream holds a lease (hasLease()) but that lease has already been freed (isLeaseFreed()). The lease enforces single-writer semantics; once freed — typically because the lease was lost, expired, or released after a failure — further writes are rejected rather than silently writing unguarded.

Source

Thrown at hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java:461

   * @throws IOException if an I/O error occurs. In particular, an IOException may be
   *                     thrown if the output stream has been closed.
   */
  @Override
  public synchronized void write(final byte[] data, final int off, final int length)
      throws IOException {
    if (closed) {
      throw new IOException(FSExceptionMessages.STREAM_IS_CLOSED);
    }
    // validate if data is not null and index out of bounds.
    DataBlocks.validateWriteArgs(data, off, length);
    maybeThrowLastError();

    if (off < 0 || length < 0 || length > data.length - off) {
      throw new IndexOutOfBoundsException();
    }

    if (hasLease() && isLeaseFreed()) {
      throw new PathIOException(path, ERR_WRITE_WITHOUT_LEASE);
    }
    if (length == 0) {
      LOG.debug("No data to write, length is 0 for path: {}", path);
      return;
    }

    AbfsBlock block = createBlockIfNeeded(position);
    int written = bufferData(block, data, off, length);
    // Update the incremental MD5 hash with the written data.
    if (isChecksumValidationEnabled()) {
      getMessageDigest().update(data, off, written);
    }
    // Update the full blob MD5 hash with the written data.
    if (isFullBlobChecksumValidationEnabled()) {
      getFullBlobContentMd5().update(data, off, written);
    }
    int remainingCapacity = block.remainingCapacity();

View on GitHub (pinned to 2add963021)

Solutions

1. Fail the write task and restart it on a fresh stream — writing without the lease would risk lost updates
2. If you must continue, acquire a new lease (free/reacquire or reopen the output stream)
3. Review lease settings (duration/renewal) and ensure the lease thread pool is healthy (fs.azure.lease.threads)
4. Eliminate competing writers/break-lease operations on the same path

Example fix

// before
out.write(buf, 0, len);   // PathIOException: write without lease

// after
try {
  out.write(buf, 0, len);
} catch (PathIOException e) {
  if (e.getMessage().contains("without lease")) {
    // lease lost: stop, then restart the writer on a fresh stream/lease
    throw new IOException("Lease lost for " + path
        + "; restart this write task", e);
  }
  throw e;
}

// Nothing to validate locally — lease loss is server-side.
// Mitigate: flush frequently so a lease loss loses little data
out.hflush();   // keep unflushed window small while leasing

try {
  out.write(buf, 0, len);
} catch (PathIOException e) {
  if (e.getMessage() != null && e.getMessage().contains("without lease")) {
    // single-writer guarantee broken: fail the task / reopen with a new lease
    throw new IOException("ABFS lease lost for " + path + "; restart writer", e);
  }
  throw e;
}

When it happens

Trigger: The lease expired because renewals (LeaseTimerTask) fell behind — fixed-duration lease too short, GC pauses, thread starvation, or network problems; the lease was broken/acquired by another client (e.g., someone ran a break-lease tool or a competing writer); a prior REST failure freed the lease and the application kept writing.

Common situations: Long-running output streams with fs.azure.lease features enabled; operators breaking 'stuck' leases while a job is still writing; competing jobs/tools writing the same path; async lease tasks failing due to the lease thread pool being saturated.

Related errors

• Lease desired but no lease threads configured, set fs.azure.
• There is already an existing lease operation
• Failed to {} {} for {} on {} because {} is already the curre
• Failed to {} {} for {} on {} because this file lease is curr
• Client (={}) is not the lease owner (={}: {} (inode {}) {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b618518d361db531. Report an issue: GitHub.