microsoft/garnet · error

Failed to schedule async IO: {}, handle {}

Error message

Failed to schedule async IO: {}, handle {}

What it means

Same failure class as [344] but in QueueFile::ScheduleOperation (QueueIoHandler / manual IOCP variant rather than the threadpool-I/O object). ReadFile/WriteFile fails with a Win32 code other than ERROR_IO_PENDING; the message appends the raw file_handle_ value for correlation. Returns Status::IOError. There is no CancelThreadpoolIo here (this path uses an IOCP + GetQueuedCompletionStatus loop, not a TP_IO), so the message omits that step.

Source

Thrown at libs/storage/Tsavorite/cc/src/device/file_windows.cc:385

  new(io_context.get()) QueueIoHandler::IoCallbackContext(offset, caller_context_copy,
      callback);

  bool success = FALSE;
  if(FileOperationType::Read == operationType) {
    success = ::ReadFile(file_handle_, buffer, length, nullptr, &io_context->parent_overlapped);
  } else {
    success = ::WriteFile(file_handle_, buffer, length, nullptr, &io_context->parent_overlapped);
  }
  if(!success) {
    DWORD win32_result = ::GetLastError();
    // Any error other than ERROR_IO_PENDING means the IO failed. Otherwise it will finish
    // asynchronously on the threadpool
    if(ERROR_IO_PENDING != win32_result) {
      std::stringstream ss;
      ss << "Failed to schedule async IO: " << FormatWin32AndHRESULT(win32_result) <<
         ", handle " << std::to_string((uint64_t)file_handle_);
      fprintf(stderr, "%s\n", ss.str().c_str());
      return Status::IOError;
    }
  }
  io_context.release();
  return Status::Ok;
}

#undef DCHECK_ALIGNMENT

}
} // namespace FASTER::environment

View on GitHub (pinned to 951b0fc683)

Solutions

1. Validate offset/length/buffer against device alignment (file.device_alignment()) before every Read/Write on unbuffered devices; use aligned allocators for buffers.
2. Match the handle value printed in the message to your open-file table to identify which segment/operation failed.
3. Block IO after Close()/Dispose with a disposed guard and a valid file_handle_ check.
4. For transient network/removable-media codes, retry with backoff; for ERROR_INVALID_PARAMETER (87) fix alignment rather than retrying.

Example fix

// before:
// file.Write(offset, length, src, ctx, cb);   // misaligned on QueueFile unbuffered device
// after:
size_t sector = file.device_alignment();
if (offset % sector != 0 || length % sector != 0 ||
    reinterpret_cast<uintptr_t>(src) % sector != 0) {
    return Status::IOError;   // fail fast with a clear cause instead of Win32(87)
}
file.Write(offset, length, src, ctx, cb);

// Same alignment/handle checks for the QueueIoHandler device variant.
size_t sector = file.device_alignment();
if (offset % sector != 0 || length % sector != 0 ||
    reinterpret_cast<uintptr_t>(buffer) % sector != 0)
    return Status::IOError;
if (file.handle() == INVALID_HANDLE_VALUE) return Status::IOError;
// Correlate failures via the handle value printed in the stderr message.

When it happens

Trigger: QueueIoHandler-based device Read/Write on an overlapped handle fails synchronously. Same root causes as [344] — unbuffered-I/O misalignment (DCHECK_ALIGNMENT only in debug), invalid/closed handle, device/network removal, disk full on write. The handle suffix in the message lets you map the failure back to the specific segment file.

Common situations: Using the QueueFile/QueueIoHandler device variant with non-sector-aligned buffers or sizes; IO attempted after Close(); removable/network storage dropping; concurrent reopen of the same segment file. Distinct from [344] only by which device implementation is in use — this variant is selected when you construct the queue-based handler.

Related errors

• Failed to schedule async IO: {}
• Failed to schedule work: {}
• Page read from storage failed, skipping page. Inner exceptio
• Slowdown: Unable to add trigger to epoch
• RetrieveCheckpointFile: unexpected state{retStateType}

AI-assisted analysis of microsoft/garnet@951b0fc683 (2026-08-13). Data as JSON: /api/errors/a265ccd06caf5fca. Report an issue: GitHub.