microsoft/FASTER · warning

Hlog compaction was not successful :( -- retry!

Error message

Hlog compaction was not successful :( -- retry!

What it means

The background auto-compaction thread called CompactWithLookup on the hybrid log and it returned false, so the log was not successfully compacted in that cycle. This is informational-ish by design (the library logs it and schedules retries), but persistent occurrences mean the log keeps growing toward its size limit.

Solutions

1. Let the library retry (it re-schedules); if the error recurs every interval, increase hlog_compaction_config.num_threads.
2. Manually run CompactWithLookup during low-traffic windows to reclaim space.
3. Reduce the auto-compaction threshold (interval_size_mb) so each cycle has less work to do.
4. If it never succeeds, disable auto-compaction and drive compaction explicitly with error handling, and investigate with a debug build.

Example fix

// before
auto cfg = CompactionConfig{}; // defaults, 1 thread
kv.StartAutoCompaction(cfg); // repeated failures
// after
CompactionConfig cfg;
cfg.num_threads = std::thread::hardware_concurrency();
cfg.interval_size_mb = 512;
kv.StartAutoCompaction(cfg);

// tune before enabling auto-compaction
CompactionConfig cfg;
cfg.num_threads = std::max(2u, std::thread::hardware_concurrency());
cfg.interval_size_mb = /* smaller bucket size, e.g. 256 */;

When it happens

Trigger: The store exceeds hlog_compaction_config_.interval_size_mb / auto-compaction thresholds, the auto-compaction task runs StartSession + CompactWithLookup(until_address, true, num_threads), and the compaction fails — usually due to concurrent mutation of records being folded, too few threads, or transient internal scan errors.

Common situations: High write throughput keeping compaction from converging; hlog_compaction_config.num_threads too small for log size; auto-compaction racing with user-triggered checkpoint/compaction; long-running sessions delaying safe truncation addresses.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

• Compact HOT log failed! :(
• Compact COLD log failed! :(
• Invalid compaction type
• Can compact only until Log.SafeReadOnlyAddress
• Unable to find valid HybridLog token

AI-assisted analysis of microsoft/FASTER@321d872eab (2026-09-15). Data as JSON: /api/errors/7b9d739c3a2a30dc. Report an issue: GitHub.

Appendix: source

Thrown at cc/src/core/faster.h:4497

    until_address = std::min(until_address, begin_address + hlog_compaction_config_.max_compacted_size);
    // do not compact in-memory region
    until_address = std::min(until_address, hlog.safe_head_address.control());
    // round down address to page bounds
    until_address = until_address - Address(until_address).offset() + Address::kMaxOffset + 1;
    assert(until_address <= hlog.safe_read_only_address.control());
    assert(until_address % hlog.kPageSize == 0);

    // perform log compaction
    log_info("Auto-compaction: [%lu %lu] -> [%lu %lu] {%lu}",
             begin_address, hlog.GetTailAddress(),
             until_address, hlog.GetTailAddress(), Size());
    StartSession();
    bool success = CompactWithLookup(until_address, true, hlog_compaction_config_.num_threads);
    StopSession();

    log_info("Auto-compaction: Size: %.3f GB", static_cast<double>(Size()) / (1 << 30));
    if (!success) {
      log_error("Hlog compaction was not successful :( -- retry!");
    }
  }
  // no more auto-compactions
  auto_compaction_scheduled_.store(false);
}

#ifdef TOML_CONFIG
template <class K, class V, class D, class H, class OH>
inline FasterKv<K, V, D, H, OH> FasterKv<K, V, D, H, OH>::FromConfigString(const std::string& config) {
  return Config::FromConfigString(config, "faster");
}

template <class K, class V, class D, class H, class OH>
inline FasterKv<K, V, D, H, OH> FasterKv<K, V, D, H, OH>::FromConfigFile(const std::string& filepath) {
  std::ifstream t(filepath);
  std::string config(
    (std::istreambuf_iterator<char>(t)), std::istreambuf_iterator<char>());
  return FasterKv<K, V, D, H, OH>::FromConfigString(config);

View on GitHub (pinned to 321d872eab)