{"record":{"id":"5ce1842b2d4ebc80","repo":"fluent/fluentd","slug":"can-t-acquire-lock-because-fluentd-lock-dir-isn-t","errorCode":null,"errorMessage":"can't acquire lock because FLUENTD_LOCK_DIR isn't set","messagePattern":"can't acquire lock because FLUENTD_LOCK_DIR isn't set","errorType":"exception","errorClass":"Fluent::InvalidLockDirectory","httpStatus":null,"severity":"error","filePath":"lib/fluent/plugin/base.rb","lineNumber":84,"sourceCode":"        self\n      end\n\n      def multi_workers_ready?\n        true\n      end\n\n      LOCK_FILE_BUCKETS = 65536\n\n      def get_lock_path(name)\n        # The mapping from a name to a bucket MUST be identical across worker processes.\n        # Ruby's String#hash is randomly seeded per process and MUST NOT be used here.\n        bucket = Zlib.crc32(name.to_s) % LOCK_FILE_BUCKETS\n        File.join(@fluentd_lock_dir, \"fluentd-bucket-#{bucket}.lock\")\n      end\n\n      def acquire_worker_lock(name)\n        if @fluentd_lock_dir.nil?\n          raise InvalidLockDirectory, \"can't acquire lock because FLUENTD_LOCK_DIR isn't set\"\n        end\n        lock_path = get_lock_path(name)\n        File.open(lock_path, \"w\") do |f|\n          f.flock(File::LOCK_EX)\n          yield\n        end\n        # Update access time to prevent tmpwatch from deleting a lock file.\n        FileUtils.touch(lock_path)\n      end\n\n      def string_safe_encoding(str)\n        unless str.valid_encoding?\n          str = str.scrub('?')\n          log.info \"invalid byte sequence is replaced in `#{str}`\" if self.respond_to?(:log)\n        end\n        yield str\n      end\n","sourceCodeStart":66,"sourceCodeEnd":102,"githubUrl":"https://github.com/fluent/fluentd/blob/dd45c6e18dc7be33b5e5a0f0767bf46307ff5626/lib/fluent/plugin/base.rb#L66-L102","documentation":"acquire_worker_lock serializes plugin work across worker processes using flock files under a lock directory taken from ENV['FLUENTD_LOCK_DIR'] (read in Fluent::Plugin::Base#initialize). The supervisor exports that env var when it boots workers (supervisor.rb); core users include out_file append mode and output flush locking (output.rb, out_file.rb). When a plugin calls the lock outside a supervised fluentd process — unit tests, standalone embedding, or an environment where the variable was scrubbed — the nil lock dir raises Fluent::InvalidLockDirectory.","triggerScenarios":"Running plugin unit tests (e.g. out_file with append) without setting FLUENTD_LOCK_DIR; embedding Fluent::Engine in a custom process instead of the fluentd supervisor; systemd/docker hardening (EnvironmentFile omission, env clear) dropping the variable inherited from the supervisor.","commonSituations":"RSpec failures appearing only when a locking code path is exercised; CI running plugins in isolation; custom launchers that exec worker code directly rather than through fluentd's supervisor.","solutions":["Set FLUENTD_LOCK_DIR to a writable directory before the code path runs: export FLUENTD_LOCK_DIR=$(mktemp -d)","In tests, set ENV['FLUENTD_LOCK_DIR'] = Dir.mktmpdir in setup (and clean up in teardown)","Run the plugin under the normal fluentd supervisor, which exports the variable itself","For containerized deployments, ensure the env var survives into the worker process (do not scrub the environment)"],"exampleFix":"# before\n# spec: plugin calls acquire_worker_lock -> Fluent::InvalidLockDirectory\nit 'appends' do\n  driver.configure(fluent_config)\nend\n\n# after\naround do |ex|\n  ENV['FLUENTD_LOCK_DIR'] = Dir.mktmpdir\n  ex.run\n  ENV.delete('FLUENTD_LOCK_DIR')\nend\nit 'appends' do\n  driver.configure(fluent_config)\nend","handlingStrategy":"validation","validationCode":"lock_dir = ENV['FLUENTD_LOCK_DIR']\nraise 'FLUENTD_LOCK_DIR unset; cannot use worker locks' if lock_dir.nil?\nrequire 'fileutils'\nFileUtils.mkdir_p(lock_dir) unless Dir.exist?(lock_dir)","typeGuard":null,"tryCatchPattern":"begin\n  plugin.acquire_worker_lock('name') { work }\nrescue Fluent::InvalidLockDirectory\n  ENV['FLUENTD_LOCK_DIR'] = Dir.mktmpdir\n  retry\nend","preventionTips":["Set FLUENTD_LOCK_DIR in test setup (Dir.mktmpdir) and clean up after","Run fluentd through its supervisor, which exports the variable","Do not scrub the environment when wrapping/launching fluentd workers"],"tags":["fluentd","environment","file-lock","testing","missing-env"],"backgroundTag":"missing-env-var","analyzedSha":"dd45c6e18dc7be33b5e5a0f0767bf46307ff5626","analyzedAt":"2026-08-21T16:22:07.332Z","schemaVersion":2},"datasetVersion":"2026-08-21T18:17:14.833Z"}