Freika/dawarich · error · StandardError

Failed to read JSON data: %{message}

Error message

Failed to read JSON data: %{message}

What it means

StandardError raised from V1Handler#process's `rescue IOError`: reading data.json (File.open + parser.load streaming) failed at the IO layer rather than the parsing layer - the stream could not be read. The IOError's message is interpolated into 'Failed to read JSON data: %{message}'. This is an infrastructure error (file gone, permissions, fd closed, disk/full filesystem fault), distinct from error 51 which is document syntax.

Source

Thrown at app/services/users/import_data/v1_handler.rb:57

    Rails.logger.info "Processing v1 format archive for user: #{user.email}"

    json_path = import_directory.join('data.json')
    raise StandardError, I18n.t('services.users.import_data.v1_handler.data_file_missing') unless File.exist?(json_path)

    initialize_stream_state

    handler = ::JsonStreamHandler.new(self)
    parser = Oj::Parser.new(:saj, handler: handler)

    File.open(json_path, 'rb') do |io|
      parser.load(io)
    end

    finalize_stream_processing
  rescue Oj::ParseError => e
    raise StandardError, I18n.t('services.users.import_data.v1_handler.invalid_json', message: e.message)
  rescue IOError => e
    raise StandardError, I18n.t('services.users.import_data.v1_handler.read_failed', message: e.message)
  end

  attr_reader :expected_counts

  # Called by JsonStreamHandler for non-streamed sections
  def handle_section(key, value)
    case key
    when 'counts'
      @expected_counts = value if value.is_a?(Hash)
      Rails.logger.info "Expected entity counts from export: #{@expected_counts}" if @expected_counts
    when 'settings'
      import_settings(value) if value.present?
    when 'areas'
      import_areas(value)
    when 'imports'
      import_imports(value)
    when 'exports'
      import_exports(value)

View on GitHub (pinned to 97fad417c5)

Solutions

1. Check the interpolated IO message plus the filesystem: free space and permissions of the tmp/import_* directory
2. Ensure nothing prunes Rails.root/tmp while imports run (systemd-tmpfiles, cron cleaners)
3. Retry the import - transient IO faults are the usual cause and the transaction rolls back cleanly
4. Give the import directory more durable storage if tmpfs is too small for the export

Example fix

# before: import runs from tmpfs that can drop files mid-read
@import_directory = Rails.root.join('tmp', "import_#{user.email}_#{Time.current.to_i}")

# after: point large imports at durable storage with room
IMPORT_ROOT = ENV.fetch('DAWARICH_IMPORT_DIR') { Rails.root.join('tmp') }
@import_directory = Pathname.new(IMPORT_ROOT).join("import_#{user.id}_#{Time.current.to_i}")

path = import_directory.join('data.json')
raise ArgumentError, 'data.json unreadable' unless path.readable?

def readable_data_file?(import_directory)
  p = import_directory.join('data.json')
  p.exist? && p.readable?
end

begin
  Users::ImportData::V1Handler.new(user, dir, stats).process
rescue StandardError => e
  if e.message.start_with?('Failed to read JSON data')
    retry if (attempts += 1) < 3   # IO faults are usually transient
  else
    raise
  end
end

When it happens

Trigger: The extracted data.json deleted or its permissions changed while the import runs, a file descriptor closed underneath the parser, or a filesystem-level fault (disk full, NFS hiccup, ephemeral container tmp mount) during the streamed read of a multi-gigabyte export.

Common situations: tmp/ cleanup jobs racing the import, containerized deployments with small or volatile tmpfs mounts, storage pressure during large restores.

Related errors

• Unknown export format: neither manifest.json nor data.json f
• Unsupported export format version: %{version}
• Data file not found in archive: data.json
• Invalid JSON format in data file: %{message}
• Manifest file not found in archive: manifest.json

AI-assisted analysis of Freika/dawarich@97fad417c5 (2026-08-21). Data as JSON: /api/errors/900b567a3dc76024. Report an issue: GitHub.