Freika/dawarich · error · StandardError

Invalid JSON format in data file: %{message}

Error message

Invalid JSON format in data file: %{message}

What it means

StandardError raised from V1Handler#process's `rescue Oj::ParseError`: data.json was found and opened for streaming SAJ parse, but the JSON document itself is syntactically invalid. The parser's message (character/line position of the violation) is interpolated into 'Invalid JSON format in data file: %{message}', so the exact break location is part of the error.

Source

Thrown at app/services/users/import_data/v1_handler.rb:55

  def process
    Rails.logger.info "Processing v1 format archive for user: #{user.email}"

    json_path = import_directory.join('data.json')
    raise StandardError, I18n.t('services.users.import_data.v1_handler.data_file_missing') unless File.exist?(json_path)

    initialize_stream_state

    handler = ::JsonStreamHandler.new(self)
    parser = Oj::Parser.new(:saj, handler: handler)

    File.open(json_path, 'rb') do |io|
      parser.load(io)
    end

    finalize_stream_processing
  rescue Oj::ParseError => e
    raise StandardError, I18n.t('services.users.import_data.v1_handler.invalid_json', message: e.message)
  rescue IOError => e
    raise StandardError, I18n.t('services.users.import_data.v1_handler.read_failed', message: e.message)
  end

  attr_reader :expected_counts

  # Called by JsonStreamHandler for non-streamed sections
  def handle_section(key, value)
    case key
    when 'counts'
      @expected_counts = value if value.is_a?(Hash)
      Rails.logger.info "Expected entity counts from export: #{@expected_counts}" if @expected_counts
    when 'settings'
      import_settings(value) if value.present?
    when 'areas'
      import_areas(value)
    when 'imports'
      import_imports(value)

View on GitHub (pinned to 97fad417c5)

Solutions

1. Read the interpolated parser message - it names where parsing failed
2. Validate locally: jq . data.json (or python -m json.tool) to confirm and locate the corruption
3. Check archive integrity (unzip -t) and re-download/re-export, then retry the import
4. If the file is truncated at the tail, re-exporting is the only sound fix - do not hand-append closing brackets for a restore

Example fix

# before: importing a suspect archive blindly
Users::ImportData.new(user, 'maybe-broken-export.zip').import

# after: pre-validate the inner JSON before committing to the import
Oj.load(File.read('tmp/extracted/data.json')) rescue => e
  # log "archive corrupt: #{e.message}" and abort before touching user data

# Cheap pre-flight for reasonable sizes: full parse before committing
begin
  Oj.load(File.read(json_path))
rescue Oj::ParseError => e
  raise ArgumentError, "archive data.json corrupt: #{e.message}"
end

def valid_export_json?(path)
  Oj.load(File.read(path))
  true
rescue Oj::ParseError, JSON::ParserError
  false
end

begin
  Users::ImportData.new(user, archive).import
rescue StandardError => e
  if e.message.start_with?('Invalid JSON format in data file')
    # the suffix is the parser telling you WHERE it broke - surface it
    notify_user("Export file is corrupt (#{e.message}). Re-export and retry.")
  else
    raise
  end
end

When it happens

Trigger: A data.json truncated by an interrupted export or partial download (large exports are the norm), encoding corruption from re-saving or text-mode transfers, or hand-editing that introduced syntax errors. The file streams via Oj::Parser(:saj), so any stray byte anywhere in the document aborts the import.

Common situations: Exports cut off by disk-full or timeout on the source instance, files round-tripped through tools that re-encode bytes, or manual JSON surgery before import.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• GPX parse error: %{message}
• Unknown export format: neither manifest.json nor data.json f
• Unsupported export format version: %{version}
• Data file not found in archive: data.json
• Failed to read JSON data: %{message}

AI-assisted analysis of Freika/dawarich@97fad417c5 (2026-08-21). Data as JSON: /api/errors/b624a92625031414. Report an issue: GitHub.