Class: Lumberjack::JsonDevice

Inherits:
Device
  • Object
show all
Defined in:
lib/lumberjack/json_device.rb

Overview

This Lumberjack device logs output to another device as JSON formatted text with one document per line. This format (JSONL) is ideal for structured logging pipelines and can be easily consumed by log aggregation services, search engines, and monitoring tools.

The device supports flexible field mapping to customize the JSON structure, datetime formatting, post-processing, and pretty printing for development use.

The mapping parameter can be used to define the JSON data structure. To define the structure pass in a hash with key indicating the log entry field and the value indicating the JSON document key.

The standard entry fields are mapped with the following keys:

  • :time
  • :severity
  • :progname
  • :pid
  • :message
  • :attributes

Any additional keys will be pulled from the attributes. If any of the standard keys are missing or have a nil mapping, the entry field will not be included in the JSON output.

You can create a nested JSON structure by specifying an array as the JSON key.

Examples:

Basic usage

device = Lumberjack::JsonDevice.new(output: STDOUT)
logger = Lumberjack::Logger.new(device)
logger.info("User logged in", user_id: 123)

Custom field mapping

device = Lumberjack::JsonDevice.new(
  output: STDOUT,
  mapping: {
    time: "timestamp",
    severity: "level",
    message: true,
    attributes: "*"
  }
)

Constant Summary collapse

VERSION = 
File.read(File.join(__dir__, "..", "..", "VERSION")).strip.freeze
DEFAULT_MAPPING =

Default mapping for standard log entry fields to JSON keys.

{
  time: true,
  severity: true,
  message: true,
  progname: true,
  pid: true,
  attributes: true
}.freeze
DEFAULT_TIME_FORMAT =

Default ISO 8601 datetime format with microsecond precision and timezone offset.

"%Y-%m-%dT%H:%M:%S.%6N%z"

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(options = {}, deprecated_options = nil) ⇒ JsonDevice

Create a new JsonDevice instance.

Parameters:

  • options (Hash<Symbol, Object>) (defaults to: {}) —

    The options for the JSON device.

  • deprecated_options (Hash<Symbol, Object>) (defaults to: nil) —

    The device options for the JSON device if the output stream or device is specified in the first argument. This is deprecated behavior for backward compatibility with version 2.x.

Options Hash (options):

  • :output (IO, Lumberjack::Device, Symbol, String, Pathname, nil) —

    The output stream or Lumberjack device to write the JSON formatted log entries to. If this is a string or Pathname, then the output will be written to that file path. The values :stdout and :stderr can be used to write to STDOUT and STDERR respectively. Defaults to STDOUT.

  • :mapping (Hash) —

    A hash where the key is the log entry field name and the value indicates how to map the field if it exists. If the value is true, the field will be mapped to the same name. If the value is a String, the field will be mapped to that key name. If the value is an Array, it will be mapped to a nested structure that follows the array elements. If the value is a callable object, it will be called with the value and is expected to return a hash that will be merged into the JSON document. If the value is false or nil, the field will not be included in the JSON output. Special value "*" for :attributes will flatten all remaining attributes to the root level.

  • :formatter (Lumberjack::Formatter) —

    An optional formatter to use for formatting the log entry data.

  • :datetime_format (String) —

    An optional datetime format string to use for formatting the log timestamp. Defaults to ISO 8601 format with microsecond precision.

  • :post_processor (Proc) —

    An optional callable object that will be called with the log entry hash before it is written to the output stream. This can be used to modify the log entry data before it is serialized to JSON. The callable should return a Hash or the result will be ignored.

  • :pretty (Boolean) —

    If true, the output will be formatted as pretty JSON with indentation and newlines. The default is false, which writes each log entry as a single line JSON document.

  • :utc (Boolean) —

    If true, all times will be converted to UTC before formatting.



126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
 
# File 'lib/lumberjack/json_device.rb', line 126

def initialize(options = {}, deprecated_options = nil)
  unless options.is_a?(Hash)
    Lumberjack::Utils.deprecated(:new, "Passing a stream or device as the first argument is no longer supported and will be removed in version 3.1; specify the output stream in the :output key of the options hash.") do
      options = (deprecated_options || {}).merge(output: options)
    end
  end

  @mutex = Mutex.new

  stream_options = options.dup
  JSON_OPTIONS.each { |key| stream_options.delete(key) }
  @output = output_stream(options[:output], stream_options)

  self.mapping = options.fetch(:mapping, DEFAULT_MAPPING)

  @force_utc = options.fetch(:utc, false)
  @formatter = default_formatter
  self.datetime_format = options.fetch(:datetime_format, DEFAULT_TIME_FORMAT)
  @formatter.include(options[:formatter]) if options[:formatter]

  @post_processor = options[:post_processor]

  @pretty = !!options[:pretty]
end

Instance Attribute Details

#datetime_format ⇒ Object

Returns the value of attribute datetime_format.



181
182
183
 
# File 'lib/lumberjack/json_device.rb', line 181

def datetime_format
  @datetime_format
end

#formatter ⇒ Lumberjack::Formatter

Returns The formatter used to format log entry values before JSON serialization.

Returns:

  • (Lumberjack::Formatter) —

    The formatter used to format log entry values before JSON serialization.



85
86
87
 
# File 'lib/lumberjack/json_device.rb', line 85

def formatter
  @formatter
end

#mapping ⇒ Object

Returns the value of attribute mapping.



97
98
99
 
# File 'lib/lumberjack/json_device.rb', line 97

def mapping
  @mapping
end

#post_processor ⇒ Proc?

Returns A callable object that can modify the log entry hash before JSON serialization.

Returns:

  • (Proc, nil) —

    A callable object that can modify the log entry hash before JSON serialization.



89
90
91
 
# File 'lib/lumberjack/json_device.rb', line 89

def post_processor
  @post_processor
end

#pretty=(value) ⇒ Object (writeonly)

Sets the attribute pretty

Parameters:

  • value —

    the value to set the attribute pretty to.



93
94
95
 
# File 'lib/lumberjack/json_device.rb', line 93

def pretty=(value)
  @pretty = value
end

Instance Method Details

#dev ⇒ Object

Get the underlying device from the output stream.

Returns:

  • (Object) —

    The underlying device.



168
169
170
 
# File 'lib/lumberjack/json_device.rb', line 168

def dev
  @output.dev
end

#entry_as_json(entry) ⇒ Hash

Convert a Lumberjack::LogEntry to a Hash using the specified field mapping.

Parameters:

  • entry (Lumberjack::LogEntry) —

    The log entry to convert.

Returns:

  • (Hash) —

    A hash representing the log entry in JSON format.



258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
 
# File 'lib/lumberjack/json_device.rb', line 258

def entry_as_json(entry)
  keys = @keys
  data = {}
  set_attribute(data, keys.time, entry.time) if keys.time
  set_attribute(data, keys.severity, entry.severity_label) if keys.severity
  set_attribute(data, keys.message, json_safe(entry.message)) if keys.message
  set_attribute(data, keys.progname, json_safe(entry.progname)) if keys.progname && entry.progname
  set_attribute(data, keys.pid, entry.pid) if keys.pid

  attributes = entry.attributes.transform_values { |value| json_safe(value) } if entry.attributes

  if keys.custom.size > 0 && attributes && !attributes&.empty?
    keys.custom.each do |name, key|
      name = name.is_a?(Array) ? name.join(".") : name.to_s
      value = attributes.delete(name)
      next if value.nil?

      value = Lumberjack::Utils.expand_attributes(value) if value.is_a?(Hash)
      set_attribute(data, key, value)
    end
  end

  if keys.attributes && !attributes&.empty?
    attributes = Lumberjack::Utils.expand_attributes(attributes)
    if keys.attributes == "*"
      attributes.each { |k, v| data[k] = v unless data.include?(k) }
    else
      set_attribute(data, keys.attributes, attributes)
    end
  end

  data = @formatter.format(data) if @formatter
  if @post_processor
    processed_result = @post_processor.call(data)
    data = processed_result if processed_result.is_a?(Hash)
  end

  data
end

#flush ⇒ void

This method returns an undefined value.

Flush the output stream.



175
176
177
 
# File 'lib/lumberjack/json_device.rb', line 175

def flush
  @output.flush
end

#map(field_mapping) ⇒ void

This method returns an undefined value.

Add a field mapping to the existing mappings.

Parameters:

  • field_mapping (Hash) —

    A hash where the key is the log entry field name and the value is the JSON key. If the value is true, the field will be mapped to the same name If the value is an array, it will be mapped to a nested structure. If the value is a callable object, it will be called with the value and should return a hash that will be merged into the JSON document. If the value is false, the field will not be included in the JSON output.



249
250
251
252
 
# File 'lib/lumberjack/json_device.rb', line 249

def map(field_mapping)
  new_mapping = field_mapping.transform_keys(&:to_sym)
  self.mapping = mapping.merge(new_mapping)
end

#pretty? ⇒ Boolean

Return true if the output is written in a multi-line pretty format. The default is to write each log entry as a single line JSON document.

Returns:

  • (Boolean) 


197
198
199
 
# File 'lib/lumberjack/json_device.rb', line 197

def pretty?
  !!@pretty
end

#write(entry) ⇒ void

This method returns an undefined value.

Write a log entry to the output stream as JSON. Each entry is written as a single line JSON document (JSONL format) unless pretty printing is enabled. Empty log entries (nil or empty message) are ignored.

Parameters:

  • entry (Lumberjack::LogEntry) —

    The log entry to write.



157
158
159
160
161
162
163
 
# File 'lib/lumberjack/json_device.rb', line 157

def write(entry)
  return if entry.empty?

  data = entry_as_json(entry)
  json = @pretty ? JSON.pretty_generate(data) : JSON.generate(data)
  @output.write("#{json}\n") # thread safety is handled by the underlying output stream
end