Class: SemanticLogger::Appender::ElasticsearchBase

Inherits:
Subscriber
  • Object
show all
Defined in:
lib/semantic_logger/appender/elasticsearch_base.rb

Overview

Abstract base appender for Elasticsearch-compatible search engines.

Implements the shared bulk-indexing pipeline used by both the Elasticsearch and OpenSearch appenders. Subclasses only need to supply the backing client class (and, optionally, whether the server version still supports document _type).

This class is internal: applications add an appender via SemanticLogger.add_appender(appender: :elasticsearch, ...) or appender: :opensearch, never by referencing this class directly.

Direct Known Subclasses

Elasticsearch, Opensearch

Instance Attribute Summary collapse

Attributes inherited from Subscriber

#application, #environment, #formatter, #host, #logger, #metrics

Instance Method Summary collapse

Methods inherited from Subscriber

#batch_by_default?, #close, #console_output?, #console_stream, #flush, #level, #should_log?

Constructor Details

#initialize(url: "http://localhost:9200", index: "semantic_logger", date_pattern: "%Y.%m.%d", type: "log", level: nil, formatter: nil, filter: nil, application: nil, environment: nil, host: nil, data_stream: false, **client_args) ⇒ ElasticsearchBase

Create an Elasticsearch-compatible appender over persistent HTTP(S).

Parameters:

index: [String]
Prefix of the index to store the logs in.
The final index appends the date so that indexes are used per day.
  I.e. The final index will look like 'semantic_logger-YYYY.MM.DD'
Default: 'semantic_logger'

date_pattern: [String]
The time format used to generate the full index name. Useful
  if you want monthly indexes ('%Y.%m') or weekly ('%Y.%W').
Default: '%Y.%m.%d'

type: [String]
Document type to associate with logs when they are written.
Deprecated in Elasticsearch 7.0.0, unused by OpenSearch.
Default: 'log'

level: [:trace | :debug | :info | :warn | :error | :fatal]
Override the log level for this appender.
Default: SemanticLogger.default_level

formatter: [Object|Proc|Symbol|Hash]
An instance of a class that implements #call, or a Proc to be used to format
the output from this appender
Default: :raw_json (See: #call)

filter: [Regexp|Proc]
RegExp: Only include log messages where the class name matches the supplied.
regular expression. All other messages will be ignored.
Proc: Only include log messages where the supplied Proc returns true
      The Proc must return true or false.

host: [String]
Name of this host to appear in log messages.
Default: SemanticLogger.host

application: [String]
Name of this application to appear in log messages.
Default: SemanticLogger.application

Client Parameters (passed through to the backing client):

url: [String]
Fully qualified address to the service.
Default: 'http://localhost:9200'

hosts: [String|Hash|Array]
Single host passed as a String or Hash, or multiple hosts
passed as an Array; `host` or `url` keys are also valid.
Note:
  :url above is ignored when supplying this option.

resurrect_after [Float]
After how many seconds a dead connection should be tried again.

reload_connections [true|false|Integer]
Reload connections after X requests.
Default: false

randomize_hosts [true|false]
Shuffle connections on initialization and reload.
Default: false

sniffer_timeout [Integer]
Timeout for reloading connections in seconds.
Default: 1

retry_on_failure [true|false|Integer]
Retry X times when request fails before raising and exception.
Default: false

retry_on_status [Array<Number>]
Retry when specific status codes are returned.

reload_on_failure [true|false]
Reload connections after failure.
Default: false

request_timeout [Integer]
The request timeout to be passed to transport in options.

adapter [Symbol]
A specific adapter for Faraday (e.g. `:patron`)

transport_options [Hash]
Options to be passed to the `Faraday::Connection` constructor.

transport_class [Constant]
A specific transport class to use, will be initialized by
the client and passed hosts and all arguments.

transport [Object]
A specific transport instance.

serializer_class [Constant]
A specific serializer class to use, will be initialized by
the transport and passed the transport instance.

selector
An instance of a connection selector strategy.

send_get_body_as [String]
Specify the HTTP method to use for GET requests with a body.
Default: 'GET'


124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 124

def initialize(url: "http://localhost:9200",
               index: "semantic_logger",
               date_pattern: "%Y.%m.%d",
               type: "log",
               level: nil,
               formatter: nil,
               filter: nil,
               application: nil,
               environment: nil,
               host: nil,
               data_stream: false,
               **client_args,
               &)
  @url                  = url
  @index                = index
  @date_pattern         = date_pattern
  @type                 = type
  @client_args          = client_args.dup
  @client_args[:url]    = url if url && !client_args[:hosts]
  @client_args[:logger] = logger
  @data_stream          = data_stream

  super(level: level, formatter: formatter, filter: filter, application: application, environment: environment, host: host, metrics: false, &)
  reopen
end

Instance Attribute Details

#batch_size ⇒ Object

Returns the value of attribute batch_size.



16
17
18
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 16

def batch_size
  @batch_size
end

#client ⇒ Object

Returns the value of attribute client.



16
17
18
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 16

def client
  @client
end

#client_args ⇒ Object

Returns the value of attribute client_args.



16
17
18
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 16

def client_args
  @client_args
end

#date_pattern ⇒ Object

Returns the value of attribute date_pattern.



16
17
18
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 16

def date_pattern
  @date_pattern
end

#flush_interval ⇒ Object

Returns the value of attribute flush_interval.



16
17
18
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 16

def flush_interval
  @flush_interval
end

#index ⇒ Object

Returns the value of attribute index.



16
17
18
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 16

def index
  @index
end

#timeout_interval ⇒ Object

Returns the value of attribute timeout_interval.



16
17
18
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 16

def timeout_interval
  @timeout_interval
end

#type ⇒ Object

Returns the value of attribute type.



16
17
18
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 16

def type
  @type
end

#url ⇒ Object

Returns the value of attribute url.



16
17
18
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 16

def url
  @url
end

Instance Method Details

#batch(logs) ⇒ Object 



161
162
163
164
165
166
167
168
169
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 161

def batch(logs)
  messages = []
  logs.each do |log|
    messages << bulk_index(log) << formatter.call(log, self)
  end

  bulk_write(messages)
  true
end

#log(log) ⇒ Object

Log to the index for today



155
156
157
158
159
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 155

def log(log)
  bulk_payload = formatter.call(log, self)
  bulk_write([bulk_index(log), bulk_payload])
  true
end

#reopen ⇒ Object 



150
151
152
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 150

def reopen
  @client = client_class.new(@client_args)
end