Class: SemanticLogger::Appender::ElasticsearchBase

Inherits:
Subscriber
  • Object
show all
Defined in:
lib/semantic_logger/appender/elasticsearch_base.rb

Overview

Abstract base appender for Elasticsearch-compatible search engines.

Implements the shared bulk-indexing pipeline used by both the Elasticsearch and OpenSearch appenders. Subclasses only need to supply the backing client class.

This class is internal: applications add an appender via SemanticLogger.add_appender(appender: :elasticsearch, ...) or appender: :opensearch, never by referencing this class directly.

Direct Known Subclasses

Elasticsearch, Opensearch

Instance Attribute Summary collapse

Attributes inherited from Subscriber

#application, #environment, #formatter, #host, #logger, #metrics

Instance Method Summary collapse

Methods inherited from Subscriber

#batch_by_default?, #close, #console_output?, #console_stream, #flush, #level, #should_log?

Constructor Details

#initialize(url: "http://localhost:9200", index: "semantic_logger", date_pattern: "%Y.%m.%d", type: nil, level: nil, formatter: nil, filter: nil, application: nil, environment: nil, host: nil, data_stream: false, **client_args) ⇒ ElasticsearchBase

Create an Elasticsearch-compatible appender over persistent HTTP(S).

Parameters:

index: [String]
Prefix of the index to store the logs in.
The final index appends the date so that indexes are used per day.
  I.e. The final index will look like 'semantic_logger-YYYY.MM.DD'
Default: 'semantic_logger'

date_pattern: [String]
The time format used to generate the full index name. Useful
  if you want monthly indexes ('%Y.%m') or weekly ('%Y.%W').
Default: '%Y.%m.%d'

type: [String]
Deprecated and ignored. Document `_type` has been unused since
Elasticsearch 7 and is ignored by OpenSearch. Passing it logs a
deprecation warning; it will be removed in v6.

level: [:trace | :debug | :info | :warn | :error | :fatal]
Override the log level for this appender.
Default: SemanticLogger.default_level

formatter: [Object|Proc|Symbol|Hash]
An instance of a class that implements #call, or a Proc to be used to format
the output from this appender
Default: :raw_json (See: #call)

filter: [Regexp|Proc]
RegExp: Only include log messages where the class name matches the supplied.
regular expression. All other messages will be ignored.
Proc: Only include log messages where the supplied Proc returns true
      The Proc must return true or false.

host: [String]
Name of this host to appear in log messages.
Default: SemanticLogger.host

application: [String]
Name of this application to appear in log messages.
Default: SemanticLogger.application

Client Parameters (passed through to the backing client):

url: [String]
Fully qualified address to the service.
Default: 'http://localhost:9200'

hosts: [String|Hash|Array]
Single host passed as a String or Hash, or multiple hosts
passed as an Array; `host` or `url` keys are also valid.
Note:
  :url above is ignored when supplying this option.

resurrect_after [Float]
After how many seconds a dead connection should be tried again.

reload_connections [true|false|Integer]
Reload connections after X requests.
Default: false

randomize_hosts [true|false]
Shuffle connections on initialization and reload.
Default: false

sniffer_timeout [Integer]
Timeout for reloading connections in seconds.
Default: 1

retry_on_failure [true|false|Integer]
Retry X times when request fails before raising and exception.
Default: false

retry_on_status [Array<Number>]
Retry when specific status codes are returned.

reload_on_failure [true|false]
Reload connections after failure.
Default: false

request_timeout [Integer]
The request timeout to be passed to transport in options.

adapter [Symbol]
A specific adapter for Faraday (e.g. `:patron`)

transport_options [Hash]
Options to be passed to the `Faraday::Connection` constructor.

transport_class [Constant]
A specific transport class to use, will be initialized by
the client and passed hosts and all arguments.

transport [Object]
A specific transport instance.

serializer_class [Constant]
A specific serializer class to use, will be initialized by
the transport and passed the transport instance.

selector
An instance of a connection selector strategy.

send_get_body_as [String]
Specify the HTTP method to use for GET requests with a body.
Default: 'GET'


123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 123

def initialize(url: "http://localhost:9200",
               index: "semantic_logger",
               date_pattern: "%Y.%m.%d",
               type: nil,
               level: nil,
               formatter: nil,
               filter: nil,
               application: nil,
               environment: nil,
               host: nil,
               data_stream: false,
               **client_args,
               &)
  if type
    Kernel.warn(
      "The Elasticsearch/OpenSearch appender `type:` parameter is deprecated and will be removed in v6. " \
      "Document `_type` has been unused since Elasticsearch 7 and is ignored by OpenSearch.",
      category: :deprecated
    )
  end

  @url                  = url
  @index                = index
  @date_pattern         = date_pattern
  @client_args          = client_args.dup
  @client_args[:url]    = url if url && !client_args[:hosts]
  @client_args[:logger] = logger
  @data_stream          = data_stream

  super(level: level, formatter: formatter, filter: filter, application: application,
        environment: environment, host: host, metrics: false, &)
  reopen
end

Instance Attribute Details

#batch_size ⇒ Object

Returns the value of attribute batch_size.



15
16
17
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 15

def batch_size
  @batch_size
end

#client ⇒ Object

Returns the value of attribute client.



15
16
17
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 15

def client
  @client
end

#client_args ⇒ Object

Returns the value of attribute client_args.



15
16
17
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 15

def client_args
  @client_args
end

#date_pattern ⇒ Object

Returns the value of attribute date_pattern.



15
16
17
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 15

def date_pattern
  @date_pattern
end

#flush_interval ⇒ Object

Returns the value of attribute flush_interval.



15
16
17
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 15

def flush_interval
  @flush_interval
end

#index ⇒ Object

Returns the value of attribute index.



15
16
17
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 15

def index
  @index
end

#timeout_interval ⇒ Object

Returns the value of attribute timeout_interval.



15
16
17
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 15

def timeout_interval
  @timeout_interval
end

#url ⇒ Object

Returns the value of attribute url.



15
16
17
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 15

def url
  @url
end

Instance Method Details

#batch(logs) ⇒ Object 



168
169
170
171
172
173
174
175
176
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 168

def batch(logs)
  messages = []
  logs.each do |log|
    messages << bulk_index(log) << formatter.call(log, self)
  end

  bulk_write(messages)
  true
end

#log(log) ⇒ Object

Log to the index for today



162
163
164
165
166
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 162

def log(log)
  bulk_payload = formatter.call(log, self)
  bulk_write([bulk_index(log), bulk_payload])
  true
end

#reopen ⇒ Object 



157
158
159
 
# File 'lib/semantic_logger/appender/elasticsearch_base.rb', line 157

def reopen
  @client = client_class.new(@client_args)
end