Class: LogStash::Filters::Date

Inherits:
Base
  • Object
show all
Defined in:
lib/logstash/filters/date.rb

Overview

The date filter is used for parsing dates from fields, and then using that date or timestamp as the logstash timestamp for the event.

For example, syslog events usually have timestamps like this:

source,ruby

“Apr 17 09:32:01”

You would use the date format ‘MMM dd HH:mm:ss` to parse this.

The date filter is especially important for sorting events and for backfilling old data. If you don’t get the date correct in your event, then searching for them later will likely sort out of order.

In the absence of this filter, logstash will choose a timestamp based on the first time it sees the event (at input time), if the timestamp is not already set in the event. For example, with file input, the timestamp is set to the time of each read.

Instance Method Summary collapse

Constructor Details

#initialize(config = {}) ⇒ Date

Returns a new instance of Date.



164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
 
# File 'lib/logstash/filters/date.rb', line 164

def initialize(config = {})
  super
  if @match.length < 2
    raise LogStash::ConfigurationError, I18n.t("logstash.agent.configuration.invalid_plugin_register",
      :plugin => "filter", :type => "date",
      :error => "The match setting should contains first a field name and at least one date format, current value is #{@match}")
  end
  if @locale
    if @locale.include? '_'
      @logger.warn("Date filter now use BCP47 format for locale, replacing underscore with dash")
      @locale.gsub!('_','-')
    end
    locale = java.util.Locale.forLanguageTag(@locale)
  end

  source = @match.first

  success_block = Proc.new do |event|
    filter_matched(event)
    metric.increment(:matches)
  end
  failure_block = Proc.new do |event|
    metric.increment(:failures)
  end

  @datefilter = org.logstash.filters.DateFilter.new(source, @target, @tag_on_failure, @precision, success_block, failure_block)

  @match[1..-1].map do |format|
    @datefilter.accept_filter_config(format, @locale, @timezone)

    # Offer a fallback parser such that if the default system Locale is non-english and that no locale is set,
    # we should try to parse english if the first local parsing fails.
    if !@locale && "en" != java.util.Locale.getDefault().getLanguage() && (format.include?("MMM") || format.include?("E"))
      @datefilter.accept_filter_config(format, "en-US", @timezone)
    end
  end

end

Instance Method Details

#filter(event) ⇒ Object 



207
208
209
 
# File 'lib/logstash/filters/date.rb', line 207

def filter(event)
  multi_filter([event]).first
end

#multi_filter(events) ⇒ Object

def initialize



203
204
205
 
# File 'lib/logstash/filters/date.rb', line 203

def multi_filter(events)
  @datefilter.receive(events)
end

#register ⇒ Object 



160
161
162
 
# File 'lib/logstash/filters/date.rb', line 160

def register
  # nothing
end