Module: Karafka::Pro::Routing::Features::AdaptiveIterator::Topic

Defined in:
lib/karafka/pro/routing/features/adaptive_iterator/topic.rb

Overview

Topic extension allowing us to enable and configure adaptive iterator

Instance Method Summary collapse

Instance Method Details

#adaptive_iterator(active: false, safety_margin: 10, marking_method: :mark_as_consumed, clean_after_yielding: true) ⇒ Object

Parameters:

  • active (Boolean) (defaults to: false) —

    should we use the automatic adaptive iterator

  • safety_margin (Integer) (defaults to: 10) —

    How big of a margin we leave ourselves so we can safely communicate back with Kafka, etc. We stop and seek back when we’ve burned 85% of the time by default. We leave 15% of time for post-processing operations so we have space before we hit max.poll.interval.ms.

  • marking_method (Symbol) (defaults to: :mark_as_consumed) —

    If we should, how should we mark

  • clean_after_yielding (Boolean) (defaults to: true) —

    Should we clean post-yielding via the cleaner API



56
57
58
59
60
61
62
63
64
65
66
67
68
 
# File 'lib/karafka/pro/routing/features/adaptive_iterator/topic.rb', line 56

def adaptive_iterator(
  active: false,
  safety_margin: 10,
  marking_method: :mark_as_consumed,
  clean_after_yielding: true
)
  @adaptive_iterator ||= Config.new(
    active: active,
    safety_margin: safety_margin,
    marking_method: marking_method,
    clean_after_yielding: clean_after_yielding
  )
end

#adaptive_iterator? ⇒ Boolean

Returns Is adaptive iterator active. It is always ‘true`, since we use it via explicit messages batch wrapper.

Returns:

  • (Boolean) —

    Is adaptive iterator active. It is always ‘true`, since we use it via explicit messages batch wrapper



72
73
74
 
# File 'lib/karafka/pro/routing/features/adaptive_iterator/topic.rb', line 72

def adaptive_iterator?
  adaptive_iterator.active?
end

#initialize ⇒ Object

This method calls the parent class initializer and then sets up the extra instance variable to nil. The explicit initialization to nil is included as an optimization for Ruby’s object shapes system, which improves memory layout and access performance.



42
43
44
45
 
# File 'lib/karafka/pro/routing/features/adaptive_iterator/topic.rb', line 42

def initialize(...)
  super
  @adaptive_iterator = nil
end

#to_h ⇒ Hash

Returns topic with all its native configuration options plus poll guarding setup configuration.

Returns:

  • (Hash) —

    topic with all its native configuration options plus poll guarding setup configuration.



78
79
80
81
82
 
# File 'lib/karafka/pro/routing/features/adaptive_iterator/topic.rb', line 78

def to_h
  super.merge(
    adaptive_iterator: adaptive_iterator.to_h
  ).freeze
end