Class: Deimos::ActiveRecordProducer

Inherits:
Producer
  • Object
show all
Defined in:
lib/deimos/active_record_producer.rb,
sig/defs.rbs

Overview

Class which automatically produces a record when given an ActiveRecord instance or a list of them. Just call send_events on a list of records and they will be auto-published. You can override generate_payload to make changes to the payload before it's published.

You can also call this with a list of hashes representing attributes. This is common when using activerecord-import.

Constant Summary collapse

MAX_BATCH_SIZE =

Returns:

  • (Integer) 

      
    

  







  
    

      Class Method Summary
      collapse
    


    
    
  


  
  
  
  
  
  
  
  
  
  
Methods inherited from Producer

  
config, determine_backend_class, karafka_config, key_encoder, partition_key, produce, produce_batch, publish, publish_list, topic


  
  
  
  
  
  
  
  

  

    
Class Method Details


    
      

  

  
    .encoder  ⇒ Object 
  

  

  


  

    
      


63
64
65
66
67

    		
    
      
# File 'lib/deimos/active_record_producer.rb', line 63

def encoder
  raise "No schema or namespace configured for #{self.name}" if karafka_config.nil?

  karafka_config.deserializers[:payload].backend
end

    		
  





    
      

  

  
    .generate_deletion_payload(record)  ⇒ Hash 
  

  

  


  

    
Deletion payload for a record by default, delegate to the
model's deletion_payload. Producers may override this method to
customize the deletion key/payload per producer.


  





  
Parameters:


    
  
    
  • 
      
        record
      
      
        (ActiveRecord::Base)
      
      
      
    
    • 
  




Returns:


    
  
    
  • 
      
      
        (Hash)
      
      
      
    
    • 
  





  

    
      


92
93
94

    		
    
      
# File 'lib/deimos/active_record_producer.rb', line 92

def generate_deletion_payload(record)
  record.deletion_payload
end

    		
  





    
      

  

  
    .generate_payload(attributes, _record)  ⇒ ::Hash[untyped, untyped] 
  

  

  


  

    
Generate the payload, given a list of attributes or a record..
Can be overridden or added to by subclasses.
is not set.


@param attributes


@param _record — May be nil if refetch_record


  





  
Parameters:


    
  
    
  • 
      
        attributes
      
      
        (::Hash[untyped, untyped])
      
      
      
    
    • 
  
    
  • 
      
        _record
      
      
        (ActiveRecord::Base)
      
      
      
    
    • 
  




Returns:


    
  
    
  • 
      
      
        (::Hash[untyped, untyped])
      
      
      
    
    • 
  





  

    
      


75
76
77
78
79
80
81
82
83
84
85

    		
    
      
# File 'lib/deimos/active_record_producer.rb', line 75

def generate_payload(attributes, _record)
  fields = self.encoder.schema_fields
  payload = attributes.stringify_keys
  payload.delete_if do |k, _|
    k.to_sym != :payload_key && !fields.map(&:name).include?(k)
  end
  return payload if self.karafka_config.use_schema_classes.nil? &&
                    !Deimos.config.schema.use_schema_classes

  Utils::SchemaClass.instance(payload, encoder.schema, encoder.namespace)
end

    		
  





    
      

  

  
    .poll_query(time_from:, time_to:, min_id:, column_name: :updated_at)  ⇒ Object 
  

  

  


  

    
Query to use when polling the database with the DbPoller. Add
includes, joins, or wheres as necessary, or replace entirely.
than this value).


@param time_from — the time to start the query from.


@param time_to — the time to end the query.


@param column_name — the column name to look for.


@param min_id — the minimum ID (i.e. all IDs must be greater


  





  



  

    
      


104
105
106
107
108
109
110
111
112
113
114
115
116
117

    		
    
      
# File 'lib/deimos/active_record_producer.rb', line 104

def poll_query(time_from:, time_to:, min_id:, column_name: :updated_at)
  klass = record_klass
  table = ActiveRecord::Base.connection.quote_table_name(klass.table_name)
  column = ActiveRecord::Base.connection.quote_column_name(column_name)
  primary = ActiveRecord::Base.connection.quote_column_name(klass.primary_key)
  klass.where(
    "((#{table}.#{column} = ? AND #{table}.#{primary} > ?) \
     OR #{table}.#{column} > ?) AND #{table}.#{column} <= ?",
    time_from,
    min_id,
    time_from,
    time_to
  )
end

    		
  





    
      

  

  
    .post_process(_records)  ⇒ Object 
  

  

  


  

    
Post process records after publishing


@param records


  





  
Parameters:


    
  
    
  • 
      
        _records
      
      
        (::Array[ActiveRecord::Base])
      
      
      
    
    • 
  




Returns:


    
  
    
  • 
      
      
        (Object)
      
      
      
    
    • 
  





  

    
      


121
122

    		
    
      
# File 'lib/deimos/active_record_producer.rb', line 121

def post_process(_records)
end

    		
  





    
      

  

  
    .record_class(klass = nil, refetch: true)  ⇒ void 
  

  

  


  

    
This method returns an undefined value.
Indicate the class this producer is working on.
a record object, refetch the record to pass into the generate_payload
method.


@param klass


@param refetch — if true, and we are given a hash instead of


  





  
Parameters:


    
  
    
  • 
      
        klass
      
      
        (Class)
      
      
        (defaults to: nil)
      
      
    
    • 
  
    
  • 
      
        refetch:
      
      
        (Boolean)
      
      
        (defaults to: true)
      
      
    
    • 
  






  

    
      


21
22
23
24
25
26

    		
    
      
# File 'lib/deimos/active_record_producer.rb', line 21

def record_class(klass=nil, refetch: true)
  return record_klass if klass.nil?

  @record_class = klass
  @refetch_record = refetch
end

    		
  





    
      

  

  
    .record_klass  ⇒ Class? 
  

  

  


  

    

  





  

Returns:


    
  
    
  • 
      
      
        (Class, nil)
      
      
      
    
    • 
  





  

    
      


29
30
31
32

    		
    
      
# File 'lib/deimos/active_record_producer.rb', line 29

def record_klass
  @record_class = @record_class.constantize if @record_class.is_a?(String)
  @record_class
end

    		
  





    
      

  

  
    .send_event(record, force_send: false)  ⇒ void 
  

  

  


  

    
This method returns an undefined value.
@param record


@param force_send


  





  
Parameters:


    
  
    
  • 
      
        record
      
      
        (ActiveRecord::Base)
      
      
      
    
    • 
  
    
  • 
      
        force_send:
      
      
        (Boolean)
      
      
        (defaults to: false)
      
      
    
    • 
  






  

    
      


37
38
39

    		
    
      
# File 'lib/deimos/active_record_producer.rb', line 37

def send_event(record, force_send: false)
  send_events([record], force_send: force_send)
end

    		
  





    
      

  

  
    .send_events(records, force_send: false)  ⇒ void 
  

  

  


  

    
This method returns an undefined value.
@param records


@param force_send


  





  
Parameters:


    
  
    
  • 
      
        records
      
      
        (::Array[ActiveRecord::Base])
      
      
      
    
    • 
  
    
  • 
      
        force_send:
      
      
        (Boolean)
      
      
        (defaults to: false)
      
      
    
    • 
  






  

    
      


44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

    		
    
      
# File 'lib/deimos/active_record_producer.rb', line 44

def send_events(records, force_send: false)
  return if Deimos.producers_disabled?(self)

  primary_key = record_klass&.primary_key
  messages = records.map do |record|
    if record.respond_to?(:attributes)
      attrs = record.attributes.with_indifferent_access
    else
      attrs = record.with_indifferent_access
      if @refetch_record && attrs[primary_key]
        record = record_klass.find(attrs[primary_key])
      end
    end
    generate_payload(attrs, record).with_indifferent_access
  end
  self.publish_list(messages, force_send: force_send)
  self.post_process(records)
end

    		
  





    
      

  

  
    .watched_attributes(_record)  ⇒ Array<String> 
  

  

  


  

    
Override this in active record producers to add
non-schema fields to check for updates


  





  
Parameters:


    
  
    
  • 
      
        _record
      
      
        (ActiveRecord::Base)
      
      
      
    
    • 
  




Returns:


    
  
    
  • 
      
      
        (Array<String>)
      
      
      
        —
        
    fields to check for updates
    
      
    
    • 
  





  

    
      


128
129
130

    		
    
      
# File 'lib/deimos/active_record_producer.rb', line 128

def watched_attributes(_record)
  self.encoder.schema_fields.map(&:name)
end