Module: ElasticGraph::DatastoreCore::IndexDefinition::Base

Included in:
Index, RolloverIndexTemplate
Defined in:
lib/elastic_graph/datastore_core/index_definition/base.rb

Overview

This module contains common implementation logic for both the rollover and non-rollover implementations of the common IndexDefinition type.

Instance Method Summary collapse

Instance Method Details

#accessible_cluster_names_to_index_into ⇒ Object 



91
92
93
94
95
 
# File 'lib/elastic_graph/datastore_core/index_definition/base.rb', line 91

def accessible_cluster_names_to_index_into
  @accessible_cluster_names_to_index_into ||= clusters_to_index_into.select do |name|
    defined_clusters.include?(name)
  end
end

#accessible_from_queries? ⇒ Boolean

Indicates whether not the index is be accessible from GraphQL queries, by virtue of the ‘cluster_to_query` being a defined cluster or not. This will be used to hide GraphQL schema elements that can’t be queried when our config omits the means to query an index (e.g. due to lacking a configured URL).

Returns:

  • (Boolean) 


101
102
103
104
 
# File 'lib/elastic_graph/datastore_core/index_definition/base.rb', line 101

def accessible_from_queries?
  return false unless (cluster = cluster_to_query)
  defined_clusters.include?(cluster)
end

#all_accessible_cluster_names ⇒ Object

Returns a list of all defined datastore clusters this index resides within.



83
84
85
86
87
88
89
 
# File 'lib/elastic_graph/datastore_core/index_definition/base.rb', line 83

def all_accessible_cluster_names
  @all_accessible_cluster_names ||=
    # Using `_` because steep doesn't understand that `compact` removes nils.
    (clusters_to_index_into + [_ = cluster_to_query]).compact.uniq.select do |name|
      defined_clusters.include?(name)
    end
end

#cluster_to_query ⇒ Object 



68
69
70
 
# File 'lib/elastic_graph/datastore_core/index_definition/base.rb', line 68

def cluster_to_query
  env_index_config.query_cluster
end

#clusters_to_index_into ⇒ Object 



72
73
74
75
76
 
# File 'lib/elastic_graph/datastore_core/index_definition/base.rb', line 72

def clusters_to_index_into
  env_index_config.index_into_clusters.tap do |clusters_to_index_into|
    raise Errors::ConfigError, "No `index_into_clusters` defined for #{self} in env_index_config" unless clusters_to_index_into
  end
end

#flattened_env_setting_overrides ⇒ Object

Returns any setting overrides for this index from the environment-specific config file, after flattening it so that it can be directly used in a create index request.



21
22
23
24
25
26
 
# File 'lib/elastic_graph/datastore_core/index_definition/base.rb', line 21

def flattened_env_setting_overrides
  @flattened_env_setting_overrides ||= Support::HashUtil.flatten_and_stringify_keys(
    env_index_config.setting_overrides,
    prefix: "index"
  )
end

#has_custom_routing? ⇒ Boolean

Returns:

  • (Boolean) 


45
46
47
 
# File 'lib/elastic_graph/datastore_core/index_definition/base.rb', line 45

def has_custom_routing?
  route_with != "id"
end

#ignored_values_for_routing ⇒ Object 



78
79
80
 
# File 'lib/elastic_graph/datastore_core/index_definition/base.rb', line 78

def ignored_values_for_routing
  env_index_config.ignore_routing_values
end

#known_related_query_rollover_indices ⇒ Object

Returns a list of indices related to this template in the datastore cluster this index definition is configured to query. Note that for performance reasons, this method memoizes the result of querying the datastore for its current list of indices, and as a result the return value may be out of date. If it is absolutely essential that you get an up-to-date list of related indices, use ‘related_rollover_indices(datastore_client`) instead of this method.

Note, however, that indices generally change very rarely (say, monthly or yearly) and as such this will very rarely be out of date, even with the memoization.



115
116
117
118
119
120
121
122
 
# File 'lib/elastic_graph/datastore_core/index_definition/base.rb', line 115

def known_related_query_rollover_indices
  @known_related_query_rollover_indices ||= cluster_to_query&.then do |name|
    # For query purposes, we only want indices that exist. If we return a query that is defined in our configuration
    # but does not exist, and that gets used in a search index expression (even for the purposes of excluding it!),
    # the datastore will return an error.
    related_rollover_indices(datastore_clients_by_name.fetch(name), only_if_exists: true)
  end || []
end

#list_counts_field_paths_for_source(source) ⇒ Object

Returns a set of all of the field paths to subfields of the special ‘LIST_COUNTS_FIELD` that contains the element counts of all list fields. The returned set is filtered based on the provided `source` to only contain the paths of fields that are populated by the given source.



128
129
130
131
 
# File 'lib/elastic_graph/datastore_core/index_definition/base.rb', line 128

def list_counts_field_paths_for_source(source)
  @list_counts_field_paths_for_source ||= {} # : ::Hash[::String, ::Set[::String]]
  @list_counts_field_paths_for_source[source] ||= identify_list_counts_field_paths_for_source(source)
end

#max_result_window ⇒ Object 



49
50
51
52
53
 
# File 'lib/elastic_graph/datastore_core/index_definition/base.rb', line 49

def max_result_window
  @max_result_window ||= flattened_env_setting_overrides.fetch("index.max_result_window") do
    env_agnostic_settings.fetch("index.max_result_window")
  end
end

#routing_value_for_prepared_record(prepared_record, route_with_path: route_with, id_path: "id") ⇒ Object

Gets the routing value for the given ‘prepared_record`. Notably, `prepared_record` must be previously prepared with an `Indexer::RecordPreparer` in order to ensure that it uses internal index field names (to align with `route_with_path`/`route_with` which also use the internal name) rather than the public field name (which can differ).



32
33
34
35
36
37
38
39
40
41
42
43
 
# File 'lib/elastic_graph/datastore_core/index_definition/base.rb', line 32

def routing_value_for_prepared_record(prepared_record, route_with_path: route_with, id_path: "id")
  return nil unless has_custom_routing?

  unless route_with_path
    raise Errors::ConfigError, "`#{self}` uses custom routing, but `route_with_path` is misconfigured (was `nil`)"
  end

  config_routing_value = Support::HashUtil.fetch_value_at_path(prepared_record, route_with_path.split(".")).to_s
  return config_routing_value unless ignored_values_for_routing.include?(config_routing_value)

  Support::HashUtil.fetch_value_at_path(prepared_record, id_path.split(".")).to_s
end

#searches_could_hit_incomplete_docs? ⇒ Boolean

Indicates if a search on this index definition may hit incomplete documents. An incomplete document can occur when multiple event types flow into the same index. An index that has had only one source type can never have incomplete documents, but an index that has had 2 or more sources can have incomplete documents when the “primary” event type hasn’t yet been received for a document.

This case is notable because we need to apply automatic filtering in order to hide documents that are not yet complete.

This flag should be set via ‘has_had_multiple_sources!` in the schema definition when `sourced_from` is used.

Returns:

  • (Boolean) 


64
65
66
 
# File 'lib/elastic_graph/datastore_core/index_definition/base.rb', line 64

def searches_could_hit_incomplete_docs?
  has_had_multiple_sources
end

#to_s ⇒ Object Also known as: inspect 



133
134
135
 
# File 'lib/elastic_graph/datastore_core/index_definition/base.rb', line 133

def to_s
  "#<#{self.class.name} #{name}>"
end