Module: ElasticGraph::DatastoreCore::IndexConfigNormalizer

Defined in:
lib/elastic_graph/datastore_core/index_config_normalizer.rb

Constant Summary collapse

READ_ONLY_SETTINGS =

These are settings that the datastore exposes when you fetch an index, but that you can never set. We need to ignore them when figuring out what settings to update.

Note: ‘index.routing.allocation.include._tier_preference` is not a read-only setting, but we want to treat it as one, because (1) Elasticsearch 7.10+ sets it and (2) we do not want to ever write it at this time.

Note: ‘index.history.uuid` is a weird setting that sometimes shows up in managed AWS OpenSearch clusters, but only on some indices. It’s not documented and we don’t want to mess with it here, so we want to treat it as a read only setting.

Note: ‘index.resize.source.name`, `index.resize.source.uuid`, and `index.routing.allocation.initial_recovery._id` are set by the datastore on indices produced by a shrink/split/clone. They are exposed on read but rejected on write (attempting to write them returns `illegal_argument_exception: unknown setting`), so we must treat them as read-only to avoid the update loop trying to clear them by writing null.

%w[
  index.creation_date
  index.history.uuid
  index.provided_name
  index.replication.type
  index.resize.source.name
  index.resize.source.uuid
  index.routing.allocation.include._tier_preference
  index.routing.allocation.initial_recovery._id
  index.uuid
  index.version.created
  index.version.upgraded
]

Class Method Summary collapse

Class Method Details

.normalize(index_config) ⇒ Object

Normalizes the provided index configuration so that it is in a stable form that we can compare to what the datastore returns when we query it for the configuration of an index. This includes:

  • Dropping read-only settings that we never interact with but that the datastore automatically sets on an index. Omitting them makes it easier for us to compare our desired configuration to what is in the datastore.

  • Converting setting values to a normalized string form. The datastore oddly returns setting values as strings (e.g. ‘“false”` or `“7”` instead of `false` or `7`), so this matches that behavior.

  • Drops ‘type: object` from a mapping when there are `properties` because the datastore omits it in that situation, treating it as the default type.



51
52
53
54
55
56
57
58
59
60
61
 
# File 'lib/elastic_graph/datastore_core/index_config_normalizer.rb', line 51

def self.normalize(index_config)
  if (settings = index_config["settings"])
    index_config = index_config.merge("settings" => normalize_settings(settings))
  end

  if (mappings = index_config["mappings"])
    index_config = index_config.merge("mappings" => normalize_mappings(mappings))
  end

  index_config
end

.normalize_mappings(mappings) ⇒ Object 



63
64
65
66
67
68
 
# File 'lib/elastic_graph/datastore_core/index_config_normalizer.rb', line 63

def self.normalize_mappings(mappings)
  return mappings unless (properties = mappings["properties"])

  mappings = mappings.except("type") if mappings["type"] == "object"
  mappings.merge("properties" => properties.transform_values { |prop| normalize_mappings(prop) })
end

.normalize_settings(settings) ⇒ Object 



70
71
72
73
74
 
# File 'lib/elastic_graph/datastore_core/index_config_normalizer.rb', line 70

def self.normalize_settings(settings)
  settings
    .except(*READ_ONLY_SETTINGS)
    .to_h { |name, value| [name, normalize_setting_value(value)] }
end