Class: OpenSearch::Sugar::Models

Inherits:
Object
  • Object
show all
Defined in:
lib/opensearch/sugar/models.rb,
sig/opensearch/sugar.rbs

Overview

Manages ML models deployed via the OpenSearch ML Commons plugin.

Provides methods to register, deploy, list, and delete models, as well as to build text-embedding ingest pipelines that use a deployed model.

Access an instance via Client#models:

Examples:

models = client.models
model  = models.register(name: "all-MiniLM-L6-v2", version: "1.0.0")
puts model.id

Defined Under Namespace

Classes: ML_INFO

Instance Method Summary collapse

Constructor Details

#initialize(os) ⇒ Models

Returns a new instance of Models.

Parameters:



26
27
28
 
# File 'lib/opensearch/sugar/models.rb', line 26

def initialize(os)
  @os = os
end

Instance Method Details

#[](id_or_fullname_or_nickname) ⇒ ML_INFO?

Looks up a model by exact name, exact id, or case-insensitive partial name.

Parameters:

  • id_or_fullname_or_nickname (String)

Returns:



80
81
82
83
84
85
86
87
88
89
90
91
 
# File 'lib/opensearch/sugar/models.rb', line 80

def [](id_or_fullname_or_nickname)
  mlm = list
  name = mlm.find { |x| x.name == id_or_fullname_or_nickname }
  return name if name

  id = mlm.find { |m| m.id == id_or_fullname_or_nickname }
  return id if id

  nickname_pattern = Regexp.new(id_or_fullname_or_nickname, "i")
  nicks = mlm.find_all { |m| nickname_pattern.match(m.name) }.sort { |a, b| b.version <=> a.version }
  nicks.first # could be nil
end

#create_pipeline(name:, model:, description:, field_map:) ⇒ Object

Creates an ingest pipeline for text-embedding using the named model.

Parameters:

  • name: (String)
  • model: (String)
  • description: (String)
  • field_map: (Hash[String, String])

Returns:

  • (Object) 


175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
 
# File 'lib/opensearch/sugar/models.rb', line 175

def create_pipeline(name:, model:, description:, field_map:)
  m = self[model]
  raise "Can't find model #{model}" unless m
  url = "/_ingest/pipeline/#{name.gsub(/\s+/, " ").gsub(/\s+/, "_")}"
  field_map_to_temp = field_map.transform_values { |actual_target| "#{actual_target}_temp" }
  temp_to_field_map = field_map.values.each_with_object({}) do |actual_target, h|
    h["#{actual_target}_temp"] = actual_target
  end

  payload = {
    description: description,
    processors: [
      {
        text_embedding: {
          model_id: m.id,
          field_map: field_map_to_temp
        }
      }
    ]
  }

  temp_to_field_map.each_pair do |tmp, real|
    payload[:processors] << {
      copy: {
        source_field: "#{tmp}.knn",
        target_field: real,
        ignore_missing: true,
        remove_source: true
      }
    }
  end
  @os.http.put(url, body: payload)
end

#delete!(name_or_id) ⇒ Hash

Undeploys and permanently deletes a model from the cluster.

Examples:

client.models.delete!("all-MiniLM-L6-v2")

Parameters:

  • name_or_id (String) —

    Model name, ID, or partial name accepted by #[]

Returns:

  • (Hash) —

    The OpenSearch delete response

Raises:

  • (NoMethodError) —

    If no model matching name_or_id is found



137
138
139
140
141
 
# File 'lib/opensearch/sugar/models.rb', line 137

def delete!(name_or_id)
  m = self[name_or_id]
  undeploy!(m.id)
  @os.http.delete("/_plugins/_ml/models/#{m.id}")
end

#delete_pipeline!(pipeline_name) ⇒ Object

Deletes an ingest pipeline by name.

Parameters:

  • pipeline_name (String)

Returns:

  • (Object) 


149
150
151
 
# File 'lib/opensearch/sugar/models.rb', line 149

def delete_pipeline!(pipeline_name)
  @os.ingest.delete_pipeline(id: pipeline_name)
end

#list ⇒ Array<ML_INFO>

Returns all deployed ML models in the cluster.

Examples:

client.models.list.each { |m| puts "#{m.name} #{m.version} (#{m.id})" }

Returns:

  • (Array<ML_INFO>) —

    Unique name/version/id triples for all deployed models



98
99
100
101
102
103
104
 
# File 'lib/opensearch/sugar/models.rb', line 98

def list
  lst = raw_list.dig("hits", "hits").map { |x| x["_source"] }.each_with_object([]) do |ml, a|
    model = ML_INFO.new(ml["name"], ml["model_version"], ml["model_id"])
    a << model
  end
  lst.uniq
end

#raw_list ⇒ Hash

Returns the raw OpenSearch response from the ML models search endpoint.

This is the unprocessed response from /_plugins/_ml/models/_search, filtered to chunk 0 to avoid returning embedding chunks. Prefer #list or #[] for normal usage.

Returns:

  • (Hash) —

    Raw OpenSearch search response



113
114
115
116
 
# File 'lib/opensearch/sugar/models.rb', line 113

def raw_list
  @os.http.get("/_plugins/_ml/models/_search",
    body: {"query" => {"term" => {"chunk_number" => 0}}})
end

#register(name:, version:, format: "TORCH_SCRIPT") ⇒ ML_INFO? Also known as: deploy

Registers and deploys an ML model. Idempotent — returns existing if found.

Parameters:

  • name: (String)
  • version: (String)
  • format: (String) (defaults to: "TORCH_SCRIPT")

Returns:



45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
 
# File 'lib/opensearch/sugar/models.rb', line 45

def register(name:, version:, format: "TORCH_SCRIPT")
  config = {
    name: name,
    version: version,
    model_format: format
  }

  current = self[name]
  return current if current

  resp = @os.http.post("/_plugins/_ml/models/_register?deploy=true", body: config)
  taskid = resp["task_id"]
  loop do
    model_install_response = @os.http.get("_plugins/_ml/tasks/#{taskid}")
    break if model_install_response["state"] == "COMPLETED"
    raise model_install_response["error"].to_s if model_install_response["state"] == "FAILED"
    sleep(5)
  end
  self[name]
end

#undeploy!(name_or_id) ⇒ Hash

Undeploys (unloads from memory) a model without deleting its registration.

Examples:

client.models.undeploy!("all-MiniLM-L6-v2")

Parameters:

  • name_or_id (String) —

    Model name, ID, or partial name accepted by #[]

Returns:

  • (Hash) —

    The OpenSearch undeploy response

Raises:

  • (NoMethodError) —

    If no model matching name_or_id is found



125
126
127
128
 
# File 'lib/opensearch/sugar/models.rb', line 125

def undeploy!(name_or_id)
  m = self[name_or_id]
  @os.http.post("/_plugins/_ml/models/#{m.id}/_undeploy")
end