Class: Telnyx::Resources::AI::Embeddings

Inherits:
Object
  • Object
show all
Defined in:
lib/telnyx/resources/ai/embeddings.rb,
lib/telnyx/resources/ai/embeddings/buckets.rb

Overview

Embed documents and perform text searches

Defined Under Namespace

Classes: Buckets

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(client:) ⇒ Embeddings

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of Embeddings.

Parameters:



189
190
191
192
 
# File 'lib/telnyx/resources/ai/embeddings.rb', line 189

def initialize(client:)
  @client = client
  @buckets = Telnyx::Resources::AI::Embeddings::Buckets.new(client: client)
end

Instance Attribute Details

#buckets ⇒ Telnyx::Resources::AI::Embeddings::Buckets (readonly)

Embed documents and perform text searches

Returns:



10
11
12
 
# File 'lib/telnyx/resources/ai/embeddings.rb', line 10

def buckets
  @buckets
end

Instance Method Details

#create(bucket_name:, document_chunk_overlap_size: nil, document_chunk_size: nil, embedding_model: nil, loader: nil, request_options: {}) ⇒ Telnyx::Models::AI::EmbeddingResponse

Perform embedding on a Telnyx Storage Bucket using the a embedding model. The current supported file types are:

  • PDF

  • HTML

  • txt/unstructured text files

  • json

  • csv

  • audio / video (mp3, mp4, mpeg, mpga, m4a, wav, or webm ) - Max of 100mb file size.

Any files not matching the above types will be attempted to be embedded as unstructured text.

This process can be slow, so it runs in the background and the user can check the status of the task using the endpoint ‘/ai/embeddings/task_id`.

**Important Note**: When you update documents in a Telnyx Storage bucket, their associated embeddings are automatically kept up to date. If you add or update a file, it is automatically embedded. If you delete a file, the embeddings are deleted for that particular file.

You can also specify a custom ‘loader` param. Currently the only supported loader value is `intercom` which loads Intercom article jsons as specified by [the Intercom article API](developers.intercom.com/docs/references/rest-api/api.intercom.io/Articles/article/) This loader will split each article into paragraphs and save additional parameters relevant to Intercom docs, such as `article_url` and `heading`. These values will be returned by the `/v2/ai/embeddings/similarity-search` endpoint in the `loader_metadata` field.

Parameters:

Returns:

See Also:



59
60
61
62
63
64
65
66
67
68
 
# File 'lib/telnyx/resources/ai/embeddings.rb', line 59

def create(params)
  parsed, options = Telnyx::AI::EmbeddingCreateParams.dump_request(params)
  @client.request(
    method: :post,
    path: "ai/embeddings",
    body: parsed,
    model: Telnyx::AI::EmbeddingResponse,
    options: options
  )
end

#list(status: nil, request_options: {}) ⇒ Telnyx::Models::AI::EmbeddingListResponse

Retrieve tasks for the user that are either ‘queued`, `processing`, `failed`, `success` or `partial_success` based on the query string. Defaults to `queued` and `processing`.

Parameters:

  • status (Array<String>) —

    List of task statuses i.e. ‘status=queued&status=processing`

  • request_options (Telnyx::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



109
110
111
112
113
114
115
116
117
118
119
 
# File 'lib/telnyx/resources/ai/embeddings.rb', line 109

def list(params = {})
  parsed, options = Telnyx::AI::EmbeddingListParams.dump_request(params)
  query = Telnyx::Internal::Util.encode_query_params(parsed)
  @client.request(
    method: :get,
    path: "ai/embeddings",
    query: query,
    model: Telnyx::Models::AI::EmbeddingListResponse,
    options: options
  )
end

#retrieve(task_id, request_options: {}) ⇒ Telnyx::Models::AI::EmbeddingRetrieveResponse

Check the status of a current embedding task. Will be one of the following:

  • ‘queued` - Task is waiting to be picked up by a worker

  • ‘processing` - The embedding task is running

  • ‘success` - Task completed successfully and the bucket is embedded

  • ‘failure` - Task failed and no files were embedded successfully

  • ‘partial_success` - Some files were embedded successfully, but at least one failed

Parameters:

Returns:

See Also:



87
88
89
90
91
92
93
94
 
# File 'lib/telnyx/resources/ai/embeddings.rb', line 87

def retrieve(task_id, params = {})
  @client.request(
    method: :get,
    path: ["ai/embeddings/%1$s", task_id],
    model: Telnyx::Models::AI::EmbeddingRetrieveResponse,
    options: params[:request_options]
  )
end

#similarity_search(bucket_name:, query:, num_of_docs: nil, request_options: {}) ⇒ Telnyx::Models::AI::EmbeddingSimilaritySearchResponse

Perform a similarity search on a Telnyx Storage Bucket, returning the most similar ‘num_docs` document chunks to the query.

Currently the only available distance metric is cosine similarity which will return a ‘distance` between 0 and 1. The lower the distance, the more similar the returned document chunks are to the query. A `certainty` will also be returned, which is a value between 0 and 1 where the higher the certainty, the more similar the document. You can read more about Weaviate distance metrics here: [Weaviate Docs](weaviate.io/developers/weaviate/config-refs/distances)

If a bucket was embedded using a custom loader, such as ‘intercom`, the additional metadata will be returned in the `loader_metadata` field.

Parameters:

  • bucket_name (String)
  • query (String)
  • num_of_docs (Integer)
  • request_options (Telnyx::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



145
146
147
148
149
150
151
152
153
154
 
# File 'lib/telnyx/resources/ai/embeddings.rb', line 145

def similarity_search(params)
  parsed, options = Telnyx::AI::EmbeddingSimilaritySearchParams.dump_request(params)
  @client.request(
    method: :post,
    path: "ai/embeddings/similarity-search",
    body: parsed,
    model: Telnyx::Models::AI::EmbeddingSimilaritySearchResponse,
    options: options
  )
end

#url(bucket_name:, url:, request_options: {}) ⇒ Telnyx::Models::AI::EmbeddingResponse

Embed website content from a specified URL, including child pages up to 5 levels deep within the same domain. The process crawls and loads content from the main URL and its linked pages into a Telnyx Cloud Storage bucket. As soon as each webpage is added to the bucket, its content is immediately processed for embeddings, that can be used for [similarity search](developers.telnyx.com/api-reference/embeddings/search-for-documents) and [clustering](developers.telnyx.com/docs/inference/clusters).

Parameters:

  • bucket_name (String) —

    Name of the bucket to store the embeddings. This bucket must already exist.

  • url (String) —

    The URL of the webpage to embed

  • request_options (Telnyx::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



175
176
177
178
179
180
181
182
183
184
 
# File 'lib/telnyx/resources/ai/embeddings.rb', line 175

def url(params)
  parsed, options = Telnyx::AI::EmbeddingURLParams.dump_request(params)
  @client.request(
    method: :post,
    path: "ai/embeddings/url",
    body: parsed,
    model: Telnyx::AI::EmbeddingResponse,
    options: options
  )
end