Class: VectorAmp::Dataset

Inherits:
Object
  • Object
show all
Defined in:
lib/vector_amp/dataset.rb

Overview

Rich dataset resource returned by DatasetsResource#create/get/list.

It keeps the raw API payload while adding convenient instance methods that delegate to the existing service-style APIs.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(data, service:, client: nil) ⇒ Dataset

Parameters:

  • data (Hash) —

    dataset API payload; id or dataset_id is required.

  • service (DatasetsResource) —

    backing dataset service.

  • client (Client, nil) (defaults to: nil) —

    optional client for convenience helpers.

Raises:

  • (ArgumentError) 


20
21
22
23
24
25
26
 
# File 'lib/vector_amp/dataset.rb', line 20

def initialize(data, service:, client: nil)
  @data = normalize_data(data)
  @service = service
  @client = client
  @id = extract_id(@data)
  raise ArgumentError, "dataset id is required" if @id.nil? || @id.to_s.empty?
end

Instance Attribute Details

#client ⇒ DatasetsResource, ... (readonly)

Returns:

  • (DatasetsResource) —

    backing dataset service.

  • (Client, nil) —

    client that created this object, required for convenience ingestion/ask helpers.

  • (String) —

    dataset id.

  • (Hash) —

    normalized raw API payload.



13
14
15
 
# File 'lib/vector_amp/dataset.rb', line 13

def client
  @client
end

#data ⇒ DatasetsResource, ... (readonly) Also known as: raw_data

Returns:

  • (DatasetsResource) —

    backing dataset service.

  • (Client, nil) —

    client that created this object, required for convenience ingestion/ask helpers.

  • (String) —

    dataset id.

  • (Hash) —

    normalized raw API payload.



13
14
15
 
# File 'lib/vector_amp/dataset.rb', line 13

def data
  @data
end

#id ⇒ DatasetsResource, ... (readonly)

Returns:

  • (DatasetsResource) —

    backing dataset service.

  • (Client, nil) —

    client that created this object, required for convenience ingestion/ask helpers.

  • (String) —

    dataset id.

  • (Hash) —

    normalized raw API payload.



13
14
15
 
# File 'lib/vector_amp/dataset.rb', line 13

def id
  @id
end

#service ⇒ DatasetsResource, ... (readonly)

Returns:

  • (DatasetsResource) —

    backing dataset service.

  • (Client, nil) —

    client that created this object, required for convenience ingestion/ask helpers.

  • (String) —

    dataset id.

  • (Hash) —

    normalized raw API payload.



13
14
15
 
# File 'lib/vector_amp/dataset.rb', line 13

def service
  @service
end

Instance Method Details

#[](key) ⇒ Object?

Read a raw dataset field by string or symbol key.

Parameters:

  • key (String, Symbol) —

    field name.

Returns:

  • (Object, nil) 


31
32
33
 
# File 'lib/vector_amp/dataset.rb', line 31

def [](key)
  @data[key.to_s]
end

#add_texts(texts_arg = nil, texts: nil, ids: nil, metadata: nil) ⇒ Hash

Embed and insert texts into this dataset.

Parameters:

  • texts_arg (Array<String>, nil) (defaults to: nil) —

    positional texts for convenience.

  • texts (Array<String>, nil) (defaults to: nil) —

    keyword texts.

  • ids (Array<String>, nil) (defaults to: nil) —

    optional ids; generated UUIDs when omitted.

  • metadata (Hash, Array<Hash>, nil) (defaults to: nil) —

    metadata applied to all texts or per-text.

Returns:

  • (Hash) —

    insert response.



80
81
82
 
# File 'lib/vector_amp/dataset.rb', line 80

def add_texts(texts_arg = nil, texts: nil, ids: nil, metadata: nil)
  service.add_texts(id, texts_arg, texts: texts, ids: ids, metadata: metadata)
end

#ask(query, **options) ⇒ Hash

Ask an intelligence question constrained to this dataset.

Parameters:

  • query (String) —

    natural-language question.

  • options (Hash) —

    forwarded to Client#ask; dataset_id is set to this dataset id.

Returns:

  • (Hash) —

    intelligence response.



117
118
119
120
 
# File 'lib/vector_amp/dataset.rb', line 117

def ask(query, **options)
  require_client!("ask")
  client.ask(query, **options.merge(dataset_id: id))
end

#delete ⇒ Hash

Delete this dataset.

Returns:

  • (Hash) —

    delete response.



86
87
88
 
# File 'lib/vector_amp/dataset.rb', line 86

def delete
  service.delete(id)
end

#download_document(document_id) ⇒ String

Download retained original bytes for a dataset source document.

Parameters:

Returns:

  • (String) —

    raw document bytes.



109
110
111
 
# File 'lib/vector_amp/dataset.rb', line 109

def download_document(document_id)
  service.download_document(id, document_id)
end

#fetch(key, *args, &block) ⇒ Object

Fetch a raw dataset field by string or symbol key.

Parameters:

  • key (String, Symbol) —

    field name.

Returns:

  • (Object) 


38
39
40
 
# File 'lib/vector_amp/dataset.rb', line 38

def fetch(key, *args, &block)
  @data.fetch(key.to_s, *args, &block)
end

#ingest_files(paths:, source_name: nil, description: nil, metadata: {}) ⇒ Hash

Upload local files by auto-creating a file_upload source, initializing presigned uploads, and completing the upload job.

Parameters:

  • paths (String, Array<String>) —

    local file paths to upload.

  • source_name (String, nil) (defaults to: nil) —

    optional source name; defaults to timestamped Ruby SDK file-upload name.

  • description (String, nil) (defaults to: nil) —

    optional source description.

  • metadata (Hash) (defaults to: {}) —

    optional source metadata; dataset_id is added automatically.

Returns:

  • (Hash) —

    upload completion/job response.



128
129
130
131
132
133
134
135
136
137
 
# File 'lib/vector_amp/dataset.rb', line 128

def ingest_files(paths:, source_name: nil, description: nil, metadata: {})
  require_client!("ingest_files")
  client.ingestion.ingest_files(
    dataset_id: id,
    paths: paths,
    source_name: source_name,
    description: description,
    metadata: metadata
  )
end

#ingest_source(source_id = nil, source: nil, pipeline_id: nil) ⇒ Hash

Start an ingestion job from an existing source into this dataset.

Parameters:

  • source_id (String, Source, Hash, nil) (defaults to: nil) —

    source id or object/hash containing an id.

  • source (Source, Hash, nil) (defaults to: nil) —

    alternate source object/hash containing an id.

  • pipeline_id (String, nil) (defaults to: nil) —

    optional pipeline id.

Returns:

  • (Hash) —

    ingestion job response.

Raises:

  • (ArgumentError) 


144
145
146
147
148
149
150
 
# File 'lib/vector_amp/dataset.rb', line 144

def ingest_source(source_id = nil, source: nil, pipeline_id: nil)
  require_client!("ingest_source")
  resolved_source_id = extract_source_id(source_id || source)
  raise ArgumentError, "source_id is required" if resolved_source_id.nil? || resolved_source_id.to_s.empty?

  client.ingestion.start_job(source_id: resolved_source_id, dataset_id: id, pipeline_id: pipeline_id)
end

#insert(vectors:) ⇒ Hash

Insert vectors into this dataset.

Parameters:

  • vectors (Array<Hash>) —

    vector records with ids, values, and optional metadata.

Returns:

  • (Hash) —

    insert response.



70
71
72
 
# File 'lib/vector_amp/dataset.rb', line 70

def insert(vectors:)
  service.insert(id, vectors: vectors)
end

#inspect ⇒ Object 



55
56
57
 
# File 'lib/vector_amp/dataset.rb', line 55

def inspect
  "#<#{self.class} id=#{id.inspect} data=#{@data.inspect}>"
end

#key?(key) ⇒ Boolean Also known as: has_key?

Returns whether the raw payload has the field.

Parameters:

  • key (String, Symbol) —

    field name.

Returns:

  • (Boolean) —

    whether the raw payload has the field.



44
45
46
 
# File 'lib/vector_amp/dataset.rb', line 44

def key?(key)
  @data.key?(key.to_s)
end

#list_documents(limit: 50, cursor: nil, status: nil) ⇒ Hash

List retained source documents for this dataset using cursor pagination.

Parameters:

  • limit (Integer, nil) (defaults to: 50) —

    maximum documents to return.

  • cursor (String, nil) (defaults to: nil) —

    cursor from a previous response's next_cursor.

  • status (String, nil) (defaults to: nil) —

    optional document status filter.

Returns:

  • (Hash) —

    response envelope with documents and next_cursor.



102
103
104
 
# File 'lib/vector_amp/dataset.rb', line 102

def list_documents(limit: 50, cursor: nil, status: nil)
  service.list_documents(id, limit: limit, cursor: cursor, status: status)
end

#search(query_text = nil, **options) ⇒ Hash

Search this dataset.

Parameters:

  • query_text (String, nil) (defaults to: nil) —

    optional text query; alternatively pass query: vector/options.

  • options (Hash) —

    forwarded to VectorAmp::DatasetsResource#search.

Returns:

  • (Hash) —

    search response.



63
64
65
 
# File 'lib/vector_amp/dataset.rb', line 63

def search(query_text = nil, **options)
  service.search(id, query_text, **options)
end

#stats ⇒ Hash

Fetch stats for this dataset.

Returns:

  • (Hash) —

    dataset statistics.



92
93
94
 
# File 'lib/vector_amp/dataset.rb', line 92

def stats
  service.stats(id)
end

#to_h ⇒ Hash Also known as: to_hash

Returns shallow copy of the raw API payload.

Returns:

  • (Hash) —

    shallow copy of the raw API payload.



50
51
52
 
# File 'lib/vector_amp/dataset.rb', line 50

def to_h
  @data.dup
end