Class: RubyLLM::Tokenizer::Analysis

Inherits:

Struct

• Object
• Struct
• RubyLLM::Tokenizer::Analysis

Defined in:

lib/ruby_llm/tokenizer/analysis.rb

Instance Attribute Summary

• #ids ⇒ Object

  Returns the value of attribute ids.

• #model ⇒ Object

  Returns the value of attribute model.

• #tokens ⇒ Object

  Returns the value of attribute tokens.

Instance Method Summary

• #count ⇒ Object
• #to_h ⇒ Object

Instance Attribute Details

#ids ⇒ Object

Returns the value of attribute ids

Returns:

• (Object) —

  the current value of ids

# File 'lib/ruby_llm/tokenizer/analysis.rb', line 5

def ids
  @ids
end

#model ⇒ Object

Returns the value of attribute model

Returns:

• (Object) —

  the current value of model

# File 'lib/ruby_llm/tokenizer/analysis.rb', line 5

def model
  @model
end

#tokens ⇒ Object

Returns the value of attribute tokens

Returns:

• (Object) —

  the current value of tokens

# File 'lib/ruby_llm/tokenizer/analysis.rb', line 5

def tokens
  @tokens
end

Instance Method Details

#count ⇒ Object

# File 'lib/ruby_llm/tokenizer/analysis.rb', line 6

def count
  ids.size
end

#to_h ⇒ Object

# File 'lib/ruby_llm/tokenizer/analysis.rb', line 10

def to_h
  { tokens: tokens, ids: ids, model: model, count: count }
end