Class: Spacy::Token

Inherits:
Object
  • Object
show all
Defined in:
lib/ruby-spacy.rb

Overview

See also spaCy Python API document for Token.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(py_token) ⇒ Token

It is recommended to use Doc#tokens or Span#tokens methods to create tokens. There is no way to generate a token from scratch but relying on a pre-exising Python Token object.

Parameters:

  • py_token (Object) —

    Python Token object



1067
1068
1069
1070
 
# File 'lib/ruby-spacy.rb', line 1067

def initialize(py_token)
  @py_token = py_token
  @text = @py_token.text
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing(name, *args) ⇒ Object

Methods defined in Python but not wrapped in ruby-spacy can be called by this dynamic method handling mechanism.



1199
1200
1201
 
# File 'lib/ruby-spacy.rb', line 1199

def method_missing(name, *args)
  Spacy.safe_py_send(@py_token, name, args)
end

Instance Attribute Details

#py_token ⇒ Object (readonly)

Returns a Python Token instance accessible via PyCall.

Returns:

  • (Object) —

    a Python Token instance accessible via PyCall



1059
1060
1061
 
# File 'lib/ruby-spacy.rb', line 1059

def py_token
  @py_token
end

#text ⇒ String (readonly)

Returns a string representing the token.

Returns:

  • (String) —

    a string representing the token



1062
1063
1064
 
# File 'lib/ruby-spacy.rb', line 1062

def text
  @text
end

Instance Method Details

#ancestors ⇒ Array<Token>

Returns the token's ancestors.

Returns:

  • (Array<Token>) —

    an array of tokens



1092
1093
1094
 
# File 'lib/ruby-spacy.rb', line 1092

def ancestors
  PyCall::List.call(@py_token.ancestors).map { |ancestor| Token.new(ancestor) }
end

#children ⇒ Array<Token>

Returns a sequence of the token's immediate syntactic children.

Returns:

  • (Array<Token>) —

    an array of tokens



1098
1099
1100
 
# File 'lib/ruby-spacy.rb', line 1098

def children
  PyCall::List.call(@py_token.children).map { |child| Token.new(child) }
end

#dep ⇒ String

Returns the dependency relation by calling dep_' of @py_token` object

Returns:

  • (String) 


1170
1171
1172
 
# File 'lib/ruby-spacy.rb', line 1170

def dep
  @py_token.dep_
end

#ent_type ⇒ String

Returns the named entity type by calling ent_type_' of @py_token` object

Returns:

  • (String) 


1188
1189
1190
 
# File 'lib/ruby-spacy.rb', line 1188

def ent_type
  @py_token.ent_type_
end

#head ⇒ Token

Returns the head token

Returns:



1080
1081
1082
 
# File 'lib/ruby-spacy.rb', line 1080

def head
  Token.new(@py_token.head)
end

#idx ⇒ Integer

Returns the character offset of the token within the parent document.

Returns:

  • (Integer) 


1074
1075
1076
 
# File 'lib/ruby-spacy.rb', line 1074

def idx
  @py_token.idx
end

#instance_variables_to_inspect ⇒ Object 



1207
1208
1209
 
# File 'lib/ruby-spacy.rb', line 1207

def instance_variables_to_inspect
  [:@text]
end

#lang ⇒ String

Returns the language by calling lang_' of @py_token` object

Returns:

  • (String) 


1176
1177
1178
 
# File 'lib/ruby-spacy.rb', line 1176

def lang
  @py_token.lang_
end

#lefts ⇒ Array<Token>

The leftward immediate children of the word in the syntactic dependency parse.

Returns:

  • (Array<Token>) —

    an array of tokens



1104
1105
1106
 
# File 'lib/ruby-spacy.rb', line 1104

def lefts
  PyCall::List.call(@py_token.lefts).map { |token| Token.new(token) }
end

#lemma ⇒ String

Returns the lemma by calling lemma_' of @py_token` object

Returns:

  • (String) 


1140
1141
1142
 
# File 'lib/ruby-spacy.rb', line 1140

def lemma
  @py_token.lemma_
end

#lexeme ⇒ Lexeme

Returns a lexeme object

Returns:



1194
1195
1196
 
# File 'lib/ruby-spacy.rb', line 1194

def lexeme
  Lexeme.new(@py_token.lex)
end

#lower ⇒ String

Returns the lowercase form by calling lower_' of @py_token` object

Returns:

  • (String) 


1146
1147
1148
 
# File 'lib/ruby-spacy.rb', line 1146

def lower
  @py_token.lower_
end

#morphology(hash: true) ⇒ Hash, String

Returns a hash or string of morphological information

Parameters:

  • hash (Boolean) (defaults to: true) —

    if true, a hash will be returned instead of a string

Returns:

  • (Hash, String) 


1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
 
# File 'lib/ruby-spacy.rb', line 1123

def morphology(hash: true)
  if @py_token.has_morph
    morph_analysis = @py_token.morph
    if hash
      morph_analysis.to_dict
    else
      morph_analysis.to_s
    end
  elsif hash
    {}
  else
    ""
  end
end

#pos ⇒ String

Returns the pos by calling pos_' of @py_token` object

Returns:

  • (String) 


1158
1159
1160
 
# File 'lib/ruby-spacy.rb', line 1158

def pos
  @py_token.pos_
end

#respond_to_missing?(sym, include_private = false) ⇒ Boolean

Returns:

  • (Boolean) 


1203
1204
1205
 
# File 'lib/ruby-spacy.rb', line 1203

def respond_to_missing?(sym, include_private = false)
  Spacy.py_hasattr?(@py_token, sym) || super
end

#rights ⇒ Array<Token>

The rightward immediate children of the word in the syntactic dependency parse.

Returns:

  • (Array<Token>) —

    an array of tokens



1110
1111
1112
 
# File 'lib/ruby-spacy.rb', line 1110

def rights
  PyCall::List.call(@py_token.rights).map { |token| Token.new(token) }
end

#shape ⇒ String

Returns the shape (e.g. "Xxxxx") by calling shape_' of @py_token` object

Returns:

  • (String) 


1152
1153
1154
 
# File 'lib/ruby-spacy.rb', line 1152

def shape
  @py_token.shape_
end

#subtree ⇒ Array<Token>

Returns the token in question and the tokens that descend from it.

Returns:

  • (Array<Token>) —

    an array of tokens



1086
1087
1088
 
# File 'lib/ruby-spacy.rb', line 1086

def subtree
  PyCall::List.call(@py_token.subtree).map { |descendant| Token.new(descendant) }
end

#tag ⇒ String

Returns the fine-grained pos by calling tag_' of @py_token` object

Returns:

  • (String) 


1164
1165
1166
 
# File 'lib/ruby-spacy.rb', line 1164

def tag
  @py_token.tag_
end

#to_s ⇒ String

String representation of the token.

Returns:

  • (String) 


1116
1117
1118
 
# File 'lib/ruby-spacy.rb', line 1116

def to_s
  @text
end

#whitespace ⇒ String

Returns the trailing space character if present by calling whitespace_' of @py_token` object

Returns:

  • (String) 


1182
1183
1184
 
# File 'lib/ruby-spacy.rb', line 1182

def whitespace
  @py_token.whitespace_
end