Class: Spacy::Token

Inherits:
Object
  • Object
show all
Defined in:
lib/ruby-spacy.rb

Overview

See also spaCy Python API document for Token.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(py_token) ⇒ Token

It is recommended to use Doc#tokens or Span#tokens methods to create tokens. There is no way to generate a token from scratch but relying on a pre-exising Python Token object.

Parameters:

  • py_token (Object) —

    Python Token object



959
960
961
962
 
# File 'lib/ruby-spacy.rb', line 959

def initialize(py_token)
  @py_token = py_token
  @text = @py_token.text
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing(name, *args) ⇒ Object

Methods defined in Python but not wrapped in ruby-spacy can be called by this dynamic method handling mechanism.



1091
1092
1093
 
# File 'lib/ruby-spacy.rb', line 1091

def method_missing(name, *args)
  @py_token.send(name, *args)
end

Instance Attribute Details

#py_token ⇒ Object (readonly)

Returns a Python Token instance accessible via PyCall.

Returns:

  • (Object) —

    a Python Token instance accessible via PyCall



951
952
953
 
# File 'lib/ruby-spacy.rb', line 951

def py_token
  @py_token
end

#text ⇒ String (readonly)

Returns a string representing the token.

Returns:

  • (String) —

    a string representing the token



954
955
956
 
# File 'lib/ruby-spacy.rb', line 954

def text
  @text
end

Instance Method Details

#ancestors ⇒ Array<Token>

Returns the token's ancestors.

Returns:

  • (Array<Token>) —

    an array of tokens



984
985
986
 
# File 'lib/ruby-spacy.rb', line 984

def ancestors
  PyCall::List.call(@py_token.ancestors).map { |ancestor| Token.new(ancestor) }
end

#children ⇒ Array<Token>

Returns a sequence of the token's immediate syntactic children.

Returns:

  • (Array<Token>) —

    an array of tokens



990
991
992
 
# File 'lib/ruby-spacy.rb', line 990

def children
  PyCall::List.call(@py_token.children).map { |child| Token.new(child) }
end

#dep ⇒ String

Returns the dependency relation by calling dep_' of @py_token` object

Returns:

  • (String) 


1062
1063
1064
 
# File 'lib/ruby-spacy.rb', line 1062

def dep
  @py_token.dep_
end

#ent_type ⇒ String

Returns the named entity type by calling ent_type_' of @py_token` object

Returns:

  • (String) 


1080
1081
1082
 
# File 'lib/ruby-spacy.rb', line 1080

def ent_type
  @py_token.ent_type_
end

#head ⇒ Token

Returns the head token

Returns:



972
973
974
 
# File 'lib/ruby-spacy.rb', line 972

def head
  Token.new(@py_token.head)
end

#idx ⇒ Integer

Returns the character offset of the token within the parent document.

Returns:

  • (Integer) 


966
967
968
 
# File 'lib/ruby-spacy.rb', line 966

def idx
  @py_token.idx
end

#instance_variables_to_inspect ⇒ Object 



1099
1100
1101
 
# File 'lib/ruby-spacy.rb', line 1099

def instance_variables_to_inspect
  [:@text]
end

#lang ⇒ String

Returns the language by calling lang_' of @py_token` object

Returns:

  • (String) 


1068
1069
1070
 
# File 'lib/ruby-spacy.rb', line 1068

def lang
  @py_token.lang_
end

#lefts ⇒ Array<Token>

The leftward immediate children of the word in the syntactic dependency parse.

Returns:

  • (Array<Token>) —

    an array of tokens



996
997
998
 
# File 'lib/ruby-spacy.rb', line 996

def lefts
  PyCall::List.call(@py_token.lefts).map { |token| Token.new(token) }
end

#lemma ⇒ String

Returns the lemma by calling lemma_' of @py_token` object

Returns:

  • (String) 


1032
1033
1034
 
# File 'lib/ruby-spacy.rb', line 1032

def lemma
  @py_token.lemma_
end

#lexeme ⇒ Lexeme

Returns a lexeme object

Returns:



1086
1087
1088
 
# File 'lib/ruby-spacy.rb', line 1086

def lexeme
  Lexeme.new(@py_token.lex)
end

#lower ⇒ String

Returns the lowercase form by calling lower_' of @py_token` object

Returns:

  • (String) 


1038
1039
1040
 
# File 'lib/ruby-spacy.rb', line 1038

def lower
  @py_token.lower_
end

#morphology(hash: true) ⇒ Hash, String

Returns a hash or string of morphological information

Parameters:

  • hash (Boolean) (defaults to: true) —

    if true, a hash will be returned instead of a string

Returns:

  • (Hash, String) 


1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
 
# File 'lib/ruby-spacy.rb', line 1015

def morphology(hash: true)
  if @py_token.has_morph
    morph_analysis = @py_token.morph
    if hash
      morph_analysis.to_dict
    else
      morph_analysis.to_s
    end
  elsif hash
    {}
  else
    ""
  end
end

#pos ⇒ String

Returns the pos by calling pos_' of @py_token` object

Returns:

  • (String) 


1050
1051
1052
 
# File 'lib/ruby-spacy.rb', line 1050

def pos
  @py_token.pos_
end

#respond_to_missing?(sym, include_private = false) ⇒ Boolean

Returns:

  • (Boolean) 


1095
1096
1097
 
# File 'lib/ruby-spacy.rb', line 1095

def respond_to_missing?(sym, include_private = false)
  Spacy.py_hasattr?(@py_token, sym) || super
end

#rights ⇒ Array<Token>

The rightward immediate children of the word in the syntactic dependency parse.

Returns:

  • (Array<Token>) —

    an array of tokens



1002
1003
1004
 
# File 'lib/ruby-spacy.rb', line 1002

def rights
  PyCall::List.call(@py_token.rights).map { |token| Token.new(token) }
end

#shape ⇒ String

Returns the shape (e.g. "Xxxxx") by calling shape_' of @py_token` object

Returns:

  • (String) 


1044
1045
1046
 
# File 'lib/ruby-spacy.rb', line 1044

def shape
  @py_token.shape_
end

#subtree ⇒ Array<Token>

Returns the token in question and the tokens that descend from it.

Returns:

  • (Array<Token>) —

    an array of tokens



978
979
980
 
# File 'lib/ruby-spacy.rb', line 978

def subtree
  PyCall::List.call(@py_token.subtree).map { |descendant| Token.new(descendant) }
end

#tag ⇒ String

Returns the fine-grained pos by calling tag_' of @py_token` object

Returns:

  • (String) 


1056
1057
1058
 
# File 'lib/ruby-spacy.rb', line 1056

def tag
  @py_token.tag_
end

#to_s ⇒ String

String representation of the token.

Returns:

  • (String) 


1008
1009
1010
 
# File 'lib/ruby-spacy.rb', line 1008

def to_s
  @text
end

#whitespace ⇒ String

Returns the trailing space character if present by calling whitespace_' of @py_token` object

Returns:

  • (String) 


1074
1075
1076
 
# File 'lib/ruby-spacy.rb', line 1074

def whitespace
  @py_token.whitespace_
end