Class: Google::Apis::BigqueryV2::SparkOptions

Inherits:
Object
  • Object
show all
Includes:
Core::Hashable, Core::JsonObjectSupport
Defined in:
lib/google/apis/bigquery_v2/classes.rb,
lib/google/apis/bigquery_v2/representations.rb,
lib/google/apis/bigquery_v2/representations.rb

Overview

Options for a user-defined Spark routine.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(**args) ⇒ SparkOptions

Returns a new instance of SparkOptions.



9869
9870
9871
 
# File 'lib/google/apis/bigquery_v2/classes.rb', line 9869

def initialize(**args)
   update!(**args)
end

Instance Attribute Details

#archive_uris ⇒ Array<String>

Archive files to be extracted into the working directory of each executor. For more information about Apache Spark, see Apache Spark. Corresponds to the JSON property archiveUris

Returns:

  • (Array<String>) 


9806
9807
9808
 
# File 'lib/google/apis/bigquery_v2/classes.rb', line 9806

def archive_uris
  @archive_uris
end

#connection ⇒ String

Fully qualified name of the user-provided Spark connection object. Format: " projects/`project_id`/locations/`location_id`/connections/`connection_id`" Corresponds to the JSON property connection

Returns:

  • (String) 


9812
9813
9814
 
# File 'lib/google/apis/bigquery_v2/classes.rb', line 9812

def connection
  @connection
end

#container_image ⇒ String

Custom container image for the runtime environment. Corresponds to the JSON property containerImage

Returns:

  • (String) 


9817
9818
9819
 
# File 'lib/google/apis/bigquery_v2/classes.rb', line 9817

def container_image
  @container_image
end

#file_uris ⇒ Array<String>

Files to be placed in the working directory of each executor. For more information about Apache Spark, see Apache Spark. Corresponds to the JSON property fileUris

Returns:

  • (Array<String>) 


9824
9825
9826
 
# File 'lib/google/apis/bigquery_v2/classes.rb', line 9824

def file_uris
  @file_uris
end

#jar_uris ⇒ Array<String>

JARs to include on the driver and executor CLASSPATH. For more information about Apache Spark, see Apache Spark. Corresponds to the JSON property jarUris

Returns:

  • (Array<String>) 


9831
9832
9833
 
# File 'lib/google/apis/bigquery_v2/classes.rb', line 9831

def jar_uris
  @jar_uris
end

#main_class ⇒ String

The fully qualified name of a class in jar_uris, for example, com.example. wordcount. Exactly one of main_class and main_jar_uri field should be set for Java/Scala language type. Corresponds to the JSON property mainClass

Returns:

  • (String) 


9838
9839
9840
 
# File 'lib/google/apis/bigquery_v2/classes.rb', line 9838

def main_class
  @main_class
end

#main_file_uri ⇒ String

The main file/jar URI of the Spark application. Exactly one of the definition_body field and the main_file_uri field must be set for Python. Exactly one of main_class and main_file_uri field should be set for Java/Scala language type. Corresponds to the JSON property mainFileUri

Returns:

  • (String) 


9846
9847
9848
 
# File 'lib/google/apis/bigquery_v2/classes.rb', line 9846

def main_file_uri
  @main_file_uri
end

#properties ⇒ Hash<String,String>

Configuration properties as a set of key/value pairs, which will be passed on to the Spark application. For more information, see Apache Spark and the procedure option list. Corresponds to the JSON property properties

Returns:

  • (Hash<String,String>) 


9855
9856
9857
 
# File 'lib/google/apis/bigquery_v2/classes.rb', line 9855

def properties
  @properties
end

#py_file_uris ⇒ Array<String>

Python files to be placed on the PYTHONPATH for PySpark application. Supported file types: .py, .egg, and .zip. For more information about Apache Spark, see Apache Spark. Corresponds to the JSON property pyFileUris

Returns:

  • (Array<String>) 


9862
9863
9864
 
# File 'lib/google/apis/bigquery_v2/classes.rb', line 9862

def py_file_uris
  @py_file_uris
end

#runtime_version ⇒ String

Runtime version. If not specified, the default runtime version is used. Corresponds to the JSON property runtimeVersion

Returns:

  • (String) 


9867
9868
9869
 
# File 'lib/google/apis/bigquery_v2/classes.rb', line 9867

def runtime_version
  @runtime_version
end

Instance Method Details

#update!(**args) ⇒ Object

Update properties of this object



9874
9875
9876
9877
9878
9879
9880
9881
9882
9883
9884
9885
 
# File 'lib/google/apis/bigquery_v2/classes.rb', line 9874

def update!(**args)
  @archive_uris = args[:archive_uris] if args.key?(:archive_uris)
  @connection = args[:connection] if args.key?(:connection)
  @container_image = args[:container_image] if args.key?(:container_image)
  @file_uris = args[:file_uris] if args.key?(:file_uris)
  @jar_uris = args[:jar_uris] if args.key?(:jar_uris)
  @main_class = args[:main_class] if args.key?(:main_class)
  @main_file_uri = args[:main_file_uri] if args.key?(:main_file_uri)
  @properties = args[:properties] if args.key?(:properties)
  @py_file_uris = args[:py_file_uris] if args.key?(:py_file_uris)
  @runtime_version = args[:runtime_version] if args.key?(:runtime_version)
end