Class: Google::Apis::DataprocV1::OrderedJob

Inherits:
Object
  • Object
show all
Includes:
Core::Hashable, Core::JsonObjectSupport
Defined in:
lib/google/apis/dataproc_v1/classes.rb,
lib/google/apis/dataproc_v1/representations.rb,
lib/google/apis/dataproc_v1/representations.rb

Overview

A job executed by the workflow.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(**args) ⇒ OrderedJob

Returns a new instance of OrderedJob.



5891
5892
5893
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 5891

def initialize(**args)
   update!(**args)
end

Instance Attribute Details

A Dataproc job for running Apache Flink applications on YARN. Corresponds to the JSON property flinkJob

Returns:



5799
5800
5801
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 5799

def flink_job
  @flink_job
end

#hadoop_job ⇒ Google::Apis::DataprocV1::HadoopJob

A Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/ docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/ MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/ docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). Corresponds to the JSON property hadoopJob

Returns:



5807
5808
5809
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 5807

def hadoop_job
  @hadoop_job
end

#hive_job ⇒ Google::Apis::DataprocV1::HiveJob

A Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN. Corresponds to the JSON property hiveJob

Returns:



5813
5814
5815
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 5813

def hive_job
  @hive_job
end

#labels ⇒ Hash<String,String>

Optional. The labels to associate with this job.Label keys must be between 1 and 63 characters long, and must conform to the following regular expression: \ pLl\pLo0,62Label values must be between 1 and 63 characters long, and must conform to the following regular expression: \pLl\pLo\pN_-0,63No more than 32 labels can be associated with a given job. Corresponds to the JSON property labels

Returns:

  • (Hash<String,String>) 


5822
5823
5824
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 5822

def labels
  @labels
end

#pig_job ⇒ Google::Apis::DataprocV1::PigJob

A Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN. Corresponds to the JSON property pigJob

Returns:



5828
5829
5830
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 5828

def pig_job
  @pig_job
end

#prerequisite_step_ids ⇒ Array<String>

Optional. The optional list of prerequisite job step_ids. If not specified, the job will start at the beginning of workflow. Corresponds to the JSON property prerequisiteStepIds

Returns:

  • (Array<String>) 


5834
5835
5836
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 5834

def prerequisite_step_ids
  @prerequisite_step_ids
end

#presto_job ⇒ Google::Apis::DataprocV1::PrestoJob

A Dataproc job for running Presto (https://prestosql.io/) queries. IMPORTANT: The Dataproc Presto Optional Component (https://cloud.google.com/dataproc/docs/ concepts/components/presto) must be enabled when the cluster is created to submit a Presto job to the cluster. Corresponds to the JSON property prestoJob

Returns:



5842
5843
5844
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 5842

def presto_job
  @presto_job
end

#pyspark_job ⇒ Google::Apis::DataprocV1::PySparkJob

A Dataproc job for running Apache PySpark (https://spark.apache.org/docs/ latest/api/python/index.html#pyspark-overview) applications on YARN. Corresponds to the JSON property pysparkJob

Returns:



5848
5849
5850
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 5848

def pyspark_job
  @pyspark_job
end

#scheduling ⇒ Google::Apis::DataprocV1::JobScheduling

Job scheduling options. Corresponds to the JSON property scheduling

Returns:



5853
5854
5855
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 5853

def scheduling
  @scheduling
end

#spark_job ⇒ Google::Apis::DataprocV1::SparkJob

A Dataproc job for running Apache Spark (https://spark.apache.org/) applications on YARN. Corresponds to the JSON property sparkJob

Returns:



5859
5860
5861
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 5859

def spark_job
  @spark_job
end

#spark_r_job ⇒ Google::Apis::DataprocV1::SparkRJob

A Dataproc job for running Apache SparkR (https://spark.apache.org/docs/latest/ sparkr.html) applications on YARN. Corresponds to the JSON property sparkRJob

Returns:



5865
5866
5867
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 5865

def spark_r_job
  @spark_r_job
end

#spark_sql_job ⇒ Google::Apis::DataprocV1::SparkSqlJob

A Dataproc job for running Apache Spark SQL (https://spark.apache.org/sql/) queries. Corresponds to the JSON property sparkSqlJob

Returns:



5871
5872
5873
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 5871

def spark_sql_job
  @spark_sql_job
end

#step_id ⇒ String

Required. The step id. The id must be unique among all jobs within the template.The step id is used as prefix for job id, as job goog-dataproc- workflow-step-id label, and in prerequisiteStepIds field from other steps.The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). Cannot begin or end with underscore or hyphen. Must consist of between 3 and 50 characters. Corresponds to the JSON property stepId

Returns:

  • (String) 


5881
5882
5883
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 5881

def step_id
  @step_id
end

#trino_job ⇒ Google::Apis::DataprocV1::TrinoJob

A Dataproc job for running Trino (https://trino.io/) queries. IMPORTANT: The Dataproc Trino Optional Component (https://cloud.google.com/dataproc/docs/ concepts/components/trino) must be enabled when the cluster is created to submit a Trino job to the cluster. Corresponds to the JSON property trinoJob

Returns:



5889
5890
5891
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 5889

def trino_job
  @trino_job
end

Instance Method Details

#update!(**args) ⇒ Object

Update properties of this object



5896
5897
5898
5899
5900
5901
5902
5903
5904
5905
5906
5907
5908
5909
5910
5911
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 5896

def update!(**args)
  @flink_job = args[:flink_job] if args.key?(:flink_job)
  @hadoop_job = args[:hadoop_job] if args.key?(:hadoop_job)
  @hive_job = args[:hive_job] if args.key?(:hive_job)
  @labels = args[:labels] if args.key?(:labels)
  @pig_job = args[:pig_job] if args.key?(:pig_job)
  @prerequisite_step_ids = args[:prerequisite_step_ids] if args.key?(:prerequisite_step_ids)
  @presto_job = args[:presto_job] if args.key?(:presto_job)
  @pyspark_job = args[:pyspark_job] if args.key?(:pyspark_job)
  @scheduling = args[:scheduling] if args.key?(:scheduling)
  @spark_job = args[:spark_job] if args.key?(:spark_job)
  @spark_r_job = args[:spark_r_job] if args.key?(:spark_r_job)
  @spark_sql_job = args[:spark_sql_job] if args.key?(:spark_sql_job)
  @step_id = args[:step_id] if args.key?(:step_id)
  @trino_job = args[:trino_job] if args.key?(:trino_job)
end