Class: Google::Cloud::Dataflow::V1beta3::Environment

Inherits:
Object
  • Object
show all
Extended by:
Protobuf::MessageExts::ClassMethods
Includes:
Protobuf::MessageExts
Defined in:
proto_docs/google/dataflow/v1beta3/environment.rb

Overview

Describes the environment in which a Dataflow Job runs.

Instance Attribute Summary collapse

Instance Attribute Details

#cluster_manager_api_service ⇒ ::String

Returns The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com".

Returns:

  • (::String) —

    The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com".



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#dataset ⇒ ::String

Returns Optional. The dataset for the current project where various workflow related tables are stored.

The supported resource type is:

Google BigQuery: bigquery.googleapis.com/{dataset}.

Returns:

  • (::String) —

    Optional. The dataset for the current project where various workflow related tables are stored.

    The supported resource type is:

    Google BigQuery: bigquery.googleapis.com/{dataset}



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#debug_options ⇒ ::Google::Cloud::Dataflow::V1beta3::DebugOptions

Returns Optional. Any debugging options to be supplied to the job.

Returns:



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#experiments ⇒ ::Array<::String>

Returns The list of experiments to enable. This field should be used for SDK related experiments and not for service related experiments. The proper field for service related experiments is service_options.

Returns:

  • (::Array<::String>) —

    The list of experiments to enable. This field should be used for SDK related experiments and not for service related experiments. The proper field for service related experiments is service_options.



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#flex_resource_scheduling_goal ⇒ ::Google::Cloud::Dataflow::V1beta3::FlexResourceSchedulingGoal

Returns Optional. Which Flexible Resource Scheduling mode to run in.

Returns:



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#internal_experiments ⇒ ::Google::Protobuf::Any

Returns Experimental settings.

Returns:



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#sdk_pipeline_options ⇒ ::Google::Protobuf::Struct

Returns The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way.

Returns:

  • (::Google::Protobuf::Struct) —

    The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way.



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#service_account_email ⇒ ::String

Returns Optional. Identity to run virtual machines as. Defaults to the default account.

Returns:

  • (::String) —

    Optional. Identity to run virtual machines as. Defaults to the default account.



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#service_kms_key_name ⇒ ::String

Returns Optional. If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK).

Format: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY.

Returns:

  • (::String) —

    Optional. If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK).

    Format: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#service_options ⇒ ::Array<::String>

Returns Optional. The list of service options to enable. This field should be used for service related experiments only. These experiments, when graduating to GA, should be replaced by dedicated fields or become default (i.e. always on).

Returns:

  • (::Array<::String>) —

    Optional. The list of service options to enable. This field should be used for service related experiments only. These experiments, when graduating to GA, should be replaced by dedicated fields or become default (i.e. always on).



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#shuffle_mode ⇒ ::Google::Cloud::Dataflow::V1beta3::ShuffleMode (readonly)

Returns Output only. The shuffle mode used for the job.

Returns:



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#streaming_mode ⇒ ::Google::Cloud::Dataflow::V1beta3::StreamingMode

Returns Optional. Specifies the Streaming Engine message processing guarantees. Reduces cost and latency but might result in duplicate messages committed to storage. Designed to run simple mapping streaming ETL jobs at the lowest cost. For example, Change Data Capture (CDC) to BigQuery is a canonical use case. For more information, see Set the pipeline streaming mode.

Returns:

  • (::Google::Cloud::Dataflow::V1beta3::StreamingMode) —

    Optional. Specifies the Streaming Engine message processing guarantees. Reduces cost and latency but might result in duplicate messages committed to storage. Designed to run simple mapping streaming ETL jobs at the lowest cost. For example, Change Data Capture (CDC) to BigQuery is a canonical use case. For more information, see Set the pipeline streaming mode.



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#temp_storage_prefix ⇒ ::String

Returns The prefix of the resources the system should use for temporary storage. The system will append the suffix "/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is:

Google Cloud Storage:

storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}.

Returns:

  • (::String) —

    The prefix of the resources the system should use for temporary storage. The system will append the suffix "/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is:

    Google Cloud Storage:

    storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#use_public_ips ⇒ ::Boolean

Returns Optional. True when any worker pool that uses public IPs is present.

Returns:

  • (::Boolean) —

    Optional. True when any worker pool that uses public IPs is present.



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#use_streaming_engine_resource_based_billing ⇒ ::Boolean (readonly)

Returns Output only. Whether the job uses the Streaming Engine resource-based billing model.

Returns:

  • (::Boolean) —

    Output only. Whether the job uses the Streaming Engine resource-based billing model.



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#user_agent ⇒ ::Google::Protobuf::Struct

Returns Optional. A description of the process that generated the request.

Returns:



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#version ⇒ ::Google::Protobuf::Struct

Returns A structure describing which components and their versions of the service are required in order to run the job.

Returns:

  • (::Google::Protobuf::Struct) —

    A structure describing which components and their versions of the service are required in order to run the job.



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#worker_pools ⇒ ::Array<::Google::Cloud::Dataflow::V1beta3::WorkerPool>

Returns The worker pools. At least one "harness" worker pool must be specified in order for the job to have workers.

Returns:



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#worker_region ⇒ ::String

Returns Optional. The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region.

Returns:



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#worker_zone ⇒ ::String

Returns Optional. The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity.

Returns:

  • (::String) —

    Optional. The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity.



136
137
138
139
 
# File 'proto_docs/google/dataflow/v1beta3/environment.rb', line 136

class Environment
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end