Class: Google::Cloud::Dataproc::V1::ClusterConfig

Inherits:
Object
  • Object
show all
Extended by:
Protobuf::MessageExts::ClassMethods
Includes:
Protobuf::MessageExts
Defined in:
proto_docs/google/cloud/dataproc/v1/clusters.rb

Overview

The cluster config.

Instance Attribute Summary collapse

Instance Attribute Details

#autoscaling_config ⇒ ::Google::Cloud::Dataproc::V1::AutoscalingConfig

Returns Optional. Autoscaling config for the policy associated with the cluster. Cluster does not autoscale if this field is unset.

Returns:



165
166
167
168
 
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 165

class ClusterConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#config_bucket ⇒ ::String

Returns Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket..

Returns:

  • (::String) —

    Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.



165
166
167
168
 
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 165

class ClusterConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#encryption_config ⇒ ::Google::Cloud::Dataproc::V1::EncryptionConfig

Returns Optional. Encryption settings for the cluster.

Returns:



165
166
167
168
 
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 165

class ClusterConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#endpoint_config ⇒ ::Google::Cloud::Dataproc::V1::EndpointConfig

Returns Optional. Port/endpoint configuration for this cluster.

Returns:



165
166
167
168
 
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 165

class ClusterConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#gce_cluster_config ⇒ ::Google::Cloud::Dataproc::V1::GceClusterConfig

Returns Optional. The shared Compute Engine config settings for all instances in a cluster.

Returns:



165
166
167
168
 
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 165

class ClusterConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#gke_cluster_config ⇒ ::Google::Cloud::Dataproc::V1::GkeClusterConfig

Returns Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.

Returns:

  • (::Google::Cloud::Dataproc::V1::GkeClusterConfig) —

    Optional. BETA. The Kubernetes Engine config for Dataproc clusters deployed to Kubernetes. Setting this is considered mutually exclusive with Compute Engine-based options such as gce_cluster_config, master_config, worker_config, secondary_worker_config, and autoscaling_config.



165
166
167
168
 
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 165

class ClusterConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#initialization_actions ⇒ ::Array<::Google::Cloud::Dataproc::V1::NodeInitializationAction>

Returns Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes. You can test a node's role metadata to run an executable on a master or worker node, as shown below using curl (you can also use wget):

ROLE=$(curl -H Metadata-Flavor:Google
http://metadata/computeMetadata/v1/instance/attributes/dataproc-role)
if [[ "${ROLE}" == 'Master' ]]; then
  ... master specific actions ...
else
  ... worker specific actions ...
fi.

Returns:

  • (::Array<::Google::Cloud::Dataproc::V1::NodeInitializationAction>) —

    Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes. You can test a node's role metadata to run an executable on a master or worker node, as shown below using curl (you can also use wget):

    ROLE=$(curl -H Metadata-Flavor:Google
http://metadata/computeMetadata/v1/instance/attributes/dataproc-role)
if [[ "${ROLE}" == 'Master' ]]; then
  ... master specific actions ...
else
  ... worker specific actions ...
fi



165
166
167
168
 
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 165

class ClusterConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#lifecycle_config ⇒ ::Google::Cloud::Dataproc::V1::LifecycleConfig

Returns Optional. Lifecycle setting for the cluster.

Returns:



165
166
167
168
 
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 165

class ClusterConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#master_config ⇒ ::Google::Cloud::Dataproc::V1::InstanceGroupConfig

Returns Optional. The Compute Engine config settings for the cluster's master instance.

Returns:



165
166
167
168
 
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 165

class ClusterConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#metastore_config ⇒ ::Google::Cloud::Dataproc::V1::MetastoreConfig

Returns Optional. Metastore configuration.

Returns:



165
166
167
168
 
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 165

class ClusterConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#secondary_worker_config ⇒ ::Google::Cloud::Dataproc::V1::InstanceGroupConfig

Returns Optional. The Compute Engine config settings for a cluster's secondary worker instances.

Returns:



165
166
167
168
 
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 165

class ClusterConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#security_config ⇒ ::Google::Cloud::Dataproc::V1::SecurityConfig

Returns Optional. Security settings for the cluster.

Returns:



165
166
167
168
 
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 165

class ClusterConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#software_config ⇒ ::Google::Cloud::Dataproc::V1::SoftwareConfig

Returns Optional. The config settings for cluster software.

Returns:



165
166
167
168
 
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 165

class ClusterConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#temp_bucket ⇒ ::String

Returns Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket..

Returns:

  • (::String) —

    Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per-location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket.



165
166
167
168
 
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 165

class ClusterConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end

#worker_config ⇒ ::Google::Cloud::Dataproc::V1::InstanceGroupConfig

Returns Optional. The Compute Engine config settings for the cluster's worker instances.

Returns:



165
166
167
168
 
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 165

class ClusterConfig
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods
end