Class: Google::Apis::DataprocV1::ClusterConfig

Inherits:
Object
  • Object
show all
Includes:
Core::Hashable, Core::JsonObjectSupport
Defined in:
lib/google/apis/dataproc_v1/classes.rb,
lib/google/apis/dataproc_v1/representations.rb,
lib/google/apis/dataproc_v1/representations.rb

Overview

The cluster config.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(**args) ⇒ ClusterConfig

Returns a new instance of ClusterConfig.



1554
1555
1556
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1554

def initialize(**args)
   update!(**args)
end

Instance Attribute Details

#autoscaling_config ⇒ Google::Apis::DataprocV1::AutoscalingConfig

Autoscaling Policy config associated with the cluster. Corresponds to the JSON property autoscalingConfig

Returns:



1420
1421
1422
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1420

def autoscaling_config
  @autoscaling_config
end

#auxiliary_node_groups ⇒ Array<Google::Apis::DataprocV1::AuxiliaryNodeGroup>

Optional. The node group settings. Corresponds to the JSON property auxiliaryNodeGroups

Returns:



1425
1426
1427
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1425

def auxiliary_node_groups
  @auxiliary_node_groups
end

#cluster_tier ⇒ String

Optional. The cluster tier. Corresponds to the JSON property clusterTier

Returns:

  • (String) 


1430
1431
1432
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1430

def cluster_tier
  @cluster_tier
end

#cluster_type ⇒ String

Optional. The type of the cluster. Corresponds to the JSON property clusterType

Returns:

  • (String) 


1435
1436
1437
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1435

def cluster_type
  @cluster_type
end

#config_bucket ⇒ String

Optional. A Cloud Storage bucket used to stage job dependencies, config files, and job driver console output. If you do not specify a staging bucket, Cloud Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's staging bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per- location bucket (see Dataproc staging and temp buckets (https://cloud.google. com/dataproc/docs/concepts/configuring-clusters/staging-bucket)). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket. Corresponds to the JSON property configBucket

Returns:

  • (String) 


1448
1449
1450
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1448

def config_bucket
  @config_bucket
end

#dataproc_metric_config ⇒ Google::Apis::DataprocV1::DataprocMetricConfig

Dataproc metric config. Corresponds to the JSON property dataprocMetricConfig

Returns:



1453
1454
1455
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1453

def dataproc_metric_config
  @dataproc_metric_config
end

#diagnostic_bucket ⇒ String

Optional. A Cloud Storage bucket used to collect checkpoint diagnostic data ( https://cloud.google.com/dataproc/docs/support/diagnose-clusters# checkpoint_diagnostic_data). If you do not specify a diagnostic bucket, Cloud Dataproc will use the Dataproc temp bucket to collect the checkpoint diagnostic data. This field requires a Cloud Storage bucket name, not a gs://.. . URI to a Cloud Storage bucket. Corresponds to the JSON property diagnosticBucket

Returns:

  • (String) 


1463
1464
1465
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1463

def diagnostic_bucket
  @diagnostic_bucket
end

#encryption_config ⇒ Google::Apis::DataprocV1::EncryptionConfig

Encryption settings for the cluster. Corresponds to the JSON property encryptionConfig

Returns:



1468
1469
1470
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1468

def encryption_config
  @encryption_config
end

#endpoint_config ⇒ Google::Apis::DataprocV1::EndpointConfig

Endpoint config for this cluster Corresponds to the JSON property endpointConfig

Returns:



1473
1474
1475
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1473

def endpoint_config
  @endpoint_config
end

#engine ⇒ String

Optional. The cluster engine. Corresponds to the JSON property engine

Returns:

  • (String) 


1478
1479
1480
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1478

def engine
  @engine
end

#gce_cluster_config ⇒ Google::Apis::DataprocV1::GceClusterConfig

Common config settings for resources of Compute Engine cluster instances, applicable to all instances in the cluster. Corresponds to the JSON property gceClusterConfig

Returns:



1484
1485
1486
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1484

def gce_cluster_config
  @gce_cluster_config
end

#gke_cluster_config ⇒ Google::Apis::DataprocV1::GkeClusterConfig

The cluster's GKE config. Corresponds to the JSON property gkeClusterConfig

Returns:



1489
1490
1491
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1489

def gke_cluster_config
  @gke_cluster_config
end

#initialization_actions ⇒ Array<Google::Apis::DataprocV1::NodeInitializationAction>

Optional. Commands to execute on each node after config is completed. By default, executables are run on master and all worker nodes. You can test a node's role metadata to run an executable on a master or worker node, as shown below using curl (you can also use wget): ROLE=$(curl -H Metadata-Flavor: Google http://metadata/computeMetadata/v1/instance/attributes/dataproc-role) if [[ "$ROLE" == 'Master' ]]; then ... master specific actions ... else ... worker specific actions ... fi Corresponds to the JSON property initializationActions

Returns:



1500
1501
1502
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1500

def initialization_actions
  @initialization_actions
end

#lifecycle_config ⇒ Google::Apis::DataprocV1::LifecycleConfig

Specifies the cluster auto-delete schedule configuration. Corresponds to the JSON property lifecycleConfig

Returns:



1505
1506
1507
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1505

def lifecycle_config
  @lifecycle_config
end

#master_config ⇒ Google::Apis::DataprocV1::InstanceGroupConfig

The config settings for Compute Engine resources in an instance group, such as a master or worker group. Corresponds to the JSON property masterConfig

Returns:



1511
1512
1513
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1511

def master_config
  @master_config
end

#metastore_config ⇒ Google::Apis::DataprocV1::MetastoreConfig

Specifies a Metastore configuration. Corresponds to the JSON property metastoreConfig

Returns:



1516
1517
1518
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1516

def metastore_config
  @metastore_config
end

#secondary_worker_config ⇒ Google::Apis::DataprocV1::InstanceGroupConfig

The config settings for Compute Engine resources in an instance group, such as a master or worker group. Corresponds to the JSON property secondaryWorkerConfig

Returns:



1522
1523
1524
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1522

def secondary_worker_config
  @secondary_worker_config
end

#security_config ⇒ Google::Apis::DataprocV1::SecurityConfig

Security related configuration, including encryption, Kerberos, etc. Corresponds to the JSON property securityConfig

Returns:



1527
1528
1529
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1527

def security_config
  @security_config
end

#software_config ⇒ Google::Apis::DataprocV1::SoftwareConfig

Specifies the selection and config of software inside the cluster. Corresponds to the JSON property softwareConfig

Returns:



1532
1533
1534
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1532

def software_config
  @software_config
end

#temp_bucket ⇒ String

Optional. A Cloud Storage bucket used to store ephemeral cluster and jobs data, such as Spark and MapReduce history files. If you do not specify a temp bucket, Dataproc will determine a Cloud Storage location (US, ASIA, or EU) for your cluster's temp bucket according to the Compute Engine zone where your cluster is deployed, and then create and manage this project-level, per- location bucket. The default bucket has a TTL of 90 days, but you can use any TTL (or none) if you specify a bucket (see Dataproc staging and temp buckets ( https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/staging- bucket)). This field requires a Cloud Storage bucket name, not a gs://... URI to a Cloud Storage bucket. Corresponds to the JSON property tempBucket

Returns:

  • (String) 


1546
1547
1548
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1546

def temp_bucket
  @temp_bucket
end

#worker_config ⇒ Google::Apis::DataprocV1::InstanceGroupConfig

The config settings for Compute Engine resources in an instance group, such as a master or worker group. Corresponds to the JSON property workerConfig

Returns:



1552
1553
1554
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1552

def worker_config
  @worker_config
end

Instance Method Details

#update!(**args) ⇒ Object

Update properties of this object



1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 1559

def update!(**args)
  @autoscaling_config = args[:autoscaling_config] if args.key?(:autoscaling_config)
  @auxiliary_node_groups = args[:auxiliary_node_groups] if args.key?(:auxiliary_node_groups)
  @cluster_tier = args[:cluster_tier] if args.key?(:cluster_tier)
  @cluster_type = args[:cluster_type] if args.key?(:cluster_type)
  @config_bucket = args[:config_bucket] if args.key?(:config_bucket)
  @dataproc_metric_config = args[:dataproc_metric_config] if args.key?(:dataproc_metric_config)
  @diagnostic_bucket = args[:diagnostic_bucket] if args.key?(:diagnostic_bucket)
  @encryption_config = args[:encryption_config] if args.key?(:encryption_config)
  @endpoint_config = args[:endpoint_config] if args.key?(:endpoint_config)
  @engine = args[:engine] if args.key?(:engine)
  @gce_cluster_config = args[:gce_cluster_config] if args.key?(:gce_cluster_config)
  @gke_cluster_config = args[:gke_cluster_config] if args.key?(:gke_cluster_config)
  @initialization_actions = args[:initialization_actions] if args.key?(:initialization_actions)
  @lifecycle_config = args[:lifecycle_config] if args.key?(:lifecycle_config)
  @master_config = args[:master_config] if args.key?(:master_config)
  @metastore_config = args[:metastore_config] if args.key?(:metastore_config)
  @secondary_worker_config = args[:secondary_worker_config] if args.key?(:secondary_worker_config)
  @security_config = args[:security_config] if args.key?(:security_config)
  @software_config = args[:software_config] if args.key?(:software_config)
  @temp_bucket = args[:temp_bucket] if args.key?(:temp_bucket)
  @worker_config = args[:worker_config] if args.key?(:worker_config)
end