Class: Google::Cloud::Dataproc::V1::InstanceGroupConfig
- Inherits:
-
Object
- Object
- Google::Cloud::Dataproc::V1::InstanceGroupConfig
- Extended by:
- Protobuf::MessageExts::ClassMethods
- Includes:
- Protobuf::MessageExts
- Defined in:
- proto_docs/google/cloud/dataproc/v1/clusters.rb
Overview
The config settings for Compute Engine resources in an instance group, such as a master or worker group.
Defined Under Namespace
Modules: Preemptibility
Instance Attribute Summary collapse
-
#accelerators ⇒ ::Array<::Google::Cloud::Dataproc::V1::AcceleratorConfig>
Optional.
-
#disk_config ⇒ ::Google::Cloud::Dataproc::V1::DiskConfig
Optional.
-
#image_uri ⇒ ::String
Optional.
-
#instance_flexibility_policy ⇒ ::Google::Cloud::Dataproc::V1::InstanceFlexibilityPolicy
Optional.
-
#instance_names ⇒ ::Array<::String>
readonly
Output only.
-
#instance_references ⇒ ::Array<::Google::Cloud::Dataproc::V1::InstanceReference>
readonly
Output only.
-
#is_preemptible ⇒ ::Boolean
readonly
Output only.
-
#machine_type_uri ⇒ ::String
Optional.
-
#managed_group_config ⇒ ::Google::Cloud::Dataproc::V1::ManagedGroupConfig
readonly
Output only.
-
#min_cpu_platform ⇒ ::String
Optional.
-
#min_num_instances ⇒ ::Integer
Optional.
-
#num_instances ⇒ ::Integer
Optional.
-
#preemptibility ⇒ ::Google::Cloud::Dataproc::V1::InstanceGroupConfig::Preemptibility
Optional.
Instance Attribute Details
#accelerators ⇒ ::Array<::Google::Cloud::Dataproc::V1::AcceleratorConfig>
Returns Optional. The Compute Engine accelerator configuration for these instances.
564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 |
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 564 class InstanceGroupConfig include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Controls the use of preemptible instances within the group. module Preemptibility # Preemptibility is unspecified, the system will choose the # appropriate setting for each instance group. PREEMPTIBILITY_UNSPECIFIED = 0 # Instances are non-preemptible. # # This option is allowed for all instance groups and is the only valid # value for Master and Worker instance groups. NON_PREEMPTIBLE = 1 # Instances are [preemptible] # (https://cloud.google.com/compute/docs/instances/preemptible). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. PREEMPTIBLE = 2 # Instances are [Spot VMs] # (https://cloud.google.com/compute/docs/instances/spot). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. Spot VMs are the latest version of [preemptible VMs] # (https://cloud.google.com/compute/docs/instances/preemptible), and # provide additional features. SPOT = 3 end end |
#disk_config ⇒ ::Google::Cloud::Dataproc::V1::DiskConfig
Returns Optional. Disk option config settings.
564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 |
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 564 class InstanceGroupConfig include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Controls the use of preemptible instances within the group. module Preemptibility # Preemptibility is unspecified, the system will choose the # appropriate setting for each instance group. PREEMPTIBILITY_UNSPECIFIED = 0 # Instances are non-preemptible. # # This option is allowed for all instance groups and is the only valid # value for Master and Worker instance groups. NON_PREEMPTIBLE = 1 # Instances are [preemptible] # (https://cloud.google.com/compute/docs/instances/preemptible). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. PREEMPTIBLE = 2 # Instances are [Spot VMs] # (https://cloud.google.com/compute/docs/instances/spot). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. Spot VMs are the latest version of [preemptible VMs] # (https://cloud.google.com/compute/docs/instances/preemptible), and # provide additional features. SPOT = 3 end end |
#image_uri ⇒ ::String
Returns Optional. The Compute Engine image resource used for cluster instances.
The URI can represent an image or image family.
Image examples:
https://www.googleapis.com/compute/v1/projects/[project_id]/global/images/[image-id]projects/[project_id]/global/images/[image-id]image-id
Image family examples. Dataproc will use the most recent image from the family:
https://www.googleapis.com/compute/v1/projects/[project_id]/global/images/family/[custom-image-family-name]projects/[project_id]/global/images/family/[custom-image-family-name]
If the URI is unspecified, it will be inferred from
SoftwareConfig.image_version or the system default.
564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 |
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 564 class InstanceGroupConfig include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Controls the use of preemptible instances within the group. module Preemptibility # Preemptibility is unspecified, the system will choose the # appropriate setting for each instance group. PREEMPTIBILITY_UNSPECIFIED = 0 # Instances are non-preemptible. # # This option is allowed for all instance groups and is the only valid # value for Master and Worker instance groups. NON_PREEMPTIBLE = 1 # Instances are [preemptible] # (https://cloud.google.com/compute/docs/instances/preemptible). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. PREEMPTIBLE = 2 # Instances are [Spot VMs] # (https://cloud.google.com/compute/docs/instances/spot). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. Spot VMs are the latest version of [preemptible VMs] # (https://cloud.google.com/compute/docs/instances/preemptible), and # provide additional features. SPOT = 3 end end |
#instance_flexibility_policy ⇒ ::Google::Cloud::Dataproc::V1::InstanceFlexibilityPolicy
Returns Optional. Instance flexibility Policy allowing a mixture of VM shapes and provisioning models.
564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 |
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 564 class InstanceGroupConfig include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Controls the use of preemptible instances within the group. module Preemptibility # Preemptibility is unspecified, the system will choose the # appropriate setting for each instance group. PREEMPTIBILITY_UNSPECIFIED = 0 # Instances are non-preemptible. # # This option is allowed for all instance groups and is the only valid # value for Master and Worker instance groups. NON_PREEMPTIBLE = 1 # Instances are [preemptible] # (https://cloud.google.com/compute/docs/instances/preemptible). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. PREEMPTIBLE = 2 # Instances are [Spot VMs] # (https://cloud.google.com/compute/docs/instances/spot). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. Spot VMs are the latest version of [preemptible VMs] # (https://cloud.google.com/compute/docs/instances/preemptible), and # provide additional features. SPOT = 3 end end |
#instance_names ⇒ ::Array<::String> (readonly)
Returns Output only. The list of instance names. Dataproc derives the names
from cluster_name, num_instances, and the instance group.
564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 |
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 564 class InstanceGroupConfig include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Controls the use of preemptible instances within the group. module Preemptibility # Preemptibility is unspecified, the system will choose the # appropriate setting for each instance group. PREEMPTIBILITY_UNSPECIFIED = 0 # Instances are non-preemptible. # # This option is allowed for all instance groups and is the only valid # value for Master and Worker instance groups. NON_PREEMPTIBLE = 1 # Instances are [preemptible] # (https://cloud.google.com/compute/docs/instances/preemptible). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. PREEMPTIBLE = 2 # Instances are [Spot VMs] # (https://cloud.google.com/compute/docs/instances/spot). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. Spot VMs are the latest version of [preemptible VMs] # (https://cloud.google.com/compute/docs/instances/preemptible), and # provide additional features. SPOT = 3 end end |
#instance_references ⇒ ::Array<::Google::Cloud::Dataproc::V1::InstanceReference> (readonly)
Returns Output only. List of references to Compute Engine instances.
564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 |
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 564 class InstanceGroupConfig include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Controls the use of preemptible instances within the group. module Preemptibility # Preemptibility is unspecified, the system will choose the # appropriate setting for each instance group. PREEMPTIBILITY_UNSPECIFIED = 0 # Instances are non-preemptible. # # This option is allowed for all instance groups and is the only valid # value for Master and Worker instance groups. NON_PREEMPTIBLE = 1 # Instances are [preemptible] # (https://cloud.google.com/compute/docs/instances/preemptible). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. PREEMPTIBLE = 2 # Instances are [Spot VMs] # (https://cloud.google.com/compute/docs/instances/spot). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. Spot VMs are the latest version of [preemptible VMs] # (https://cloud.google.com/compute/docs/instances/preemptible), and # provide additional features. SPOT = 3 end end |
#is_preemptible ⇒ ::Boolean (readonly)
Returns Output only. Specifies that this instance group contains preemptible instances.
564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 |
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 564 class InstanceGroupConfig include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Controls the use of preemptible instances within the group. module Preemptibility # Preemptibility is unspecified, the system will choose the # appropriate setting for each instance group. PREEMPTIBILITY_UNSPECIFIED = 0 # Instances are non-preemptible. # # This option is allowed for all instance groups and is the only valid # value for Master and Worker instance groups. NON_PREEMPTIBLE = 1 # Instances are [preemptible] # (https://cloud.google.com/compute/docs/instances/preemptible). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. PREEMPTIBLE = 2 # Instances are [Spot VMs] # (https://cloud.google.com/compute/docs/instances/spot). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. Spot VMs are the latest version of [preemptible VMs] # (https://cloud.google.com/compute/docs/instances/preemptible), and # provide additional features. SPOT = 3 end end |
#machine_type_uri ⇒ ::String
Returns Optional. The Compute Engine machine type used for cluster instances.
A full URL, partial URI, or short name are valid. Examples:
https://www.googleapis.com/compute/v1/projects/[project_id]/zones/[zone]/machineTypes/n1-standard-2projects/[project_id]/zones/[zone]/machineTypes/n1-standard-2n1-standard-2
Auto Zone Exception: If you are using the Dataproc
Auto Zone
Placement
feature, you must use the short name of the machine type
resource, for example, n1-standard-2.
564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 |
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 564 class InstanceGroupConfig include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Controls the use of preemptible instances within the group. module Preemptibility # Preemptibility is unspecified, the system will choose the # appropriate setting for each instance group. PREEMPTIBILITY_UNSPECIFIED = 0 # Instances are non-preemptible. # # This option is allowed for all instance groups and is the only valid # value for Master and Worker instance groups. NON_PREEMPTIBLE = 1 # Instances are [preemptible] # (https://cloud.google.com/compute/docs/instances/preemptible). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. PREEMPTIBLE = 2 # Instances are [Spot VMs] # (https://cloud.google.com/compute/docs/instances/spot). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. Spot VMs are the latest version of [preemptible VMs] # (https://cloud.google.com/compute/docs/instances/preemptible), and # provide additional features. SPOT = 3 end end |
#managed_group_config ⇒ ::Google::Cloud::Dataproc::V1::ManagedGroupConfig (readonly)
Returns Output only. The config for Compute Engine Instance Group Manager that manages this group. This is only used for preemptible instance groups.
564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 |
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 564 class InstanceGroupConfig include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Controls the use of preemptible instances within the group. module Preemptibility # Preemptibility is unspecified, the system will choose the # appropriate setting for each instance group. PREEMPTIBILITY_UNSPECIFIED = 0 # Instances are non-preemptible. # # This option is allowed for all instance groups and is the only valid # value for Master and Worker instance groups. NON_PREEMPTIBLE = 1 # Instances are [preemptible] # (https://cloud.google.com/compute/docs/instances/preemptible). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. PREEMPTIBLE = 2 # Instances are [Spot VMs] # (https://cloud.google.com/compute/docs/instances/spot). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. Spot VMs are the latest version of [preemptible VMs] # (https://cloud.google.com/compute/docs/instances/preemptible), and # provide additional features. SPOT = 3 end end |
#min_cpu_platform ⇒ ::String
Returns Optional. Specifies the minimum cpu platform for the Instance Group. See Dataproc -> Minimum CPU Platform.
564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 |
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 564 class InstanceGroupConfig include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Controls the use of preemptible instances within the group. module Preemptibility # Preemptibility is unspecified, the system will choose the # appropriate setting for each instance group. PREEMPTIBILITY_UNSPECIFIED = 0 # Instances are non-preemptible. # # This option is allowed for all instance groups and is the only valid # value for Master and Worker instance groups. NON_PREEMPTIBLE = 1 # Instances are [preemptible] # (https://cloud.google.com/compute/docs/instances/preemptible). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. PREEMPTIBLE = 2 # Instances are [Spot VMs] # (https://cloud.google.com/compute/docs/instances/spot). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. Spot VMs are the latest version of [preemptible VMs] # (https://cloud.google.com/compute/docs/instances/preemptible), and # provide additional features. SPOT = 3 end end |
#min_num_instances ⇒ ::Integer
Returns Optional. The minimum number of instances to create. If min_num_instances is set, min_num_instances is used for a criteria to decide the cluster. Cluster creation will be failed by being an error state if the total number of instances created is less than the min_num_instances. For example, given that num_instances = 5 and min_num_instances = 3,
- if 4 instances are created and then registered successfully but one instance is failed, the failed VM will be deleted and the cluster will be resized to 4 instances in running state.
- if 2 instances are created successfully and 3 instances are failed, the cluster will be in an error state and does not delete failed VMs for debugging.
- if 2 instance are created and then registered successfully but 3 instances are failed to initialize, the cluster will be in an error state and does not delete failed VMs for debugging. NB: This can only be set for primary workers now.
564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 |
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 564 class InstanceGroupConfig include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Controls the use of preemptible instances within the group. module Preemptibility # Preemptibility is unspecified, the system will choose the # appropriate setting for each instance group. PREEMPTIBILITY_UNSPECIFIED = 0 # Instances are non-preemptible. # # This option is allowed for all instance groups and is the only valid # value for Master and Worker instance groups. NON_PREEMPTIBLE = 1 # Instances are [preemptible] # (https://cloud.google.com/compute/docs/instances/preemptible). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. PREEMPTIBLE = 2 # Instances are [Spot VMs] # (https://cloud.google.com/compute/docs/instances/spot). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. Spot VMs are the latest version of [preemptible VMs] # (https://cloud.google.com/compute/docs/instances/preemptible), and # provide additional features. SPOT = 3 end end |
#num_instances ⇒ ::Integer
Returns Optional. The number of VM instances in the instance group. For HA cluster master_config groups, must be set to 3. For standard cluster master_config groups, must be set to 1.
564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 |
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 564 class InstanceGroupConfig include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Controls the use of preemptible instances within the group. module Preemptibility # Preemptibility is unspecified, the system will choose the # appropriate setting for each instance group. PREEMPTIBILITY_UNSPECIFIED = 0 # Instances are non-preemptible. # # This option is allowed for all instance groups and is the only valid # value for Master and Worker instance groups. NON_PREEMPTIBLE = 1 # Instances are [preemptible] # (https://cloud.google.com/compute/docs/instances/preemptible). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. PREEMPTIBLE = 2 # Instances are [Spot VMs] # (https://cloud.google.com/compute/docs/instances/spot). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. Spot VMs are the latest version of [preemptible VMs] # (https://cloud.google.com/compute/docs/instances/preemptible), and # provide additional features. SPOT = 3 end end |
#preemptibility ⇒ ::Google::Cloud::Dataproc::V1::InstanceGroupConfig::Preemptibility
Returns Optional. Specifies the preemptibility of the instance group.
The default value for master and worker groups is
NON_PREEMPTIBLE. This default cannot be changed.
The default value for secondary instances is
PREEMPTIBLE.
564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 |
# File 'proto_docs/google/cloud/dataproc/v1/clusters.rb', line 564 class InstanceGroupConfig include ::Google::Protobuf::MessageExts extend ::Google::Protobuf::MessageExts::ClassMethods # Controls the use of preemptible instances within the group. module Preemptibility # Preemptibility is unspecified, the system will choose the # appropriate setting for each instance group. PREEMPTIBILITY_UNSPECIFIED = 0 # Instances are non-preemptible. # # This option is allowed for all instance groups and is the only valid # value for Master and Worker instance groups. NON_PREEMPTIBLE = 1 # Instances are [preemptible] # (https://cloud.google.com/compute/docs/instances/preemptible). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. PREEMPTIBLE = 2 # Instances are [Spot VMs] # (https://cloud.google.com/compute/docs/instances/spot). # # This option is allowed only for [secondary worker] # (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms) # groups. Spot VMs are the latest version of [preemptible VMs] # (https://cloud.google.com/compute/docs/instances/preemptible), and # provide additional features. SPOT = 3 end end |