Class: Google::Apis::DataflowV1b3::WorkerPool

Inherits:
Object
  • Object
show all
Includes:
Core::Hashable, Core::JsonObjectSupport
Defined in:
lib/google/apis/dataflow_v1b3/classes.rb,
lib/google/apis/dataflow_v1b3/representations.rb,
lib/google/apis/dataflow_v1b3/representations.rb

Overview

Describes one particular pool of Cloud Dataflow workers to be instantiated by the Cloud Dataflow service in order to perform the computations required by a job. Note that a workflow job may use multiple pools, in order to match the various computational requirements of the various stages of the job.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(**args) ⇒ WorkerPool

Returns a new instance of WorkerPool.



8039
8040
8041
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 8039

def initialize(**args)
   update!(**args)
end

Instance Attribute Details

#autoscaling_settings ⇒ Google::Apis::DataflowV1b3::AutoscalingSettings

Settings for WorkerPool autoscaling. Corresponds to the JSON property autoscalingSettings

Returns:



7894
7895
7896
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7894

def autoscaling_settings
  @autoscaling_settings
end

#data_disks ⇒ Array<Google::Apis::DataflowV1b3::Disk>

Data disks that are used by a VM in this workflow. Corresponds to the JSON property dataDisks

Returns:



7899
7900
7901
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7899

def data_disks
  @data_disks
end

#default_package_set ⇒ String

The default package set to install. This allows the service to select a default set of packages which are useful to worker harnesses written in a particular language. Corresponds to the JSON property defaultPackageSet

Returns:

  • (String) 


7906
7907
7908
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7906

def default_package_set
  @default_package_set
end

#disk_provisioned_iops ⇒ Fixnum

Optional. IOPS provisioned for the root disk for VMs. Corresponds to the JSON property diskProvisionedIops

Returns:

  • (Fixnum) 


7911
7912
7913
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7911

def disk_provisioned_iops
  @disk_provisioned_iops
end

#disk_provisioned_throughput_mibps ⇒ Fixnum

Optional. Throughput provisioned for the root disk for VMs. Corresponds to the JSON property diskProvisionedThroughputMibps

Returns:

  • (Fixnum) 


7916
7917
7918
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7916

def disk_provisioned_throughput_mibps
  @disk_provisioned_throughput_mibps
end

#disk_size_gb ⇒ Fixnum

Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default. Corresponds to the JSON property diskSizeGb

Returns:

  • (Fixnum) 


7922
7923
7924
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7922

def disk_size_gb
  @disk_size_gb
end

#disk_source_image ⇒ String

Fully qualified source image for disks. Corresponds to the JSON property diskSourceImage

Returns:

  • (String) 


7927
7928
7929
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7927

def disk_source_image
  @disk_source_image
end

#disk_type ⇒ String

Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default. Corresponds to the JSON property diskType

Returns:

  • (String) 


7933
7934
7935
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7933

def disk_type
  @disk_type
end

#ip_configuration ⇒ String

Configuration for VM IPs. Corresponds to the JSON property ipConfiguration

Returns:

  • (String) 


7938
7939
7940
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7938

def ip_configuration
  @ip_configuration
end

#kind ⇒ String

The kind of the worker pool; currently only harness and shuffle are supported. Corresponds to the JSON property kind

Returns:

  • (String) 


7944
7945
7946
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7944

def kind
  @kind
end

#machine_type ⇒ String

Machine type (e.g. "n1-standard-1"). If empty or unspecified, the service will attempt to choose a reasonable default. Corresponds to the JSON property machineType

Returns:

  • (String) 


7950
7951
7952
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7950

def machine_type
  @machine_type
end

#metadata ⇒ Hash<String,String>

Metadata to set on the Google Compute Engine VMs. Corresponds to the JSON property metadata

Returns:

  • (Hash<String,String>) 


7955
7956
7957
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7955

def metadata
  @metadata
end

#network ⇒ String

Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default". Corresponds to the JSON property network

Returns:

  • (String) 


7961
7962
7963
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7961

def network
  @network
end

#num_threads_per_worker ⇒ Fixnum

The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming). Corresponds to the JSON property numThreadsPerWorker

Returns:

  • (Fixnum) 


7968
7969
7970
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7968

def num_threads_per_worker
  @num_threads_per_worker
end

#num_workers ⇒ Fixnum

Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default. Corresponds to the JSON property numWorkers

Returns:

  • (Fixnum) 


7975
7976
7977
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7975

def num_workers
  @num_workers
end

#on_host_maintenance ⇒ String

The action to take on host maintenance, as defined by the Google Compute Engine API. Corresponds to the JSON property onHostMaintenance

Returns:

  • (String) 


7981
7982
7983
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7981

def on_host_maintenance
  @on_host_maintenance
end

#packages ⇒ Array<Google::Apis::DataflowV1b3::Package>

Packages to be installed on workers. Corresponds to the JSON property packages

Returns:



7986
7987
7988
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7986

def packages
  @packages
end

#pool_args ⇒ Hash<String,Object>

Extra arguments for this worker pool. Corresponds to the JSON property poolArgs

Returns:

  • (Hash<String,Object>) 


7991
7992
7993
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7991

def pool_args
  @pool_args
end

#sdk_harness_container_images ⇒ Array<Google::Apis::DataflowV1b3::SdkHarnessContainerImage>

Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries. Corresponds to the JSON property sdkHarnessContainerImages

Returns:



7998
7999
8000
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 7998

def sdk_harness_container_images
  @sdk_harness_container_images
end

#subnetwork ⇒ String

Subnetwork to which VMs will be assigned, if desired. Expected to be of the form "regions/REGION/subnetworks/SUBNETWORK". Corresponds to the JSON property subnetwork

Returns:

  • (String) 


8004
8005
8006
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 8004

def subnetwork
  @subnetwork
end

#taskrunner_settings ⇒ Google::Apis::DataflowV1b3::TaskRunnerSettings

Taskrunner configuration settings. Corresponds to the JSON property taskrunnerSettings

Returns:



8009
8010
8011
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 8009

def taskrunner_settings
  @taskrunner_settings
end

#teardown_policy ⇒ String

Sets the policy for determining when to turndown worker pool. Allowed values are: TEARDOWN_ALWAYS, TEARDOWN_ON_SUCCESS, and TEARDOWN_NEVER. TEARDOWN_ALWAYS means workers are always torn down regardless of whether the job succeeds. TEARDOWN_ON_SUCCESS means workers are torn down if the job succeeds. TEARDOWN_NEVER means the workers are never torn down. If the workers are not torn down by the service, they will continue to run and use Google Compute Engine VM resources in the user's project until they are explicitly terminated by the user. Because of this, Google recommends using the TEARDOWN_ALWAYS policy except for small, manually supervised test jobs. If unknown or unspecified, the service will attempt to choose a reasonable default. Corresponds to the JSON property teardownPolicy

Returns:

  • (String) 


8024
8025
8026
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 8024

def teardown_policy
  @teardown_policy
end

#worker_harness_container_image ⇒ String

Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry. Deprecated for the Fn API path. Use sdk_harness_container_images instead. Corresponds to the JSON property workerHarnessContainerImage

Returns:

  • (String) 


8031
8032
8033
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 8031

def worker_harness_container_image
  @worker_harness_container_image
end

#zone ⇒ String

Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default. Corresponds to the JSON property zone

Returns:

  • (String) 


8037
8038
8039
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 8037

def zone
  @zone
end

Instance Method Details

#update!(**args) ⇒ Object

Update properties of this object



8044
8045
8046
8047
8048
8049
8050
8051
8052
8053
8054
8055
8056
8057
8058
8059
8060
8061
8062
8063
8064
8065
8066
8067
8068
8069
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 8044

def update!(**args)
  @autoscaling_settings = args[:autoscaling_settings] if args.key?(:autoscaling_settings)
  @data_disks = args[:data_disks] if args.key?(:data_disks)
  @default_package_set = args[:default_package_set] if args.key?(:default_package_set)
  @disk_provisioned_iops = args[:disk_provisioned_iops] if args.key?(:disk_provisioned_iops)
  @disk_provisioned_throughput_mibps = args[:disk_provisioned_throughput_mibps] if args.key?(:disk_provisioned_throughput_mibps)
  @disk_size_gb = args[:disk_size_gb] if args.key?(:disk_size_gb)
  @disk_source_image = args[:disk_source_image] if args.key?(:disk_source_image)
  @disk_type = args[:disk_type] if args.key?(:disk_type)
  @ip_configuration = args[:ip_configuration] if args.key?(:ip_configuration)
  @kind = args[:kind] if args.key?(:kind)
  @machine_type = args[:machine_type] if args.key?(:machine_type)
  @metadata = args[:metadata] if args.key?(:metadata)
  @network = args[:network] if args.key?(:network)
  @num_threads_per_worker = args[:num_threads_per_worker] if args.key?(:num_threads_per_worker)
  @num_workers = args[:num_workers] if args.key?(:num_workers)
  @on_host_maintenance = args[:on_host_maintenance] if args.key?(:on_host_maintenance)
  @packages = args[:packages] if args.key?(:packages)
  @pool_args = args[:pool_args] if args.key?(:pool_args)
  @sdk_harness_container_images = args[:sdk_harness_container_images] if args.key?(:sdk_harness_container_images)
  @subnetwork = args[:subnetwork] if args.key?(:subnetwork)
  @taskrunner_settings = args[:taskrunner_settings] if args.key?(:taskrunner_settings)
  @teardown_policy = args[:teardown_policy] if args.key?(:teardown_policy)
  @worker_harness_container_image = args[:worker_harness_container_image] if args.key?(:worker_harness_container_image)
  @zone = args[:zone] if args.key?(:zone)
end