Class: Google::Apis::DataflowV1b3::FlexTemplateRuntimeEnvironment

Inherits:
Object
  • Object
show all
Includes:
Core::Hashable, Core::JsonObjectSupport
Defined in:
lib/google/apis/dataflow_v1b3/classes.rb,
lib/google/apis/dataflow_v1b3/representations.rb,
lib/google/apis/dataflow_v1b3/representations.rb

Overview

The environment values to be set at runtime for flex template.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(**args) ⇒ FlexTemplateRuntimeEnvironment

Returns a new instance of FlexTemplateRuntimeEnvironment.



1946
1947
1948
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1946

def initialize(**args)
   update!(**args)
end

Instance Attribute Details

#additional_experiments ⇒ Array<String>

Additional experiment flags for the job. Corresponds to the JSON property additionalExperiments

Returns:

  • (Array<String>) 


1776
1777
1778
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1776

def additional_experiments
  @additional_experiments
end

#additional_pipeline_options ⇒ Array<String>

Optional. Additional pipeline option flags for the job. Corresponds to the JSON property additionalPipelineOptions

Returns:

  • (Array<String>) 


1781
1782
1783
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1781

def additional_pipeline_options
  @additional_pipeline_options
end

#additional_user_labels ⇒ Hash<String,String>

Additional user labels to be specified for the job. Keys and values must follow the restrictions specified in the labeling restrictions page. An object containing a list of "key": value pairs. Example: "name": "wrench", "mass": " 1kg", "count": "3". Corresponds to the JSON property additionalUserLabels

Returns:

  • (Hash<String,String>) 


1790
1791
1792
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1790

def additional_user_labels
  @additional_user_labels
end

#autoscaling_algorithm ⇒ String

The algorithm to use for autoscaling Corresponds to the JSON property autoscalingAlgorithm

Returns:

  • (String) 


1795
1796
1797
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1795

def autoscaling_algorithm
  @autoscaling_algorithm
end

#disk_size_gb ⇒ Fixnum

Worker disk size, in gigabytes. Corresponds to the JSON property diskSizeGb

Returns:

  • (Fixnum) 


1800
1801
1802
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1800

def disk_size_gb
  @disk_size_gb
end

#dump_heap_on_oom ⇒ Boolean Also known as: dump_heap_on_oom?

If true, when processing time is spent almost entirely on garbage collection ( GC), saves a heap dump before ending the thread or process. If false, ends the thread or process without saving a heap dump. Does not save a heap dump when the Java Virtual Machine (JVM) has an out of memory error during processing. The location of the heap file is either echoed back to the user, or the user is given the opportunity to download the heap file. Corresponds to the JSON property dumpHeapOnOom

Returns:

  • (Boolean) 


1810
1811
1812
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1810

def dump_heap_on_oom
  @dump_heap_on_oom
end

#enable_launcher_vm_serial_port_logging ⇒ Boolean Also known as: enable_launcher_vm_serial_port_logging?

If true serial port logging will be enabled for the launcher VM. Corresponds to the JSON property enableLauncherVmSerialPortLogging

Returns:

  • (Boolean) 


1816
1817
1818
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1816

def enable_launcher_vm_serial_port_logging
  @enable_launcher_vm_serial_port_logging
end

#enable_streaming_engine ⇒ Boolean Also known as: enable_streaming_engine?

Whether to enable Streaming Engine for the job. Corresponds to the JSON property enableStreamingEngine

Returns:

  • (Boolean) 


1822
1823
1824
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1822

def enable_streaming_engine
  @enable_streaming_engine
end

#flexrs_goal ⇒ String

Set FlexRS goal for the job. https://cloud.google.com/dataflow/docs/guides/ flexrs Corresponds to the JSON property flexrsGoal

Returns:

  • (String) 


1829
1830
1831
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1829

def flexrs_goal
  @flexrs_goal
end

#ip_configuration ⇒ String

Configuration for VM IPs. Corresponds to the JSON property ipConfiguration

Returns:

  • (String) 


1834
1835
1836
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1834

def ip_configuration
  @ip_configuration
end

#kms_key_name ⇒ String

Name for the Cloud KMS key for the job. Key format is: projects//locations// keyRings//cryptoKeys/ Corresponds to the JSON property kmsKeyName

Returns:

  • (String) 


1840
1841
1842
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1840

def kms_key_name
  @kms_key_name
end

#launcher_machine_type ⇒ String

The machine type to use for launching the job. If not set, Dataflow will select a default machine type. Corresponds to the JSON property launcherMachineType

Returns:

  • (String) 


1846
1847
1848
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1846

def launcher_machine_type
  @launcher_machine_type
end

#machine_type ⇒ String

The machine type to use for the job. Defaults to the value from the template if not specified. Corresponds to the JSON property machineType

Returns:

  • (String) 


1852
1853
1854
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1852

def machine_type
  @machine_type
end

#max_workers ⇒ Fixnum

The maximum number of Google Compute Engine instances to be made available to your pipeline during execution, from 1 to 1000. Corresponds to the JSON property maxWorkers

Returns:

  • (Fixnum) 


1858
1859
1860
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1858

def max_workers
  @max_workers
end

#network ⇒ String

Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default". Corresponds to the JSON property network

Returns:

  • (String) 


1864
1865
1866
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1864

def network
  @network
end

#num_workers ⇒ Fixnum

The initial number of Google Compute Engine instances for the job. Corresponds to the JSON property numWorkers

Returns:

  • (Fixnum) 


1869
1870
1871
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1869

def num_workers
  @num_workers
end

#save_heap_dumps_to_gcs_path ⇒ String

Cloud Storage bucket (directory) to upload heap dumps to. Enabling this field implies that dump_heap_on_oom is set to true. Corresponds to the JSON property saveHeapDumpsToGcsPath

Returns:

  • (String) 


1875
1876
1877
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1875

def save_heap_dumps_to_gcs_path
  @save_heap_dumps_to_gcs_path
end

#sdk_container_image ⇒ String

Docker registry location of container image to use for the 'worker harness. Default is the container for the version of the SDK. Note this field is only valid for portable pipelines. Corresponds to the JSON property sdkContainerImage

Returns:

  • (String) 


1882
1883
1884
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1882

def sdk_container_image
  @sdk_container_image
end

#service_account_email ⇒ String

The email address of the service account to run the job as. Corresponds to the JSON property serviceAccountEmail

Returns:

  • (String) 


1887
1888
1889
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1887

def 
  @service_account_email
end

#staging_location ⇒ String

The Cloud Storage path for staging local files. Must be a valid Cloud Storage URL, beginning with gs://. Corresponds to the JSON property stagingLocation

Returns:

  • (String) 


1893
1894
1895
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1893

def staging_location
  @staging_location
end

#streaming_mode ⇒ String

Optional. Specifies the Streaming Engine message processing guarantees. Reduces cost and latency but might result in duplicate messages committed to storage. Designed to run simple mapping streaming ETL jobs at the lowest cost. For example, Change Data Capture (CDC) to BigQuery is a canonical use case. For more information, see Set the pipeline streaming mode. Corresponds to the JSON property streamingMode

Returns:

  • (String) 


1903
1904
1905
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1903

def streaming_mode
  @streaming_mode
end

#subnetwork ⇒ String

Subnetwork to which VMs will be assigned, if desired. You can specify a subnetwork using either a complete URL or an abbreviated path. Expected to be of the form "https://www.googleapis.com/compute/v1/projects/HOST_PROJECT_ID/ regions/REGION/subnetworks/SUBNETWORK" or "regions/REGION/subnetworks/ SUBNETWORK". If the subnetwork is located in a Shared VPC network, you must use the complete URL. Corresponds to the JSON property subnetwork

Returns:

  • (String) 


1913
1914
1915
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1913

def subnetwork
  @subnetwork
end

#temp_location ⇒ String

The Cloud Storage path to use for temporary files. Must be a valid Cloud Storage URL, beginning with gs://. Corresponds to the JSON property tempLocation

Returns:

  • (String) 


1919
1920
1921
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1919

def temp_location
  @temp_location
end

#worker_region ⇒ String

The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/ regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region. Corresponds to the JSON property workerRegion

Returns:

  • (String) 


1927
1928
1929
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1927

def worker_region
  @worker_region
end

#worker_zone ⇒ String

The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/ regions-zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity. If both worker_zone and zone are set, worker_zone takes precedence. Corresponds to the JSON property workerZone

Returns:

  • (String) 


1937
1938
1939
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1937

def worker_zone
  @worker_zone
end

#zone ⇒ String

The Compute Engine availability zone for launching worker instances to run your pipeline. In the future, worker_zone will take precedence. Corresponds to the JSON property zone

Returns:

  • (String) 


1944
1945
1946
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1944

def zone
  @zone
end

Instance Method Details

#update!(**args) ⇒ Object

Update properties of this object



1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1951

def update!(**args)
  @additional_experiments = args[:additional_experiments] if args.key?(:additional_experiments)
  @additional_pipeline_options = args[:additional_pipeline_options] if args.key?(:additional_pipeline_options)
  @additional_user_labels = args[:additional_user_labels] if args.key?(:additional_user_labels)
  @autoscaling_algorithm = args[:autoscaling_algorithm] if args.key?(:autoscaling_algorithm)
  @disk_size_gb = args[:disk_size_gb] if args.key?(:disk_size_gb)
  @dump_heap_on_oom = args[:dump_heap_on_oom] if args.key?(:dump_heap_on_oom)
  @enable_launcher_vm_serial_port_logging = args[:enable_launcher_vm_serial_port_logging] if args.key?(:enable_launcher_vm_serial_port_logging)
  @enable_streaming_engine = args[:enable_streaming_engine] if args.key?(:enable_streaming_engine)
  @flexrs_goal = args[:flexrs_goal] if args.key?(:flexrs_goal)
  @ip_configuration = args[:ip_configuration] if args.key?(:ip_configuration)
  @kms_key_name = args[:kms_key_name] if args.key?(:kms_key_name)
  @launcher_machine_type = args[:launcher_machine_type] if args.key?(:launcher_machine_type)
  @machine_type = args[:machine_type] if args.key?(:machine_type)
  @max_workers = args[:max_workers] if args.key?(:max_workers)
  @network = args[:network] if args.key?(:network)
  @num_workers = args[:num_workers] if args.key?(:num_workers)
  @save_heap_dumps_to_gcs_path = args[:save_heap_dumps_to_gcs_path] if args.key?(:save_heap_dumps_to_gcs_path)
  @sdk_container_image = args[:sdk_container_image] if args.key?(:sdk_container_image)
  @service_account_email = args[:service_account_email] if args.key?(:service_account_email)
  @staging_location = args[:staging_location] if args.key?(:staging_location)
  @streaming_mode = args[:streaming_mode] if args.key?(:streaming_mode)
  @subnetwork = args[:subnetwork] if args.key?(:subnetwork)
  @temp_location = args[:temp_location] if args.key?(:temp_location)
  @worker_region = args[:worker_region] if args.key?(:worker_region)
  @worker_zone = args[:worker_zone] if args.key?(:worker_zone)
  @zone = args[:zone] if args.key?(:zone)
end