Class: Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction

Inherits:
Object
  • Object
show all
Extended by:
Protobuf::MessageExts::ClassMethods
Includes:
Protobuf::MessageExts
Defined in:
proto_docs/google/cloud/dataform/v1beta1/dataform.rb

Overview

Represents a workflow action that will run a Data Preparation.

Defined Under Namespace

Classes: ActionErrorTable, ActionIncrementalLoadMode, ActionLoadConfig, ActionSimpleLoadMode, ActionSqlDefinition

Instance Attribute Summary collapse

Instance Attribute Details

#contents_sql ⇒ ::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionSqlDefinition

Returns SQL definition for a Data Preparation. Contains a SQL query and additional context information.

Note: The following fields are mutually exclusive: contents_sql, contents_yaml. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:



2565
2566
2567
2568
2569
2570
2571
2572
2573
2574
2575
2576
2577
2578
2579
2580
2581
2582
2583
2584
2585
2586
2587
2588
2589
2590
2591
2592
2593
2594
2595
2596
2597
2598
2599
2600
2601
2602
2603
2604
2605
2606
2607
2608
2609
2610
2611
2612
2613
2614
2615
2616
2617
2618
2619
2620
2621
2622
2623
2624
2625
2626
2627
2628
2629
2630
2631
2632
2633
2634
2635
2636
2637
2638
2639
2640
2641
 
# File 'proto_docs/google/cloud/dataform/v1beta1/dataform.rb', line 2565

class DataPreparationAction
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # Definition of a SQL Data Preparation
  # @!attribute [rw] query
  #   @return [::String]
  #     The SQL query representing the data preparation steps. Formatted as a
  #     Pipe SQL query statement.
  # @!attribute [rw] error_table
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionErrorTable]
  #     Error table configuration,
  # @!attribute [rw] load_config
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionLoadConfig]
  #     Load configuration.
  class ActionSqlDefinition
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Error table information, used to write error data into a BigQuery
  # table.
  # @!attribute [rw] target
  #   @return [::Google::Cloud::Dataform::V1beta1::Target]
  #     Error Table target.
  # @!attribute [rw] retention_days
  #   @return [::Integer]
  #     Error table partition expiration in days. Only positive values are
  #     allowed.
  class ActionErrorTable
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Simplified load configuration for actions
  # @!attribute [rw] replace
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionSimpleLoadMode]
  #     Replace destination table
  #
  #     Note: The following fields are mutually exclusive: `replace`, `append`, `maximum`, `unique`. If a field in that set is populated, all other fields in the set will automatically be cleared.
  # @!attribute [rw] append
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionSimpleLoadMode]
  #     Append into destination table
  #
  #     Note: The following fields are mutually exclusive: `append`, `replace`, `maximum`, `unique`. If a field in that set is populated, all other fields in the set will automatically be cleared.
  # @!attribute [rw] maximum
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionIncrementalLoadMode]
  #     Insert records where the value exceeds the previous maximum value for
  #     a column in the destination table
  #
  #     Note: The following fields are mutually exclusive: `maximum`, `replace`, `append`, `unique`. If a field in that set is populated, all other fields in the set will automatically be cleared.
  # @!attribute [rw] unique
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionIncrementalLoadMode]
  #     Insert records where the value of a column is not already present in
  #     the destination table
  #
  #     Note: The following fields are mutually exclusive: `unique`, `replace`, `append`, `maximum`. If a field in that set is populated, all other fields in the set will automatically be cleared.
  class ActionLoadConfig
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Simple load definition
  class ActionSimpleLoadMode
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Load definition for incremental load modes
  # @!attribute [rw] column
  #   @return [::String]
  #     Column name for incremental load modes
  class ActionIncrementalLoadMode
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#contents_yaml ⇒ ::String (readonly)

Returns Output only. YAML representing the contents of the data preparation. Can be used to show the customer what the input was to their workflow.

Note: The following fields are mutually exclusive: contents_yaml, contents_sql. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns:

  • (::String) —

    Output only. YAML representing the contents of the data preparation. Can be used to show the customer what the input was to their workflow.

    Note: The following fields are mutually exclusive: contents_yaml, contents_sql. If a field in that set is populated, all other fields in the set will automatically be cleared.



2565
2566
2567
2568
2569
2570
2571
2572
2573
2574
2575
2576
2577
2578
2579
2580
2581
2582
2583
2584
2585
2586
2587
2588
2589
2590
2591
2592
2593
2594
2595
2596
2597
2598
2599
2600
2601
2602
2603
2604
2605
2606
2607
2608
2609
2610
2611
2612
2613
2614
2615
2616
2617
2618
2619
2620
2621
2622
2623
2624
2625
2626
2627
2628
2629
2630
2631
2632
2633
2634
2635
2636
2637
2638
2639
2640
2641
 
# File 'proto_docs/google/cloud/dataform/v1beta1/dataform.rb', line 2565

class DataPreparationAction
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # Definition of a SQL Data Preparation
  # @!attribute [rw] query
  #   @return [::String]
  #     The SQL query representing the data preparation steps. Formatted as a
  #     Pipe SQL query statement.
  # @!attribute [rw] error_table
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionErrorTable]
  #     Error table configuration,
  # @!attribute [rw] load_config
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionLoadConfig]
  #     Load configuration.
  class ActionSqlDefinition
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Error table information, used to write error data into a BigQuery
  # table.
  # @!attribute [rw] target
  #   @return [::Google::Cloud::Dataform::V1beta1::Target]
  #     Error Table target.
  # @!attribute [rw] retention_days
  #   @return [::Integer]
  #     Error table partition expiration in days. Only positive values are
  #     allowed.
  class ActionErrorTable
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Simplified load configuration for actions
  # @!attribute [rw] replace
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionSimpleLoadMode]
  #     Replace destination table
  #
  #     Note: The following fields are mutually exclusive: `replace`, `append`, `maximum`, `unique`. If a field in that set is populated, all other fields in the set will automatically be cleared.
  # @!attribute [rw] append
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionSimpleLoadMode]
  #     Append into destination table
  #
  #     Note: The following fields are mutually exclusive: `append`, `replace`, `maximum`, `unique`. If a field in that set is populated, all other fields in the set will automatically be cleared.
  # @!attribute [rw] maximum
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionIncrementalLoadMode]
  #     Insert records where the value exceeds the previous maximum value for
  #     a column in the destination table
  #
  #     Note: The following fields are mutually exclusive: `maximum`, `replace`, `append`, `unique`. If a field in that set is populated, all other fields in the set will automatically be cleared.
  # @!attribute [rw] unique
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionIncrementalLoadMode]
  #     Insert records where the value of a column is not already present in
  #     the destination table
  #
  #     Note: The following fields are mutually exclusive: `unique`, `replace`, `append`, `maximum`. If a field in that set is populated, all other fields in the set will automatically be cleared.
  class ActionLoadConfig
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Simple load definition
  class ActionSimpleLoadMode
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Load definition for incremental load modes
  # @!attribute [rw] column
  #   @return [::String]
  #     Column name for incremental load modes
  class ActionIncrementalLoadMode
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#generated_sql ⇒ ::String (readonly)

Returns Output only. The generated BigQuery SQL script that will be executed. For reference only.

Returns:

  • (::String) —

    Output only. The generated BigQuery SQL script that will be executed. For reference only.



2565
2566
2567
2568
2569
2570
2571
2572
2573
2574
2575
2576
2577
2578
2579
2580
2581
2582
2583
2584
2585
2586
2587
2588
2589
2590
2591
2592
2593
2594
2595
2596
2597
2598
2599
2600
2601
2602
2603
2604
2605
2606
2607
2608
2609
2610
2611
2612
2613
2614
2615
2616
2617
2618
2619
2620
2621
2622
2623
2624
2625
2626
2627
2628
2629
2630
2631
2632
2633
2634
2635
2636
2637
2638
2639
2640
2641
 
# File 'proto_docs/google/cloud/dataform/v1beta1/dataform.rb', line 2565

class DataPreparationAction
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # Definition of a SQL Data Preparation
  # @!attribute [rw] query
  #   @return [::String]
  #     The SQL query representing the data preparation steps. Formatted as a
  #     Pipe SQL query statement.
  # @!attribute [rw] error_table
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionErrorTable]
  #     Error table configuration,
  # @!attribute [rw] load_config
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionLoadConfig]
  #     Load configuration.
  class ActionSqlDefinition
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Error table information, used to write error data into a BigQuery
  # table.
  # @!attribute [rw] target
  #   @return [::Google::Cloud::Dataform::V1beta1::Target]
  #     Error Table target.
  # @!attribute [rw] retention_days
  #   @return [::Integer]
  #     Error table partition expiration in days. Only positive values are
  #     allowed.
  class ActionErrorTable
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Simplified load configuration for actions
  # @!attribute [rw] replace
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionSimpleLoadMode]
  #     Replace destination table
  #
  #     Note: The following fields are mutually exclusive: `replace`, `append`, `maximum`, `unique`. If a field in that set is populated, all other fields in the set will automatically be cleared.
  # @!attribute [rw] append
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionSimpleLoadMode]
  #     Append into destination table
  #
  #     Note: The following fields are mutually exclusive: `append`, `replace`, `maximum`, `unique`. If a field in that set is populated, all other fields in the set will automatically be cleared.
  # @!attribute [rw] maximum
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionIncrementalLoadMode]
  #     Insert records where the value exceeds the previous maximum value for
  #     a column in the destination table
  #
  #     Note: The following fields are mutually exclusive: `maximum`, `replace`, `append`, `unique`. If a field in that set is populated, all other fields in the set will automatically be cleared.
  # @!attribute [rw] unique
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionIncrementalLoadMode]
  #     Insert records where the value of a column is not already present in
  #     the destination table
  #
  #     Note: The following fields are mutually exclusive: `unique`, `replace`, `append`, `maximum`. If a field in that set is populated, all other fields in the set will automatically be cleared.
  class ActionLoadConfig
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Simple load definition
  class ActionSimpleLoadMode
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Load definition for incremental load modes
  # @!attribute [rw] column
  #   @return [::String]
  #     Column name for incremental load modes
  class ActionIncrementalLoadMode
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end

#job_id ⇒ ::String (readonly)

Returns Output only. The ID of the BigQuery job that executed the SQL in sql_script. Only set once the job has started to run.

Returns:

  • (::String) —

    Output only. The ID of the BigQuery job that executed the SQL in sql_script. Only set once the job has started to run.



2565
2566
2567
2568
2569
2570
2571
2572
2573
2574
2575
2576
2577
2578
2579
2580
2581
2582
2583
2584
2585
2586
2587
2588
2589
2590
2591
2592
2593
2594
2595
2596
2597
2598
2599
2600
2601
2602
2603
2604
2605
2606
2607
2608
2609
2610
2611
2612
2613
2614
2615
2616
2617
2618
2619
2620
2621
2622
2623
2624
2625
2626
2627
2628
2629
2630
2631
2632
2633
2634
2635
2636
2637
2638
2639
2640
2641
 
# File 'proto_docs/google/cloud/dataform/v1beta1/dataform.rb', line 2565

class DataPreparationAction
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  # Definition of a SQL Data Preparation
  # @!attribute [rw] query
  #   @return [::String]
  #     The SQL query representing the data preparation steps. Formatted as a
  #     Pipe SQL query statement.
  # @!attribute [rw] error_table
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionErrorTable]
  #     Error table configuration,
  # @!attribute [rw] load_config
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionLoadConfig]
  #     Load configuration.
  class ActionSqlDefinition
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Error table information, used to write error data into a BigQuery
  # table.
  # @!attribute [rw] target
  #   @return [::Google::Cloud::Dataform::V1beta1::Target]
  #     Error Table target.
  # @!attribute [rw] retention_days
  #   @return [::Integer]
  #     Error table partition expiration in days. Only positive values are
  #     allowed.
  class ActionErrorTable
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Simplified load configuration for actions
  # @!attribute [rw] replace
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionSimpleLoadMode]
  #     Replace destination table
  #
  #     Note: The following fields are mutually exclusive: `replace`, `append`, `maximum`, `unique`. If a field in that set is populated, all other fields in the set will automatically be cleared.
  # @!attribute [rw] append
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionSimpleLoadMode]
  #     Append into destination table
  #
  #     Note: The following fields are mutually exclusive: `append`, `replace`, `maximum`, `unique`. If a field in that set is populated, all other fields in the set will automatically be cleared.
  # @!attribute [rw] maximum
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionIncrementalLoadMode]
  #     Insert records where the value exceeds the previous maximum value for
  #     a column in the destination table
  #
  #     Note: The following fields are mutually exclusive: `maximum`, `replace`, `append`, `unique`. If a field in that set is populated, all other fields in the set will automatically be cleared.
  # @!attribute [rw] unique
  #   @return [::Google::Cloud::Dataform::V1beta1::WorkflowInvocationAction::DataPreparationAction::ActionIncrementalLoadMode]
  #     Insert records where the value of a column is not already present in
  #     the destination table
  #
  #     Note: The following fields are mutually exclusive: `unique`, `replace`, `append`, `maximum`. If a field in that set is populated, all other fields in the set will automatically be cleared.
  class ActionLoadConfig
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Simple load definition
  class ActionSimpleLoadMode
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end

  # Load definition for incremental load modes
  # @!attribute [rw] column
  #   @return [::String]
  #     Column name for incremental load modes
  class ActionIncrementalLoadMode
    include ::Google::Protobuf::MessageExts
    extend ::Google::Protobuf::MessageExts::ClassMethods
  end
end