Class: Aws::Glue::Types::SparkSQL

Inherits:
Struct
  • Object
show all
Includes:
Structure
Defined in:
lib/aws-sdk-glue/types.rb

Overview

Note:

When making an API call, you may pass SparkSQL data as a hash:

{
  name: "NodeName", # required
  inputs: ["NodeId"], # required
  sql_query: "SqlQuery", # required
  sql_aliases: [ # required
    {
      from: "NodeId", # required
      alias: "EnclosedInStringPropertyWithQuote", # required
    },
  ],
  output_schemas: [
    {
      columns: [
        {
          name: "GlueStudioColumnNameString", # required
          type: "ColumnTypeString",
        },
      ],
    },
  ],
}

Specifies a transform where you enter a SQL query using Spark SQL syntax to transform the data. The output is a single `DynamicFrame`.

See Also:

Constant Summary collapse

SENSITIVE = 
[]

Instance Attribute Summary collapse

Instance Attribute Details

#inputs ⇒ Array<String>

The data inputs identified by their node names. You can associate a table name with each input node to use in the SQL query. The name you choose must meet the Spark SQL naming restrictions.

Returns:

  • (Array<String>) 


21911
21912
21913
21914
21915
21916
21917
21918
21919
 
# File 'lib/aws-sdk-glue/types.rb', line 21911

class SparkSQL < Struct.new(
  :name,
  :inputs,
  :sql_query,
  :sql_aliases,
  :output_schemas)
  SENSITIVE = []
  include Aws::Structure
end

#name ⇒ String

The name of the transform node.

Returns:

  • (String) 


21911
21912
21913
21914
21915
21916
21917
21918
21919
 
# File 'lib/aws-sdk-glue/types.rb', line 21911

class SparkSQL < Struct.new(
  :name,
  :inputs,
  :sql_query,
  :sql_aliases,
  :output_schemas)
  SENSITIVE = []
  include Aws::Structure
end

#output_schemas ⇒ Array<Types::GlueSchema>

Specifies the data schema for the SparkSQL transform.

Returns:



21911
21912
21913
21914
21915
21916
21917
21918
21919
 
# File 'lib/aws-sdk-glue/types.rb', line 21911

class SparkSQL < Struct.new(
  :name,
  :inputs,
  :sql_query,
  :sql_aliases,
  :output_schemas)
  SENSITIVE = []
  include Aws::Structure
end

#sql_aliases ⇒ Array<Types::SqlAlias>

A list of aliases. An alias allows you to specify what name to use in the SQL for a given input. For example, you have a datasource named “MyDataSource”. If you specify `From` as MyDataSource, and `Alias` as SqlName, then in your SQL you can do:

`select * from SqlName`

and that gets data from MyDataSource.

Returns:



21911
21912
21913
21914
21915
21916
21917
21918
21919
 
# File 'lib/aws-sdk-glue/types.rb', line 21911

class SparkSQL < Struct.new(
  :name,
  :inputs,
  :sql_query,
  :sql_aliases,
  :output_schemas)
  SENSITIVE = []
  include Aws::Structure
end

#sql_query ⇒ String

A SQL query that must use Spark SQL syntax and return a single data set.

Returns:

  • (String) 


21911
21912
21913
21914
21915
21916
21917
21918
21919
 
# File 'lib/aws-sdk-glue/types.rb', line 21911

class SparkSQL < Struct.new(
  :name,
  :inputs,
  :sql_query,
  :sql_aliases,
  :output_schemas)
  SENSITIVE = []
  include Aws::Structure
end