Class: Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRule
- Inherits:
-
Object
- Object
- Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRule
- Includes:
- Core::Hashable, Core::JsonObjectSupport
- Defined in:
- lib/google/apis/dataplex_v1/classes.rb,
lib/google/apis/dataplex_v1/representations.rb,
lib/google/apis/dataplex_v1/representations.rb
Overview
A rule captures data quality intent about a data source.
Instance Attribute Summary collapse
-
#column ⇒ String
Optional.
-
#description ⇒ String
Optional.
-
#dimension ⇒ String
Required.
-
#ignore_null ⇒ Boolean
(also: #ignore_null?)
Optional.
-
#name ⇒ String
Optional.
-
#non_null_expectation ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleNonNullExpectation
Evaluates whether each column value is null.
-
#range_expectation ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleRangeExpectation
Evaluates whether each column value lies between a specified range.
-
#regex_expectation ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleRegexExpectation
Evaluates whether each column value matches a specified regex.
-
#row_condition_expectation ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleRowConditionExpectation
Evaluates whether each row passes the specified condition.The SQL expression needs to use BigQuery standard SQL syntax and should produce a boolean value per row as the result.Example: col1 >= 0 AND col2 < 10 Corresponds to the JSON property
rowConditionExpectation
. -
#set_expectation ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleSetExpectation
Evaluates whether each column value is contained by a specified set.
-
#sql_assertion ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleSqlAssertion
A SQL statement that is evaluated to return rows that match an invalid state.
-
#statistic_range_expectation ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleStatisticRangeExpectation
Evaluates whether the column aggregate statistic lies between a specified range.
-
#table_condition_expectation ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleTableConditionExpectation
Evaluates whether the provided expression is true.The SQL expression needs to use BigQuery standard SQL syntax and should produce a scalar boolean result.
-
#threshold ⇒ Float
Optional.
-
#uniqueness_expectation ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleUniquenessExpectation
Evaluates whether the column has duplicates.
Instance Method Summary collapse
-
#initialize(**args) ⇒ GoogleCloudDataplexV1DataQualityRule
constructor
A new instance of GoogleCloudDataplexV1DataQualityRule.
-
#update!(**args) ⇒ Object
Update properties of this object.
Constructor Details
#initialize(**args) ⇒ GoogleCloudDataplexV1DataQualityRule
Returns a new instance of GoogleCloudDataplexV1DataQualityRule.
2321 2322 2323 |
# File 'lib/google/apis/dataplex_v1/classes.rb', line 2321 def initialize(**args) update!(**args) end |
Instance Attribute Details
#column ⇒ String
Optional. The unnested column which this rule is evaluated against.
Corresponds to the JSON property column
2226 2227 2228 |
# File 'lib/google/apis/dataplex_v1/classes.rb', line 2226 def column @column end |
#description ⇒ String
Optional. Description of the rule. The maximum length is 1,024 characters.
Corresponds to the JSON property description
2231 2232 2233 |
# File 'lib/google/apis/dataplex_v1/classes.rb', line 2231 def description @description end |
#dimension ⇒ String
Required. The dimension a rule belongs to. Results are also aggregated at the
dimension level. Supported dimensions are "COMPLETENESS", "ACCURACY", "
CONSISTENCY", "VALIDITY", "UNIQUENESS", "INTEGRITY"
Corresponds to the JSON property dimension
2238 2239 2240 |
# File 'lib/google/apis/dataplex_v1/classes.rb', line 2238 def dimension @dimension end |
#ignore_null ⇒ Boolean Also known as: ignore_null?
Optional. Rows with null values will automatically fail a rule, unless
ignore_null is true. In that case, such null rows are trivially considered
passing.This field is only valid for the following type of rules:
RangeExpectation RegexExpectation SetExpectation UniquenessExpectation
Corresponds to the JSON property ignoreNull
2246 2247 2248 |
# File 'lib/google/apis/dataplex_v1/classes.rb', line 2246 def ignore_null @ignore_null end |
#name ⇒ String
Optional. A mutable name for the rule. The name must contain only letters (a-z,
A-Z), numbers (0-9), or hyphens (-). The maximum length is 63 characters.
Must start with a letter. Must end with a number or a letter.
Corresponds to the JSON property name
2254 2255 2256 |
# File 'lib/google/apis/dataplex_v1/classes.rb', line 2254 def name @name end |
#non_null_expectation ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleNonNullExpectation
Evaluates whether each column value is null.
Corresponds to the JSON property nonNullExpectation
2259 2260 2261 |
# File 'lib/google/apis/dataplex_v1/classes.rb', line 2259 def non_null_expectation @non_null_expectation end |
#range_expectation ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleRangeExpectation
Evaluates whether each column value lies between a specified range.
Corresponds to the JSON property rangeExpectation
2264 2265 2266 |
# File 'lib/google/apis/dataplex_v1/classes.rb', line 2264 def range_expectation @range_expectation end |
#regex_expectation ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleRegexExpectation
Evaluates whether each column value matches a specified regex.
Corresponds to the JSON property regexExpectation
2269 2270 2271 |
# File 'lib/google/apis/dataplex_v1/classes.rb', line 2269 def regex_expectation @regex_expectation end |
#row_condition_expectation ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleRowConditionExpectation
Evaluates whether each row passes the specified condition.The SQL expression
needs to use BigQuery standard SQL syntax and should produce a boolean value
per row as the result.Example: col1 >= 0 AND col2 < 10
Corresponds to the JSON property rowConditionExpectation
2276 2277 2278 |
# File 'lib/google/apis/dataplex_v1/classes.rb', line 2276 def row_condition_expectation @row_condition_expectation end |
#set_expectation ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleSetExpectation
Evaluates whether each column value is contained by a specified set.
Corresponds to the JSON property setExpectation
2281 2282 2283 |
# File 'lib/google/apis/dataplex_v1/classes.rb', line 2281 def set_expectation @set_expectation end |
#sql_assertion ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleSqlAssertion
A SQL statement that is evaluated to return rows that match an invalid state.
If any rows are are returned, this rule fails.The SQL statement must use
BigQuery standard SQL syntax, and must not contain any semicolons.You can use
the data reference parameter $data()
to reference the source table with all
of its precondition filters applied. Examples of precondition filters include
row filters, incremental data filters, and sampling. For more information, see
Data reference parameter (https://cloud.google.com/dataplex/docs/auto-data-
quality-overview#data-reference-parameter).Example: SELECT * FROM $data()
WHERE price < 0
Corresponds to the JSON property sqlAssertion
2294 2295 2296 |
# File 'lib/google/apis/dataplex_v1/classes.rb', line 2294 def sql_assertion @sql_assertion end |
#statistic_range_expectation ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleStatisticRangeExpectation
Evaluates whether the column aggregate statistic lies between a specified
range.
Corresponds to the JSON property statisticRangeExpectation
2300 2301 2302 |
# File 'lib/google/apis/dataplex_v1/classes.rb', line 2300 def statistic_range_expectation @statistic_range_expectation end |
#table_condition_expectation ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleTableConditionExpectation
Evaluates whether the provided expression is true.The SQL expression needs to
use BigQuery standard SQL syntax and should produce a scalar boolean result.
Example: MIN(col1) >= 0
Corresponds to the JSON property tableConditionExpectation
2307 2308 2309 |
# File 'lib/google/apis/dataplex_v1/classes.rb', line 2307 def table_condition_expectation @table_condition_expectation end |
#threshold ⇒ Float
Optional. The minimum ratio of passing_rows / total_rows required to pass this
rule, with a range of 0.0, 1.0.0 indicates default value (i.e. 1.0).This field
is only valid for row-level type rules.
Corresponds to the JSON property threshold
2314 2315 2316 |
# File 'lib/google/apis/dataplex_v1/classes.rb', line 2314 def threshold @threshold end |
#uniqueness_expectation ⇒ Google::Apis::DataplexV1::GoogleCloudDataplexV1DataQualityRuleUniquenessExpectation
Evaluates whether the column has duplicates.
Corresponds to the JSON property uniquenessExpectation
2319 2320 2321 |
# File 'lib/google/apis/dataplex_v1/classes.rb', line 2319 def uniqueness_expectation @uniqueness_expectation end |
Instance Method Details
#update!(**args) ⇒ Object
Update properties of this object
2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 |
# File 'lib/google/apis/dataplex_v1/classes.rb', line 2326 def update!(**args) @column = args[:column] if args.key?(:column) @description = args[:description] if args.key?(:description) @dimension = args[:dimension] if args.key?(:dimension) @ignore_null = args[:ignore_null] if args.key?(:ignore_null) @name = args[:name] if args.key?(:name) @non_null_expectation = args[:non_null_expectation] if args.key?(:non_null_expectation) @range_expectation = args[:range_expectation] if args.key?(:range_expectation) @regex_expectation = args[:regex_expectation] if args.key?(:regex_expectation) @row_condition_expectation = args[:row_condition_expectation] if args.key?(:row_condition_expectation) @set_expectation = args[:set_expectation] if args.key?(:set_expectation) @sql_assertion = args[:sql_assertion] if args.key?(:sql_assertion) @statistic_range_expectation = args[:statistic_range_expectation] if args.key?(:statistic_range_expectation) @table_condition_expectation = args[:table_condition_expectation] if args.key?(:table_condition_expectation) @threshold = args[:threshold] if args.key?(:threshold) @uniqueness_expectation = args[:uniqueness_expectation] if args.key?(:uniqueness_expectation) end |