Class: GitLab::SecretDetection::Core::Scanner

Inherits:
Object
  • Object
show all
Defined in:
lib/gitlab/secret_detection/core/scanner.rb

Overview

Scan is responsible for running Secret Detection scan operation

Constant Summary collapse

RulesetParseError =

RulesetParseError is thrown when the code fails to parse the ruleset file from the given path

Class.new(StandardError)
RulesetCompilationError =

RulesetCompilationError is thrown when the code fails to compile the predefined rulesets

Class.new(StandardError)
DEFAULT_SCAN_TIMEOUT_SECS =

default time limit(in seconds) for running the scan operation per invocation

180
DEFAULT_PAYLOAD_TIMEOUT_SECS =

default time limit(in seconds) for running the scan operation on a single payload

30
DEFAULT_PATTERN_MATCHER_TAGS =

Tags used for creating default pattern matcher

['gitlab_blocking'].freeze

Instance Method Summary collapse

Constructor Details

#initialize(rules:, logger: Logger.new($stdout)) ⇒ Scanner

Initializes the instance with logger along with following operations:

  1. Extract keywords from the parsed ruleset to use it for matching keywords before regex operation.

  2. Build and Compile rule regex patterns obtained from the ruleset with DEFAULT_PATTERN_MATCHER_TAGS

tags. Raises RulesetCompilationError in case the regex pattern compilation fails.



32
33
34
35
36
37
38
39
40
41
42
43
44
 
# File 'lib/gitlab/secret_detection/core/scanner.rb', line 32

def initialize(rules:, logger: Logger.new($stdout))
  @logger = logger
  @rules = rules
  @keywords = create_keywords(rules)
  @default_keyword_matcher = build_keyword_matcher(
    tags: DEFAULT_PATTERN_MATCHER_TAGS,
    include_missing_tags: false
  )
  @default_pattern_matcher = build_pattern_matcher(
    tags: DEFAULT_PATTERN_MATCHER_TAGS,
    include_missing_tags: false
  ) # includes only gitlab_blocking rules
end

Instance Method Details

#secrets_scan(payloads, timeout: DEFAULT_SCAN_TIMEOUT_SECS, payload_timeout: DEFAULT_PAYLOAD_TIMEOUT_SECS, raw_value_exclusions: [], rule_exclusions: [], tags: DEFAULT_PATTERN_MATCHER_TAGS) ⇒ Object

Runs Secret Detection scan on the list of given payloads. Both the total scan duration and the duration for each payload is time bound via timeout and payload_timeout respectively.

payloads

Array of payloads where each payload should have ‘id` and `data` properties.

timeout

No of seconds(accepts floating point for smaller time values) to limit the total scan duration

payload_timeout

No of seconds(accepts floating point for smaller time values) to limit

the scan duration on each payload
raw_value_exclusions:

Array of raw values to exclude from the scan.

rule_exclusions

Array of rules to exclude from the ruleset used for the scan. Each rule is represented

by its ID. For example: `gitlab_personal_access_token` for representing GitLab Personal Access
Token. By default, no rule is excluded from the ruleset.
tags

Array of tag values to filter from the default ruleset when determining the rules used for the scan.

For example: Add `gitlab_blocking` to include only rules for Push Protection. Defaults to
[`gitlab_blocking`] (+DEFAULT_PATTERN_MATCHER_TAGS+).

Returns an instance of GitLab::SecretDetection::Core::Response by following below structure: 

status: One of the Core::Status values
results: [SecretDetection::Finding]




67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
 
# File 'lib/gitlab/secret_detection/core/scanner.rb', line 67

def secrets_scan(
  payloads,
  timeout: DEFAULT_SCAN_TIMEOUT_SECS,
  payload_timeout: DEFAULT_PAYLOAD_TIMEOUT_SECS,
  raw_value_exclusions: [],
  rule_exclusions: [],
  tags: DEFAULT_PATTERN_MATCHER_TAGS
)

  return Core::Response.new(Core::Status::INPUT_ERROR) unless validate_scan_input(payloads)

  # assign defaults since grpc passing zero timeout value to `Timeout.timeout(..)` makes it effectively useless.
  timeout = DEFAULT_SCAN_TIMEOUT_SECS unless timeout.positive?
  payload_timeout = DEFAULT_PAYLOAD_TIMEOUT_SECS unless payload_timeout.positive?
  tags = DEFAULT_PATTERN_MATCHER_TAGS if tags.empty?

  Timeout.timeout(timeout) do
    keyword_matcher = build_keyword_matcher(tags:)

    matched_payloads = filter_by_keywords(keyword_matcher, payloads)

    next Core::Response.new(Core::Status::NOT_FOUND) if matched_payloads.empty?

    secrets = run_scan(
      payloads: matched_payloads, payload_timeout:,
      pattern_matcher: build_pattern_matcher(tags:),
      raw_value_exclusions:, rule_exclusions:
    )

    scan_status = overall_scan_status(secrets)

    Core::Response.new(scan_status, secrets)
  end
rescue Timeout::Error => e
  logger.error "Secret detection operation timed out: #{e}"

  Core::Response.new(Core::Status::SCAN_TIMEOUT)
end