Class: Fbe::Iterate

Inherits:
Object
  • Object
show all
Defined in:
lib/fbe/iterate.rb

Overview

An iterator.

Here, you go through all repositories defined by the repositories option in the $options, trying to run the provided query for each of them. If the query returns an integer that is different from the previously seen, the function keeps repeating the cycle. Otherwise, it will restart from the beginning.

Author

Yegor Bugayenko (yegor256@gmail.com)

Copyright

Copyright © 2024 Zerocracy

License

MIT

Instance Method Summary collapse

Constructor Details

#initialize(fb:, loog:, options:, global:) ⇒ Iterate

Ctor.

Parameters:

  • fb (Factbase) —

    The factbase

  • loog (Loog) —

    The logging facility

  • options (Judges::Options) —

    The options coming from the judges tool

  • global (Hash) —

    The hash for global caching



62
63
64
65
66
67
68
69
70
71
72
 
# File 'lib/fbe/iterate.rb', line 62

def initialize(fb:, loog:, options:, global:)
  @fb = fb
  @loog = loog
  @options = options
  @global = global
  @label = nil
  @since = 0
  @query = nil
  @repeats = 1
  @quota_aware = false
end

Instance Method Details

#as(label) ⇒ nil

Sets the label to use in the “marker” fact.

Parameters:

  • label (String) —

    The label

Returns:

  • (nil) —

    Nothing



107
108
109
110
111
 
# File 'lib/fbe/iterate.rb', line 107

def as(label)
  raise 'Label is already set' unless @label.nil?
  raise 'Cannot set "label" to nil' if label.nil?
  @label = label
end

#by(query) ⇒ nil

Sets the query to run.

Parameters:

  • query (String) —

    The query

Returns:

  • (nil) —

    Nothing



97
98
99
100
101
 
# File 'lib/fbe/iterate.rb', line 97

def by(query)
  raise 'Query is already set' unless @query.nil?
  raise 'Cannot set query to nil' if query.nil?
  @query = query
end

#over(timeout: 2 * 60) {|Array<Integer, Integer>| ... } ⇒ nil

It makes a number of repeats of going through all repositories provided by the repositories configuration option. In each “repeat” it yields the repository ID and a number that is retrieved by the query. The query is supplied with two parameter: $before the value from the previous repeat and $repository (GitHub repo ID).

Parameters:

  • timeout (Float) (defaults to: 2 * 60) —

    How many seconds to spend as a maximum

Yields:

  • (Array<Integer, Integer>) —

    Repository ID and the next number to be considered

Returns:

  • (nil) —

    Nothing



122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
 
# File 'lib/fbe/iterate.rb', line 122

def over(timeout: 2 * 60, &)
  raise 'Use "as" first' if @label.nil?
  raise 'Use "by" first' if @query.nil?
  seen = {}
  oct = Fbe.octo(loog: @loog, options: @options, global: @global)
  repos = Fbe.unmask_repos(loog: @loog, options: @options, global: @global)
  restarted = []
  start = Time.now
  loop do
    repos.each do |repo|
      if Time.now - start > timeout
        $loog.info("We are doing this for #{start.ago} already, won't check #{repo}")
        next
      end
      next if restarted.include?(repo)
      seen[repo] = 0 if seen[repo].nil?
      if seen[repo] >= @repeats
        @loog.debug("We've seen too many (#{seen[repo]}) in #{repo}, let's see next one")
        next
      end
      rid = oct.repo_id_by_name(repo)
      before = @fb.query(
        "(agg (and (eq what '#{@label}') (eq where 'github') (eq repository #{rid})) (first latest))"
      ).one
      @fb.query("(and (eq what '#{@label}') (eq where 'github') (eq repository #{rid}))").delete!
      before = before.nil? ? @since : before.first
      nxt = @fb.query(@query).one(before:, repository: rid)
      after =
        if nxt.nil?
          @loog.debug("Next element after ##{before} not suggested, re-starting from ##{@since}: #{@query}")
          restarted << repo
          @since
        else
          @loog.debug("Next is ##{nxt}, starting from it...")
          yield(rid, nxt)
        end
      raise "Iterator must return an Integer, while #{after.class} returned" unless after.is_a?(Integer)
      f = @fb.insert
      f.where = 'github'
      f.repository = rid
      f.latest =
        if after.nil?
          @loog.debug("After is nil at #{repo}, setting the 'latest' to ##{nxt}")
          nxt
        else
          @loog.debug("After is ##{after} at #{repo}, setting the 'latest' to it")
          after
        end
      f.what = @label
      seen[repo] += 1
      if oct.off_quota
        @loog.debug('We are off GitHub quota, time to stop')
        break
      end
    end
    if oct.off_quota
      @loog.info("We are off GitHub quota, time to stop after #{start.ago}")
      break
    end
    unless seen.any? { |r, v| v < @repeats && !restarted.include?(r) }
      @loog.debug("No more repos to scan (out of #{repos.size}), quitting after #{start.ago}")
      break
    end
    if restarted.size == repos.size
      @loog.debug("All #{repos.size} repos restarted, quitting after #{start.ago}")
      break
    end
  end
  @loog.debug("Finished scanning #{repos.size} repos in #{start.ago}: #{seen.map { |k, v| "#{k}:#{v}" }.join(', ')}")
end

#quota_aware ⇒ nil

Make this block aware of GitHub API quota.

When the quota is reached, the loop will gracefully stop.

Returns:

  • (nil) —

    Nothing



79
80
81
 
# File 'lib/fbe/iterate.rb', line 79

def quota_aware
  @quota_aware = true
end

#repeats(repeats) ⇒ nil

Sets the total counter of repeats to make.

Parameters:

  • repeats (Integer) —

    The total count of them

Returns:

  • (nil) —

    Nothing



87
88
89
90
91
 
# File 'lib/fbe/iterate.rb', line 87

def repeats(repeats)
  raise 'Cannot set "repeats" to nil' if repeats.nil?
  raise 'The "repeats" must be a positive integer' unless repeats.positive?
  @repeats = repeats
end