Class: Fbe::Iterate

Inherits:
Object
  • Object
show all
Defined in:
lib/fbe/iterate.rb

Overview

An iterator.

Here, you go through all repositories defined by the repositories option in the $options, trying to run the provided query for each of them. If the query returns an integer that is different from the previously seen, the function keeps repeating the cycle. Otherwise, it will restart from the beginning.

Author

Yegor Bugayenko (yegor256@gmail.com)

Copyright

Copyright © 2024-2025 Zerocracy

License

MIT

Instance Method Summary collapse

Constructor Details

#initialize(fb:, loog:, options:, global:) ⇒ Iterate

Ctor.

Parameters:

  • fb (Factbase) —

    The factbase

  • loog (Loog) —

    The logging facility

  • options (Judges::Options) —

    The options coming from the judges tool

  • global (Hash) —

    The hash for global caching



43
44
45
46
47
48
49
50
51
52
53
 
# File 'lib/fbe/iterate.rb', line 43

def initialize(fb:, loog:, options:, global:)
  @fb = fb
  @loog = loog
  @options = options
  @global = global
  @label = nil
  @since = 0
  @query = nil
  @repeats = 1
  @quota_aware = false
end

Instance Method Details

#as(label) ⇒ nil

Sets the label to use in the “marker” fact.

Parameters:

  • label (String) —

    The label

Returns:

  • (nil) —

    Nothing



88
89
90
91
92
 
# File 'lib/fbe/iterate.rb', line 88

def as(label)
  raise 'Label is already set' unless @label.nil?
  raise 'Cannot set "label" to nil' if label.nil?
  @label = label
end

#by(query) ⇒ nil

Sets the query to run.

Parameters:

  • query (String) —

    The query

Returns:

  • (nil) —

    Nothing



78
79
80
81
82
 
# File 'lib/fbe/iterate.rb', line 78

def by(query)
  raise 'Query is already set' unless @query.nil?
  raise 'Cannot set query to nil' if query.nil?
  @query = query
end

#over(timeout: 2 * 60) {|Array<Integer, Integer>| ... } ⇒ nil

It makes a number of repeats of going through all repositories provided by the repositories configuration option. In each “repeat” it yields the repository ID and a number that is retrieved by the query. The query is supplied with two parameter: $before the value from the previous repeat and $repository (GitHub repo ID).

Parameters:

  • timeout (Float) (defaults to: 2 * 60) —

    How many seconds to spend as a maximum

Yields:

  • (Array<Integer, Integer>) —

    Repository ID and the next number to be considered

Returns:

  • (nil) —

    Nothing



103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
 
# File 'lib/fbe/iterate.rb', line 103

def over(timeout: 2 * 60, &)
  raise 'Use "as" first' if @label.nil?
  raise 'Use "by" first' if @query.nil?
  seen = {}
  oct = Fbe.octo(loog: @loog, options: @options, global: @global)
  repos = Fbe.unmask_repos(loog: @loog, options: @options, global: @global)
  restarted = []
  start = Time.now
  loop do
    repos.each do |repo|
      if Time.now - start > timeout
        $loog.info("We are doing this for #{start.ago} already, won't check #{repo}")
        next
      end
      next if restarted.include?(repo)
      seen[repo] = 0 if seen[repo].nil?
      if seen[repo] >= @repeats
        @loog.debug("We've seen too many (#{seen[repo]}) in #{repo}, let's see next one")
        next
      end
      rid = oct.repo_id_by_name(repo)
      before = @fb.query(
        "(agg (and (eq what '#{@label}') (eq where 'github') (eq repository #{rid})) (first latest))"
      ).one
      @fb.query("(and (eq what '#{@label}') (eq where 'github') (eq repository #{rid}))").delete!
      before = before.nil? ? @since : before.first
      nxt = @fb.query(@query).one(@fb, before:, repository: rid)
      after =
        if nxt.nil?
          @loog.debug("Next element after ##{before} not suggested, re-starting from ##{@since}: #{@query}")
          restarted << repo
          @since
        else
          @loog.debug("Next is ##{nxt}, starting from it...")
          yield(rid, nxt)
        end
      raise "Iterator must return an Integer, while #{after.class} returned" unless after.is_a?(Integer)
      f = @fb.insert
      f.where = 'github'
      f.repository = rid
      f.latest =
        if after.nil?
          @loog.debug("After is nil at #{repo}, setting the 'latest' to ##{nxt}")
          nxt
        else
          @loog.debug("After is ##{after} at #{repo}, setting the 'latest' to it")
          after
        end
      f.what = @label
      seen[repo] += 1
      if oct.off_quota
        @loog.debug('We are off GitHub quota, time to stop')
        break
      end
    end
    if oct.off_quota
      @loog.info("We are off GitHub quota, time to stop after #{start.ago}")
      break
    end
    unless seen.any? { |r, v| v < @repeats && !restarted.include?(r) }
      @loog.debug("No more repos to scan (out of #{repos.size}), quitting after #{start.ago}")
      break
    end
    if restarted.size == repos.size
      @loog.debug("All #{repos.size} repos restarted, quitting after #{start.ago}")
      break
    end
    if Time.now - start > timeout
      $loog.info("We are iterating for #{start.ago} already, time to give up")
      break
    end
  end
  @loog.debug("Finished scanning #{repos.size} repos in #{start.ago}: #{seen.map { |k, v| "#{k}:#{v}" }.join(', ')}")
end

#quota_aware ⇒ nil

Make this block aware of GitHub API quota.

When the quota is reached, the loop will gracefully stop.

Returns:

  • (nil) —

    Nothing



60
61
62
 
# File 'lib/fbe/iterate.rb', line 60

def quota_aware
  @quota_aware = true
end

#repeats(repeats) ⇒ nil

Sets the total counter of repeats to make.

Parameters:

  • repeats (Integer) —

    The total count of them

Returns:

  • (nil) —

    Nothing



68
69
70
71
72
 
# File 'lib/fbe/iterate.rb', line 68

def repeats(repeats)
  raise 'Cannot set "repeats" to nil' if repeats.nil?
  raise 'The "repeats" must be a positive integer' unless repeats.positive?
  @repeats = repeats
end