Module: Fbe
- Defined in:
- lib/fbe.rb
Overview
The main and only module of this gem.
- Author
Yegor Bugayenko (yegor256@gmail.com)
- Copyright
Copyright (c) 2024-2026 Zerocracy
- License
MIT
Defined Under Namespace
Modules: Middleware Classes: Award, Conclude, Error, FakeOctokit, Graph, Iterate, OffQuota, Tombstone
Constant Summary collapse
- VERSION =
'0.48.7'- SEARCH_METHODS =
%i[ search_issues search_commits search_repositories search_users search_code search_topics ].freeze
Class Method Summary collapse
-
.bylaws(anger: 2, love: 2, paranoia: 2) ⇒ Hash<String, String>
Generates policies/bylaws from Liquid templates.
-
.conclude(fb: Fbe.fb, judge: $judge, loog: $loog, options: $options, global: $global, epoch: $epoch || Time.now, kickoff: $kickoff || Time.now) {|Factbase::Fact| ... } ⇒ Object
Creates an instance of Conclude and evals it with the block provided.
-
.consider(query, fb: Fbe.fb, judge: $judge, loog: $loog, options: $options, global: $global, epoch: $epoch || Time.now, kickoff: $kickoff || Time.now, lifetime_aware: true, timeout_aware: true, quota_aware: true) {|Factbase::Fact| ... } ⇒ Object
Creates an instance of Conclude and then runs "consider" in it.
-
.copy(source, target, except: []) ⇒ Integer
Makes a copy of a fact, moving all properties to a new fact.
-
.delete(fact, *props, fb: Fbe.fb, id: '_id') ⇒ nil
Delete properties from a fact by creating a new fact without them.
-
.delete_one(fact, prop, value, fb: Fbe.fb, id: '_id') ⇒ nil
Delete one value of a property.
-
.enter(badge, why, options: $options, loog: $loog) { ... } ⇒ Object
Enter a new valve in the Zerocracy system.
-
.fb(fb: $fb, global: $global, options: $options, loog: $loog) ⇒ Factbase
Returns an instance of
Factbase(cached). -
.github_graph(options: $options, global: $global, loog: $loog) ⇒ Fbe::Graph
Creates an instance of Graph.
-
.if_absent(fb: Fbe.fb, always: false) {|Factbase::Fact| ... } ⇒ nil, Factbase::Fact
Injects a fact if it's absent in the factbase, otherwise returns nil.
-
.issue(fact, options: $options, global: $global, loog: $loog) ⇒ String
Converts GitHub repository and issue IDs into a formatted issue reference.
-
.iterate(fb: Fbe.fb, loog: $loog, options: $options, global: $global, epoch: $epoch || Time.now, kickoff: $kickoff || Time.now) { ... } ⇒ Object
Creates an instance of Iterate and evaluates it with the provided block.
-
.just_one(fb: Fbe.fb) {|Factbase::Fact| ... } ⇒ Factbase::Fact
Ensures exactly one fact exists with the specified attributes in the factbase.
-
.kill_if(facts, fb: Fbe.fb, fid: '_id') ⇒ Object
Delete a few facts, knowing their IDs.
-
.mask_to_regex(mask) ⇒ Regexp
Converts a repository mask pattern to a regular expression.
-
.octo(options: $options, global: $global, loog: $loog) ⇒ Hash
Makes a call to the GitHub API.
-
.over?(global: $global, options: $options, loog: $loog, epoch: $epoch || Time.now, kickoff: $kickoff || Time.now, quota_aware: true, lifetime_aware: true, timeout_aware: true) ⇒ Boolean
Check GitHub API quota, lifetime, and timeout.
-
.overwrite(fact, property_or_hash, values = nil, fb: Fbe.fb, fid: '_id') ⇒ nil
Overwrites a property in the fact by recreating the entire fact.
-
.pmp(fb: Fbe.fb, global: $global, options: $options, loog: $loog) ⇒ Object
Takes configuration parameter from the "PMP" fact.
-
.regularly(area, p_every_days, p_since_days = nil, fb: Fbe.fb, judge: $judge, loog: $loog) {|Factbase::Fact| ... } ⇒ nil
Run the block provided every X days based on PMP configuration.
-
.repeatedly(area, p_every_hours, fb: Fbe.fb, judge: $judge, loog: $loog) {|Factbase::Fact| ... } ⇒ nil
Run the block provided every X hours based on PMP configuration.
-
.sec(fact, prop = :seconds) ⇒ String
Converts number of seconds into human-readable time format.
-
.unmask_repos(options: $options, global: $global, loog: $loog, epoch: $epoch || Time.now, kickoff: $kickoff || Time.now, quota_aware: true, lifetime_aware: true, timeout_aware: true) ⇒ Array<String>
Resolves repository masks to actual GitHub repository names.
-
.who(fact, prop = :who, options: $options, global: $global, loog: $loog) ⇒ String
Converts a GitHub user ID into a formatted username string.
Class Method Details
.bylaws(anger: 2, love: 2, paranoia: 2) ⇒ Hash<String, String>
Generates policies/bylaws from Liquid templates.
Using the templates stored in the assets/bylaws directory, this function
creates a hash where keys are bylaw names (derived from filenames) and values
are the rendered formulas. Templates can use three parameters to control
the strictness and generosity of the bylaws.
38 39 40 41 42 43 44 45 46 47 48 49 50 |
# File 'lib/fbe/bylaws.rb', line 38 def Fbe.bylaws(anger: 2, love: 2, paranoia: 2) raise(Fbe::Error, "The 'anger' must be in the [0..4] interval: #{anger.inspect}") unless !anger.negative? && anger < 5 raise(Fbe::Error, "The 'love' must be in the [0..4] interval: #{love.inspect}") unless !love.negative? && love < 5 unless paranoia.positive? && paranoia < 5 raise(Fbe::Error, "The 'paranoia' must be in the [1..4] interval: #{paranoia.inspect}") end home = File.join(__dir__, '../../assets/bylaws') raise(Fbe::Error, "The directory with templates is absent #{home.inspect}") unless File.exist?(home) Dir[File.join(home, '*.fe.liquid')].to_h do |f| formula = Liquid::Template.parse(File.read(f)).render('anger' => anger, 'love' => love, 'paranoia' => paranoia) [File.basename(f).gsub(/\.fe.liquid$/, ''), formula] end end |
.conclude(fb: Fbe.fb, judge: $judge, loog: $loog, options: $options, global: $global, epoch: $epoch || Time.now, kickoff: $kickoff || Time.now) {|Factbase::Fact| ... } ⇒ Object
Creates an instance of Conclude and evals it with the block provided.
23 24 25 26 27 28 29 30 31 32 33 34 |
# File 'lib/fbe/conclude.rb', line 23 def Fbe.conclude( fb: Fbe.fb, judge: $judge, loog: $loog, options: $options, global: $global, epoch: $epoch || Time.now, kickoff: $kickoff || Time.now, & ) raise(Fbe::Error, 'The fb is nil') if fb.nil? raise(Fbe::Error, 'The $judge is not set') if judge.nil? raise(Fbe::Error, 'The $global is not set') if global.nil? raise(Fbe::Error, 'The $options is not set') if .nil? raise(Fbe::Error, 'The $loog is not set') if loog.nil? c = Fbe::Conclude.new(fb:, judge:, loog:, options:, global:, epoch:, kickoff:) c.instance_eval(&) end |
.consider(query, fb: Fbe.fb, judge: $judge, loog: $loog, options: $options, global: $global, epoch: $epoch || Time.now, kickoff: $kickoff || Time.now, lifetime_aware: true, timeout_aware: true, quota_aware: true) {|Factbase::Fact| ... } ⇒ Object
Creates an instance of Conclude and then runs "consider" in it.
24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
# File 'lib/fbe/consider.rb', line 24 def Fbe.consider( query, fb: Fbe.fb, judge: $judge, loog: $loog, options: $options, global: $global, epoch: $epoch || Time.now, kickoff: $kickoff || Time.now, lifetime_aware: true, timeout_aware: true, quota_aware: true, & ) Fbe.conclude(fb:, judge:, loog:, options:, global:, epoch:, kickoff:) do on(query) timeout_unaware unless timeout_aware lifetime_unaware unless lifetime_aware quota_unaware unless quota_aware consider(&) end end |
.copy(source, target, except: []) ⇒ Integer
Existing properties in target are preserved (not overwritten)
Makes a copy of a fact, moving all properties to a new fact.
All properties from the source will be copied to the target, except those
listed in the except array. Only copies properties that don't already exist
in the target. Multi-valued properties are copied with all their values.
26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
# File 'lib/fbe/copy.rb', line 26 def Fbe.copy(source, target, except: []) raise(Fbe::Error, 'The source is nil') if source.nil? raise(Fbe::Error, 'The target is nil') if target.nil? raise(Fbe::Error, 'The except is nil') if except.nil? copied = 0 source.all_properties.each do |k| next unless target[k].nil? next if except.include?(k) source[k].each do |v| target.public_send(:"#{k}=", v) copied += 1 end end copied end |
.delete(fact, *props, fb: Fbe.fb, id: '_id') ⇒ nil
Delete properties from a fact by creating a new fact without them.
This method doesn't modify the original fact. Instead, it deletes the existing fact from the factbase and creates a new one with all properties except those specified for deletion.
25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
# File 'lib/fbe/delete.rb', line 25 def Fbe.delete(fact, *props, fb: Fbe.fb, id: '_id') raise(Fbe::Error, 'The fact is nil') if fact.nil? return if props.all? { |k| fact[k].nil? } i = fact[id] raise(Fbe::Error, "There is no #{id.inspect} in the fact") if i.nil? i = i.first before = {} fact.all_properties.each do |k| next if props.include?(k) before[k] = fact[k] end fb.query("(eq #{id} #{i})").delete! fb.txn do |fbt| c = fbt.insert f = c while f.instance_variable_defined?(:@fact) || f.instance_variable_defined?(:@origin) iv = f.instance_variable_defined?(:@fact) ? :@fact : :@origin f = f.instance_variable_get(iv) end map = f.instance_variable_get(:@map) %w[_id _time _version _job].each { |k| map.delete(k) if before.key?(k) } before.each do |k, vv| next unless c[k].nil? vv.each do |v| c.public_send(:"#{k}=", v) end end end nil end |
.delete_one(fact, prop, value, fb: Fbe.fb, id: '_id') ⇒ nil
Delete one value of a property.
This method doesn't modify the original fact. Instead, it deletes the existing fact from the factbase and creates a new one with all properties except the one specified for deletion.
21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
# File 'lib/fbe/delete_one.rb', line 21 def Fbe.delete_one(fact, prop, value, fb: Fbe.fb, id: '_id') raise(Fbe::Error, 'The fact is nil') if fact.nil? i = fact[id] raise(Fbe::Error, "There is no #{id.inspect} in the fact") if i.nil? i = i.first before = {} fact.all_properties.each do |k| before[k] = fact[k] end return unless before[prop] nv = before[prop] - [value] return if nv == before[prop] before[prop] = nv before.delete(prop) if nv.empty? fb.query("(eq #{id} #{i})").delete! fb.txn do |fbt| c = fbt.insert f = c while f.instance_variable_defined?(:@fact) || f.instance_variable_defined?(:@origin) iv = f.instance_variable_defined?(:@fact) ? :@fact : :@origin f = f.instance_variable_get(iv) end map = f.instance_variable_get(:@map) %w[_id _time _version _job].each { |k| map.delete(k) if before.key?(k) } before.each do |k, vv| next unless c[k].nil? vv.each do |v| c.public_send(:"#{k}=", v) end end end nil end |
.enter(badge, why, options: $options, loog: $loog) { ... } ⇒ Object
Requires $options and $loog global variables to be set
In testing mode (options.testing != nil), bypasses valve recording
Enter a new valve in the Zerocracy system.
A valve is a checkpoint or gate in the processing pipeline. This method records the entry into a valve with a reason, unless in testing mode.
28 29 30 31 32 33 34 35 36 |
# File 'lib/fbe/enter.rb', line 28 def Fbe.enter(badge, why, options: $options, loog: $loog, &) raise(Fbe::Error, 'The badge is nil') if badge.nil? raise(Fbe::Error, 'The why is nil') if why.nil? raise(Fbe::Error, 'The $options is not set') if .nil? raise(Fbe::Error, 'The $loog is not set') if loog.nil? return yield unless .testing.nil? baza = BazaRb.new('api.zerocracy.com', 443, .zerocracy_token, loog:) baza.enter(.job_name, badge, why, .job_id.nil? ? 0 : Integer(.job_id.to_s, 10), &) end |
.fb(fb: $fb, global: $global, options: $options, loog: $loog) ⇒ Factbase
Returns an instance of Factbase (cached).
Instead of using $fb directly, it is recommended to use this utility
method. It will not only return the global factbase, but will also
make sure it's properly decorated and cached.
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
# File 'lib/fbe/fb.rb', line 29 def Fbe.fb(fb: $fb, global: $global, options: $options, loog: $loog) raise(Fbe::Error, 'The fb is nil') if fb.nil? raise(Fbe::Error, 'The $global is not set') if global.nil? raise(Fbe::Error, 'The $options is not set') if .nil? raise(Fbe::Error, 'The $loog is not set') if loog.nil? global[:mutex] ||= Mutex.new global[:mutex].synchronize do global[:fb] ||= begin rules = Dir.glob(File.join(File.join(__dir__, '../../rules'), '*.fe')).map { |f| File.read(f) } fbe = Factbase::Rules.new( fb, "(and \n#{rules.join("\n")}\n)", uid: '_id' ) fbe = Factbase::Pre.new(fbe) do |f, fbt| max = fbt.query('(max _id)').one f._id = (max.nil? ? 0 : max) + 1 f._time = Time.now f._version = [Factbase::VERSION, Judges::VERSION, .action_version].compact.join('/') f._job = Integer(.job_id.to_s, 10) if .job_id end Factbase::Impatient.new( Factbase::Logged.new( Factbase::SyncFactbase.new( Factbase::IndexedFactbase.new( Factbase::CachedFactbase.new( fbe ) ) ), loog ), timeout: 60 ) end end end |
.github_graph(options: $options, global: $global, loog: $loog) ⇒ Fbe::Graph
Creates an instance of Graph.
17 18 19 20 21 22 23 24 25 26 27 28 |
# File 'lib/fbe/github_graph.rb', line 17 def Fbe.github_graph(options: $options, global: $global, loog: $loog) global[:mutex] ||= Mutex.new global[:mutex].synchronize do global[:github_graph] ||= if .testing.nil? Fbe::Graph.new(token: .github_token || ENV.fetch('GITHUB_TOKEN', nil)) else loog.debug('The connection to GitHub GraphQL API is mocked') Fbe::Graph::Fake.new end end end |
.if_absent(fb: Fbe.fb, always: false) {|Factbase::Fact| ... } ⇒ nil, Factbase::Fact
String values are properly escaped in queries
Time values are converted to UTC ISO8601 format for comparison
Injects a fact if it's absent in the factbase, otherwise returns nil.
Checks if a fact with the same property values already exists. If not, creates a new fact. System properties (_id, _time, _version) are excluded from the uniqueness check.
Here is what you do when you want to add a fact to the factbase, but don't want to make a duplicate of an existing one:
require 'fbe/if_absent' n = Fbe.if_absent do |f| f.what = 'something' f.details = 'important' end return if n.nil? # Fact already existed n.when = Time.now # Add additional properties to the new fact
This code will definitely create one fact with what equals to something
and details equals to important, while the when will be equal to the
time of its first creation.
49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 |
# File 'lib/fbe/if_absent.rb', line 49 def Fbe.if_absent(fb: Fbe.fb, always: false) attrs = {} f = others(map: attrs) do |*args| k = args[0] if k.end_with?('=') k = k[0..-2].to_sym v = args[1] raise(Fbe::Error, "Can't set #{k} to nil") if v.nil? raise(Fbe::Error, "Can't set #{k} to empty string") if v.is_a?(String) && v.empty? @map[k] = v else @map[k.to_sym] end end yield(f) q = attrs.except(:_id, :_time, :_version).map do |k, v| vv = v.to_s if v.is_a?(String) vv = "'#{vv.gsub('"', '\\\\"').gsub("'", "\\\\'")}'" elsif v.is_a?(Time) vv = v.utc.iso8601 end "(eq #{k} #{vv})" end.join(' ') q = "(and #{q})" before = fb.query(q).each.first return before if before && always return nil if before n = fb.insert attrs.each { |k, v| n.public_send(:"#{k}=", v) } n end |
.issue(fact, options: $options, global: $global, loog: $loog) ⇒ String
Requires 'repository' and 'issue' properties in the fact
Repository names are cached to reduce GitHub API calls
Converts GitHub repository and issue IDs into a formatted issue reference.
Takes the repository and issue properties from the provided fact,
queries the GitHub API to get the repository's full name, and formats it
as a standard GitHub issue reference (e.g., "zerocracy/fbe#42").
Results are cached globally to minimize API calls.
30 31 32 33 34 35 36 37 38 39 40 41 42 |
# File 'lib/fbe/issue.rb', line 30 def Fbe.issue(fact, options: $options, global: $global, loog: $loog) raise(Fbe::Error, 'The fact is nil') if fact.nil? raise(Fbe::Error, 'The $global is not set') if global.nil? raise(Fbe::Error, 'The $options is not set') if .nil? raise(Fbe::Error, 'The $loog is not set') if loog.nil? rid = fact['repository'] raise(Fbe::Error, "There is no 'repository' property") if rid.nil? rid = Integer(rid.first.to_s, 10) issue = fact['issue'] raise(Fbe::Error, "There is no 'issue' property") if issue.nil? issue = Integer(issue.first.to_s, 10) "#{Fbe.octo(global:, options:, loog:).repo_name_by_id(rid)}##{issue}" end |
.iterate(fb: Fbe.fb, loog: $loog, options: $options, global: $global, epoch: $epoch || Time.now, kickoff: $kickoff || Time.now) { ... } ⇒ Object
Creates an instance of Iterate and evaluates it with the provided block.
This is a convenience method that creates an iterator instance and evaluates the DSL block within its context. The iterator processes repositories defined in options.repositories, executing queries and managing state for each.
41 42 43 44 45 46 47 48 49 50 51 |
# File 'lib/fbe/iterate.rb', line 41 def Fbe.iterate( fb: Fbe.fb, loog: $loog, options: $options, global: $global, epoch: $epoch || Time.now, kickoff: $kickoff || Time.now, & ) raise(Fbe::Error, 'The fb is nil') if fb.nil? raise(Fbe::Error, 'The $global is not set') if global.nil? raise(Fbe::Error, 'The $options is not set') if .nil? raise(Fbe::Error, 'The $loog is not set') if loog.nil? c = Fbe::Iterate.new(fb:, loog:, options:, global:, epoch:, kickoff:) c.instance_eval(&) end |
.just_one(fb: Fbe.fb) {|Factbase::Fact| ... } ⇒ Factbase::Fact
System attributes (_id, _time, _version) are ignored when matching
Ensures exactly one fact exists with the specified attributes in the factbase.
This method creates a new fact if none exists with the given attributes, or returns an existing fact if one already matches. Useful for preventing duplicate facts while ensuring required facts exist.
35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
# File 'lib/fbe/just_one.rb', line 35 def Fbe.just_one(fb: Fbe.fb) attrs = {} f = others(map: attrs) do |*args| k = args[0] if k.end_with?('=') v = args[1] raise(Fbe::Error, "Can't set #{k[0..-2]} to nil") if v.nil? raise(Fbe::Error, "Can't set #{k[0..-2]} to empty string") if v.is_a?(String) && v.empty? @map[k[0..-2].to_sym] = v else @map[k.to_sym] end end yield(f) q = attrs.except(:_id, :_time, :_version).map do |k, v| vv = v.to_s if v.is_a?(String) vv = "'#{vv.gsub('"', '\\\\"').gsub("'", "\\\\'")}'" elsif v.is_a?(Time) vv = v.utc.iso8601 end "(eq #{k} #{vv})" end.join(' ') q = "(and #{q})" before = fb.query(q).each.first return before unless before.nil? n = fb.insert attrs.each { |k, v| n.public_send(:"#{k}=", v) } n end |
.kill_if(facts, fb: Fbe.fb, fid: '_id') ⇒ Object
Delete a few facts, knowing their IDs.
13 14 15 16 17 18 19 20 21 22 23 24 |
# File 'lib/fbe/kill_if.rb', line 13 def Fbe.kill_if(facts, fb: Fbe.fb, fid: '_id') ids = [] facts.each do |f| if block_given? t = yield(f) next unless t end ids << f[fid].first end return 0 if ids.empty? fb.query("(or #{ids.map { |id| "(eq #{fid} #{id})" }.join})").delete! end |
.mask_to_regex(mask) ⇒ Regexp
Converts a repository mask pattern to a regular expression.
24 25 26 27 28 |
# File 'lib/fbe/unmask_repos.rb', line 24 def Fbe.mask_to_regex(mask) org, repo = mask.split('/') raise(Fbe::Error, "Org '#{org}' can't have an asterisk") if org.include?('*') Regexp.compile("\\A#{Regexp.escape(org)}/#{Regexp.escape(repo).gsub('\\*', '.*')}\\z", Regexp::IGNORECASE) end |
.octo(options: $options, global: $global, loog: $loog) ⇒ Hash
Makes a call to the GitHub API.
It is supposed to be used instead of Octokit::Client, because it
is pre-configured and enables additional features, such as retrying,
logging, and caching.
49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 |
# File 'lib/fbe/octo.rb', line 49 def Fbe.octo(options: $options, global: $global, loog: $loog) # rubocop:disable Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/MethodLength, Metrics/PerceivedComplexity raise(Fbe::Error, 'The $global is not set') if global.nil? raise(Fbe::Error, 'The $options is not set') if .nil? raise(Fbe::Error, 'The $loog is not set') if loog.nil? global[:mutex] ||= Mutex.new global[:mutex].synchronize do # rubocop:disable Metrics/BlockLength global[:octo] ||= begin loog.info("Fbe version is #{Fbe::VERSION}") trace = [] limits = {} if .testing.nil? o = Octokit::Client.new token = .github_token if token.nil? loog.debug("The 'github_token' option is not provided") token = ENV.fetch('GITHUB_TOKEN', nil) if token.nil? loog.debug("The 'GITHUB_TOKEN' environment variable is not set") else loog.debug("The 'GITHUB_TOKEN' environment was provided") end else loog.debug("The 'github_token' option was provided (#{token.length} chars)") end if token.nil? loog.warn('Accessing GitHub API without a token!') elsif token.empty? loog.warn('The GitHub API token is an empty string, won\'t use it') else o = Octokit::Client.new(access_token: token) end o.auto_paginate = true o.per_page = 100 o. = { request: { open_timeout: 15, timeout: 15 } } stack = Faraday::RackBuilder.new do |builder| builder.use( Faraday::Retry::Middleware, exceptions: Faraday::Retry::Middleware::DEFAULT_EXCEPTIONS + [ Octokit::TooManyRequests, Octokit::ServiceUnavailable ], max: 4, interval: ENV['RACK_ENV'] == 'test' ? 0.01 : 4, methods: [:get], backoff_factor: 2 ) builder.use(Octokit::Response::RaiseError) builder.use(Faraday::Response::Logger, loog, formatter: Fbe::Middleware::Formatter) builder.use(Fbe::Middleware::RateLimit, limits) builder.use(Fbe::Middleware::Trace, trace, ignores: [:fresh]) if .sqlite_cache maxsize = Integer(Filesize.from(.sqlite_cache_maxsize || '100M')) maxvsize = Integer(Filesize.from(.sqlite_cache_maxvsize || '100K')) minage = .sqlite_cache_min_age.nil? ? nil : Integer(.sqlite_cache_min_age.to_s, 10) store = Fbe::Middleware::SqliteStore.new( .sqlite_cache, Fbe::VERSION, loog:, maxsize:, maxvsize:, ttl: 24, cache_min_age: minage ) loog.info( "Using HTTP cache in SQLite file: #{store.path} (" \ "#{File.exist?(store.path) ? Filesize.from(File.size(store.path).to_s).pretty : 'file is absent'}, " \ "max size: #{Filesize.from(maxsize.to_s).pretty}, max vsize: #{Filesize.from(maxvsize.to_s).pretty})" ) builder.use(Faraday::HttpCache, store:, serializer: JSON, shared_cache: false, logger: Loog::NULL) else loog.info("No HTTP cache in SQLite file, because 'sqlite_cache' option is not provided") builder.use(Faraday::HttpCache, serializer: Marshal, shared_cache: false, logger: Loog::NULL) end builder.adapter(Faraday.default_adapter) end o.middleware = stack o = Verbose.new(o, log: loog) unless token.nil? || token.empty? loog.info( "Accessing GitHub API with a token (#{token.length} chars, ending by #{token[-4..].inspect}, " \ "#{o.rate_limit.remaining} quota remaining)" ) end else loog.debug('The connection to GitHub API is mocked') o = Fbe::FakeOctokit.new end o = decoor(o, loog:, trace:, limits:) do # rubocop:disable Metrics/BlockLength def print_trace!(all: false, max: 5) if @trace.empty? @loog.debug('GitHub API trace is empty') else grouped = @trace.select { |e| e[:duration] > 0.05 || all }.group_by do |entry| uri = URI.parse(entry[:url]) query = uri.query query = "?#{query.ellipsized(40)}" if query "#{uri.scheme}://#{uri.host}#{uri.path}#{query}" end = grouped .sort_by { |_path, entries| -entries.count } .map do |path, entries| [ ' ', path.gsub(%r{^https://api.github.com/}, '/'), ': ', entries.count, " (#{entries.sum { |e| e[:duration] }.seconds})" ].join end .take(max) .join("\n") @loog.info( "GitHub API trace (#{grouped.count} URLs vs #{@trace.count} requests, " \ "#{@origin.rate_limit!.remaining} quota left):\n#{}" ) @trace.clear end end def off_quota?(threshold: nil, resource: :core) # rubocop:disable Layout/EmptyLineBetweenDefs threshold ||= resource == :search ? 5 : 50 label = resource == :search ? 'GitHub Search API' : 'GitHub API' rate = @origin.rate_limit! left = @limits[:rate_limit]&.remaining(resource) got = !left.nil? left = rate.remaining unless got if resource == :search && !got @loog.warn( "Search-quota check fell back to core remaining (#{left}); " \ 'search count unavailable in rate-limit middleware' ) end if left < threshold @loog.info("Too much #{label} quota consumed already (#{left} < #{threshold})") true else @loog.debug("Still #{left} #{label} quota left (>#{threshold})") false end end def user_name_by_id(id) # rubocop:disable Layout/EmptyLineBetweenDefs raise(Fbe::Error, 'The ID of the user is nil') if id.nil? raise(Fbe::Error, 'The ID of the user must be an Integer') unless id.is_a?(Integer) json = @origin.user(id) name = json[:login].downcase @loog.debug("GitHub user ##{id} has a name: @#{name}") name end def repo_id_by_name(name) # rubocop:disable Layout/EmptyLineBetweenDefs raise(Fbe::Error, 'The name of the repo is nil') if name.nil? json = @origin.repository(name) id = json[:id] raise(Fbe::Error, "Repository #{name} not found") if id.nil? @loog.debug("GitHub repository #{name.inspect} has an ID: ##{id}") id end def repo_name_by_id(id) # rubocop:disable Layout/EmptyLineBetweenDefs raise(Fbe::Error, 'The ID of the repo is nil') if id.nil? raise(Fbe::Error, 'The ID of the repo must be an Integer') unless id.is_a?(Integer) json = @origin.repository(id) name = json[:full_name].downcase @loog.debug("GitHub repository ##{id} has a name: #{name}") name end # Disable auto pagination for octokit client called in block # # @yield [octo] Give octokit client with disabled auto pagination # @yieldparam [Octokit::Client, Fbe::FakeOctokit] Octokit client # @return [Object] Last value in block # @example # issue = # Fbe.octo.with_disable_auto_paginate do |octo| # octo.list_issue('zerocracy/fbe', per_page: 1).first # end def with_disable_auto_paginate # rubocop:disable Layout/EmptyLineBetweenDefs ap = @origin.auto_paginate @origin.auto_paginate = false yield(self) if block_given? ensure @origin.auto_paginate = ap end end o = intercepted(o) do |e, m, _args, _r| next unless e == :before next if %i[off_quota? print_trace! rate_limit].include?(m) if Fbe::SEARCH_METHODS.include?(m) raise(Fbe::OffQuota, "We are off-quota on the search resource, can't do #{m}()") if o.off_quota?(resource: :search) elsif o.off_quota? raise(Fbe::OffQuota, "We are off-quota (remaining: #{o.rate_limit.remaining}), can't do #{m}()") end end o.instance_eval do def send(...) __send__(...) end def public_send(...) # rubocop:disable Layout/EmptyLineBetweenDefs, Elegant/GoodMethodName __send__(...) end end o end end end |
.over?(global: $global, options: $options, loog: $loog, epoch: $epoch || Time.now, kickoff: $kickoff || Time.now, quota_aware: true, lifetime_aware: true, timeout_aware: true) ⇒ Boolean
Check GitHub API quota, lifetime, and timeout.
20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
# File 'lib/fbe/over.rb', line 20 def Fbe.over?( global: $global, options: $options, loog: $loog, epoch: $epoch || Time.now, kickoff: $kickoff || Time.now, quota_aware: true, lifetime_aware: true, timeout_aware: true ) if quota_aware octo = Fbe.octo(loog:, options:, global:) if octo.off_quota?(threshold: 100, resource: :core) || octo.off_quota?(resource: :search, threshold: 10) loog.info('We are off GitHub quota, time to stop') return true end end if lifetime_aware && .lifetime && Time.now - epoch > .lifetime * 0.9 loog.info("We ran out of lifetime (#{epoch.ago} already), must stop here") return true end if timeout_aware && .timeout && Time.now - kickoff > .timeout * 0.9 loog.info("We've spent more than #{kickoff.ago}, must stop here") return true end false end |
.overwrite(fact, property_or_hash, values = nil, fb: Fbe.fb, fid: '_id') ⇒ nil
This operation preserves all other properties during recreation
If property already has the same single value, no changes are made
Overwrites a property in the fact by recreating the entire fact.
If the property doesn't exist in the fact, it will be added. If it does exist, the entire fact will be destroyed, a new fact created with all existing properties, and the specified property set with the new value.
It is important that the fact has the _id property. If it doesn't,
an exception will be raised.
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 |
# File 'lib/fbe/overwrite.rb', line 34 def Fbe.overwrite(fact, property_or_hash, values = nil, fb: Fbe.fb, fid: '_id') # rubocop:disable Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity raise(Fbe::Error, 'The fact is nil') if fact.nil? raise(Fbe::Error, 'The fb is nil') if fb.nil? if property_or_hash.is_a?(Hash) before = {} fact.all_properties.each do |k| before[k.to_s] = fact[k] end modified = false overwrites = false property_or_hash.each do |k, vv| sk = k.to_s raise(Fbe::Error, "The value for #{k} is nil") if vv.nil? vv = [vv] unless vv.is_a?(Array) existing = before[sk] next if existing == vv before[sk] = vv modified = true overwrites = true unless existing.nil? end return fact unless modified unless overwrites property_or_hash.each do |k, vv| sk = k.to_s next unless fact[sk].nil? vv = [vv] unless vv.is_a?(Array) vv.each do |v| fact.public_send(:"#{sk}=", v) end end return end id = fact[fid]&.first raise(Fbe::Error, "There is no #{fid} in the fact, cannot use Fbe.overwrite") if id.nil? raise(Fbe::Error, "No facts by #{fid} = #{id}") if fb.query("(eq #{fid} #{id})").delete!.zero? fb.txn do |fbt| n = fbt.insert f = n while f.instance_variable_defined?(:@fact) || f.instance_variable_defined?(:@origin) iv = f.instance_variable_defined?(:@fact) ? :@fact : :@origin f = f.instance_variable_get(iv) end map = f.instance_variable_get(:@map) %w[_id _time _version _job].each { |k| map.delete(k) if before.key?(k) } before.each do |k, vv| next unless n[k].nil? vv.each do |v| n.public_send(:"#{k}=", v) end end end return end property = property_or_hash raise(Fbe::Error, "The property is not a String but #{property.class} (#{property})") unless property.is_a?(String) raise(Fbe::Error, 'The values is nil') if values.nil? values = [values] unless values.is_a?(Array) return fact if !fact[property].nil? && fact[property].one? && values.one? && fact[property].first == values.first if fact[property].nil? values.each do |v| fact.public_send(:"#{property}=", v) end return end before = {} fact.all_properties.each do |k| before[k.to_s] = fact[k] end id = fact[fid]&.first raise(Fbe::Error, "There is no #{fid} in the fact, cannot use Fbe.overwrite") if id.nil? raise(Fbe::Error, "No facts by #{fid} = #{id}") if fb.query("(eq #{fid} #{id})").delete!.zero? fb.txn do |fbt| n = fbt.insert f = n while f.instance_variable_defined?(:@fact) || f.instance_variable_defined?(:@origin) iv = f.instance_variable_defined?(:@fact) ? :@fact : :@origin f = f.instance_variable_get(iv) end map = f.instance_variable_get(:@map) %w[_id _time _version _job].each { |k| map.delete(k) if before.key?(k) } before[property.to_s] = values before.each do |k, vv| next unless n[k].nil? vv.each do |v| n.public_send(:"#{k}=", v) end end end nil end |
.pmp(fb: Fbe.fb, global: $global, options: $options, loog: $loog) ⇒ Object
Takes configuration parameter from the "PMP" fact.
The factbase may have a few facts with the what set to pmp (stands for the
"project management plan"). These facts contain information that configures
the project. It is expected that every fact with the what set to pmp also
contains the area property, which is set to one of nine values: scope,
time, cost, etc. (the nine process areas in the PMBOK).
If a proper pmp fact is not found or the property is absent in the fact, this method throws an exception. The factbase must contain PMP-related facts. Most probably, a special judge must fill it up with such a fact.
The method uses a double nested others block to create a chainable interface
that allows accessing configuration like:
Fbe.pmp.hr.reward_points
Fbe.pmp.cost.hourly_rate
Fbe.pmp.time.deadline
Custom areas (not defined in assets/pmp.xml) are also supported. When
accessing a property through a custom area, the returned value is read
directly from the factbase without XML defaults or type coercion. The
returned Pmpv object will have nil for default, type, and memo.
Fbe.pmp.my_custom.my_prop # reads from factbase, no XML defaults
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 |
# File 'lib/fbe/pmp.rb', line 55 def Fbe.pmp(fb: Fbe.fb, global: $global, options: $options, loog: $loog) # rubocop:disable Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity xml = Nokogiri::XML(File.read(File.join(__dir__, '../../assets/pmp.xml'))) pmpv = Class.new(SimpleDelegator) do attr_reader :default, :type, :memo, :value # rubocop:disable Layout/EmptyLinesAroundAttributeAccessor def initialize(value, default, type, memo) super(value) @value = value @default = default @type = type @memo = memo end end query = ->(area) { Fbe.fb(global:, fb:, options:, loog:).query("(and (eq what 'pmp') (eq area '#{area}'))") } Class.new do define_method(:areas) do xml.xpath('/pmp/area/@name').map(&:value) end others do |*args1| area = args1.first.to_s node = xml.at_xpath("/pmp/area[@name='#{area}']") if node.nil? Class.new do define_method(:properties) do query.call(area).each.first&.all_properties&.map(&:to_s) || [] end others do |*args2| param = args2.first.to_s result = query.call(area).each.first&.[](param)&.first pmpv.new(result, nil, nil, nil) end end.new else Class.new do define_method(:properties) do node.xpath('p/name').map(&:text) end others do |*args2| param = args2.first.to_s result = query.call(area).each.first&.[](param)&.first prop = node.at_xpath("p[name='#{param}']") default = nil type = nil memo = nil if prop default = prop.at_xpath('default').text type = prop.at_xpath('type').text memo = prop.at_xpath('memo').text default = case type when 'int' then Integer(default, 10) when 'float' then Float(default) when 'bool' then default == 'true' else default end end result ||= default result = case type when 'int' then Integer(Float(result).truncate) when 'float' then Float(result) when 'bool' then result.to_s == 'true' else result end pmpv.new(result, default, type, memo) end end.new end end end.new end |
.regularly(area, p_every_days, p_since_days = nil, fb: Fbe.fb, judge: $judge, loog: $loog) {|Factbase::Fact| ... } ⇒ nil
Skips execution if judge was run within the interval period
The 'since' property is added to the fact when p_since_days is provided
Run the block provided every X days based on PMP configuration.
Executes a block periodically based on PMP (Project Management Plan) settings. The block will only run if it hasn't been executed within the specified interval. Creates a fact recording when the judge was last run.
32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
# File 'lib/fbe/regularly.rb', line 32 def Fbe.regularly(area, p_every_days, p_since_days = nil, fb: Fbe.fb, judge: $judge, loog: $loog, &) raise(Fbe::Error, 'The area is nil') if area.nil? raise(Fbe::Error, 'The p_every_days is nil') if p_every_days.nil? raise(Fbe::Error, 'The fb is nil') if fb.nil? raise(Fbe::Error, 'The $judge is not set') if judge.nil? raise(Fbe::Error, 'The $loog is not set') if loog.nil? pmp = fb.query("(and (eq what 'pmp') (eq area '#{area}') (exists #{p_every_days}))").each.first interval = pmp.nil? ? 7 : pmp[p_every_days].first recent = fb.query( "(and (eq what '#{judge}') (gt when (minus (to_time (env 'TODAY' '#{Time.now.utc.iso8601}')) '#{interval} days')))" ).each.first if recent loog.info( "#{judge} statistics were collected #{recent.when.ago} ago, " \ "skipping now (we run it every #{interval} days)" ) return end loog.info("#{judge} statistics weren't collected for the last #{interval} days") fb.txn do |fbt| f = fbt.insert f.what = judge f.when = Time.now unless p_since_days.nil? days = pmp.nil? || pmp[p_since_days].nil? ? 28 : pmp[p_since_days].first since = Time.now - (days * 24 * 60 * 60) f.since = since end yield(f) end nil end |
.repeatedly(area, p_every_hours, fb: Fbe.fb, judge: $judge, loog: $loog) {|Factbase::Fact| ... } ⇒ nil
Skips execution if judge was run within the interval period
Overwrites the 'when' property of existing judge fact
Run the block provided every X hours based on PMP configuration.
Similar to Fbe.regularly but works with hour intervals instead of days. Executes a block periodically, maintaining a single fact that tracks the last execution time. The fact is overwritten on each run rather than creating new facts.
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 |
# File 'lib/fbe/repeatedly.rb', line 34 def Fbe.repeatedly(area, p_every_hours, fb: Fbe.fb, judge: $judge, loog: $loog, &) raise(Fbe::Error, 'The area is nil') if area.nil? raise(Fbe::Error, 'The p_every_hours is nil') if p_every_hours.nil? raise(Fbe::Error, 'The fb is nil') if fb.nil? raise(Fbe::Error, 'The $judge is not set') if judge.nil? raise(Fbe::Error, 'The $loog is not set') if loog.nil? pmp = fb.query("(and (eq what 'pmp') (eq area '#{area}') (exists #{p_every_hours}))").each.first hours = pmp.nil? ? 24 : pmp[p_every_hours].first recent = fb.query( "(and (eq what '#{judge}') (gt when (minus (to_time (env 'TODAY' '#{Time.now.utc.iso8601}')) '#{hours} hours')))" ).each.first if recent loog.info("#{judge} was executed #{recent.when.ago} ago, skipping now (we run it every #{hours} hours)") return end f = fb.query("(and (eq what '#{judge}'))").each.first if f.nil? f = fb.insert f.what = judge end yield(fb.query("(and (eq what '#{judge}'))").each.first) Fbe.overwrite(f, 'when', Time.now) nil end |
.sec(fact, prop = :seconds) ⇒ String
Uses the tago gem's ago method for formatting
Converts number of seconds into human-readable time format.
The number of seconds is taken from the fact provided, usually stored
there in the seconds property. The seconds are formatted into a
compact human-readable string like "3d" or "5h" using the tago gem.
24 25 26 27 28 29 |
# File 'lib/fbe/sec.rb', line 24 def Fbe.sec(fact, prop = :seconds) s = fact[prop.to_s] raise(Fbe::Error, "There is no #{prop.inspect} property") if s.nil? s = Integer(s.first.to_s, 10) (Time.now - s).ago end |
.unmask_repos(options: $options, global: $global, loog: $loog, epoch: $epoch || Time.now, kickoff: $kickoff || Time.now, quota_aware: true, lifetime_aware: true, timeout_aware: true) ⇒ Array<String>
Exclusion patterns must start with '-' (e.g., '-org/pattern*')
Results are shuffled to distribute load when processing
Resolves repository masks to actual GitHub repository names.
Takes a comma-separated list of repository masks from options and expands wildcards by querying GitHub API. Supports inclusion and exclusion patterns. Archived repositories are automatically filtered out.
62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 |
# File 'lib/fbe/unmask_repos.rb', line 62 def Fbe.unmask_repos( # rubocop:disable Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity options: $options, global: $global, loog: $loog, epoch: $epoch || Time.now, kickoff: $kickoff || Time.now, quota_aware: true, lifetime_aware: true, timeout_aware: true ) raise(Fbe::Error, 'Repositories mask is not specified') unless .repositories raise(Fbe::Error, 'Repositories mask is empty') if .repositories.empty? return if block_given? && Fbe.over?( global:, options:, loog:, epoch:, kickoff:, quota_aware:, lifetime_aware:, timeout_aware: ) repos = [] octo = Fbe.octo(loog:, global:, options:) masks = (.repositories || '').split(',') masks.reject { |m| m.start_with?('-') }.each do |mask| unless mask.include?('*') repos << mask next end re = Fbe.mask_to_regex(mask) org = mask.split('/')[0] list = begin octo.organization_repositories(org, type: 'all') rescue Octokit::NotFound octo.repositories(org) end list.each do |r| repos << r[:full_name] if re.match?(r[:full_name]) end end masks.select { |m| m.start_with?('-') }.each do |mask| re = Fbe.mask_to_regex(mask[1..]) repos.reject! { |r| re.match?(r) } end repos.reject! { |repo| octo.repository(repo)[:archived] } raise(Fbe::Error, "No repos found matching: #{.repositories.inspect}") if repos.empty? repos.shuffle! loog.debug("Scanning #{repos.size} repositories: #{repos.joined}...") return repos unless block_given? repos.each do |repo| break if Fbe.over?(global:, options:, loog:, epoch:, kickoff:, quota_aware:, lifetime_aware:, timeout_aware:) yield(repo) end end |
.who(fact, prop = :who, options: $options, global: $global, loog: $loog) ⇒ String
Results are cached to reduce GitHub API calls
Subject to GitHub API rate limits
Converts a GitHub user ID into a formatted username string.
The ID of the user (integer) is expected to be stored in the who property of the
provided fact. This function makes a live request to GitHub API to
retrieve the username. The result is cached globally to minimize API calls.
For example, the ID 526301 will be converted to "@yegor256".
29 30 31 32 33 34 |
# File 'lib/fbe/who.rb', line 29 def Fbe.who(fact, prop = :who, options: $options, global: $global, loog: $loog) id = fact[prop.to_s] raise(Fbe::Error, "There is no #{prop.inspect} property") if id.nil? id = Integer(id.first.to_s, 10) "@#{Fbe.octo(options:, global:, loog:).user_name_by_id(id)}" end |