Class: Insika::EdgeLimiter

Inherits:
Middleware show all
Defined in:
lib/insika/edge_limiter.rb

Overview

The production edge: THE named place where volume/cost abuse is cut. A Middleware with two independent limits, both OPT-IN (nil/0 = off — a bare wiring behaves exactly as before):

· chat rate limit    — turn ATTEMPTS per chat per window. Counted on entry
(a blocked attempt still counts), so a flood keeps hitting the wall.
· agent token ceiling — total tokens per agent per window. Checked on entry
against the accumulated ledger; the turn's own usage is recorded AFTER the
terminal returns (the Middleware wraps stages 5-9, so state.usage is set).
· calendar budget    — WS2: `AgentProfile#budget` caps the spend per
(tenant, agent) over CALENDAR windows (daily/monthly), on the
BudgetLedger. Hard (default): crossing the cap raises the typed
Insika::BudgetExceeded (the envelope quotes budget_exceeded +
retry_after); soft: crossing warns instead — ONE budget_warning event
per window plus a note injected into the context. Crossing `alert_at`
(default 0.8 of the cap) warns the same way, before the wall. The turn's
billed spend (input+output+cached+cache_creation — the A4 rule) lands on
the windows after the terminal.

Config resolution, per turn (configuration over convention):

profile.limits[:chat_rate_limit / :agent_token_ceiling]  — per-agent override
settings["edge"]                                          — platform default

A per-agent 0 explicitly disables a platform default for that agent.

On breach it uses the graceful-halt contract: halt_response (the safe reply) + guardrail_block (audit -> :guardrail_blocked) and does NOT call nxt — the turn completes with ZERO LLM calls. It sits BEFORE the InputGuardrail in the stack so a flood can't spend the LLM moderator either. The BUDGET breach is the ONE deliberate exception: it is a typed failure (BudgetExceeded), not a customer-facing reply — the operator wants the envelope to say "budget" and quote when the window rolls, not to hand the customer a cost message.

Constant Summary collapse

CHAT_KIND = 
"chat"
TOKENS_KIND = 
"tokens"
DEFAULT_CHAT_WINDOW =

seconds

60
DEFAULT_TOKEN_WINDOW =

seconds (daily ceiling)

86_400
DEFAULT_RESPONSE =

Neutral fallback, same contract as Safety::SafeResponses (pt-BR — the pilot's language; override via settings edge.limit_response).

"Estou recebendo muitas mensagens agora. Aguarde um " \
"momento e tente novamente, por favor."

Instance Method Summary collapse

Constructor Details

#initialize(ledger:, settings_store: nil, budget_ledger: nil, event_stream: nil) ⇒ EdgeLimiter

Returns a new instance of EdgeLimiter.



51
52
53
54
55
56
57
58
 
# File 'lib/insika/edge_limiter.rb', line 51

def initialize(ledger:, settings_store: nil, budget_ledger: nil, event_stream: nil)
  @ledger = ledger
  @settings = settings_store
  # WS2: the calendar-window ledger. nil = budget off (parity — the bare
  # wiring is byte-identical to before).
  @budget_ledger = budget_ledger
  @event_stream = event_stream
end

Instance Method Details

#call(state, &nxt) ⇒ Object 



60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
 
# File 'lib/insika/edge_limiter.rb', line 60

def call(state, &nxt)
  edge = platform_edge
  limits = state.profile.limits || {}
  # A resume (crash/pause recovery) re-enters the pipeline for a turn that was
  # ALREADY admitted: re-counting it would swallow a legitimate message with
  # the rate-limit reply exactly when the window is saturated. Entry checks
  # are skipped; the turn's usage still lands on the ledger below.
  resumed = state.resumed

  if !resumed && (limit = positive(limits.key?(:chat_rate_limit) ? limits[:chat_rate_limit] : edge["chat_rate_limit"]))
    breach = check_chat_rate(state, limit, edge)
    return block(state, edge, **breach) if breach
  end

  # NB: a per-agent key PRESENT with nil (e.g. an imported pack carrying
  # `"chat_rate_limit": null`) reads as OFF for that agent, not "inherit".
  if (ceiling = positive(limits.key?(:agent_token_ceiling) ? limits[:agent_token_ceiling] : edge["agent_token_ceiling"]))
    token_window = positive(edge["agent_token_window"]) || DEFAULT_TOKEN_WINDOW
    unless resumed
      spent = @ledger.count(TOKENS_KIND, state.profile.id.to_s, window: token_window)
      if spent >= ceiling
        return block(state, edge, category: :token_ceiling,
                                  detail: "agent #{state.profile.id}: #{spent}/#{ceiling} tokens per #{token_window}s")
      end
    end

    record_after = token_window
  end

  # WS2: calendar budgets. Entry — a HARD budget at/over the cap raises the
  # typed error (never a customer-facing reply); the alert_at warning and
  # the SOFT over-cap both warn once per window + inject a context note.
  # A resumed turn (crash/pause replay) was already admitted: it is never
  # refused twice — its spend still lands on the ledger below.
  budget_on = budget_configured?(state)
  budget_enforce(state) unless resumed

  result = begin
    nxt.call(state)
  ensure
    # A turn that FAILED after burning tokens still SPENT them: record the
    # usage the state captured before the error propagates. The ask's usage
    # lands on state.usage before any later stage (guardrail block, tool
    # error, workflow schema) can fail the turn — a failed turn must count
    # against the budget like a completed one (WS2).
    record_usage(state, record_after) if record_after
    record_budget_usage(state) if budget_on
  end
  result
end