Module: PWN::AI::Agent::Learning

Defined in:: lib/pwn/ai/agent/learning.rb

Overview

PWN::AI::Agent::Learning is the self-improvement engine that closes the pwn-ai feedback loop. It captures task outcomes, mines session transcripts for durable lessons, promotes successful workflows into reusable skills, and keeps ~/.pwn lean (memory + learning.jsonl + mistakes + sessions) so the agent gets sharper over time instead of accumulating noise.

Data flows:

Loop.run --(tool telemetry)--> Metrics.record
Loop.run --(final answer)----> Learning.auto_introspect (opt-in)
auto_introspect --(throttled)--> Learning.gc_stores!  # ~/.pwn lean
model    --(tool calls)------> learning_note_outcome / _distill_skill
PromptBuilder <----------------- Learning.to_context + Metrics.to_context

Everything is file-backed under ~/.pwn so it survives across REPL restarts and is shared by every future session.

Constant Summary collapse

LEARNING_FILE =

File.join(Dir.home, '.pwn', 'learning.jsonl')

FINETUNE_DIR =

File.join(Dir.home, '.pwn', 'finetune')

INTROSPECT_SOFT_MS = P0 — post-answer introspect must not train "stop early" while spending the iteration budget after the final. Soft cap skips expensive stages (tool critic, PRM-LLM, reflect, extrospect); hard cap keeps only note_outcome + judge(heuristic) + sentinel.

2_500

INTROSPECT_HARD_MS =

8_000

INTROSPECT_MIN_STAGES =

%i[judge note_outcome fold_judge sentinel].freeze

MAX_MEMORY_ENTRIES =

MAX_OUTCOME_ROWS = Lean outcome retention — keep gold RL signal, drop bulk auto noise.

OUTCOME_RETAIN_DAYS =

OUTCOME_RECENT_DAYS =

OUTCOME_DETAILS_MAX =

GOLD_MIN_SCORE =

0.6

EXEMPLARS_POOL_MIN =

FAILURE_WINDOW_MIN =

HIGH_VALUE_TAGS =

%w[
  needs_human extro_verify sdr gqrx rl pwn-ai curriculum hindsight her
].freeze

LOW_VALUE_ONLY_TAGS =

%w[
  auto loop partial wrong solved offline_judge plan_cover_high
].freeze

PRUNE_EVERY_N_APPENDS =

CLAIM_METRIC_WORDS = E3/P26 — only CVE-ids or software-name + full semver (x.y.z). Two-part floats ("cap 0.2", "proxy 1.0", "judge 37.0") are RL metric crumbs that were scraped by verify_as_reward and flooded learning.jsonl with extro_verify :unknown failures.

%w[
  cap share proxy judge success only now clears gap score rate mean
  brier overconf distrust trajectory_fraction handler orm prm delta
  limit window pct percent ms iter budget conf confidence n
  ruby python linux kernel host cwd e.g e.g.
].freeze

CLAIM_RX =

/
  CVE-\d{4}-\d{4,7}
  |
  \b
  (?!
    (?:cap|share|proxy|judge|success|only|now|clears|gap|score|rate|mean|
       brier|overconf|distrust|trajectory_fraction|handler|orm|prm|delta|
       limit|window|pct|percent|ms|iter|budget|conf|confidence|
       ruby|python|linux|kernel|host|cwd|e\.g)
    \b
  )
  [A-Za-z][\w.+-]{2,}
  \s+
  v?\d+\.\d+\.\d+(?:[-+][\w.]+)?
  \b
/x

SFT_MIN_SCORE = P12 — SFT quality gate (as hard as DPO source-cap): drop HER/soft, low judge_score, auto-only noise without score, and PRM-compress trajectories so LoRA is not 5MB of "how we flailed".

0.6

SFT_MAX_TOOL_CHARS =

1_200

PROCESS_SOP_RX = M4.1 — keyword gate for "process / hygiene" SOPs the operator keeps re-requesting (rubocop, rake, rspec, conventions). These must become durable Memory lessons, not just learning.jsonl rows.

%r{\b(rubocop|rake\b|rspec|bundle\s+exec|conventions?|lint(?:ing)?|style/|code\s*hygiene|post[- ]?patch|after\s+(?:every\s+)?(?:patch|change|edit))\b}i

FAILURE_FINAL_RX =

/\[pwn-ai\] (iteration budget exhausted|engine returned no message)|\b(i (was )?unable to|i could not|i couldn'?t|cannot proceed|failed to)\b/i

Class Method Summary collapse

.authors ⇒ Object

Author(s)

0day Inc.
.auto_introspect(opts = {}) ⇒ Object

Supported Method Parameters

PWN::AI::Agent::Learning.auto_introspect( session_id: 'required - id of the just-completed session', request: 'optional - original user request (for outcome logging)', final: 'optional - final assistant answer (for outcome logging)' ).
.consolidate(opts = {}) ⇒ Object

Supported Method Parameters

removed = PWN::AI::Agent::Learning.consolidate( max_entries: 'optional - hard cap on PWN::Memory size (default MAX_MEMORY_ENTRIES)' ).
.distill_skill(opts = {}) ⇒ Object

Supported Method Parameters

skill = PWN::AI::Agent::Learning.distill_skill( name: 'required - snake_case name for the new skill', session_id: 'optional - PWN::Sessions id to mine (uses its transcript)', content: 'optional - explicit markdown body; overrides transcript mining', references: 'optional - Array of reference URLs / CWE / CVE / ATT&CK ids' ).
.exemplars_for(opts = {}) ⇒ Object

Supported Method Parameters

msgs = PWN::AI::Agent::Learning.exemplars_for( request: 'required - current user request', limit: 'optional - max exemplar traces to return (default 1)', max_msgs: 'optional - cap on messages per exemplar (default 6)' ).
.export_finetune(opts = {}) ⇒ Object
.flip_last_outcome(opts = {}) ⇒ Object
.gc_stores!(opts = {}) ⇒ Object
One-shot lean across memory + learning + mistakes + sessions.
.help ⇒ Object
Display Usage for this Module.
.lean!(opts = {}) ⇒ Object
Memory lean + outcome prune.
.note_outcome(opts = {}) ⇒ Object

Supported Method Parameters

entry = PWN::AI::Agent::Learning.note_outcome( task: 'required - short description of what was attempted', success: 'required - Boolean, did the attempt achieve its goal', details: 'optional - free-form notes / error / evidence', session_id: 'optional - PWN::Sessions id this outcome belongs to', tags: 'optional - Array of String labels for later retrieval' ).
.outcomes(opts = {}) ⇒ Object

Supported Method Parameters

rows = PWN::AI::Agent::Learning.outcomes( limit: 'optional - max entries returned newest-first (default 50)', success: 'optional - filter by Boolean outcome', tag: 'optional - filter by tag substring' ).
.prune_outcomes!(opts = {}) ⇒ Object

Supported Method Parameters

result = PWN::AI::Agent::Learning.prune_outcomes!( dry_run: 'optional - Boolean (default false)', max_rows: 'optional - hard cap (default MAX_OUTCOME_ROWS)', retain_days: 'optional - age floor for low-value drop', recent_days: 'optional - always keep newer than this' ).
.purge_noise ⇒ Object

Supported Method Parameters

PWN::AI::Agent::Learning.purge_noise.
.reconcile_verdict_tags!(opts = {}) ⇒ Object
One-shot / on-load repair: rewrite tags+details where verdict label disagrees with score (solved @ 0.3 etc.).
.reflect(opts = {}) ⇒ Object

Supported Method Parameters

report = PWN::AI::Agent::Learning.reflect( session_id: 'required - PWN::Sessions id to analyse', dry_run: 'optional - when true, do not write to Memory/Skills (default false)' ).
.reset ⇒ Object

Supported Method Parameters

PWN::AI::Agent::Learning.reset.
.stats ⇒ Object

Supported Method Parameters

stats = PWN::AI::Agent::Learning.stats.
.to_context(opts = {}) ⇒ Object

Supported Method Parameters

ctx = PWN::AI::Agent::Learning.to_context( limit: 'optional - number of recent outcomes to surface (default 5)' ).
.update_skill(opts = {}) ⇒ Object
Fold RL artefacts (mistakes / structured_fix / an explicit lesson) into an existing skill.

Class Method Details

.authors ⇒ `Object`

Author(s): 0day Inc. support@0dayinc.com



1759
1760
1761

# File 'lib/pwn/ai/agent/learning.rb', line 1759

public_class_method def self.authors
  "AUTHOR(S):\n  0day Inc. <support@0dayinc.com>\n"
end

.auto_introspect(opts = {}) ⇒ `Object`

Supported Method Parameters

PWN::AI::Agent::Learning.auto_introspect( session_id: 'required - id of the just-completed session', request: 'optional - original user request (for outcome logging)', final: 'optional - final assistant answer (for outcome logging)' )

Called by Loop.run when PWN::Env[:agent][:auto_introspect] is truthy. Never raises — learning must not break the primary loop.

# File 'lib/pwn/ai/agent/learning.rb', line 517

public_class_method def self.auto_introspect(opts = {})
  session_id = opts[:session_id]
  return unless session_id
  return unless auto_introspect_enabled?

  # Hermes split: Loop.run is on the stack → defer critic/judge/PRM/HER
  # onto a daemon thread so the user already has the reply. Specs,
  # cron, and tools pass through (inline:true or no user-path depth).
  return TurnFinalizer.defer(opts.merge(inline: true)) if defined?(TurnFinalizer) && !opts[:inline] && TurnFinalizer.should_defer?

  t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  stages_run = []
  stages_skipped = []
  budget_hot = begin
    defined?(Loop) && Loop.respond_to?(:budget_exhaustion_hot?, true) &&
      Loop.send(:budget_exhaustion_hot?)
  rescue StandardError
    false
  end

  elapsed_ms = lambda do
    ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0) * 1000).round
  end
  # soft = skip expensive; hard = stop almost everything
  over_soft = lambda do
    ms = elapsed_ms.call
    ms >= INTROSPECT_SOFT_MS || budget_hot
  end
  over_hard = lambda do
    ms = elapsed_ms.call
    ms >= INTROSPECT_HARD_MS
  end

  proxy_ok = infer_success(session_id: session_id, final: opts[:final])

  crit = { verdict: :pass, source: :skipped }
  # Live-turn critic is always text-only. A tool-armed persona is
  # another Loop.run of the same goal (opens more browsers, never
  # returns the operator answer). Cron/practice still call
  # Curriculum.critic without text_only.
  if !defined?(Curriculum) || over_hard.call
    stages_skipped << :critic
  else
    stages_run << :critic_text_only
    crit = Curriculum.critic(
      request: opts[:request],
      final: opts[:final],
      session_id: session_id,
      text_only: true
    )
  end

  # R1 judge — always attempt (heuristic is cheap; LLM gated inside)
  stages_run << :judge
  v = Reward.judge(request: opts[:request], final: opts[:final], session_id: session_id, proxy_ok: proxy_ok, predicted: opts[:predicted], plan: opts[:plan]) if defined?(Reward)
  v ||= { score: proxy_ok ? 1.0 : 0.0, success: proxy_ok, verdict: proxy_ok ? :solved : :wrong }
  v[:score] = [v[:score], 0.3].min if crit[:verdict] == :flaw
  # P29 — critic floor used to leave stale verdict=:solved at score=0.3,
  # producing learning.jsonl rows tagged "solved" with success=false
  # (116+ rows). Always resync verdict/success from the final score.
  v[:verdict] = verdict_for_score(score: v[:score])
  v[:success] = v[:score].to_f >= 0.6
  ok = v[:success]

  # W1 pending user_correction pair
  pend = Thread.current[:pwn_pending_pref]
  if pend && ok && defined?(Reward)
    stages_run << :user_correction_pref
    Reward.record_preference(
      prompt: pend[:prompt],
      rejected: pend[:rejected],
      chosen: opts[:final].to_s,
      source: :user_correction,
      shape: :revised_answer,
      force: true
    )
    Thread.current[:pwn_pending_pref] = nil
  end

  # Soft plan-quality feature (W3) — tag only; not full DPO.
  plan_cov = nil
  if defined?(Reward) && Reward.respond_to?(:plan_coverage)
    begin
      plan_for_cov = opts[:plan]
      plan_for_cov = opts[:ts_state][:plan] if plan_for_cov.nil? && opts[:ts_state].is_a?(Hash)
      if plan_for_cov.nil? && defined?(TaskSummarizer)
        # Recover numbered tasks from the final/request only when caller
        # did not pass a plan — still keeps TaskSummarizer out of the
        # credit path (parse is pure text).
        plan_for_cov = nil
      end
      if !plan_for_cov.nil? || opts[:final].to_s.length.positive?
        plan_cov = Reward.plan_coverage(
          plan: plan_for_cov || [],
          final: opts[:final],
          request: opts[:request],
          session_id: session_id
        )
        stages_run << :plan_coverage if plan_cov && plan_cov[:total].to_i.positive?
      end
    rescue StandardError => e
      warn "[pwn-ai/learning] plan_coverage swallowed: #{e.class}: #{e.message}"
    end
  end

  stages_run << :note_outcome
  outcome_tags = ['auto', 'loop', v[:verdict].to_s]
  outcome_tags << plan_cov[:tag] if plan_cov && plan_cov[:tag]
  outcome_tags << "plan_cover=#{plan_cov[:score]}" if plan_cov && plan_cov[:total].to_i.positive?
  # P29 — persist the bare user ask (strip REQUEST:/GOAL: envelopes at write time)
  task_txt = display_task(task: opts[:request].to_s)
  task_txt = opts[:request].to_s[0, 100] if task_txt.empty?
  note_outcome(
    task: task_txt,
    success: ok,
    score: v[:score],
    details: "#{v[:verdict]}(#{v[:score].to_f.round(2)}) #{v[:rationale]} | #{opts[:final].to_s[0, 200]}",
    session_id: session_id,
    tags: outcome_tags,
    judge_source: v[:source]
  )

  stages_run << :fold_judge
  fold_judge_into_metrics(session_id: session_id, score: v[:score], confidence: v[:confidence])
  # R5 — close the live MDP episode with the ORM terminal reward.
  if defined?(PWN::AI::Agent::Policy) && Policy.respond_to?(:finish)
    stages_run << :policy
    Policy.finish(
      session_id: session_id,
      score: v[:score],
      verdict: v[:verdict],
      proxy_ok: ok,
      final: opts[:final],
      ts_state: opts[:ts_state]
    )
  end

  # R2 PRM — skip under hard cap (expensive LLM); keep under soft if heuristic path
  if over_hard.call
    stages_skipped << :prm
  elsif defined?(Reward)
    stages_run << :prm
    Reward.prm(request: opts[:request], session_id: session_id)
  end

  # C3 HER — only on failure; skip hard
  if !ok && defined?(Curriculum) && !over_hard.call
    stages_run << :hindsight
    Curriculum.hindsight(request: opts[:request], final: opts[:final], session_id: session_id)
  else
    stages_skipped << :hindsight unless ok
  end

  # W3 calibrate — cheap; always write so Metrics.calibration is never empty.
  predicted = opts[:predicted]
  predicted = recover_predicted_from_session(session_id: session_id) if predicted.nil?
  stages_run << :calibrate if defined?(Curriculum)

  # reflect on success — skip soft/hard (LLM + memory writes)
  # M4.1 — also reflect when the request/final is a process SOP
  # (code hygiene) even if judge score < 0.6, so rubocop/rake
  # lessons still land in PWN::Memory.
  process_sop = process_sop_text?(text: "#{opts[:request]} #{opts[:final]}")
  if (ok || process_sop) && !over_soft.call
    stages_run << :reflect
    reflect(session_id: session_id)
  elsif ok || process_sop
    stages_skipped << :reflect
    # Cheap path under soft budget: still promote a canned process lesson
    if process_sop && defined?(PWN::Memory)
      promote_process_lesson(
        entry: {
          task: opts[:request].to_s[0, 120],
          success: ok,
          score: v[:score],
          details: opts[:final].to_s[0, 500],
          tags: %w[auto process_sop]
        }
      )
    end
  end

  # R3 sentinel — cheap disk math; always
  if defined?(Reward)
    stages_run << :sentinel
    Reward.sentinel
  end

  # E ambient extrospect — skip soft (can launch probes)
  if defined?(Extrospection) && !over_soft.call
    stages_run << :extrospect
    Extrospection.auto_extrospect(session_id: session_id)
  else
    stages_skipped << :extrospect
  end

  # Keep ~/.pwn RL stores lean on the feedback path (memory +
  # learning.jsonl + mistakes + sessions). Throttled; never raises.
  # Disk-only work: skip only hard budget / budget_hot.
  begin
    if !over_hard.call && !budget_hot && should_gc_stores?
      stages_run << :lean_gc
      gc_stores!(current_session_id: session_id)
    elsif should_gc_stores?
      stages_skipped << :lean_gc
    end
  rescue StandardError => e
    warn "[pwn-ai/learning] post-introspect lean swallowed: #{e.class}: #{e.message}"
  end

  {
    ok: ok,
    score: v[:score],
    elapsed_ms: elapsed_ms.call,
    budget_hot: budget_hot,
    stages_run: stages_run,
    stages_skipped: stages_skipped
  }
rescue StandardError => e
  warn "[pwn-ai/learning] auto_introspect swallowed: #{e.class}: #{e.message}"
  nil
end

.consolidate(opts = {}) ⇒ `Object`

Supported Method Parameters

removed = PWN::AI::Agent::Learning.consolidate( max_entries: 'optional - hard cap on PWN::Memory size (default MAX_MEMORY_ENTRIES)' )

Deduplicates near-identical lesson values and prunes the oldest entries once the cap is exceeded so the injected MEMORY block stays high-signal.

# File 'lib/pwn/ai/agent/learning.rb', line 772

public_class_method def self.consolidate(opts = {})
  cap = opts[:max_entries] || MAX_MEMORY_ENTRIES
  return { removed: 0 } unless defined?(PWN::Memory)

  mem = nil
  load_err = nil
  begin
    mem = PWN::Memory.load
  rescue StandardError => e
    load_err = e
  end
  if load_err
    warn "[pwn-ai/learning] consolidate aborted (memory load failed): #{load_err.class}: #{load_err.message}"
    return { removed: 0, aborted: true, error: "#{load_err.class}: #{load_err.message}" }
  end
  removed = []

  # M1 — semantic clustering: embed :lesson entries, greedy-merge
  # near-duplicates (cosine ≥ 0.92) via Reflect into ONE imperative
  # lesson. Falls back to sha-dedup when no embed backend.
  removed.concat(semantic_merge(mem: mem)) if defined?(PWN::MemoryIndex) && PWN::MemoryIndex.available?

  seen = {}
  mem.each do |k, v|
    sig = Digest::SHA256.hexdigest(v[:value].to_s.strip.downcase)[0, 16]
    seen[sig] ? removed << k : seen[sig] = k
  end
  removed.uniq.each { |k| mem.delete(k) }

  # M3 — evict by (age/ttl) / (importance × confidence), NOT
  # oldest-first. Hand-written high-value lessons survive; low-
  # confidence :heuristic auto-gen self-evicts first.
  if mem.size > cap
    now = Time.now.utc
    scored = mem.map do |k, v|
      if defined?(PWN::Memory) && PWN::Memory.respond_to?(:protected_entry?) &&
         PWN::Memory.protected_entry?(key: k, entry: v)
        next [k, Float::INFINITY]
      end

      age_d = (now - Time.parse(v[:timestamp].to_s)) / 86_400.0
      ttl_d = (v[:ttl].to_f / 86_400.0)
      imp   = (v[:importance] || 0.5).to_f.clamp(0.05, 1.0)
      conf  = (v[:confidence] || (v[:source].to_s == 'human' ? 0.95 : 0.5)).to_f.clamp(0.05, 1.0)
      staleness = ttl_d.positive? ? age_d / ttl_d : age_d / 90.0
      # lower score = drop first; Infinity protected sorts last
      [k, -(staleness / (imp * conf))]
    rescue StandardError
      [k, 0.0]
    end
    # sort ascending by score so lowest (most stale/low-imp) first
    ordered = scored.sort_by { |_k, s| s }
    drop = []
    ordered.each do |pair|
      k = pair[0]
      break if mem.size - drop.size <= cap
      next if defined?(PWN::Memory) && PWN::Memory.respond_to?(:protected_entry?) &&
              PWN::Memory.protected_entry?(key: k, entry: mem[k])

      drop << k
    end
    drop.each { |k| mem.delete(k) }
    removed.concat(drop)
  end
  PWN::Memory.save(mem: mem, force: mem.empty?)
  if PWN::Memory.respond_to?(:lean!)
    begin
      PWN::Memory.lean!
    rescue StandardError => e
      warn "[pwn-ai/learning] post-consolidate memory.lean! swallowed: #{e.class}: #{e.message}"
    end
  end
  { removed: removed.uniq.length, remaining: mem.size }
end

.distill_skill(opts = {}) ⇒ `Object`

Supported Method Parameters

skill = PWN::AI::Agent::Learning.distill_skill( name: 'required - snake_case name for the new skill', session_id: 'optional - PWN::Sessions id to mine (uses its transcript)', content: 'optional - explicit markdown body; overrides transcript mining', references: 'optional - Array of reference URLs / CWE / CVE / ATT&CK ids' )

# File 'lib/pwn/ai/agent/learning.rb', line 391

public_class_method def self.distill_skill(opts = {})
  raise 'ERROR: name is required' if opts[:name].to_s.strip.empty?

  body = opts[:content].to_s
  body = build_skill_from_session(session_id: opts[:session_id], name: opts[:name]) if body.strip.empty? && opts[:session_id]
  raise 'ERROR: content or session_id is required' if body.strip.empty?

  root = skills_dir
  out  = PWN::Config.write_skill(
    name: opts[:name],
    description: opts[:description],
    content: body,
    references: opts[:references],
    pwn_skills_path: root
  )
  PWN::Config.load_skills(pwn_skills_path: root) if PWN::Config.respond_to?(:load_skills)
  note_outcome(task: "distill_skill:#{out[:name]}", success: true, details: "Saved #{out[:path]}", tags: %w[skill auto])
  out.merge(saved: true)
end

.exemplars_for(opts = {}) ⇒ `Object`

Supported Method Parameters

msgs = PWN::AI::Agent::Learning.exemplars_for( request: 'required - current user request', limit: 'optional - max exemplar traces to return (default 1)', max_msgs: 'optional - cap on messages per exemplar (default 6)' )

Retrieval-augmented BEHAVIOUR: keyword-matches request against prior successful outcomes in learning.jsonl, loads the matching session, and compresses its (user, tool, assistant) trace into a short few-shot exemplar Loop.run splices between system and user. Local models are dramatically better with 1 concrete example than with 25 abstract lessons.

# File 'lib/pwn/ai/agent/learning.rb', line 258

public_class_method def self.exemplars_for(opts = {})
  request  = opts[:request].to_s.downcase
  limit    = (opts[:limit]    || 1).to_i
  max_msgs = (opts[:max_msgs] || 6).to_i
  return [] if request.strip.empty?

  tokens = request.scan(/[a-z0-9_]{3,}/).uniq
  return [] if tokens.empty?

  now = Time.now.utc
  # C2 — prioritized replay: priority = judge_score × recency_decay × keyword_sim
  # C2 — strict success:true only (excludes HER success:'soft'). Also
  # down-weight any residual hindsight-tagged rows so partial failures
  # never launder into full-strength few-shot exemplars.
  # P20 — strict success:true AND prefer high judge scores. Drop rows
  # with explicit low ORM score so proxy-true / judge-low cannot be few-shot.
  pool = outcomes(limit: 500, success: true).reject { |r| r[:session_id].to_s.empty? }
  pool = pool.reject { |r| r.key?(:score) && r[:score].to_f < 0.6 }
  scored = pool.map do |r|
    sim   = tokens.count { |t| r[:task].to_s.downcase.include?(t) }.to_f / tokens.length
    age_d = (now - Time.parse(r[:timestamp].to_s)) / 86_400.0
    decay = Math.exp(-age_d / 30.0)
    score = (r[:score] || 1.0).to_f
    tags  = Array(r[:tags]).map(&:to_s)
    # HER / soft / hindsight → 0.35× so they cannot dominate C2 priority
    score *= 0.35 if r[:success].to_s == 'soft' || tags.intersect?(%w[hindsight her soft])
    [r, sim * decay * score]
  rescue StandardError
    [r, 0.0]
  end
  hits = scored.reject { |_, pr| pr <= 0.0 }.sort_by { |_, pr| -pr }.first(limit).map(&:first)

  hits.flat_map { |r| compress_exemplar(session_id: r[:session_id], max_msgs: max_msgs) }
rescue StandardError
  []
end

.export_finetune(opts = {}) ⇒ `Object`

# File 'lib/pwn/ai/agent/learning.rb', line 315

public_class_method def self.export_finetune(opts = {})
  fmt       = (opts[:format] || :sharegpt).to_sym
  min_tools = (opts[:min_tools] || 1).to_i
  min_score = (opts[:min_score] || SFT_MIN_SCORE).to_f
  compress  = opts.key?(:compress) ? opts[:compress] : true
  FileUtils.mkdir_p(FINETUNE_DIR)
  out = opts[:out] || File.join(FINETUNE_DIR, "pwn-#{Time.now.utc.strftime('%Y%m%d')}.jsonl")

  # 4.1 / P12 — exclude HER soft-success + low-score + untagged auto flail
  gold = outcomes(limit: 10_000, success: true).reject do |r|
    tags = Array(r[:tags]).map(&:to_s)
    soft = r[:success].to_s == 'soft' || tags.intersect?(%w[hindsight her soft])
    low  = !r[:score].nil? && r[:score].to_f < min_score
    # require a score when present in corpus era that has scores
    soft || low
  end
  # prefer highest-score outcome per session
  by_sid = {}
  gold.each do |r|
    sid = r[:session_id].to_s
    next if sid.empty?

    prev = by_sid[sid]
    by_sid[sid] = r if prev.nil? || r[:score].to_f >= prev[:score].to_f
  end
  sids = by_sid.keys
  rows = 0
  dropped = { tools: 0, empty: 0, load: 0 }
  File.open(out, 'w') do |f|
    sids.each do |sid|
      t = begin
        PWN::Sessions.load(session_id: sid)
      rescue StandardError
        dropped[:load] += 1
        next
      end
      tool_n = t.count { |e| e[:role].to_s == 'tool' }
      if tool_n < min_tools
        dropped[:tools] += 1
        next
      end

      conv = if compress
               compress_finetune_trace(transcript: t, max_tool_chars: SFT_MAX_TOOL_CHARS)
             else
               t.map { |e| { role: e[:role].to_s, content: e[:content].to_s } }
                .reject { |e| e[:role] == 'system' && e[:content].start_with?('Session started') }
             end
      if conv.nil? || conv.empty? || conv.none? { |m| m[:role].to_s == 'assistant' }
        dropped[:empty] += 1
        next
      end

      line = case fmt
             when :openai_jsonl then { messages: conv }
             else { conversations: conv.map { |m| { from: sharegpt_role(role: m[:role]), value: m[:content] } } }
             end
      f.puts(JSON.generate(line))
      rows += 1
    end
  end
  {
    path: out, format: fmt, sessions: sids.length, samples: rows,
    bytes: File.size(out), min_score: min_score, compressed: compress,
    dropped: dropped
  }
end

.flip_last_outcome(opts = {}) ⇒ `Object`

# File 'lib/pwn/ai/agent/learning.rb', line 740

public_class_method def self.flip_last_outcome(opts = {})
  return { flipped: false } unless File.exist?(LEARNING_FILE)

  lines = File.readlines(LEARNING_FILE)
  return { flipped: false } if lines.empty?

  last = JSON.parse(lines.last, symbolize_names: true)
  return { flipped: false } if opts[:session_id] && last[:session_id] && last[:session_id] != opts[:session_id]
  return { flipped: false } unless last[:success]

  last[:success]    = false
  last[:flipped_by] = 'user_correction'
  last[:details]    = "#{last[:details]} | CORRECTED: #{opts[:reason].to_s[0, 200]}".strip
  last[:score]      = 0.0
  lines[-1] = "#{JSON.generate(last)}\n"
  File.write(LEARNING_FILE, lines.join)
  # W1 — the (rejected_prev_answer, chosen_next_answer) pair is
  # captured by Mistakes.check_user_correction which has both.
  { flipped: true, id: last[:id], rejected: last[:details].to_s[0, 2_000] }
rescue StandardError
  { flipped: false }
end

.gc_stores!(opts = {}) ⇒ `Object`

One-shot lean across memory + learning + mistakes + sessions. Called from auto_introspect (throttled) so the RL feedback loop keeps ~/.pwn high-signal without a manual learning_gc_stores turn.

Supported Method Parameters: result = PWN::AI::Agent::Learning.gc_stores!( dry_run: 'optional - Boolean (default false)', current_session_id: 'optional - never delete this sessions id', max_entries: 'optional - Memory consolidate cap', max_rows: 'optional - learning.jsonl cap', retain_days: 'optional - outcome / session age floor' )

# File 'lib/pwn/ai/agent/learning.rb', line 1645

public_class_method def self.gc_stores!(opts = {})
  dry = opts[:dry_run] ? true : false
  res = lean!(
    dry_run: dry,
    max_entries: opts[:max_entries],
    max_rows: opts[:max_rows],
    retain_days: opts[:retain_days],
    recent_days: opts[:recent_days],
    details_max: opts[:details_max],
    gold_min_score: opts[:gold_min_score]
  )
  res[:mistakes] = if defined?(Mistakes) && Mistakes.respond_to?(:lean!)
                     Mistakes.lean!(dry_run: dry)
                   else
                     { skipped: true }
                   end
  res[:policy] = if defined?(PWN::AI::Agent::Policy) && Policy.respond_to?(:lean!)
                   Policy.lean!(dry_run: dry)
                 else
                   { skipped: true }
                 end
  res[:sessions] = if defined?(PWN::Sessions) && PWN::Sessions.respond_to?(:lean!)
                     sess_opts = { dry_run: dry }
                     sid = opts[:current_session_id].to_s
                     sess_opts[:current_session_id] = sid unless sid.empty?
                     sess_opts[:retain_days] = opts[:retain_days] if opts.key?(:retain_days)
                     sess_opts[:max_files] = opts[:max_files] if opts.key?(:max_files)
                     PWN::Sessions.lean!(**sess_opts)
                   else
                     { skipped: true }
                   end
  res
end

.help ⇒ `Object`

Display Usage for this Module

# File 'lib/pwn/ai/agent/learning.rb', line 1765

public_class_method def self.help
  puts <<~USAGE
    USAGE:
      PWN::AI::Agent::Learning.note_outcome(task: 'nmap sweep 10.0.0.0/24', success: true, details: '12 hosts up')
      PWN::AI::Agent::Learning.outcomes(limit: 20, success: false)
      PWN::AI::Agent::Learning.reflect(session_id: sid)              # LLM or heuristic → PWN::Memory
      PWN::AI::Agent::Learning.auto_introspect(session_id: sid, request: req, final: text)
      PWN::AI::Agent::Learning.distill_skill(name: 'quick_recon', session_id: sid)
      PWN::AI::Agent::Learning.exemplars_for(request: 'nmap sweep 10/8')  # few-shot for Loop.run
      PWN::AI::Agent::Learning.export_finetune(format: :sharegpt)        # -> ~/.pwn/finetune/*.jsonl
      PWN::AI::Agent::Learning.consolidate(max_entries: 200)         # M1 semantic-merge + M3 importance-evict
      PWN::AI::Agent::Learning.lean!                                 # memory + learning.jsonl prune
      PWN::AI::Agent::Learning.gc_stores!                            # full ~/.pwn RL lean (mem/learn/mistakes/sessions)
      PWN::AI::Agent::Learning.prune_outcomes!                       # learning.jsonl gold-keep cap
      PWN::AI::Agent::Learning.purge_noise                            # one-shot GC of pre-R1 garbage lessons
      PWN::AI::Agent::Learning.to_context(limit: 5)                  # injected by PromptBuilder
      PWN::AI::Agent::Learning.stats
      PWN::AI::Agent::Learning.reset

      Enable end-of-run auto-learning with:
        PWN::Env[:ai][:agent][:auto_introspect] = true
      # auto_introspect throttles gc_stores! every PRUNE_EVERY_N_APPENDS
      # outcomes so ~/.pwn stays lean without a manual GC turn.

      #{self}.authors
  USAGE
end

.lean!(opts = {}) ⇒ `Object`

Memory lean + outcome prune.

# File 'lib/pwn/ai/agent/learning.rb', line 1614

public_class_method def self.lean!(opts = {})
  dry = opts[:dry_run] ? true : false
  out = { dry_run: dry }
  out[:memory] = if defined?(PWN::Memory) && PWN::Memory.respond_to?(:lean!)
                   PWN::Memory.lean!(dry_run: dry)
                 else
                   { skipped: true }
                 end
  out[:memory_consolidate] = consolidate(max_entries: opts[:max_entries] || MAX_MEMORY_ENTRIES) unless dry
  out[:learning] = prune_outcomes!(
    dry_run: dry,
    max_rows: opts[:max_rows],
    retain_days: opts[:retain_days],
    recent_days: opts[:recent_days],
    details_max: opts[:details_max],
    gold_min_score: opts[:gold_min_score]
  )
  out
end

.note_outcome(opts = {}) ⇒ `Object`

Supported Method Parameters

entry = PWN::AI::Agent::Learning.note_outcome( task: 'required - short description of what was attempted', success: 'required - Boolean, did the attempt achieve its goal', details: 'optional - free-form notes / error / evidence', session_id: 'optional - PWN::Sessions id this outcome belongs to', tags: 'optional - Array of String labels for later retrieval' )

# File 'lib/pwn/ai/agent/learning.rb', line 91

public_class_method def self.note_outcome(opts = {})
  task    = opts[:task].to_s
  # 4.1 — allow success: 'soft' (HER) distinct from true/false
  raw_ok  = opts[:success]
  success = if ['soft', :soft].include?(raw_ok)
              'soft'
            else
              raw_ok ? true : false
            end
  raise 'ERROR: task is required' if task.strip.empty?

  tags = Array(opts[:tags]).map(&:to_s)
  details = opts[:details].to_s[0, OUTCOME_DETAILS_MAX]
  # Refuse solved-at-fail (and the reverse): when a score is present,
  # tags / details / boolean success must match verdict_for_score.
  # Soft HER stays soft so C3 positives are not flipped to true/false.
  if opts.key?(:score)
    score = opts[:score].to_f
    want = verdict_for_score(score: score).to_s
    tags = (tags - %w[solved partial wrong unknown]) << want
    details = details.sub(
      /\A(solved|partial|wrong|unknown)\(\d+(?:\.\d+)?\)/i,
      "#{want}(#{format('%.2f', score)})"
    )
    success = (score >= 0.6) if [true, false].include?(success)
  end

  entry = {
    id: Digest::SHA256.hexdigest("#{task}-#{Time.now.to_f}")[0, 12],
    task: task,
    success: success,
    details: details,
    session_id: opts[:session_id],
    tags: tags,
    timestamp: Time.now.utc.iso8601
  }
  entry[:score] = opts[:score].to_f if opts.key?(:score)
  src = opts[:judge_source].to_s
  entry[:judge_source] = src unless src.empty?
  FileUtils.mkdir_p(File.dirname(LEARNING_FILE))
  File.open(LEARNING_FILE, 'a') { |f| f.puts(JSON.generate(entry)) }
  maybe_prune_outcomes!

  # M4 — default: outcomes live in learning.jsonl ONLY.
  # M4.1 — PROCESS SOPs (rubocop/rake/spec after code changes, etc.)
  # are promoted into PWN::Memory[:lesson] so PromptBuilder recall
  # survives across sessions. Without this, the agent re-learns
  # "run rubocop after every patch" every turn (empty memory.json).
  promote_process_lesson(entry: entry) if defined?(PWN::Memory)
  entry
end

.outcomes(opts = {}) ⇒ `Object`

Supported Method Parameters

rows = PWN::AI::Agent::Learning.outcomes( limit: 'optional - max entries returned newest-first (default 50)', success: 'optional - filter by Boolean outcome', tag: 'optional - filter by tag substring' )

# File 'lib/pwn/ai/agent/learning.rb', line 150

public_class_method def self.outcomes(opts = {})
  limit   = opts[:limit] || 50
  want_ok = opts.key?(:success) ? !opts[:success].nil? && opts[:success] != false : nil
  tag     = opts[:tag].to_s.downcase
  return [] unless File.exist?(LEARNING_FILE)

  rows = File.readlines(LEARNING_FILE).map do |l|
    JSON.parse(l, symbolize_names: true)
  rescue StandardError
    nil
  end
  rows.compact!
  rows.select! { |r| want_ok == true ? r[:success] == true : r[:success] == want_ok } unless want_ok.nil?
  rows.select! { |r| Array(r[:tags]).any? { |t| t.to_s.downcase.include?(tag) } } unless tag.empty?
  rows.reverse.first(limit)
end

.prune_outcomes!(opts = {}) ⇒ `Object`

Supported Method Parameters

result = PWN::AI::Agent::Learning.prune_outcomes!( dry_run: 'optional - Boolean (default false)', max_rows: 'optional - hard cap (default MAX_OUTCOME_ROWS)', retain_days: 'optional - age floor for low-value drop', recent_days: 'optional - always keep newer than this' )

Keep gold RL rows (success+score>=0.6+session_id), recent window, high-value tags, and near-miss failures. Dedupe by task+success keeping best score. Truncate details. Never sacrifices exemplar pool.

# File 'lib/pwn/ai/agent/learning.rb', line 1487

public_class_method def self.prune_outcomes!(opts = {})
  dry = opts[:dry_run] ? true : false
  max_rows = (opts[:max_rows] || MAX_OUTCOME_ROWS).to_i
  retain_days = (opts[:retain_days] || OUTCOME_RETAIN_DAYS).to_f
  recent_days = (opts[:recent_days] || OUTCOME_RECENT_DAYS).to_f
  details_max = (opts[:details_max] || OUTCOME_DETAILS_MAX).to_i
  gold_min = (opts[:gold_min_score] || GOLD_MIN_SCORE).to_f

  return { kept: 0, removed: 0, skipped: true } unless File.exist?(LEARNING_FILE)

  before_bytes = File.size(LEARNING_FILE)
  rows = File.readlines(LEARNING_FILE).map do |l|
    JSON.parse(l, symbolize_names: true)
  rescue StandardError
    nil
  end.compact

  now = Time.now.utc
  age_days = lambda do |r|
    (now - Time.parse(r[:timestamp].to_s)) / 86_400.0
  rescue StandardError
    999.0
  end

  protected_row = lambda do |r|
    tags = Array(r[:tags]).map(&:to_s)
    a = age_days.call(r)
    return true if a <= recent_days
    return true if r[:success] == true && r.key?(:score) && r[:score].to_f >= gold_min && r[:session_id].to_s != ''
    return true if r[:success] == true && !r.key?(:score) && r[:session_id].to_s != '' && a <= retain_days
    return true if tags.intersect?(HIGH_VALUE_TAGS)
    return true if r[:success] == false && r.key?(:score) && r[:score].to_f >= 0.5

    false
  end

  rows.reject! { |r| r[:task].to_s.strip.empty? }

  best = {}
  rows.each do |r|
    key = [r[:task].to_s.strip.downcase.gsub(/\s+/, ' ')[0, 160], r[:success].to_s]
    prev = best[key]
    if prev.nil?
      best[key] = r
    else
      ps = prev.key?(:score) ? prev[:score].to_f : -1.0
      rs = r.key?(:score) ? r[:score].to_f : -1.0
      better = rs > ps || (rs == ps && r[:timestamp].to_s > prev[:timestamp].to_s)
      better ||= protected_row.call(r) && !protected_row.call(prev)
      best[key] = r if better
    end
  end
  deduped = best.values
  removed_dupes = rows.size - deduped.size

  truncated = 0
  deduped.each do |r|
    d = r[:details].to_s
    next if d.bytesize <= details_max

    r[:details] = "#{d[0, details_max]}…[compacted]"
    truncated += 1
  end

  protected, unprotected = deduped.partition { |r| protected_row.call(r) }

  kept_unprot = unprotected.reject do |r|
    a = age_days.call(r)
    tags = Array(r[:tags]).map(&:to_s)
    score = r.key?(:score) ? r[:score].to_f : 1.0
    noise_tags = (tags - LOW_VALUE_ONLY_TAGS).empty? && tags.any?
    a > retain_days && noise_tags && score < 0.4
  end

  gold = (protected + kept_unprot).select do |r|
    r[:success] == true && r[:session_id].to_s != '' &&
      (!r.key?(:score) || r[:score].to_f >= gold_min)
  end
  if gold.size < EXEMPLARS_POOL_MIN
    need = EXEMPLARS_POOL_MIN - gold.size
    extra = unprotected.select { |r| r[:success] == true && r[:session_id].to_s != '' }
                       .sort_by { |r| r[:timestamp].to_s }
                       .last(need)
    kept_unprot = (kept_unprot + extra).uniq
  end

  fails = (protected + kept_unprot).reject { |r| r[:success] == true }
  if fails.size < FAILURE_WINDOW_MIN
    need = FAILURE_WINDOW_MIN - fails.size
    extra = unprotected.reject { |r| r[:success] == true }
                       .sort_by { |r| r[:timestamp].to_s }
                       .last(need)
    kept_unprot = (kept_unprot + extra).uniq
  end

  kept = (protected + kept_unprot).uniq
  if kept.size > max_rows
    prot_ids = protected.map { |r| r[:id] }.compact
    over = kept.size - max_rows
    victims = kept.reject { |r| prot_ids.include?(r[:id]) }
                  .sort_by { |r| r[:timestamp].to_s }
                  .first(over)
    v_ids = victims.map { |r| r[:id] }
    kept = kept.reject { |r| v_ids.include?(r[:id]) }
  end

  kept = kept.sort_by { |r| r[:timestamp].to_s }

  atomic_jsonl_write(path: LEARNING_FILE, rows: kept) unless dry

  {
    kept: kept.size,
    removed: (rows.size - kept.size) + removed_dupes,
    deduped: removed_dupes,
    truncated_details: truncated,
    protected: protected.size,
    bytes_before: before_bytes,
    bytes_after: if dry
                   before_bytes
                 else
                   (File.exist?(LEARNING_FILE) ? File.size(LEARNING_FILE) : 0)
                 end,
    dry_run: dry
  }
end

.purge_noise ⇒ `Object`

Supported Method Parameters

PWN::AI::Agent::Learning.purge_noise

One-shot GC of the pre-R1 garbage: drops every PWN::Memory entry matching the old SUCCESS: <req> — <final> / Avoid repeating failure pattern from <tool>: {"success":true shapes. Run once after upgrading; subsequent writes never produce these.

# File 'lib/pwn/ai/agent/learning.rb', line 1731

public_class_method def self.purge_noise
  return { removed: 0 } unless defined?(PWN::Memory)

  mem = nil
  load_err = nil
  begin
    mem = PWN::Memory.load
  rescue StandardError => e
    load_err = e
  end
  if load_err
    warn "[pwn-ai/learning] purge_noise aborted (memory load failed): #{load_err.class}: #{load_err.message}"
    return { removed: 0, aborted: true, error: "#{load_err.class}: #{load_err.message}" }
  end
  before = mem.size
  mem.reject! do |_k, v|
    next false unless v[:category].to_s == 'lesson'

    val = v[:value].to_s
    val.start_with?('SUCCESS: ', 'FAILURE: ') ||
      val.match?(/\AAvoid repeating failure pattern from \w+: .{0,5}\{"success":true/)
  end
  PWN::Memory.save(mem: mem, force: mem.empty?)
  { removed: before - mem.size, remaining: mem.size }
end

.reconcile_verdict_tags!(opts = {}) ⇒ `Object`

One-shot / on-load repair: rewrite tags+details where verdict label disagrees with score (solved @ 0.3 etc.). Safe to call repeatedly.

# File 'lib/pwn/ai/agent/learning.rb', line 1058

public_class_method def self.reconcile_verdict_tags!(opts = {})
  return { repaired: 0 } unless File.exist?(LEARNING_FILE)

  dry = opts[:dry_run] ? true : false
  repaired = 0
  lines = File.readlines(LEARNING_FILE)
  out = lines.map do |l|
    r = JSON.parse(l, symbolize_names: true)
    score = r.key?(:score) ? r[:score].to_f : nil
    next l if score.nil?

    want = verdict_for_score(score: score).to_s
    tags = Array(r[:tags]).map(&:to_s)
    stale = tags & %w[solved partial wrong unknown]
    next l if stale.empty? || stale.include?(want)

    repaired += 1
    next l if dry

    cleaned = tags - %w[solved partial wrong unknown]
    cleaned << want
    r[:tags] = cleaned
    # Fix leading "solved(0.3)" style details head when present
    det = r[:details].to_s
    r[:details] = det.sub(
      /\A(solved|partial|wrong|unknown)\(\d+(?:\.\d+)?\)/i,
      "#{want}(#{format('%.2f', score)})"
    )
    r[:success] = (score >= 0.6) if [true, false].include?(r[:success])
    "#{JSON.generate(r)}\n"
  rescue StandardError
    l
  end
  File.write(LEARNING_FILE, out.join) if !dry && repaired.positive?
  { repaired: repaired, dry_run: dry }
rescue StandardError => e
  { repaired: 0, error: "#{e.class}: #{e.message}" }
end

.reflect(opts = {}) ⇒ `Object`

Supported Method Parameters

report = PWN::AI::Agent::Learning.reflect( session_id: 'required - PWN::Sessions id to analyse', dry_run: 'optional - when true, do not write to Memory/Skills (default false)' )

Uses PWN::AI::Agent::Reflect (when available) to LLM-summarise the session into structured lessons. Falls back to a heuristic extractor when module_reflection is disabled so learning never stops.

# File 'lib/pwn/ai/agent/learning.rb', line 480

public_class_method def self.reflect(opts = {})
  session_id = opts[:session_id]
  dry_run    = opts[:dry_run] ? true : false
  raise 'ERROR: session_id is required' if session_id.to_s.empty?

  transcript = PWN::Sessions.load(session_id: session_id)
  return { session_id: session_id, lessons: [], reason: 'empty transcript' } if transcript.empty?

  lessons = introspective_lessons(transcript: transcript)
  source, conf = lessons.empty? ? [:heuristic, 0.3] : [:reflect, 0.8]
  lessons = heuristic_lessons(transcript: transcript) if lessons.empty?

  saved = []
  lessons.each do |l|
    next if l.to_s.strip.empty?

    key = :"reflect_#{session_id}_#{Digest::SHA256.hexdigest(l)[0, 8]}"
    # M3 — provenance + confidence + ttl so consolidate evicts
    # low-confidence heuristic lessons before hand-written ones.
    PWN::Memory.remember(key: key, value: l, category: :lesson, source: source, confidence: conf, importance: conf, ttl: source == :heuristic ? 7 * 86_400 : nil) unless dry_run
    saved << { key: key, lesson: l }
  end
  consolidate unless dry_run

  { session_id: session_id, lessons: saved, count: saved.length, dry_run: dry_run }
end

.reset ⇒ `Object`

Supported Method Parameters

PWN::AI::Agent::Learning.reset

# File 'lib/pwn/ai/agent/learning.rb', line 850

public_class_method def self.reset
  FileUtils.rm_f(LEARNING_FILE)
  { cleared: true }
end

.stats ⇒ `Object`

Supported Method Parameters

stats = PWN::AI::Agent::Learning.stats

# File 'lib/pwn/ai/agent/learning.rb', line 170

public_class_method def self.stats
  rows   = outcomes(limit: 10_000)
  total  = rows.length
  ok     = rows.count { |r| r[:success] == true }
  skills = defined?(PWN::Skills) && PWN::Skills.is_a?(Hash) ? PWN::Skills.keys.length : 0
  mem    = defined?(PWN::Memory) ? PWN::Memory.load.keys.length : 0
  raw    = total.positive? ? (ok.to_f / total).round(3) : 0.0
  jmean  = total.positive? ? weighted_judge_mean(rows: rows) : nil
  distrust = 0.0
  distrust = Reward.proxy_distrust.to_f.clamp(0.0, 1.0) if defined?(Reward) && Reward.respond_to?(:proxy_distrust)
  {
    total_outcomes: total,
    successes: ok,
    failures: total - ok,
    success_rate: raw,
    adjusted_success_rate: discount_success_rate(proxy: raw, judge: jmean, distrust: distrust),
    proxy_distrust: distrust,
    skills_known: skills,
    memory_entries: mem,
    judge_mean: jmean,
    reward_sentinel: (Reward.sentinel if defined?(Reward)),
    calibration: (Metrics.calibration if defined?(Metrics) && Metrics.respond_to?(:calibration)),
    preference_pairs: (Reward.preferences(limit: 100_000).length if defined?(Reward)),
    tool_metrics: (Metrics.summary(limit: 5) if defined?(Metrics)),
    extrospection: (Extrospection.stats if defined?(Extrospection))
  }
end

.to_context(opts = {}) ⇒ `Object`

Supported Method Parameters

ctx = PWN::AI::Agent::Learning.to_context( limit: 'optional - number of recent outcomes to surface (default 5)' )

# File 'lib/pwn/ai/agent/learning.rb', line 203

public_class_method def self.to_context(opts = {})
  limit = opts[:limit] || 5
  # Fetch a wider window so prefer_primary_tasks can drop critic/red_team
  # envelope rows (REQUEST:/GOAL: prefixes) without starving the block.
  rows  = prefer_primary_tasks(rows: outcomes(limit: limit * 4)).first(limit)
  fails = prefer_primary_tasks(rows: outcomes(limit: 200, success: false))
  # Do not mirror the same ids under both headings — that doubled the
  # failure signal and made RECENT OUTCOMES == RECENT FAILURES when the
  # last N attempts all failed (the injected block looked "stuck").
  row_ids = rows.map { |r| r[:id] }.compact
  fails = fails.reject { |r| row_ids.include?(r[:id]) }.first(limit)
  return '' if rows.empty? && fails.empty?

  fmt = lambda do |r|
    flag = case r[:success]
           when true then '✓'
           when 'soft', :soft then '∼'
           else '✗'
           end
    score = r.key?(:score) ? format('%.2f', r[:score].to_f) : '-'
    task  = display_task(task: r[:task])
    line  = "  #{flag} [#{score}] #{task} (#{r[:timestamp]})"
    # Surface a one-line cause crumb so the agent can actually learn
    # from failures instead of only seeing that they failed.
    if r[:success] != true
      crumb = cause_crumb(details: r[:details])
      line += "\n      cause: #{crumb}" unless crumb.empty?
    end
    line
  end
  s   = stats
  jm  = s[:judge_mean]
  d   = s[:proxy_distrust].to_f
  rate = d > 0.05 ? s[:adjusted_success_rate] : s[:success_rate]
  tag  = d > 0.05 ? ' adj' : ''
  hdr = "RECENT OUTCOMES (success_rate=#{(rate.to_f * 100).round(1)}%#{tag}#{" judge_mean=#{jm}" if jm} over #{s[:total_outcomes]} attempts)"
  out = "#{hdr}\n#{rows.map(&fmt).join("\n")}\n"
  out += "RECENT FAILURES (learn from these — do not repeat)\n#{fails.map(&fmt).join("\n")}\n" unless fails.empty?
  "#{out}\n"
end

.update_skill(opts = {}) ⇒ `Object`

Fold RL artefacts (mistakes / structured_fix / an explicit lesson) into an existing skill. Does not create skills (use distill_skill / skill_create). Does not write loop-law. Dedupes by mistake signature.

# File 'lib/pwn/ai/agent/learning.rb', line 414

public_class_method def self.update_skill(opts = {})
  dry = opts[:dry_run] ? true : false
  lesson = opts[:lesson].to_s.strip
  notes = rl_skill_notes(
    signature: opts[:signature],
    request: opts[:request] || opts[:query],
    lesson: lesson
  )
  return { updated: false, reason: 'no rl notes' } if notes.empty?

  target = locate_skill_for_update(
    name: opts[:name],
    query: opts[:query] || opts[:request] || notes.map { |n| n[:text] }.join(' ')
  )
  return { updated: false, reason: 'no matching skill', notes: notes } unless target

  body = skill_body_without_frontmatter(meta: target[:meta])
  added = []
  notes.each do |note|
    next if body.include?(note[:id])

    added << note
  end
  return { updated: false, name: target[:name], reason: 'already folded', notes: notes } if added.empty?

  block = added.map { |n| "- [#{n[:id]}] #{n[:text]}" }.join("\n")
  body = if body.match?(/^\#{1,3}\s*RL feedback\s*$/i)
           "#{body.rstrip}\n#{block}\n"
         else
           "#{body.rstrip}\n\n## RL feedback\n#{block}\n"
         end
  return { updated: false, dry_run: true, name: target[:name], added: added.map { |n| n[:text] } } if dry

  root = skills_dir
  out = PWN::Config.write_skill(
    name: target[:name],
    description: target[:meta][:description],
    content: body,
    references: target[:meta][:references],
    license: (target[:meta][:frontmatter] || {})['license'],
    allowed_tools: target[:meta][:allowed_tools],
    metadata: (target[:meta][:frontmatter] || {})['metadata'],
    pwn_skills_path: root
  )
  PWN::Config.load_skills(pwn_skills_path: root) if PWN::Config.respond_to?(:load_skills)
  note_outcome(
    task: "skills_update:#{target[:name]}",
    success: true,
    details: "Folded #{added.length} RL note(s)",
    tags: %w[skill rl]
  )
  out.merge(updated: true, added: added.map { |n| n[:id] })
rescue StandardError => e
  { updated: false, error: "#{e.class}: #{e.message}" }
end

Module: PWN::AI::Agent::Learning

Overview

Constant Summary collapse

Class Method Summary collapse

Class Method Details

.authors ⇒ Object

.auto_introspect(opts = {}) ⇒ Object

.consolidate(opts = {}) ⇒ Object

.distill_skill(opts = {}) ⇒ Object

.exemplars_for(opts = {}) ⇒ Object

.export_finetune(opts = {}) ⇒ Object

.flip_last_outcome(opts = {}) ⇒ Object

.gc_stores!(opts = {}) ⇒ Object

.help ⇒ Object

.lean!(opts = {}) ⇒ Object

.note_outcome(opts = {}) ⇒ Object

.outcomes(opts = {}) ⇒ Object

.prune_outcomes!(opts = {}) ⇒ Object

.purge_noise ⇒ Object

.reconcile_verdict_tags!(opts = {}) ⇒ Object

.reflect(opts = {}) ⇒ Object

.reset ⇒ Object

.stats ⇒ Object

.to_context(opts = {}) ⇒ Object

.update_skill(opts = {}) ⇒ Object