Module: PWN::AI::Agent::Learning

Defined in:
lib/pwn/ai/agent/learning.rb

Overview

PWN::AI::Agent::Learning is the self-improvement engine that closes the pwn-ai feedback loop. It captures task outcomes, mines session transcripts for durable lessons, promotes successful workflows into reusable skills, and keeps ~/.pwn lean (memory + learning.jsonl + mistakes + sessions) so the agent gets sharper over time instead of accumulating noise.

Data flows:

Loop.run --(tool telemetry)--> Metrics.record
Loop.run --(final answer)----> Learning.auto_introspect (opt-in)
auto_introspect --(throttled)--> Learning.gc_stores!  # ~/.pwn lean
model    --(tool calls)------> learning_note_outcome / _distill_skill
PromptBuilder <----------------- Learning.to_context + Metrics.to_context

Everything is file-backed under ~/.pwn so it survives across REPL restarts and is shared by every future session.

Constant Summary collapse

LEARNING_FILE = 
File.join(Dir.home, '.pwn', 'learning.jsonl')
FINETUNE_DIR = 
File.join(Dir.home, '.pwn', 'finetune')
INTROSPECT_SOFT_MS =

P0 — post-answer introspect must not train "stop early" while spending the iteration budget after the final. Soft cap skips expensive stages (tool critic, PRM-LLM, reflect, extrospect); hard cap keeps only note_outcome + judge(heuristic) + sentinel.

2_500
INTROSPECT_HARD_MS = 
8_000
INTROSPECT_MIN_STAGES = 
%i[judge note_outcome fold_judge sentinel].freeze
MAX_MEMORY_ENTRIES = 
200
MAX_OUTCOME_ROWS =

Lean outcome retention — keep gold RL signal, drop bulk auto noise.

800
OUTCOME_RETAIN_DAYS = 
45
OUTCOME_RECENT_DAYS = 
14
OUTCOME_DETAILS_MAX = 
800
GOLD_MIN_SCORE = 
0.6
EXEMPLARS_POOL_MIN = 
200
FAILURE_WINDOW_MIN = 
200
HIGH_VALUE_TAGS = 
%w[
  needs_human extro_verify sdr gqrx rl pwn-ai curriculum hindsight her
].freeze
LOW_VALUE_ONLY_TAGS = 
%w[
  auto loop partial wrong solved offline_judge plan_cover_high
].freeze
PRUNE_EVERY_N_APPENDS = 
25
CLAIM_METRIC_WORDS =

E3/P26 — only CVE-ids or software-name + full semver (x.y.z). Two-part floats ("cap 0.2", "proxy 1.0", "judge 37.0") are RL metric crumbs that were scraped by verify_as_reward and flooded learning.jsonl with extro_verify :unknown failures.

%w[
  cap share proxy judge success only now clears gap score rate mean
  brier overconf distrust trajectory_fraction handler orm prm delta
  limit window pct percent ms iter budget conf confidence n
  ruby python linux kernel host cwd e.g e.g.
].freeze
CLAIM_RX = 
/
  CVE-\d{4}-\d{4,7}
  |
  \b
  (?!
    (?:cap|share|proxy|judge|success|only|now|clears|gap|score|rate|mean|
       brier|overconf|distrust|trajectory_fraction|handler|orm|prm|delta|
       limit|window|pct|percent|ms|iter|budget|conf|confidence|
       ruby|python|linux|kernel|host|cwd|e\.g)
    \b
  )
  [A-Za-z][\w.+-]{2,}
  \s+
  v?\d+\.\d+\.\d+(?:[-+][\w.]+)?
  \b
/x
SFT_MIN_SCORE =

P12 — SFT quality gate (as hard as DPO source-cap): drop HER/soft, low judge_score, auto-only noise without score, and PRM-compress trajectories so LoRA is not 5MB of "how we flailed".

0.6
SFT_MAX_TOOL_CHARS = 
1_200
PROCESS_SOP_RX =

M4.1 — keyword gate for "process / hygiene" SOPs the operator keeps re-requesting (rubocop, rake, rspec, conventions). These must become durable Memory lessons, not just learning.jsonl rows.

%r{\b(rubocop|rake\b|rspec|bundle\s+exec|conventions?|lint(?:ing)?|style/|code\s*hygiene|post[- ]?patch|after\s+(?:every\s+)?(?:patch|change|edit))\b}i
FAILURE_FINAL_RX = 
/\[pwn-ai\] (iteration budget exhausted|engine returned no message)|\b(i (was )?unable to|i could not|i couldn'?t|cannot proceed|failed to)\b/i

Class Method Summary collapse

Class Method Details

.authors ⇒ Object

Author(s)

0day Inc. support@0dayinc.com



1549
1550
1551
 
# File 'lib/pwn/ai/agent/learning.rb', line 1549

public_class_method def self.authors
  "AUTHOR(S):\n  0day Inc. <support@0dayinc.com>\n"
end

.auto_introspect(opts = {}) ⇒ Object

Supported Method Parameters

PWN::AI::Agent::Learning.auto_introspect( session_id: 'required - id of the just-completed session', request: 'optional - original user request (for outcome logging)', final: 'optional - final assistant answer (for outcome logging)' )

Called by Loop.run when PWN::Env[:agent][:auto_introspect] is truthy. Never raises — learning must not break the primary loop.



432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
 
# File 'lib/pwn/ai/agent/learning.rb', line 432

public_class_method def self.auto_introspect(opts = {})
  session_id = opts[:session_id]
  return unless session_id
  return unless auto_introspect_enabled?

  t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  stages_run = []
  stages_skipped = []
  budget_hot = begin
    defined?(Loop) && Loop.respond_to?(:budget_exhaustion_hot?, true) &&
      Loop.send(:budget_exhaustion_hot?)
  rescue StandardError
    false
  end

  elapsed_ms = lambda do
    ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0) * 1000).round
  end
  # soft = skip expensive; hard = stop almost everything
  over_soft = lambda do
    ms = elapsed_ms.call
    ms >= INTROSPECT_SOFT_MS || budget_hot
  end
  over_hard = lambda do
    ms = elapsed_ms.call
    ms >= INTROSPECT_HARD_MS
  end

  proxy_ok = infer_success(session_id: session_id, final: opts[:final])

  # S3 critic — BEFORE reward so verdict is evidence.
  # P24/P0 — budget_hot or soft-cap → text_only or skip.
  force_critic = begin
    eng = (PWN::Env.dig(:ai, :active) if defined?(PWN::Env))
    cal = defined?(Metrics) ? Metrics.calibration(engine: eng) : { n: 0 }
    cal[:n].to_i >= 8 && (cal[:brier].to_f > 0.35 || cal[:overconfidence].to_f > 0.25)
  rescue StandardError
    false
  end
  # P0 — when W1 mix urgently needs :critic pairs, prefer running critic
  need_critic_mix = begin
    mix = defined?(Reward) && Reward.respond_to?(:generator_mix) ? Reward.generator_mix : {}
    Array(mix[:urgent]).include?('critic')
  rescue StandardError
    false
  end

  crit = { verdict: :pass, source: :skipped }
  # Single skip path (Lint/DuplicateBranch): hard-budget OR no Curriculum.
  if !defined?(Curriculum) || (over_hard.call && !need_critic_mix)
    stages_skipped << :critic
  elsif budget_hot || (over_soft.call && !force_critic && !need_critic_mix)
    stages_run << :critic_text_only
    crit = Curriculum.critic(
      request: opts[:request],
      final: opts[:final],
      session_id: session_id,
      text_only: true
    )
  elsif force_critic
    stages_run << :critic_forced
    prev = (PWN::Env[:ai][:agent][:critic] if defined?(PWN::Env) && PWN::Env[:ai].is_a?(Hash) && PWN::Env[:ai][:agent].is_a?(Hash))
    begin
      PWN::Env[:ai][:agent][:critic] = true if defined?(PWN::Env) && PWN::Env[:ai].is_a?(Hash) && PWN::Env[:ai][:agent].is_a?(Hash) && !PWN::Env[:ai][:agent].frozen?
      crit = Curriculum.critic(request: opts[:request], final: opts[:final], session_id: session_id)
    ensure
      PWN::Env[:ai][:agent][:critic] = prev if defined?(PWN::Env) && PWN::Env[:ai].is_a?(Hash) && PWN::Env[:ai][:agent].is_a?(Hash) && !PWN::Env[:ai][:agent].frozen?
    end
  else
    stages_run << :critic
    crit = Curriculum.critic(request: opts[:request], final: opts[:final], session_id: session_id)
  end

  # R1 judge — always attempt (heuristic is cheap; LLM gated inside)
  stages_run << :judge
  v = Reward.judge(request: opts[:request], final: opts[:final], session_id: session_id, proxy_ok: proxy_ok) if defined?(Reward)
  v ||= { score: proxy_ok ? 1.0 : 0.0, success: proxy_ok, verdict: proxy_ok ? :solved : :wrong }
  v[:score] = [v[:score], 0.3].min if crit[:verdict] == :flaw
  # P29 — critic floor used to leave stale verdict=:solved at score=0.3,
  # producing learning.jsonl rows tagged "solved" with success=false
  # (116+ rows). Always resync verdict/success from the final score.
  v[:verdict] = verdict_for_score(score: v[:score])
  v[:success] = v[:score].to_f >= 0.6
  ok = v[:success]

  # W1 pending user_correction pair
  pend = Thread.current[:pwn_pending_pref]
  if pend && ok && defined?(Reward)
    stages_run << :user_correction_pref
    Reward.record_preference(
      prompt: pend[:prompt],
      rejected: pend[:rejected],
      chosen: opts[:final].to_s,
      source: :user_correction,
      shape: :revised_answer,
      force: true
    )
    Thread.current[:pwn_pending_pref] = nil
  end

  # Soft plan-quality feature (W3) — tag only; not full DPO.
  plan_cov = nil
  if defined?(Reward) && Reward.respond_to?(:plan_coverage)
    begin
      plan_for_cov = opts[:plan]
      plan_for_cov = opts[:ts_state][:plan] if plan_for_cov.nil? && opts[:ts_state].is_a?(Hash)
      if plan_for_cov.nil? && defined?(TaskSummarizer)
        # Recover numbered tasks from the final/request only when caller
        # did not pass a plan — still keeps TaskSummarizer out of the
        # credit path (parse is pure text).
        plan_for_cov = nil
      end
      if !plan_for_cov.nil? || opts[:final].to_s.length.positive?
        plan_cov = Reward.plan_coverage(
          plan: plan_for_cov || [],
          final: opts[:final],
          request: opts[:request],
          session_id: session_id
        )
        stages_run << :plan_coverage if plan_cov && plan_cov[:total].to_i.positive?
      end
    rescue StandardError => e
      warn "[pwn-ai/learning] plan_coverage swallowed: #{e.class}: #{e.message}"
    end
  end

  stages_run << :note_outcome
  outcome_tags = ['auto', 'loop', v[:verdict].to_s]
  outcome_tags << plan_cov[:tag] if plan_cov && plan_cov[:tag]
  outcome_tags << "plan_cover=#{plan_cov[:score]}" if plan_cov && plan_cov[:total].to_i.positive?
  # P29 — persist the bare user ask (strip REQUEST:/GOAL: envelopes at write time)
  task_txt = display_task(task: opts[:request].to_s)
  task_txt = opts[:request].to_s[0, 100] if task_txt.empty?
  note_outcome(
    task: task_txt,
    success: ok,
    score: v[:score],
    details: "#{v[:verdict]}(#{v[:score].to_f.round(2)}) #{v[:rationale]} | #{opts[:final].to_s[0, 200]}",
    session_id: session_id,
    tags: outcome_tags
  )

  stages_run << :fold_judge
  fold_judge_into_metrics(session_id: session_id, score: v[:score], confidence: v[:confidence])

  # R2 PRM — skip under hard cap (expensive LLM); keep under soft if heuristic path
  if over_hard.call
    stages_skipped << :prm
  elsif defined?(Reward)
    stages_run << :prm
    Reward.prm(request: opts[:request], session_id: session_id)
  end

  # C3 HER — only on failure; skip hard
  if !ok && defined?(Curriculum) && !over_hard.call
    stages_run << :hindsight
    Curriculum.hindsight(request: opts[:request], final: opts[:final], session_id: session_id)
  else
    stages_skipped << :hindsight unless ok
  end

  # W3 calibrate — cheap; always when predicted available
  predicted = opts[:predicted]
  predicted = recover_predicted_from_session(session_id: session_id) if predicted.nil?
  if !predicted.nil? && defined?(Curriculum)
    stages_run << :calibrate
    Curriculum.calibrate(predicted: predicted, actual: v[:score], engine: PWN::Env.dig(:ai, :active))
  end

  # reflect on success — skip soft/hard (LLM + memory writes)
  # M4.1 — also reflect when the request/final is a process SOP
  # (code hygiene) even if judge score < 0.6, so rubocop/rake
  # lessons still land in PWN::Memory.
  process_sop = process_sop_text?(text: "#{opts[:request]} #{opts[:final]}")
  if (ok || process_sop) && !over_soft.call
    stages_run << :reflect
    reflect(session_id: session_id)
  elsif ok || process_sop
    stages_skipped << :reflect
    # Cheap path under soft budget: still promote a canned process lesson
    if process_sop && defined?(PWN::Memory)
      promote_process_lesson(
        entry: {
          task: opts[:request].to_s[0, 120],
          success: ok,
          score: v[:score],
          details: opts[:final].to_s[0, 500],
          tags: %w[auto process_sop]
        }
      )
    end
  end

  # R3 sentinel — cheap disk math; always
  if defined?(Reward)
    stages_run << :sentinel
    Reward.sentinel
  end

  # E ambient extrospect — skip soft (can launch probes)
  if defined?(Extrospection) && !over_soft.call
    stages_run << :extrospect
    Extrospection.auto_extrospect(session_id: session_id)
  else
    stages_skipped << :extrospect
  end

  # Keep ~/.pwn RL stores lean on the feedback path (memory +
  # learning.jsonl + mistakes + sessions). Throttled; never raises.
  # Disk-only work: skip only hard budget / budget_hot.
  begin
    if !over_hard.call && !budget_hot && should_gc_stores?
      stages_run << :lean_gc
      gc_stores!(current_session_id: session_id)
    elsif should_gc_stores?
      stages_skipped << :lean_gc
    end
  rescue StandardError => e
    warn "[pwn-ai/learning] post-introspect lean swallowed: #{e.class}: #{e.message}"
  end

  {
    ok: ok,
    score: v[:score],
    elapsed_ms: elapsed_ms.call,
    budget_hot: budget_hot,
    stages_run: stages_run,
    stages_skipped: stages_skipped
  }
rescue StandardError => e
  warn "[pwn-ai/learning] auto_introspect swallowed: #{e.class}: #{e.message}"
  nil
end

.consolidate(opts = {}) ⇒ Object

Supported Method Parameters

removed = PWN::AI::Agent::Learning.consolidate( max_entries: 'optional - hard cap on PWN::Memory size (default MAX_MEMORY_ENTRIES)' )

Deduplicates near-identical lesson values and prunes the oldest entries once the cap is exceeded so the injected MEMORY block stays high-signal.



698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
 
# File 'lib/pwn/ai/agent/learning.rb', line 698

public_class_method def self.consolidate(opts = {})
  cap = opts[:max_entries] || MAX_MEMORY_ENTRIES
  return { removed: 0 } unless defined?(PWN::Memory)

  mem = nil
  load_err = nil
  begin
    mem = PWN::Memory.load
  rescue StandardError => e
    load_err = e
  end
  if load_err
    warn "[pwn-ai/learning] consolidate aborted (memory load failed): #{load_err.class}: #{load_err.message}"
    return { removed: 0, aborted: true, error: "#{load_err.class}: #{load_err.message}" }
  end
  removed = []

  # M1 — semantic clustering: embed :lesson entries, greedy-merge
  # near-duplicates (cosine ≥ 0.92) via Reflect into ONE imperative
  # lesson. Falls back to sha-dedup when no embed backend.
  removed.concat(semantic_merge(mem: mem)) if defined?(PWN::MemoryIndex) && PWN::MemoryIndex.available?

  seen = {}
  mem.each do |k, v|
    sig = Digest::SHA256.hexdigest(v[:value].to_s.strip.downcase)[0, 16]
    seen[sig] ? removed << k : seen[sig] = k
  end
  removed.uniq.each { |k| mem.delete(k) }

  # M3 — evict by (age/ttl) / (importance × confidence), NOT
  # oldest-first. Hand-written high-value lessons survive; low-
  # confidence :heuristic auto-gen self-evicts first.
  if mem.size > cap
    now = Time.now.utc
    scored = mem.map do |k, v|
      if defined?(PWN::Memory) && PWN::Memory.respond_to?(:protected_entry?) &&
         PWN::Memory.protected_entry?(key: k, entry: v)
        next [k, Float::INFINITY]
      end

      age_d = (now - Time.parse(v[:timestamp].to_s)) / 86_400.0
      ttl_d = (v[:ttl].to_f / 86_400.0)
      imp   = (v[:importance] || 0.5).to_f.clamp(0.05, 1.0)
      conf  = (v[:confidence] || (v[:source].to_s == 'human' ? 0.95 : 0.5)).to_f.clamp(0.05, 1.0)
      staleness = ttl_d.positive? ? age_d / ttl_d : age_d / 90.0
      # lower score = drop first; Infinity protected sorts last
      [k, -(staleness / (imp * conf))]
    rescue StandardError
      [k, 0.0]
    end
    # sort ascending by score so lowest (most stale/low-imp) first
    ordered = scored.sort_by { |_k, s| s }
    drop = []
    ordered.each do |pair|
      k = pair[0]
      break if mem.size - drop.size <= cap
      next if defined?(PWN::Memory) && PWN::Memory.respond_to?(:protected_entry?) &&
              PWN::Memory.protected_entry?(key: k, entry: mem[k])

      drop << k
    end
    drop.each { |k| mem.delete(k) }
    removed.concat(drop)
  end
  PWN::Memory.save(mem: mem, force: mem.empty?)
  if PWN::Memory.respond_to?(:lean!)
    begin
      PWN::Memory.lean!
    rescue StandardError => e
      warn "[pwn-ai/learning] post-consolidate memory.lean! swallowed: #{e.class}: #{e.message}"
    end
  end
  { removed: removed.uniq.length, remaining: mem.size }
end

.distill_skill(opts = {}) ⇒ Object

Supported Method Parameters

skill = PWN::AI::Agent::Learning.distill_skill( name: 'required - snake_case name for the new skill', session_id: 'optional - PWN::Sessions id to mine (uses its transcript)', content: 'optional - explicit markdown body; overrides transcript mining', references: 'optional - Array of reference URLs / CWE / CVE / ATT&CK ids' )



365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
 
# File 'lib/pwn/ai/agent/learning.rb', line 365

public_class_method def self.distill_skill(opts = {})
  raise 'ERROR: name is required' if opts[:name].to_s.strip.empty?

  body = opts[:content].to_s
  body = build_skill_from_session(session_id: opts[:session_id], name: opts[:name]) if body.strip.empty? && opts[:session_id]
  raise 'ERROR: content or session_id is required' if body.strip.empty?

  root = skills_dir
  out  = PWN::Config.write_skill(
    name: opts[:name],
    description: opts[:description],
    content: body,
    references: opts[:references],
    pwn_skills_path: root
  )
  PWN::Config.load_skills(pwn_skills_path: root) if PWN::Config.respond_to?(:load_skills)
  note_outcome(task: "distill_skill:#{out[:name]}", success: true, details: "Saved #{out[:path]}", tags: %w[skill auto])
  out.merge(saved: true)
end

.exemplars_for(opts = {}) ⇒ Object

Supported Method Parameters

msgs = PWN::AI::Agent::Learning.exemplars_for( request: 'required - current user request', limit: 'optional - max exemplar traces to return (default 1)', max_msgs: 'optional - cap on messages per exemplar (default 6)' )

Retrieval-augmented BEHAVIOUR: keyword-matches request against prior successful outcomes in learning.jsonl, loads the matching session, and compresses its (user, tool, assistant) trace into a short few-shot exemplar Loop.run splices between system and user. Local models are dramatically better with 1 concrete example than with 25 abstract lessons.



232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
 
# File 'lib/pwn/ai/agent/learning.rb', line 232

public_class_method def self.exemplars_for(opts = {})
  request  = opts[:request].to_s.downcase
  limit    = (opts[:limit]    || 1).to_i
  max_msgs = (opts[:max_msgs] || 6).to_i
  return [] if request.strip.empty?

  tokens = request.scan(/[a-z0-9_]{3,}/).uniq
  return [] if tokens.empty?

  now = Time.now.utc
  # C2 — prioritized replay: priority = judge_score × recency_decay × keyword_sim
  # C2 — strict success:true only (excludes HER success:'soft'). Also
  # down-weight any residual hindsight-tagged rows so partial failures
  # never launder into full-strength few-shot exemplars.
  # P20 — strict success:true AND prefer high judge scores. Drop rows
  # with explicit low ORM score so proxy-true / judge-low cannot be few-shot.
  pool = outcomes(limit: 500, success: true).reject { |r| r[:session_id].to_s.empty? }
  pool = pool.reject { |r| r.key?(:score) && r[:score].to_f < 0.6 }
  scored = pool.map do |r|
    sim   = tokens.count { |t| r[:task].to_s.downcase.include?(t) }.to_f / tokens.length
    age_d = (now - Time.parse(r[:timestamp].to_s)) / 86_400.0
    decay = Math.exp(-age_d / 30.0)
    score = (r[:score] || 1.0).to_f
    tags  = Array(r[:tags]).map(&:to_s)
    # HER / soft / hindsight → 0.35× so they cannot dominate C2 priority
    score *= 0.35 if r[:success].to_s == 'soft' || tags.intersect?(%w[hindsight her soft])
    [r, sim * decay * score]
  rescue StandardError
    [r, 0.0]
  end
  hits = scored.reject { |_, pr| pr <= 0.0 }.sort_by { |_, pr| -pr }.first(limit).map(&:first)

  hits.flat_map { |r| compress_exemplar(session_id: r[:session_id], max_msgs: max_msgs) }
rescue StandardError
  []
end

.export_finetune(opts = {}) ⇒ Object 



289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
 
# File 'lib/pwn/ai/agent/learning.rb', line 289

public_class_method def self.export_finetune(opts = {})
  fmt       = (opts[:format] || :sharegpt).to_sym
  min_tools = (opts[:min_tools] || 1).to_i
  min_score = (opts[:min_score] || SFT_MIN_SCORE).to_f
  compress  = opts.key?(:compress) ? opts[:compress] : true
  FileUtils.mkdir_p(FINETUNE_DIR)
  out = opts[:out] || File.join(FINETUNE_DIR, "pwn-#{Time.now.utc.strftime('%Y%m%d')}.jsonl")

  # 4.1 / P12 — exclude HER soft-success + low-score + untagged auto flail
  gold = outcomes(limit: 10_000, success: true).reject do |r|
    tags = Array(r[:tags]).map(&:to_s)
    soft = r[:success].to_s == 'soft' || tags.intersect?(%w[hindsight her soft])
    low  = !r[:score].nil? && r[:score].to_f < min_score
    # require a score when present in corpus era that has scores
    soft || low
  end
  # prefer highest-score outcome per session
  by_sid = {}
  gold.each do |r|
    sid = r[:session_id].to_s
    next if sid.empty?

    prev = by_sid[sid]
    by_sid[sid] = r if prev.nil? || r[:score].to_f >= prev[:score].to_f
  end
  sids = by_sid.keys
  rows = 0
  dropped = { tools: 0, empty: 0, load: 0 }
  File.open(out, 'w') do |f|
    sids.each do |sid|
      t = begin
        PWN::Sessions.load(session_id: sid)
      rescue StandardError
        dropped[:load] += 1
        next
      end
      tool_n = t.count { |e| e[:role].to_s == 'tool' }
      if tool_n < min_tools
        dropped[:tools] += 1
        next
      end

      conv = if compress
               compress_finetune_trace(transcript: t, max_tool_chars: SFT_MAX_TOOL_CHARS)
             else
               t.map { |e| { role: e[:role].to_s, content: e[:content].to_s } }
                .reject { |e| e[:role] == 'system' && e[:content].start_with?('Session started') }
             end
      if conv.nil? || conv.empty? || conv.none? { |m| m[:role].to_s == 'assistant' }
        dropped[:empty] += 1
        next
      end

      line = case fmt
             when :openai_jsonl then { messages: conv }
             else { conversations: conv.map { |m| { from: sharegpt_role(role: m[:role]), value: m[:content] } } }
             end
      f.puts(JSON.generate(line))
      rows += 1
    end
  end
  {
    path: out, format: fmt, sessions: sids.length, samples: rows,
    bytes: File.size(out), min_score: min_score, compressed: compress,
    dropped: dropped
  }
end

.flip_last_outcome(opts = {}) ⇒ Object 



666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
 
# File 'lib/pwn/ai/agent/learning.rb', line 666

public_class_method def self.flip_last_outcome(opts = {})
  return { flipped: false } unless File.exist?(LEARNING_FILE)

  lines = File.readlines(LEARNING_FILE)
  return { flipped: false } if lines.empty?

  last = JSON.parse(lines.last, symbolize_names: true)
  return { flipped: false } if opts[:session_id] && last[:session_id] && last[:session_id] != opts[:session_id]
  return { flipped: false } unless last[:success]

  last[:success]    = false
  last[:flipped_by] = 'user_correction'
  last[:details]    = "#{last[:details]} | CORRECTED: #{opts[:reason].to_s[0, 200]}".strip
  last[:score]      = 0.0
  lines[-1] = "#{JSON.generate(last)}\n"
  File.write(LEARNING_FILE, lines.join)
  # W1 — the (rejected_prev_answer, chosen_next_answer) pair is
  # captured by Mistakes.check_user_correction which has both.
  { flipped: true, id: last[:id], rejected: last[:details].to_s[0, 2_000] }
rescue StandardError
  { flipped: false }
end

.gc_stores!(opts = {}) ⇒ Object

One-shot lean across memory + learning + mistakes + sessions. Called from auto_introspect (throttled) so the RL feedback loop keeps ~/.pwn high-signal without a manual learning_gc_stores turn.

Supported Method Parameters

result = PWN::AI::Agent::Learning.gc_stores!( dry_run: 'optional - Boolean (default false)', current_session_id: 'optional - never delete this sessions id', max_entries: 'optional - Memory consolidate cap', max_rows: 'optional - learning.jsonl cap', retain_days: 'optional - outcome / session age floor' )



1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
 
# File 'lib/pwn/ai/agent/learning.rb', line 1440

public_class_method def self.gc_stores!(opts = {})
  dry = opts[:dry_run] ? true : false
  res = lean!(
    dry_run: dry,
    max_entries: opts[:max_entries],
    max_rows: opts[:max_rows],
    retain_days: opts[:retain_days],
    recent_days: opts[:recent_days],
    details_max: opts[:details_max],
    gold_min_score: opts[:gold_min_score]
  )
  res[:mistakes] = if defined?(Mistakes) && Mistakes.respond_to?(:lean!)
                     Mistakes.lean!(dry_run: dry)
                   else
                     { skipped: true }
                   end
  res[:sessions] = if defined?(PWN::Sessions) && PWN::Sessions.respond_to?(:lean!)
                     sess_opts = { dry_run: dry }
                     sid = opts[:current_session_id].to_s
                     sess_opts[:current_session_id] = sid unless sid.empty?
                     sess_opts[:retain_days] = opts[:retain_days] if opts.key?(:retain_days)
                     sess_opts[:max_files] = opts[:max_files] if opts.key?(:max_files)
                     PWN::Sessions.lean!(**sess_opts)
                   else
                     { skipped: true }
                   end
  res
end

.help ⇒ Object

Display Usage for this Module



1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
 
# File 'lib/pwn/ai/agent/learning.rb', line 1555

public_class_method def self.help
  puts <<~USAGE
    USAGE:
      PWN::AI::Agent::Learning.note_outcome(task: 'nmap sweep 10.0.0.0/24', success: true, details: '12 hosts up')
      PWN::AI::Agent::Learning.outcomes(limit: 20, success: false)
      PWN::AI::Agent::Learning.reflect(session_id: sid)              # LLM or heuristic → PWN::Memory
      PWN::AI::Agent::Learning.auto_introspect(session_id: sid, request: req, final: text)
      PWN::AI::Agent::Learning.distill_skill(name: 'quick_recon', session_id: sid)
      PWN::AI::Agent::Learning.exemplars_for(request: 'nmap sweep 10/8')  # few-shot for Loop.run
      PWN::AI::Agent::Learning.export_finetune(format: :sharegpt)        # -> ~/.pwn/finetune/*.jsonl
      PWN::AI::Agent::Learning.consolidate(max_entries: 200)         # M1 semantic-merge + M3 importance-evict
      PWN::AI::Agent::Learning.lean!                                 # memory + learning.jsonl prune
      PWN::AI::Agent::Learning.gc_stores!                            # full ~/.pwn RL lean (mem/learn/mistakes/sessions)
      PWN::AI::Agent::Learning.prune_outcomes!                       # learning.jsonl gold-keep cap
      PWN::AI::Agent::Learning.purge_noise                            # one-shot GC of pre-R1 garbage lessons
      PWN::AI::Agent::Learning.to_context(limit: 5)                  # injected by PromptBuilder
      PWN::AI::Agent::Learning.stats
      PWN::AI::Agent::Learning.reset

      Enable end-of-run auto-learning with:
        PWN::Env[:ai][:agent][:auto_introspect] = true
      # auto_introspect throttles gc_stores! every PRUNE_EVERY_N_APPENDS
      # outcomes so ~/.pwn stays lean without a manual GC turn.

      #{self}.authors
  USAGE
end

.lean!(opts = {}) ⇒ Object

Memory lean + outcome prune.



1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
 
# File 'lib/pwn/ai/agent/learning.rb', line 1409

public_class_method def self.lean!(opts = {})
  dry = opts[:dry_run] ? true : false
  out = { dry_run: dry }
  out[:memory] = if defined?(PWN::Memory) && PWN::Memory.respond_to?(:lean!)
                   PWN::Memory.lean!(dry_run: dry)
                 else
                   { skipped: true }
                 end
  out[:memory_consolidate] = consolidate(max_entries: opts[:max_entries] || MAX_MEMORY_ENTRIES) unless dry
  out[:learning] = prune_outcomes!(
    dry_run: dry,
    max_rows: opts[:max_rows],
    retain_days: opts[:retain_days],
    recent_days: opts[:recent_days],
    details_max: opts[:details_max],
    gold_min_score: opts[:gold_min_score]
  )
  out
end

.note_outcome(opts = {}) ⇒ Object

Supported Method Parameters

entry = PWN::AI::Agent::Learning.note_outcome( task: 'required - short description of what was attempted', success: 'required - Boolean, did the attempt achieve its goal', details: 'optional - free-form notes / error / evidence', session_id: 'optional - PWN::Sessions id this outcome belongs to', tags: 'optional - Array of String labels for later retrieval' )



91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
 
# File 'lib/pwn/ai/agent/learning.rb', line 91

public_class_method def self.note_outcome(opts = {})
  task    = opts[:task].to_s
  # 4.1 — allow success: 'soft' (HER) distinct from true/false
  raw_ok  = opts[:success]
  success = if ['soft', :soft].include?(raw_ok)
              'soft'
            else
              raw_ok ? true : false
            end
  raise 'ERROR: task is required' if task.strip.empty?

  entry = {
    id: Digest::SHA256.hexdigest("#{task}-#{Time.now.to_f}")[0, 12],
    task: task,
    success: success,
    details: opts[:details].to_s[0, OUTCOME_DETAILS_MAX],
    session_id: opts[:session_id],
    tags: Array(opts[:tags]).map(&:to_s),
    timestamp: Time.now.utc.iso8601
  }
  entry[:score] = opts[:score].to_f if opts.key?(:score)
  FileUtils.mkdir_p(File.dirname(LEARNING_FILE))
  File.open(LEARNING_FILE, 'a') { |f| f.puts(JSON.generate(entry)) }
  maybe_prune_outcomes!

  # M4 — default: outcomes live in learning.jsonl ONLY.
  # M4.1 — PROCESS SOPs (rubocop/rake/spec after code changes, etc.)
  # are promoted into PWN::Memory[:lesson] so PromptBuilder recall
  # survives across sessions. Without this, the agent re-learns
  # "run rubocop after every patch" every turn (empty memory.json).
  promote_process_lesson(entry: entry) if defined?(PWN::Memory)
  entry
end

.outcomes(opts = {}) ⇒ Object

Supported Method Parameters

rows = PWN::AI::Agent::Learning.outcomes( limit: 'optional - max entries returned newest-first (default 50)', success: 'optional - filter by Boolean outcome', tag: 'optional - filter by tag substring' )



132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
 
# File 'lib/pwn/ai/agent/learning.rb', line 132

public_class_method def self.outcomes(opts = {})
  limit   = opts[:limit] || 50
  want_ok = opts.key?(:success) ? !opts[:success].nil? && opts[:success] != false : nil
  tag     = opts[:tag].to_s.downcase
  return [] unless File.exist?(LEARNING_FILE)

  rows = File.readlines(LEARNING_FILE).map do |l|
    JSON.parse(l, symbolize_names: true)
  rescue StandardError
    nil
  end
  rows.compact!
  rows.select! { |r| want_ok == true ? r[:success] == true : r[:success] == want_ok } unless want_ok.nil?
  rows.select! { |r| Array(r[:tags]).any? { |t| t.to_s.downcase.include?(tag) } } unless tag.empty?
  rows.reverse.first(limit)
end

.prune_outcomes!(opts = {}) ⇒ Object

Supported Method Parameters

result = PWN::AI::Agent::Learning.prune_outcomes!( dry_run: 'optional - Boolean (default false)', max_rows: 'optional - hard cap (default MAX_OUTCOME_ROWS)', retain_days: 'optional - age floor for low-value drop', recent_days: 'optional - always keep newer than this' )

Keep gold RL rows (success+score>=0.6+session_id), recent window, high-value tags, and near-miss failures. Dedupe by task+success keeping best score. Truncate details. Never sacrifices exemplar pool.



1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
 
# File 'lib/pwn/ai/agent/learning.rb', line 1282

public_class_method def self.prune_outcomes!(opts = {})
  dry = opts[:dry_run] ? true : false
  max_rows = (opts[:max_rows] || MAX_OUTCOME_ROWS).to_i
  retain_days = (opts[:retain_days] || OUTCOME_RETAIN_DAYS).to_f
  recent_days = (opts[:recent_days] || OUTCOME_RECENT_DAYS).to_f
  details_max = (opts[:details_max] || OUTCOME_DETAILS_MAX).to_i
  gold_min = (opts[:gold_min_score] || GOLD_MIN_SCORE).to_f

  return { kept: 0, removed: 0, skipped: true } unless File.exist?(LEARNING_FILE)

  before_bytes = File.size(LEARNING_FILE)
  rows = File.readlines(LEARNING_FILE).map do |l|
    JSON.parse(l, symbolize_names: true)
  rescue StandardError
    nil
  end.compact

  now = Time.now.utc
  age_days = lambda do |r|
    (now - Time.parse(r[:timestamp].to_s)) / 86_400.0
  rescue StandardError
    999.0
  end

  protected_row = lambda do |r|
    tags = Array(r[:tags]).map(&:to_s)
    a = age_days.call(r)
    return true if a <= recent_days
    return true if r[:success] == true && r.key?(:score) && r[:score].to_f >= gold_min && r[:session_id].to_s != ''
    return true if r[:success] == true && !r.key?(:score) && r[:session_id].to_s != '' && a <= retain_days
    return true if tags.intersect?(HIGH_VALUE_TAGS)
    return true if r[:success] == false && r.key?(:score) && r[:score].to_f >= 0.5

    false
  end

  rows.reject! { |r| r[:task].to_s.strip.empty? }

  best = {}
  rows.each do |r|
    key = [r[:task].to_s.strip.downcase.gsub(/\s+/, ' ')[0, 160], r[:success].to_s]
    prev = best[key]
    if prev.nil?
      best[key] = r
    else
      ps = prev.key?(:score) ? prev[:score].to_f : -1.0
      rs = r.key?(:score) ? r[:score].to_f : -1.0
      better = rs > ps || (rs == ps && r[:timestamp].to_s > prev[:timestamp].to_s)
      better ||= protected_row.call(r) && !protected_row.call(prev)
      best[key] = r if better
    end
  end
  deduped = best.values
  removed_dupes = rows.size - deduped.size

  truncated = 0
  deduped.each do |r|
    d = r[:details].to_s
    next if d.bytesize <= details_max

    r[:details] = "#{d[0, details_max]}…[compacted]"
    truncated += 1
  end

  protected, unprotected = deduped.partition { |r| protected_row.call(r) }

  kept_unprot = unprotected.reject do |r|
    a = age_days.call(r)
    tags = Array(r[:tags]).map(&:to_s)
    score = r.key?(:score) ? r[:score].to_f : 1.0
    noise_tags = (tags - LOW_VALUE_ONLY_TAGS).empty? && tags.any?
    a > retain_days && noise_tags && score < 0.4
  end

  gold = (protected + kept_unprot).select do |r|
    r[:success] == true && r[:session_id].to_s != '' &&
      (!r.key?(:score) || r[:score].to_f >= gold_min)
  end
  if gold.size < EXEMPLARS_POOL_MIN
    need = EXEMPLARS_POOL_MIN - gold.size
    extra = unprotected.select { |r| r[:success] == true && r[:session_id].to_s != '' }
                       .sort_by { |r| r[:timestamp].to_s }
                       .last(need)
    kept_unprot = (kept_unprot + extra).uniq
  end

  fails = (protected + kept_unprot).reject { |r| r[:success] == true }
  if fails.size < FAILURE_WINDOW_MIN
    need = FAILURE_WINDOW_MIN - fails.size
    extra = unprotected.reject { |r| r[:success] == true }
                       .sort_by { |r| r[:timestamp].to_s }
                       .last(need)
    kept_unprot = (kept_unprot + extra).uniq
  end

  kept = (protected + kept_unprot).uniq
  if kept.size > max_rows
    prot_ids = protected.map { |r| r[:id] }.compact
    over = kept.size - max_rows
    victims = kept.reject { |r| prot_ids.include?(r[:id]) }
                  .sort_by { |r| r[:timestamp].to_s }
                  .first(over)
    v_ids = victims.map { |r| r[:id] }
    kept = kept.reject { |r| v_ids.include?(r[:id]) }
  end

  kept = kept.sort_by { |r| r[:timestamp].to_s }

  atomic_jsonl_write(path: LEARNING_FILE, rows: kept) unless dry

  {
    kept: kept.size,
    removed: (rows.size - kept.size) + removed_dupes,
    deduped: removed_dupes,
    truncated_details: truncated,
    protected: protected.size,
    bytes_before: before_bytes,
    bytes_after: if dry
                   before_bytes
                 else
                   (File.exist?(LEARNING_FILE) ? File.size(LEARNING_FILE) : 0)
                 end,
    dry_run: dry
  }
end

.purge_noise ⇒ Object

Supported Method Parameters

PWN::AI::Agent::Learning.purge_noise

One-shot GC of the pre-R1 garbage: drops every PWN::Memory entry matching the old SUCCESS: <req> — <final> / Avoid repeating failure pattern from <tool>: {"success":true shapes. Run once after upgrading; subsequent writes never produce these.



1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
 
# File 'lib/pwn/ai/agent/learning.rb', line 1521

public_class_method def self.purge_noise
  return { removed: 0 } unless defined?(PWN::Memory)

  mem = nil
  load_err = nil
  begin
    mem = PWN::Memory.load
  rescue StandardError => e
    load_err = e
  end
  if load_err
    warn "[pwn-ai/learning] purge_noise aborted (memory load failed): #{load_err.class}: #{load_err.message}"
    return { removed: 0, aborted: true, error: "#{load_err.class}: #{load_err.message}" }
  end
  before = mem.size
  mem.reject! do |_k, v|
    next false unless v[:category].to_s == 'lesson'

    val = v[:value].to_s
    val.start_with?('SUCCESS: ', 'FAILURE: ') ||
      val.match?(/\AAvoid repeating failure pattern from \w+: .{0,5}\{"success":true/)
  end
  PWN::Memory.save(mem: mem, force: mem.empty?)
  { removed: before - mem.size, remaining: mem.size }
end

.reconcile_verdict_tags!(opts = {}) ⇒ Object

One-shot / on-load repair: rewrite tags+details where verdict label disagrees with score (solved @ 0.3 etc.). Safe to call repeatedly.



939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
 
# File 'lib/pwn/ai/agent/learning.rb', line 939

public_class_method def self.reconcile_verdict_tags!(opts = {})
  return { repaired: 0 } unless File.exist?(LEARNING_FILE)

  dry = opts[:dry_run] ? true : false
  repaired = 0
  lines = File.readlines(LEARNING_FILE)
  out = lines.map do |l|
    r = JSON.parse(l, symbolize_names: true)
    score = r.key?(:score) ? r[:score].to_f : nil
    next l if score.nil?

    want = verdict_for_score(score: score).to_s
    tags = Array(r[:tags]).map(&:to_s)
    stale = tags & %w[solved partial wrong unknown]
    next l if stale.empty? || stale.include?(want)

    repaired += 1
    next l if dry

    cleaned = tags - %w[solved partial wrong unknown]
    cleaned << want
    r[:tags] = cleaned
    # Fix leading "solved(0.3)" style details head when present
    det = r[:details].to_s
    r[:details] = det.sub(
      /\A(solved|partial|wrong|unknown)\(\d+(?:\.\d+)?\)/i,
      "#{want}(#{format('%.2f', score)})"
    )
    r[:success] = (score >= 0.6) if [true, false].include?(r[:success])
    "#{JSON.generate(r)}\n"
  rescue StandardError
    l
  end
  File.write(LEARNING_FILE, out.join) if !dry && repaired.positive?
  { repaired: repaired, dry_run: dry }
rescue StandardError => e
  { repaired: 0, error: "#{e.class}: #{e.message}" }
end

.reflect(opts = {}) ⇒ Object

Supported Method Parameters

report = PWN::AI::Agent::Learning.reflect( session_id: 'required - PWN::Sessions id to analyse', dry_run: 'optional - when true, do not write to Memory/Skills (default false)' )

Uses PWN::AI::Agent::Reflect (when available) to LLM-summarise the session into structured lessons. Falls back to a heuristic extractor when module_reflection is disabled so learning never stops.



395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
 
# File 'lib/pwn/ai/agent/learning.rb', line 395

public_class_method def self.reflect(opts = {})
  session_id = opts[:session_id]
  dry_run    = opts[:dry_run] ? true : false
  raise 'ERROR: session_id is required' if session_id.to_s.empty?

  transcript = PWN::Sessions.load(session_id: session_id)
  return { session_id: session_id, lessons: [], reason: 'empty transcript' } if transcript.empty?

  lessons = introspective_lessons(transcript: transcript)
  source, conf = lessons.empty? ? [:heuristic, 0.3] : [:reflect, 0.8]
  lessons = heuristic_lessons(transcript: transcript) if lessons.empty?

  saved = []
  lessons.each do |l|
    next if l.to_s.strip.empty?

    key = :"reflect_#{session_id}_#{Digest::SHA256.hexdigest(l)[0, 8]}"
    # M3 — provenance + confidence + ttl so consolidate evicts
    # low-confidence heuristic lessons before hand-written ones.
    PWN::Memory.remember(key: key, value: l, category: :lesson, source: source, confidence: conf, importance: conf, ttl: source == :heuristic ? 7 * 86_400 : nil) unless dry_run
    saved << { key: key, lesson: l }
  end
  consolidate unless dry_run

  { session_id: session_id, lessons: saved, count: saved.length, dry_run: dry_run }
end

.reset ⇒ Object

Supported Method Parameters

PWN::AI::Agent::Learning.reset



776
777
778
779
 
# File 'lib/pwn/ai/agent/learning.rb', line 776

public_class_method def self.reset
  FileUtils.rm_f(LEARNING_FILE)
  { cleared: true }
end

.stats ⇒ Object

Supported Method Parameters

stats = PWN::AI::Agent::Learning.stats



152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
 
# File 'lib/pwn/ai/agent/learning.rb', line 152

public_class_method def self.stats
  rows   = outcomes(limit: 10_000)
  total  = rows.length
  ok     = rows.count { |r| r[:success] == true }
  jsum   = rows.sum { |r| r[:score] ? r[:score].to_f : { true => 1.0, false => 0.0 }[r[:success]] }
  skills = defined?(PWN::Skills) && PWN::Skills.is_a?(Hash) ? PWN::Skills.keys.length : 0
  mem    = defined?(PWN::Memory) ? PWN::Memory.load.keys.length : 0
  {
    total_outcomes: total,
    successes: ok,
    failures: total - ok,
    success_rate: total.positive? ? (ok.to_f / total).round(3) : 0.0,
    skills_known: skills,
    memory_entries: mem,
    judge_mean: total.positive? ? (jsum / total).round(3) : nil,
    reward_sentinel: (Reward.sentinel if defined?(Reward)),
    calibration: (Metrics.calibration if defined?(Metrics) && Metrics.respond_to?(:calibration)),
    preference_pairs: (Reward.preferences(limit: 100_000).length if defined?(Reward)),
    tool_metrics: (Metrics.summary(limit: 5) if defined?(Metrics)),
    extrospection: (Extrospection.stats if defined?(Extrospection))
  }
end

.to_context(opts = {}) ⇒ Object

Supported Method Parameters

ctx = PWN::AI::Agent::Learning.to_context( limit: 'optional - number of recent outcomes to surface (default 5)' )



180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
 
# File 'lib/pwn/ai/agent/learning.rb', line 180

public_class_method def self.to_context(opts = {})
  limit = opts[:limit] || 5
  # Fetch a wider window so prefer_primary_tasks can drop critic/red_team
  # envelope rows (REQUEST:/GOAL: prefixes) without starving the block.
  rows  = prefer_primary_tasks(rows: outcomes(limit: limit * 4)).first(limit)
  fails = prefer_primary_tasks(rows: outcomes(limit: 200, success: false))
  # Do not mirror the same ids under both headings — that doubled the
  # failure signal and made RECENT OUTCOMES == RECENT FAILURES when the
  # last N attempts all failed (the injected block looked "stuck").
  row_ids = rows.map { |r| r[:id] }.compact
  fails = fails.reject { |r| row_ids.include?(r[:id]) }.first(limit)
  return '' if rows.empty? && fails.empty?

  fmt = lambda do |r|
    flag = case r[:success]
           when true then '✓'
           when 'soft', :soft then '∼'
           else '✗'
           end
    score = r.key?(:score) ? format('%.2f', r[:score].to_f) : '-'
    task  = display_task(task: r[:task])
    line  = "  #{flag} [#{score}] #{task} (#{r[:timestamp]})"
    # Surface a one-line cause crumb so the agent can actually learn
    # from failures instead of only seeing that they failed.
    if r[:success] != true
      crumb = cause_crumb(details: r[:details])
      line += "\n      cause: #{crumb}" unless crumb.empty?
    end
    line
  end
  s   = stats
  jm  = s[:judge_mean]
  hdr = "RECENT OUTCOMES (success_rate=#{(s[:success_rate] * 100).round(1)}%#{" judge_mean=#{jm}" if jm} over #{s[:total_outcomes]} attempts)"
  out = "#{hdr}\n#{rows.map(&fmt).join("\n")}\n"
  out += "RECENT FAILURES (learn from these — do not repeat)\n#{fails.map(&fmt).join("\n")}\n" unless fails.empty?
  "#{out}\n"
end