Module: PWN::AI::Agent::Learning

Defined in:
lib/pwn/ai/agent/learning.rb

Overview

PWN::AI::Agent::Learning is the self-improvement engine that closes the pwn-ai feedback loop. It captures task outcomes, mines session transcripts for durable lessons, promotes successful workflows into reusable skills, and keeps ~/.pwn lean (memory + learning.jsonl + mistakes + sessions) so the agent gets sharper over time instead of accumulating noise.

Data flows:

Loop.run --(tool telemetry)--> Metrics.record
Loop.run --(final answer)----> Learning.auto_introspect (opt-in)
auto_introspect --(throttled)--> Learning.gc_stores!  # ~/.pwn lean
model    --(tool calls)------> learning_note_outcome / _distill_skill
PromptBuilder <----------------- Learning.to_context + Metrics.to_context

Everything is file-backed under ~/.pwn so it survives across REPL restarts and is shared by every future session.

Constant Summary collapse

LEARNING_FILE = 
File.join(Dir.home, '.pwn', 'learning.jsonl')
FINETUNE_DIR = 
File.join(Dir.home, '.pwn', 'finetune')
INTROSPECT_SOFT_MS =

P0 — post-answer introspect must not train "stop early" while spending the iteration budget after the final. Soft cap skips expensive stages (tool critic, PRM-LLM, reflect, extrospect); hard cap keeps only note_outcome + judge(heuristic) + sentinel.

2_500
INTROSPECT_HARD_MS = 
8_000
INTROSPECT_MIN_STAGES = 
%i[judge note_outcome fold_judge sentinel].freeze
MAX_MEMORY_ENTRIES = 
200
MAX_OUTCOME_ROWS =

Lean outcome retention — keep gold RL signal, drop bulk auto noise.

800
OUTCOME_RETAIN_DAYS = 
45
OUTCOME_RECENT_DAYS = 
14
OUTCOME_DETAILS_MAX = 
800
GOLD_MIN_SCORE = 
0.6
EXEMPLARS_POOL_MIN = 
200
FAILURE_WINDOW_MIN = 
200
HIGH_VALUE_TAGS = 
%w[
  needs_human extro_verify sdr gqrx rl pwn-ai curriculum hindsight her
].freeze
LOW_VALUE_ONLY_TAGS = 
%w[
  auto loop partial wrong solved offline_judge plan_cover_high
].freeze
PRUNE_EVERY_N_APPENDS = 
25
CLAIM_METRIC_WORDS =

E3/P26 — only CVE-ids or software-name + full semver (x.y.z). Two-part floats ("cap 0.2", "proxy 1.0", "judge 37.0") are RL metric crumbs that were scraped by verify_as_reward and flooded learning.jsonl with extro_verify :unknown failures.

%w[
  cap share proxy judge success only now clears gap score rate mean
  brier overconf distrust trajectory_fraction handler orm prm delta
  limit window pct percent ms iter budget conf confidence n
  ruby python linux kernel host cwd e.g e.g.
].freeze
CLAIM_RX = 
/
  CVE-\d{4}-\d{4,7}
  |
  \b
  (?!
    (?:cap|share|proxy|judge|success|only|now|clears|gap|score|rate|mean|
       brier|overconf|distrust|trajectory_fraction|handler|orm|prm|delta|
       limit|window|pct|percent|ms|iter|budget|conf|confidence|
       ruby|python|linux|kernel|host|cwd|e\.g)
    \b
  )
  [A-Za-z][\w.+-]{2,}
  \s+
  v?\d+\.\d+\.\d+(?:[-+][\w.]+)?
  \b
/x
SFT_MIN_SCORE =

P12 — SFT quality gate (as hard as DPO source-cap): drop HER/soft, low judge_score, auto-only noise without score, and PRM-compress trajectories so LoRA is not 5MB of "how we flailed".

0.6
SFT_MAX_TOOL_CHARS = 
1_200
PROCESS_SOP_RX =

M4.1 — keyword gate for "process / hygiene" SOPs the operator keeps re-requesting (rubocop, rake, rspec, conventions). These must become durable Memory lessons, not just learning.jsonl rows.

%r{\b(rubocop|rake\b|rspec|bundle\s+exec|conventions?|lint(?:ing)?|style/|code\s*hygiene|post[- ]?patch|after\s+(?:every\s+)?(?:patch|change|edit))\b}i
FAILURE_FINAL_RX = 
/\[pwn-ai\] (iteration budget exhausted|engine returned no message)|\b(i (was )?unable to|i could not|i couldn'?t|cannot proceed|failed to)\b/i

Class Method Summary collapse

Class Method Details

.authors ⇒ Object

Author(s)

0day Inc. support@0dayinc.com



1636
1637
1638
 
# File 'lib/pwn/ai/agent/learning.rb', line 1636

public_class_method def self.authors
  "AUTHOR(S):\n  0day Inc. <support@0dayinc.com>\n"
end

.auto_introspect(opts = {}) ⇒ Object

Supported Method Parameters

PWN::AI::Agent::Learning.auto_introspect( session_id: 'required - id of the just-completed session', request: 'optional - original user request (for outcome logging)', final: 'optional - final assistant answer (for outcome logging)' )

Called by Loop.run when PWN::Env[:agent][:auto_introspect] is truthy. Never raises — learning must not break the primary loop.



458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
 
# File 'lib/pwn/ai/agent/learning.rb', line 458

public_class_method def self.auto_introspect(opts = {})
  session_id = opts[:session_id]
  return unless session_id
  return unless auto_introspect_enabled?

  t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  stages_run = []
  stages_skipped = []
  budget_hot = begin
    defined?(Loop) && Loop.respond_to?(:budget_exhaustion_hot?, true) &&
      Loop.send(:budget_exhaustion_hot?)
  rescue StandardError
    false
  end

  elapsed_ms = lambda do
    ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0) * 1000).round
  end
  # soft = skip expensive; hard = stop almost everything
  over_soft = lambda do
    ms = elapsed_ms.call
    ms >= INTROSPECT_SOFT_MS || budget_hot
  end
  over_hard = lambda do
    ms = elapsed_ms.call
    ms >= INTROSPECT_HARD_MS
  end

  proxy_ok = infer_success(session_id: session_id, final: opts[:final])

  # S3 critic — BEFORE reward so verdict is evidence.
  # P24/P0 — budget_hot or soft-cap → text_only or skip.
  force_critic = begin
    eng = (PWN::Env.dig(:ai, :active) if defined?(PWN::Env))
    cal = defined?(Metrics) ? Metrics.calibration(engine: eng) : { n: 0 }
    cal[:n].to_i >= 8 && (cal[:brier].to_f > 0.35 || cal[:overconfidence].to_f > 0.25)
  rescue StandardError
    false
  end
  # P0 — when W1 mix urgently needs :critic pairs, prefer running critic
  need_critic_mix = begin
    mix = defined?(Reward) && Reward.respond_to?(:generator_mix) ? Reward.generator_mix : {}
    Array(mix[:urgent]).include?('critic')
  rescue StandardError
    false
  end

  crit = { verdict: :pass, source: :skipped }
  # Single skip path (Lint/DuplicateBranch): hard-budget OR no Curriculum.
  if !defined?(Curriculum) || (over_hard.call && !need_critic_mix)
    stages_skipped << :critic
  elsif budget_hot || (over_soft.call && !force_critic && !need_critic_mix)
    stages_run << :critic_text_only
    crit = Curriculum.critic(
      request: opts[:request],
      final: opts[:final],
      session_id: session_id,
      text_only: true
    )
  elsif force_critic
    stages_run << :critic_forced
    prev = (PWN::Env[:ai][:agent][:critic] if defined?(PWN::Env) && PWN::Env[:ai].is_a?(Hash) && PWN::Env[:ai][:agent].is_a?(Hash))
    begin
      PWN::Env[:ai][:agent][:critic] = true if defined?(PWN::Env) && PWN::Env[:ai].is_a?(Hash) && PWN::Env[:ai][:agent].is_a?(Hash) && !PWN::Env[:ai][:agent].frozen?
      crit = Curriculum.critic(request: opts[:request], final: opts[:final], session_id: session_id)
    ensure
      PWN::Env[:ai][:agent][:critic] = prev if defined?(PWN::Env) && PWN::Env[:ai].is_a?(Hash) && PWN::Env[:ai][:agent].is_a?(Hash) && !PWN::Env[:ai][:agent].frozen?
    end
  else
    stages_run << :critic
    crit = Curriculum.critic(request: opts[:request], final: opts[:final], session_id: session_id)
  end

  # R1 judge — always attempt (heuristic is cheap; LLM gated inside)
  stages_run << :judge
  v = Reward.judge(request: opts[:request], final: opts[:final], session_id: session_id, proxy_ok: proxy_ok, predicted: opts[:predicted], plan: opts[:plan]) if defined?(Reward)
  v ||= { score: proxy_ok ? 1.0 : 0.0, success: proxy_ok, verdict: proxy_ok ? :solved : :wrong }
  v[:score] = [v[:score], 0.3].min if crit[:verdict] == :flaw
  # P29 — critic floor used to leave stale verdict=:solved at score=0.3,
  # producing learning.jsonl rows tagged "solved" with success=false
  # (116+ rows). Always resync verdict/success from the final score.
  v[:verdict] = verdict_for_score(score: v[:score])
  v[:success] = v[:score].to_f >= 0.6
  ok = v[:success]

  # W1 pending user_correction pair
  pend = Thread.current[:pwn_pending_pref]
  if pend && ok && defined?(Reward)
    stages_run << :user_correction_pref
    Reward.record_preference(
      prompt: pend[:prompt],
      rejected: pend[:rejected],
      chosen: opts[:final].to_s,
      source: :user_correction,
      shape: :revised_answer,
      force: true
    )
    Thread.current[:pwn_pending_pref] = nil
  end

  # Soft plan-quality feature (W3) — tag only; not full DPO.
  plan_cov = nil
  if defined?(Reward) && Reward.respond_to?(:plan_coverage)
    begin
      plan_for_cov = opts[:plan]
      plan_for_cov = opts[:ts_state][:plan] if plan_for_cov.nil? && opts[:ts_state].is_a?(Hash)
      if plan_for_cov.nil? && defined?(TaskSummarizer)
        # Recover numbered tasks from the final/request only when caller
        # did not pass a plan — still keeps TaskSummarizer out of the
        # credit path (parse is pure text).
        plan_for_cov = nil
      end
      if !plan_for_cov.nil? || opts[:final].to_s.length.positive?
        plan_cov = Reward.plan_coverage(
          plan: plan_for_cov || [],
          final: opts[:final],
          request: opts[:request],
          session_id: session_id
        )
        stages_run << :plan_coverage if plan_cov && plan_cov[:total].to_i.positive?
      end
    rescue StandardError => e
      warn "[pwn-ai/learning] plan_coverage swallowed: #{e.class}: #{e.message}"
    end
  end

  stages_run << :note_outcome
  outcome_tags = ['auto', 'loop', v[:verdict].to_s]
  outcome_tags << plan_cov[:tag] if plan_cov && plan_cov[:tag]
  outcome_tags << "plan_cover=#{plan_cov[:score]}" if plan_cov && plan_cov[:total].to_i.positive?
  # P29 — persist the bare user ask (strip REQUEST:/GOAL: envelopes at write time)
  task_txt = display_task(task: opts[:request].to_s)
  task_txt = opts[:request].to_s[0, 100] if task_txt.empty?
  note_outcome(
    task: task_txt,
    success: ok,
    score: v[:score],
    details: "#{v[:verdict]}(#{v[:score].to_f.round(2)}) #{v[:rationale]} | #{opts[:final].to_s[0, 200]}",
    session_id: session_id,
    tags: outcome_tags,
    judge_source: v[:source]
  )

  stages_run << :fold_judge
  fold_judge_into_metrics(session_id: session_id, score: v[:score], confidence: v[:confidence])
  # R5 — close the live MDP episode with the ORM terminal reward.
  if defined?(PWN::AI::Agent::Policy) && Policy.respond_to?(:finish)
    stages_run << :policy
    Policy.finish(
      session_id: session_id,
      score: v[:score],
      verdict: v[:verdict],
      proxy_ok: ok,
      final: opts[:final],
      ts_state: opts[:ts_state]
    )
  end

  # R2 PRM — skip under hard cap (expensive LLM); keep under soft if heuristic path
  if over_hard.call
    stages_skipped << :prm
  elsif defined?(Reward)
    stages_run << :prm
    Reward.prm(request: opts[:request], session_id: session_id)
  end

  # C3 HER — only on failure; skip hard
  if !ok && defined?(Curriculum) && !over_hard.call
    stages_run << :hindsight
    Curriculum.hindsight(request: opts[:request], final: opts[:final], session_id: session_id)
  else
    stages_skipped << :hindsight unless ok
  end

  # W3 calibrate — cheap; always write so Metrics.calibration is never empty.
  predicted = opts[:predicted]
  predicted = recover_predicted_from_session(session_id: session_id) if predicted.nil?
  stages_run << :calibrate if defined?(Curriculum)

  # reflect on success — skip soft/hard (LLM + memory writes)
  # M4.1 — also reflect when the request/final is a process SOP
  # (code hygiene) even if judge score < 0.6, so rubocop/rake
  # lessons still land in PWN::Memory.
  process_sop = process_sop_text?(text: "#{opts[:request]} #{opts[:final]}")
  if (ok || process_sop) && !over_soft.call
    stages_run << :reflect
    reflect(session_id: session_id)
  elsif ok || process_sop
    stages_skipped << :reflect
    # Cheap path under soft budget: still promote a canned process lesson
    if process_sop && defined?(PWN::Memory)
      promote_process_lesson(
        entry: {
          task: opts[:request].to_s[0, 120],
          success: ok,
          score: v[:score],
          details: opts[:final].to_s[0, 500],
          tags: %w[auto process_sop]
        }
      )
    end
  end

  # R3 sentinel — cheap disk math; always
  if defined?(Reward)
    stages_run << :sentinel
    Reward.sentinel
  end

  # E ambient extrospect — skip soft (can launch probes)
  if defined?(Extrospection) && !over_soft.call
    stages_run << :extrospect
    Extrospection.auto_extrospect(session_id: session_id)
  else
    stages_skipped << :extrospect
  end

  # Keep ~/.pwn RL stores lean on the feedback path (memory +
  # learning.jsonl + mistakes + sessions). Throttled; never raises.
  # Disk-only work: skip only hard budget / budget_hot.
  begin
    if !over_hard.call && !budget_hot && should_gc_stores?
      stages_run << :lean_gc
      gc_stores!(current_session_id: session_id)
    elsif should_gc_stores?
      stages_skipped << :lean_gc
    end
  rescue StandardError => e
    warn "[pwn-ai/learning] post-introspect lean swallowed: #{e.class}: #{e.message}"
  end

  {
    ok: ok,
    score: v[:score],
    elapsed_ms: elapsed_ms.call,
    budget_hot: budget_hot,
    stages_run: stages_run,
    stages_skipped: stages_skipped
  }
rescue StandardError => e
  warn "[pwn-ai/learning] auto_introspect swallowed: #{e.class}: #{e.message}"
  nil
end

.consolidate(opts = {}) ⇒ Object

Supported Method Parameters

removed = PWN::AI::Agent::Learning.consolidate( max_entries: 'optional - hard cap on PWN::Memory size (default MAX_MEMORY_ENTRIES)' )

Deduplicates near-identical lesson values and prunes the oldest entries once the cap is exceeded so the injected MEMORY block stays high-signal.



734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
 
# File 'lib/pwn/ai/agent/learning.rb', line 734

public_class_method def self.consolidate(opts = {})
  cap = opts[:max_entries] || MAX_MEMORY_ENTRIES
  return { removed: 0 } unless defined?(PWN::Memory)

  mem = nil
  load_err = nil
  begin
    mem = PWN::Memory.load
  rescue StandardError => e
    load_err = e
  end
  if load_err
    warn "[pwn-ai/learning] consolidate aborted (memory load failed): #{load_err.class}: #{load_err.message}"
    return { removed: 0, aborted: true, error: "#{load_err.class}: #{load_err.message}" }
  end
  removed = []

  # M1 — semantic clustering: embed :lesson entries, greedy-merge
  # near-duplicates (cosine ≥ 0.92) via Reflect into ONE imperative
  # lesson. Falls back to sha-dedup when no embed backend.
  removed.concat(semantic_merge(mem: mem)) if defined?(PWN::MemoryIndex) && PWN::MemoryIndex.available?

  seen = {}
  mem.each do |k, v|
    sig = Digest::SHA256.hexdigest(v[:value].to_s.strip.downcase)[0, 16]
    seen[sig] ? removed << k : seen[sig] = k
  end
  removed.uniq.each { |k| mem.delete(k) }

  # M3 — evict by (age/ttl) / (importance × confidence), NOT
  # oldest-first. Hand-written high-value lessons survive; low-
  # confidence :heuristic auto-gen self-evicts first.
  if mem.size > cap
    now = Time.now.utc
    scored = mem.map do |k, v|
      if defined?(PWN::Memory) && PWN::Memory.respond_to?(:protected_entry?) &&
         PWN::Memory.protected_entry?(key: k, entry: v)
        next [k, Float::INFINITY]
      end

      age_d = (now - Time.parse(v[:timestamp].to_s)) / 86_400.0
      ttl_d = (v[:ttl].to_f / 86_400.0)
      imp   = (v[:importance] || 0.5).to_f.clamp(0.05, 1.0)
      conf  = (v[:confidence] || (v[:source].to_s == 'human' ? 0.95 : 0.5)).to_f.clamp(0.05, 1.0)
      staleness = ttl_d.positive? ? age_d / ttl_d : age_d / 90.0
      # lower score = drop first; Infinity protected sorts last
      [k, -(staleness / (imp * conf))]
    rescue StandardError
      [k, 0.0]
    end
    # sort ascending by score so lowest (most stale/low-imp) first
    ordered = scored.sort_by { |_k, s| s }
    drop = []
    ordered.each do |pair|
      k = pair[0]
      break if mem.size - drop.size <= cap
      next if defined?(PWN::Memory) && PWN::Memory.respond_to?(:protected_entry?) &&
              PWN::Memory.protected_entry?(key: k, entry: mem[k])

      drop << k
    end
    drop.each { |k| mem.delete(k) }
    removed.concat(drop)
  end
  PWN::Memory.save(mem: mem, force: mem.empty?)
  if PWN::Memory.respond_to?(:lean!)
    begin
      PWN::Memory.lean!
    rescue StandardError => e
      warn "[pwn-ai/learning] post-consolidate memory.lean! swallowed: #{e.class}: #{e.message}"
    end
  end
  { removed: removed.uniq.length, remaining: mem.size }
end

.distill_skill(opts = {}) ⇒ Object

Supported Method Parameters

skill = PWN::AI::Agent::Learning.distill_skill( name: 'required - snake_case name for the new skill', session_id: 'optional - PWN::Sessions id to mine (uses its transcript)', content: 'optional - explicit markdown body; overrides transcript mining', references: 'optional - Array of reference URLs / CWE / CVE / ATT&CK ids' )



391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
 
# File 'lib/pwn/ai/agent/learning.rb', line 391

public_class_method def self.distill_skill(opts = {})
  raise 'ERROR: name is required' if opts[:name].to_s.strip.empty?

  body = opts[:content].to_s
  body = build_skill_from_session(session_id: opts[:session_id], name: opts[:name]) if body.strip.empty? && opts[:session_id]
  raise 'ERROR: content or session_id is required' if body.strip.empty?

  root = skills_dir
  out  = PWN::Config.write_skill(
    name: opts[:name],
    description: opts[:description],
    content: body,
    references: opts[:references],
    pwn_skills_path: root
  )
  PWN::Config.load_skills(pwn_skills_path: root) if PWN::Config.respond_to?(:load_skills)
  note_outcome(task: "distill_skill:#{out[:name]}", success: true, details: "Saved #{out[:path]}", tags: %w[skill auto])
  out.merge(saved: true)
end

.exemplars_for(opts = {}) ⇒ Object

Supported Method Parameters

msgs = PWN::AI::Agent::Learning.exemplars_for( request: 'required - current user request', limit: 'optional - max exemplar traces to return (default 1)', max_msgs: 'optional - cap on messages per exemplar (default 6)' )

Retrieval-augmented BEHAVIOUR: keyword-matches request against prior successful outcomes in learning.jsonl, loads the matching session, and compresses its (user, tool, assistant) trace into a short few-shot exemplar Loop.run splices between system and user. Local models are dramatically better with 1 concrete example than with 25 abstract lessons.



258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
 
# File 'lib/pwn/ai/agent/learning.rb', line 258

public_class_method def self.exemplars_for(opts = {})
  request  = opts[:request].to_s.downcase
  limit    = (opts[:limit]    || 1).to_i
  max_msgs = (opts[:max_msgs] || 6).to_i
  return [] if request.strip.empty?

  tokens = request.scan(/[a-z0-9_]{3,}/).uniq
  return [] if tokens.empty?

  now = Time.now.utc
  # C2 — prioritized replay: priority = judge_score × recency_decay × keyword_sim
  # C2 — strict success:true only (excludes HER success:'soft'). Also
  # down-weight any residual hindsight-tagged rows so partial failures
  # never launder into full-strength few-shot exemplars.
  # P20 — strict success:true AND prefer high judge scores. Drop rows
  # with explicit low ORM score so proxy-true / judge-low cannot be few-shot.
  pool = outcomes(limit: 500, success: true).reject { |r| r[:session_id].to_s.empty? }
  pool = pool.reject { |r| r.key?(:score) && r[:score].to_f < 0.6 }
  scored = pool.map do |r|
    sim   = tokens.count { |t| r[:task].to_s.downcase.include?(t) }.to_f / tokens.length
    age_d = (now - Time.parse(r[:timestamp].to_s)) / 86_400.0
    decay = Math.exp(-age_d / 30.0)
    score = (r[:score] || 1.0).to_f
    tags  = Array(r[:tags]).map(&:to_s)
    # HER / soft / hindsight → 0.35× so they cannot dominate C2 priority
    score *= 0.35 if r[:success].to_s == 'soft' || tags.intersect?(%w[hindsight her soft])
    [r, sim * decay * score]
  rescue StandardError
    [r, 0.0]
  end
  hits = scored.reject { |_, pr| pr <= 0.0 }.sort_by { |_, pr| -pr }.first(limit).map(&:first)

  hits.flat_map { |r| compress_exemplar(session_id: r[:session_id], max_msgs: max_msgs) }
rescue StandardError
  []
end

.export_finetune(opts = {}) ⇒ Object 



315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
 
# File 'lib/pwn/ai/agent/learning.rb', line 315

public_class_method def self.export_finetune(opts = {})
  fmt       = (opts[:format] || :sharegpt).to_sym
  min_tools = (opts[:min_tools] || 1).to_i
  min_score = (opts[:min_score] || SFT_MIN_SCORE).to_f
  compress  = opts.key?(:compress) ? opts[:compress] : true
  FileUtils.mkdir_p(FINETUNE_DIR)
  out = opts[:out] || File.join(FINETUNE_DIR, "pwn-#{Time.now.utc.strftime('%Y%m%d')}.jsonl")

  # 4.1 / P12 — exclude HER soft-success + low-score + untagged auto flail
  gold = outcomes(limit: 10_000, success: true).reject do |r|
    tags = Array(r[:tags]).map(&:to_s)
    soft = r[:success].to_s == 'soft' || tags.intersect?(%w[hindsight her soft])
    low  = !r[:score].nil? && r[:score].to_f < min_score
    # require a score when present in corpus era that has scores
    soft || low
  end
  # prefer highest-score outcome per session
  by_sid = {}
  gold.each do |r|
    sid = r[:session_id].to_s
    next if sid.empty?

    prev = by_sid[sid]
    by_sid[sid] = r if prev.nil? || r[:score].to_f >= prev[:score].to_f
  end
  sids = by_sid.keys
  rows = 0
  dropped = { tools: 0, empty: 0, load: 0 }
  File.open(out, 'w') do |f|
    sids.each do |sid|
      t = begin
        PWN::Sessions.load(session_id: sid)
      rescue StandardError
        dropped[:load] += 1
        next
      end
      tool_n = t.count { |e| e[:role].to_s == 'tool' }
      if tool_n < min_tools
        dropped[:tools] += 1
        next
      end

      conv = if compress
               compress_finetune_trace(transcript: t, max_tool_chars: SFT_MAX_TOOL_CHARS)
             else
               t.map { |e| { role: e[:role].to_s, content: e[:content].to_s } }
                .reject { |e| e[:role] == 'system' && e[:content].start_with?('Session started') }
             end
      if conv.nil? || conv.empty? || conv.none? { |m| m[:role].to_s == 'assistant' }
        dropped[:empty] += 1
        next
      end

      line = case fmt
             when :openai_jsonl then { messages: conv }
             else { conversations: conv.map { |m| { from: sharegpt_role(role: m[:role]), value: m[:content] } } }
             end
      f.puts(JSON.generate(line))
      rows += 1
    end
  end
  {
    path: out, format: fmt, sessions: sids.length, samples: rows,
    bytes: File.size(out), min_score: min_score, compressed: compress,
    dropped: dropped
  }
end

.flip_last_outcome(opts = {}) ⇒ Object 



702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
 
# File 'lib/pwn/ai/agent/learning.rb', line 702

public_class_method def self.flip_last_outcome(opts = {})
  return { flipped: false } unless File.exist?(LEARNING_FILE)

  lines = File.readlines(LEARNING_FILE)
  return { flipped: false } if lines.empty?

  last = JSON.parse(lines.last, symbolize_names: true)
  return { flipped: false } if opts[:session_id] && last[:session_id] && last[:session_id] != opts[:session_id]
  return { flipped: false } unless last[:success]

  last[:success]    = false
  last[:flipped_by] = 'user_correction'
  last[:details]    = "#{last[:details]} | CORRECTED: #{opts[:reason].to_s[0, 200]}".strip
  last[:score]      = 0.0
  lines[-1] = "#{JSON.generate(last)}\n"
  File.write(LEARNING_FILE, lines.join)
  # W1 — the (rejected_prev_answer, chosen_next_answer) pair is
  # captured by Mistakes.check_user_correction which has both.
  { flipped: true, id: last[:id], rejected: last[:details].to_s[0, 2_000] }
rescue StandardError
  { flipped: false }
end

.gc_stores!(opts = {}) ⇒ Object

One-shot lean across memory + learning + mistakes + sessions. Called from auto_introspect (throttled) so the RL feedback loop keeps ~/.pwn high-signal without a manual learning_gc_stores turn.

Supported Method Parameters

result = PWN::AI::Agent::Learning.gc_stores!( dry_run: 'optional - Boolean (default false)', current_session_id: 'optional - never delete this sessions id', max_entries: 'optional - Memory consolidate cap', max_rows: 'optional - learning.jsonl cap', retain_days: 'optional - outcome / session age floor' )



1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
 
# File 'lib/pwn/ai/agent/learning.rb', line 1522

public_class_method def self.gc_stores!(opts = {})
  dry = opts[:dry_run] ? true : false
  res = lean!(
    dry_run: dry,
    max_entries: opts[:max_entries],
    max_rows: opts[:max_rows],
    retain_days: opts[:retain_days],
    recent_days: opts[:recent_days],
    details_max: opts[:details_max],
    gold_min_score: opts[:gold_min_score]
  )
  res[:mistakes] = if defined?(Mistakes) && Mistakes.respond_to?(:lean!)
                     Mistakes.lean!(dry_run: dry)
                   else
                     { skipped: true }
                   end
  res[:policy] = if defined?(PWN::AI::Agent::Policy) && Policy.respond_to?(:lean!)
                   Policy.lean!(dry_run: dry)
                 else
                   { skipped: true }
                 end
  res[:sessions] = if defined?(PWN::Sessions) && PWN::Sessions.respond_to?(:lean!)
                     sess_opts = { dry_run: dry }
                     sid = opts[:current_session_id].to_s
                     sess_opts[:current_session_id] = sid unless sid.empty?
                     sess_opts[:retain_days] = opts[:retain_days] if opts.key?(:retain_days)
                     sess_opts[:max_files] = opts[:max_files] if opts.key?(:max_files)
                     PWN::Sessions.lean!(**sess_opts)
                   else
                     { skipped: true }
                   end
  res
end

.help ⇒ Object

Display Usage for this Module



1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
 
# File 'lib/pwn/ai/agent/learning.rb', line 1642

public_class_method def self.help
  puts <<~USAGE
    USAGE:
      PWN::AI::Agent::Learning.note_outcome(task: 'nmap sweep 10.0.0.0/24', success: true, details: '12 hosts up')
      PWN::AI::Agent::Learning.outcomes(limit: 20, success: false)
      PWN::AI::Agent::Learning.reflect(session_id: sid)              # LLM or heuristic → PWN::Memory
      PWN::AI::Agent::Learning.auto_introspect(session_id: sid, request: req, final: text)
      PWN::AI::Agent::Learning.distill_skill(name: 'quick_recon', session_id: sid)
      PWN::AI::Agent::Learning.exemplars_for(request: 'nmap sweep 10/8')  # few-shot for Loop.run
      PWN::AI::Agent::Learning.export_finetune(format: :sharegpt)        # -> ~/.pwn/finetune/*.jsonl
      PWN::AI::Agent::Learning.consolidate(max_entries: 200)         # M1 semantic-merge + M3 importance-evict
      PWN::AI::Agent::Learning.lean!                                 # memory + learning.jsonl prune
      PWN::AI::Agent::Learning.gc_stores!                            # full ~/.pwn RL lean (mem/learn/mistakes/sessions)
      PWN::AI::Agent::Learning.prune_outcomes!                       # learning.jsonl gold-keep cap
      PWN::AI::Agent::Learning.purge_noise                            # one-shot GC of pre-R1 garbage lessons
      PWN::AI::Agent::Learning.to_context(limit: 5)                  # injected by PromptBuilder
      PWN::AI::Agent::Learning.stats
      PWN::AI::Agent::Learning.reset

      Enable end-of-run auto-learning with:
        PWN::Env[:ai][:agent][:auto_introspect] = true
      # auto_introspect throttles gc_stores! every PRUNE_EVERY_N_APPENDS
      # outcomes so ~/.pwn stays lean without a manual GC turn.

      #{self}.authors
  USAGE
end

.lean!(opts = {}) ⇒ Object

Memory lean + outcome prune.



1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
 
# File 'lib/pwn/ai/agent/learning.rb', line 1491

public_class_method def self.lean!(opts = {})
  dry = opts[:dry_run] ? true : false
  out = { dry_run: dry }
  out[:memory] = if defined?(PWN::Memory) && PWN::Memory.respond_to?(:lean!)
                   PWN::Memory.lean!(dry_run: dry)
                 else
                   { skipped: true }
                 end
  out[:memory_consolidate] = consolidate(max_entries: opts[:max_entries] || MAX_MEMORY_ENTRIES) unless dry
  out[:learning] = prune_outcomes!(
    dry_run: dry,
    max_rows: opts[:max_rows],
    retain_days: opts[:retain_days],
    recent_days: opts[:recent_days],
    details_max: opts[:details_max],
    gold_min_score: opts[:gold_min_score]
  )
  out
end

.note_outcome(opts = {}) ⇒ Object

Supported Method Parameters

entry = PWN::AI::Agent::Learning.note_outcome( task: 'required - short description of what was attempted', success: 'required - Boolean, did the attempt achieve its goal', details: 'optional - free-form notes / error / evidence', session_id: 'optional - PWN::Sessions id this outcome belongs to', tags: 'optional - Array of String labels for later retrieval' )



91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
 
# File 'lib/pwn/ai/agent/learning.rb', line 91

public_class_method def self.note_outcome(opts = {})
  task    = opts[:task].to_s
  # 4.1 — allow success: 'soft' (HER) distinct from true/false
  raw_ok  = opts[:success]
  success = if ['soft', :soft].include?(raw_ok)
              'soft'
            else
              raw_ok ? true : false
            end
  raise 'ERROR: task is required' if task.strip.empty?

  tags = Array(opts[:tags]).map(&:to_s)
  details = opts[:details].to_s[0, OUTCOME_DETAILS_MAX]
  # Refuse solved-at-fail (and the reverse): when a score is present,
  # tags / details / boolean success must match verdict_for_score.
  # Soft HER stays soft so C3 positives are not flipped to true/false.
  if opts.key?(:score)
    score = opts[:score].to_f
    want = verdict_for_score(score: score).to_s
    tags = (tags - %w[solved partial wrong unknown]) << want
    details = details.sub(
      /\A(solved|partial|wrong|unknown)\(\d+(?:\.\d+)?\)/i,
      "#{want}(#{format('%.2f', score)})"
    )
    success = (score >= 0.6) if [true, false].include?(success)
  end

  entry = {
    id: Digest::SHA256.hexdigest("#{task}-#{Time.now.to_f}")[0, 12],
    task: task,
    success: success,
    details: details,
    session_id: opts[:session_id],
    tags: tags,
    timestamp: Time.now.utc.iso8601
  }
  entry[:score] = opts[:score].to_f if opts.key?(:score)
  src = opts[:judge_source].to_s
  entry[:judge_source] = src unless src.empty?
  FileUtils.mkdir_p(File.dirname(LEARNING_FILE))
  File.open(LEARNING_FILE, 'a') { |f| f.puts(JSON.generate(entry)) }
  maybe_prune_outcomes!

  # M4 — default: outcomes live in learning.jsonl ONLY.
  # M4.1 — PROCESS SOPs (rubocop/rake/spec after code changes, etc.)
  # are promoted into PWN::Memory[:lesson] so PromptBuilder recall
  # survives across sessions. Without this, the agent re-learns
  # "run rubocop after every patch" every turn (empty memory.json).
  promote_process_lesson(entry: entry) if defined?(PWN::Memory)
  entry
end

.outcomes(opts = {}) ⇒ Object

Supported Method Parameters

rows = PWN::AI::Agent::Learning.outcomes( limit: 'optional - max entries returned newest-first (default 50)', success: 'optional - filter by Boolean outcome', tag: 'optional - filter by tag substring' )



150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
 
# File 'lib/pwn/ai/agent/learning.rb', line 150

public_class_method def self.outcomes(opts = {})
  limit   = opts[:limit] || 50
  want_ok = opts.key?(:success) ? !opts[:success].nil? && opts[:success] != false : nil
  tag     = opts[:tag].to_s.downcase
  return [] unless File.exist?(LEARNING_FILE)

  rows = File.readlines(LEARNING_FILE).map do |l|
    JSON.parse(l, symbolize_names: true)
  rescue StandardError
    nil
  end
  rows.compact!
  rows.select! { |r| want_ok == true ? r[:success] == true : r[:success] == want_ok } unless want_ok.nil?
  rows.select! { |r| Array(r[:tags]).any? { |t| t.to_s.downcase.include?(tag) } } unless tag.empty?
  rows.reverse.first(limit)
end

.prune_outcomes!(opts = {}) ⇒ Object

Supported Method Parameters

result = PWN::AI::Agent::Learning.prune_outcomes!( dry_run: 'optional - Boolean (default false)', max_rows: 'optional - hard cap (default MAX_OUTCOME_ROWS)', retain_days: 'optional - age floor for low-value drop', recent_days: 'optional - always keep newer than this' )

Keep gold RL rows (success+score>=0.6+session_id), recent window, high-value tags, and near-miss failures. Dedupe by task+success keeping best score. Truncate details. Never sacrifices exemplar pool.



1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
 
# File 'lib/pwn/ai/agent/learning.rb', line 1364

public_class_method def self.prune_outcomes!(opts = {})
  dry = opts[:dry_run] ? true : false
  max_rows = (opts[:max_rows] || MAX_OUTCOME_ROWS).to_i
  retain_days = (opts[:retain_days] || OUTCOME_RETAIN_DAYS).to_f
  recent_days = (opts[:recent_days] || OUTCOME_RECENT_DAYS).to_f
  details_max = (opts[:details_max] || OUTCOME_DETAILS_MAX).to_i
  gold_min = (opts[:gold_min_score] || GOLD_MIN_SCORE).to_f

  return { kept: 0, removed: 0, skipped: true } unless File.exist?(LEARNING_FILE)

  before_bytes = File.size(LEARNING_FILE)
  rows = File.readlines(LEARNING_FILE).map do |l|
    JSON.parse(l, symbolize_names: true)
  rescue StandardError
    nil
  end.compact

  now = Time.now.utc
  age_days = lambda do |r|
    (now - Time.parse(r[:timestamp].to_s)) / 86_400.0
  rescue StandardError
    999.0
  end

  protected_row = lambda do |r|
    tags = Array(r[:tags]).map(&:to_s)
    a = age_days.call(r)
    return true if a <= recent_days
    return true if r[:success] == true && r.key?(:score) && r[:score].to_f >= gold_min && r[:session_id].to_s != ''
    return true if r[:success] == true && !r.key?(:score) && r[:session_id].to_s != '' && a <= retain_days
    return true if tags.intersect?(HIGH_VALUE_TAGS)
    return true if r[:success] == false && r.key?(:score) && r[:score].to_f >= 0.5

    false
  end

  rows.reject! { |r| r[:task].to_s.strip.empty? }

  best = {}
  rows.each do |r|
    key = [r[:task].to_s.strip.downcase.gsub(/\s+/, ' ')[0, 160], r[:success].to_s]
    prev = best[key]
    if prev.nil?
      best[key] = r
    else
      ps = prev.key?(:score) ? prev[:score].to_f : -1.0
      rs = r.key?(:score) ? r[:score].to_f : -1.0
      better = rs > ps || (rs == ps && r[:timestamp].to_s > prev[:timestamp].to_s)
      better ||= protected_row.call(r) && !protected_row.call(prev)
      best[key] = r if better
    end
  end
  deduped = best.values
  removed_dupes = rows.size - deduped.size

  truncated = 0
  deduped.each do |r|
    d = r[:details].to_s
    next if d.bytesize <= details_max

    r[:details] = "#{d[0, details_max]}…[compacted]"
    truncated += 1
  end

  protected, unprotected = deduped.partition { |r| protected_row.call(r) }

  kept_unprot = unprotected.reject do |r|
    a = age_days.call(r)
    tags = Array(r[:tags]).map(&:to_s)
    score = r.key?(:score) ? r[:score].to_f : 1.0
    noise_tags = (tags - LOW_VALUE_ONLY_TAGS).empty? && tags.any?
    a > retain_days && noise_tags && score < 0.4
  end

  gold = (protected + kept_unprot).select do |r|
    r[:success] == true && r[:session_id].to_s != '' &&
      (!r.key?(:score) || r[:score].to_f >= gold_min)
  end
  if gold.size < EXEMPLARS_POOL_MIN
    need = EXEMPLARS_POOL_MIN - gold.size
    extra = unprotected.select { |r| r[:success] == true && r[:session_id].to_s != '' }
                       .sort_by { |r| r[:timestamp].to_s }
                       .last(need)
    kept_unprot = (kept_unprot + extra).uniq
  end

  fails = (protected + kept_unprot).reject { |r| r[:success] == true }
  if fails.size < FAILURE_WINDOW_MIN
    need = FAILURE_WINDOW_MIN - fails.size
    extra = unprotected.reject { |r| r[:success] == true }
                       .sort_by { |r| r[:timestamp].to_s }
                       .last(need)
    kept_unprot = (kept_unprot + extra).uniq
  end

  kept = (protected + kept_unprot).uniq
  if kept.size > max_rows
    prot_ids = protected.map { |r| r[:id] }.compact
    over = kept.size - max_rows
    victims = kept.reject { |r| prot_ids.include?(r[:id]) }
                  .sort_by { |r| r[:timestamp].to_s }
                  .first(over)
    v_ids = victims.map { |r| r[:id] }
    kept = kept.reject { |r| v_ids.include?(r[:id]) }
  end

  kept = kept.sort_by { |r| r[:timestamp].to_s }

  atomic_jsonl_write(path: LEARNING_FILE, rows: kept) unless dry

  {
    kept: kept.size,
    removed: (rows.size - kept.size) + removed_dupes,
    deduped: removed_dupes,
    truncated_details: truncated,
    protected: protected.size,
    bytes_before: before_bytes,
    bytes_after: if dry
                   before_bytes
                 else
                   (File.exist?(LEARNING_FILE) ? File.size(LEARNING_FILE) : 0)
                 end,
    dry_run: dry
  }
end

.purge_noise ⇒ Object

Supported Method Parameters

PWN::AI::Agent::Learning.purge_noise

One-shot GC of the pre-R1 garbage: drops every PWN::Memory entry matching the old SUCCESS: <req> — <final> / Avoid repeating failure pattern from <tool>: {"success":true shapes. Run once after upgrading; subsequent writes never produce these.



1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
 
# File 'lib/pwn/ai/agent/learning.rb', line 1608

public_class_method def self.purge_noise
  return { removed: 0 } unless defined?(PWN::Memory)

  mem = nil
  load_err = nil
  begin
    mem = PWN::Memory.load
  rescue StandardError => e
    load_err = e
  end
  if load_err
    warn "[pwn-ai/learning] purge_noise aborted (memory load failed): #{load_err.class}: #{load_err.message}"
    return { removed: 0, aborted: true, error: "#{load_err.class}: #{load_err.message}" }
  end
  before = mem.size
  mem.reject! do |_k, v|
    next false unless v[:category].to_s == 'lesson'

    val = v[:value].to_s
    val.start_with?('SUCCESS: ', 'FAILURE: ') ||
      val.match?(/\AAvoid repeating failure pattern from \w+: .{0,5}\{"success":true/)
  end
  PWN::Memory.save(mem: mem, force: mem.empty?)
  { removed: before - mem.size, remaining: mem.size }
end

.reconcile_verdict_tags!(opts = {}) ⇒ Object

One-shot / on-load repair: rewrite tags+details where verdict label disagrees with score (solved @ 0.3 etc.). Safe to call repeatedly.



1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
 
# File 'lib/pwn/ai/agent/learning.rb', line 1020

public_class_method def self.reconcile_verdict_tags!(opts = {})
  return { repaired: 0 } unless File.exist?(LEARNING_FILE)

  dry = opts[:dry_run] ? true : false
  repaired = 0
  lines = File.readlines(LEARNING_FILE)
  out = lines.map do |l|
    r = JSON.parse(l, symbolize_names: true)
    score = r.key?(:score) ? r[:score].to_f : nil
    next l if score.nil?

    want = verdict_for_score(score: score).to_s
    tags = Array(r[:tags]).map(&:to_s)
    stale = tags & %w[solved partial wrong unknown]
    next l if stale.empty? || stale.include?(want)

    repaired += 1
    next l if dry

    cleaned = tags - %w[solved partial wrong unknown]
    cleaned << want
    r[:tags] = cleaned
    # Fix leading "solved(0.3)" style details head when present
    det = r[:details].to_s
    r[:details] = det.sub(
      /\A(solved|partial|wrong|unknown)\(\d+(?:\.\d+)?\)/i,
      "#{want}(#{format('%.2f', score)})"
    )
    r[:success] = (score >= 0.6) if [true, false].include?(r[:success])
    "#{JSON.generate(r)}\n"
  rescue StandardError
    l
  end
  File.write(LEARNING_FILE, out.join) if !dry && repaired.positive?
  { repaired: repaired, dry_run: dry }
rescue StandardError => e
  { repaired: 0, error: "#{e.class}: #{e.message}" }
end

.reflect(opts = {}) ⇒ Object

Supported Method Parameters

report = PWN::AI::Agent::Learning.reflect( session_id: 'required - PWN::Sessions id to analyse', dry_run: 'optional - when true, do not write to Memory/Skills (default false)' )

Uses PWN::AI::Agent::Reflect (when available) to LLM-summarise the session into structured lessons. Falls back to a heuristic extractor when module_reflection is disabled so learning never stops.



421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
 
# File 'lib/pwn/ai/agent/learning.rb', line 421

public_class_method def self.reflect(opts = {})
  session_id = opts[:session_id]
  dry_run    = opts[:dry_run] ? true : false
  raise 'ERROR: session_id is required' if session_id.to_s.empty?

  transcript = PWN::Sessions.load(session_id: session_id)
  return { session_id: session_id, lessons: [], reason: 'empty transcript' } if transcript.empty?

  lessons = introspective_lessons(transcript: transcript)
  source, conf = lessons.empty? ? [:heuristic, 0.3] : [:reflect, 0.8]
  lessons = heuristic_lessons(transcript: transcript) if lessons.empty?

  saved = []
  lessons.each do |l|
    next if l.to_s.strip.empty?

    key = :"reflect_#{session_id}_#{Digest::SHA256.hexdigest(l)[0, 8]}"
    # M3 — provenance + confidence + ttl so consolidate evicts
    # low-confidence heuristic lessons before hand-written ones.
    PWN::Memory.remember(key: key, value: l, category: :lesson, source: source, confidence: conf, importance: conf, ttl: source == :heuristic ? 7 * 86_400 : nil) unless dry_run
    saved << { key: key, lesson: l }
  end
  consolidate unless dry_run

  { session_id: session_id, lessons: saved, count: saved.length, dry_run: dry_run }
end

.reset ⇒ Object

Supported Method Parameters

PWN::AI::Agent::Learning.reset



812
813
814
815
 
# File 'lib/pwn/ai/agent/learning.rb', line 812

public_class_method def self.reset
  FileUtils.rm_f(LEARNING_FILE)
  { cleared: true }
end

.stats ⇒ Object

Supported Method Parameters

stats = PWN::AI::Agent::Learning.stats



170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
 
# File 'lib/pwn/ai/agent/learning.rb', line 170

public_class_method def self.stats
  rows   = outcomes(limit: 10_000)
  total  = rows.length
  ok     = rows.count { |r| r[:success] == true }
  skills = defined?(PWN::Skills) && PWN::Skills.is_a?(Hash) ? PWN::Skills.keys.length : 0
  mem    = defined?(PWN::Memory) ? PWN::Memory.load.keys.length : 0
  raw    = total.positive? ? (ok.to_f / total).round(3) : 0.0
  jmean  = total.positive? ? weighted_judge_mean(rows: rows) : nil
  distrust = 0.0
  distrust = Reward.proxy_distrust.to_f.clamp(0.0, 1.0) if defined?(Reward) && Reward.respond_to?(:proxy_distrust)
  {
    total_outcomes: total,
    successes: ok,
    failures: total - ok,
    success_rate: raw,
    adjusted_success_rate: discount_success_rate(proxy: raw, judge: jmean, distrust: distrust),
    proxy_distrust: distrust,
    skills_known: skills,
    memory_entries: mem,
    judge_mean: jmean,
    reward_sentinel: (Reward.sentinel if defined?(Reward)),
    calibration: (Metrics.calibration if defined?(Metrics) && Metrics.respond_to?(:calibration)),
    preference_pairs: (Reward.preferences(limit: 100_000).length if defined?(Reward)),
    tool_metrics: (Metrics.summary(limit: 5) if defined?(Metrics)),
    extrospection: (Extrospection.stats if defined?(Extrospection))
  }
end

.to_context(opts = {}) ⇒ Object

Supported Method Parameters

ctx = PWN::AI::Agent::Learning.to_context( limit: 'optional - number of recent outcomes to surface (default 5)' )



203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
 
# File 'lib/pwn/ai/agent/learning.rb', line 203

public_class_method def self.to_context(opts = {})
  limit = opts[:limit] || 5
  # Fetch a wider window so prefer_primary_tasks can drop critic/red_team
  # envelope rows (REQUEST:/GOAL: prefixes) without starving the block.
  rows  = prefer_primary_tasks(rows: outcomes(limit: limit * 4)).first(limit)
  fails = prefer_primary_tasks(rows: outcomes(limit: 200, success: false))
  # Do not mirror the same ids under both headings — that doubled the
  # failure signal and made RECENT OUTCOMES == RECENT FAILURES when the
  # last N attempts all failed (the injected block looked "stuck").
  row_ids = rows.map { |r| r[:id] }.compact
  fails = fails.reject { |r| row_ids.include?(r[:id]) }.first(limit)
  return '' if rows.empty? && fails.empty?

  fmt = lambda do |r|
    flag = case r[:success]
           when true then '✓'
           when 'soft', :soft then '∼'
           else '✗'
           end
    score = r.key?(:score) ? format('%.2f', r[:score].to_f) : '-'
    task  = display_task(task: r[:task])
    line  = "  #{flag} [#{score}] #{task} (#{r[:timestamp]})"
    # Surface a one-line cause crumb so the agent can actually learn
    # from failures instead of only seeing that they failed.
    if r[:success] != true
      crumb = cause_crumb(details: r[:details])
      line += "\n      cause: #{crumb}" unless crumb.empty?
    end
    line
  end
  s   = stats
  jm  = s[:judge_mean]
  d   = s[:proxy_distrust].to_f
  rate = d > 0.05 ? s[:adjusted_success_rate] : s[:success_rate]
  tag  = d > 0.05 ? ' adj' : ''
  hdr = "RECENT OUTCOMES (success_rate=#{(rate.to_f * 100).round(1)}%#{tag}#{" judge_mean=#{jm}" if jm} over #{s[:total_outcomes]} attempts)"
  out = "#{hdr}\n#{rows.map(&fmt).join("\n")}\n"
  out += "RECENT FAILURES (learn from these — do not repeat)\n#{fails.map(&fmt).join("\n")}\n" unless fails.empty?
  "#{out}\n"
end