Module: PWN::AI::Agent::Curriculum

Defined in:
lib/pwn/ai/agent/curriculum.rb

Overview

PWN::AI::Agent::Curriculum is Tier 4/5 of the pwn-ai reinforcement loop — the SELF-PLAY layer that turns the agent from a passive experience-recorder into an active learner:

S1  .practice        — Mistake-driven auto-curriculum. Reads
                     Mistakes.top(unresolved), asks Reflect to
                     generate 3 minimal reproducer prompts per
                     signature, self-plays each under Loop.run,
                     and auto-mistakes_resolve when Reward.judge
                     says the practice run solved it. THE AGENT
                     PRACTISES ITS OWN WEAKNESSES OVERNIGHT.
S2  .counterfactual  — On a repeated in-turn failure, forks: branch
                     A continues with the correction_hint, branch
                     B asks an alt persona for a different tool.
                     Reward.judge picks the winner; (loser,
                     winner) → Reward.record_preference. Real
                     advantage estimation, not imagined rollouts.
S3  .critic          — Constitutional critic persona with TOOL
                     ACCESS (can shell/extro_verify the claim).
                     Runs BEFORE note_outcome; its verdict feeds
                     Reward.judge and its concrete flaw becomes a
                     preference pair when the agent self-corrects.
S4  .red_team_plan   — After plan_first, an adversarial persona
                     reviews the plan against THIS host's
                     Metrics/Mistakes/extro_drift and injects a
                     pre-emptive correction_hint on the step it
                     predicts will fail.
C3  .hindsight       — Hindsight Experience Replay. On failure,
                     asks the judge "what DID this trajectory
                     accomplish?", relabels the episode with the
                     achieved-goal as success:true. Free positive
                     samples from failures — first HER on real
                     tool traces.
W2  .train_and_gate  — export_finetune + export_dpo → local LoRA
                     (unsloth/axolotl if installed) → replay
                     Mistakes.top on vN vs vN+1 → promote iff
                     resolved(N+1) > resolved(N). Fully autonomous
                     weight-level self-improvement with a
                     regression gate.
W3  .calibrate       — Tracks plan_first predicted p(success) vs
                     actual outcome → Brier score in Metrics.

All entry points are cron-safe (never raise into the caller) and depth-guarded via Swarm's Thread.current so a curriculum run cannot recurse into itself.

Constant Summary collapse

CURRICULUM_DIR = 
File.join(Dir.home, '.pwn', 'curriculum')
MODELS_FILE = 
File.join(CURRICULUM_DIR, 'models.json')
CRITIC_NAME = 
'pwn_critic'
RED_TEAM_NAME = 
'pwn_red_team'
ALT_NAME = 
'pwn_alt'
COOLDOWN_FAIL_NIGHTS =

Priority-fix 1 — N-night cooldown window for thrashing signatures and stale "needs_human" tags. Signatures that score ~0 for COOLDOWN_FAIL_NIGHTS consecutive practice nights are parked so the curriculum stops burning cycles on them.

3
COOLDOWN_FILE = 
File.join(CURRICULUM_DIR, 'cooldown.json')
TRAINER_CLI =

W2 trainer discovery/execution is pure Ruby filesystem + argv spawn. Never shell-out via backticks or shell-string Open3 for probe/install checks. unsloth/axolotl presence is inferred from PATH CLIs and site-packages layout; any external process uses argv arrays only.

{
  unsloth: %w[unsloth],
  axolotl: %w[axolotl]
}.freeze
TRAINER_MODULE_HINTS = 
{
  unsloth: %w[unsloth/__init__.py unsloth/models/__init__.py],
  axolotl: %w[axolotl/__init__.py axolotl/cli/train.py]
}.freeze
DPO_DIR_CONST = 
File.join(Dir.home, '.pwn', 'finetune')

Class Method Summary collapse

Class Method Details

.authors ⇒ Object

Author(s)

0day Inc. support@0dayinc.com



1111
1112
1113
 
# File 'lib/pwn/ai/agent/curriculum.rb', line 1111

public_class_method def self.authors
  "AUTHOR(S):\n  0day Inc. <support@0dayinc.com>\n"
end

.calibrate(opts = {}) ⇒ Object

Supported Method Parameters

PWN::AI::Agent::Curriculum.calibrate(predicted:, actual:, engine:)



487
488
489
490
491
492
493
 
# File 'lib/pwn/ai/agent/curriculum.rb', line 487

public_class_method def self.calibrate(opts = {})
  p = opts[:predicted].to_f.clamp(0.0, 1.0)
  a = opts[:actual].to_f.clamp(0.0, 1.0)
  brier = (p - a)**2
  Metrics.record_calibration(predicted: p, actual: a, brier: brier, engine: opts[:engine]) if defined?(Metrics) && Metrics.respond_to?(:record_calibration)
  { predicted: p, actual: a, brier: brier.round(4) }
end

.counterfactual(opts = {}) ⇒ Object 



281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
 
# File 'lib/pwn/ai/agent/curriculum.rb', line 281

public_class_method def self.counterfactual(opts = {})
  return nil unless enabled?(key: :counterfactual)
  return nil if in_curriculum?

  request = opts[:request].to_s
  ensure_persona(name: ALT_NAME, role: 'You are an alternative-approach generator for pwn-ai. Given a failing tool call, propose ONE concrete DIFFERENT tool + args that would achieve the same sub-goal on this host. Reply with the tool call only, no prose.')

  branch_a = opts[:hint].to_s.strip
  branch_a = "retry #{opts[:name]} with corrected args" if branch_a.empty?
  branch_b = with_curriculum_guard do
    ask_persona(name: ALT_NAME, request: "Goal: #{request[0, 300]}\nFailing: #{opts[:name]}(#{opts[:args].to_s[0, 200]}) → #{opts[:error].to_s[0, 200]}\nPropose ONE different tool+args.")
  end
  return nil if branch_b.to_s.strip.empty?

  sa = score_branch(request: request, branch: branch_a)
  sb = score_branch(request: request, branch: branch_b)
  winner, loser, tag = sb > sa ? [branch_b, branch_a, :b] : [branch_a, branch_b, :a]
  Reward.record_preference(prompt: "#{request} | failing: #{opts[:name]} → #{opts[:error]}", rejected: loser, chosen: winner, source: :counterfactual) if defined?(Reward)
  log(event: :counterfactual, data: { branch: tag, a: sa, b: sb, tool: opts[:name].to_s })
  { branch: tag, content: winner, score: [sa, sb].max, a: sa, b: sb }
rescue StandardError => e
  warn "[pwn-ai/curriculum] counterfactual swallowed: #{e.class}: #{e.message}"
  nil
end

.critic(opts = {}) ⇒ Object

Supported Method Parameters

v = PWN::AI::Agent::Curriculum.critic( request: 'required - user request', final: 'required - candidate final answer', session_id: 'optional - for evidence lookup' )

Returns { verdict: :pass|:flaw, flaw:, confidence: }. On :flaw the (final, flaw) pair is recorded as a preference so a future self-correction becomes DPO signal.



321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
 
# File 'lib/pwn/ai/agent/curriculum.rb', line 321

public_class_method def self.critic(opts = {})
  return { verdict: :pass, source: :disabled } unless enabled?(key: :critic)
  return { verdict: :pass, source: :recursion } if in_curriculum?

  ensure_persona(name: CRITIC_NAME, role: "You are pwn-ai's constitutional critic. Given a REQUEST and a candidate ANSWER, find ONE concrete, verifiable flaw (wrong fact, missing step, unsupported claim, broken command). You MAY call shell / extro_verify / pwn_eval to check. If none found reply exactly: PASS. Otherwise reply: FLAW: <one line>.")
  reply = with_curriculum_guard do
    ask_persona(name: CRITIC_NAME, request: "REQUEST:\n#{opts[:request].to_s[0, 800]}\n\nANSWER:\n#{opts[:final].to_s[0, 2_000]}")
  end
  if reply.to_s.strip.upcase.start_with?('PASS')
    log(event: :critic, data: { verdict: :pass })
    { verdict: :pass, confidence: 0.7 }
  else
    flaw = reply.to_s.sub(/\AFLAW:\s*/i, '').strip[0, 300]
    Mistakes.record(tool: 'assistant_answer', error: "critic: #{flaw}", args: opts[:final].to_s[0, 200], session_id: opts[:session_id], source: :model) if defined?(Mistakes)
    # P5 — critic flaws are free DPO signal (rejected=answer, chosen=correction)
    if defined?(Reward) && !flaw.to_s.empty?
      Reward.record_preference(
        prompt: opts[:request].to_s[0, 1_000],
        rejected: opts[:final].to_s[0, 2_000],
        chosen: "CORRECTION: #{flaw}",
        source: :critic
      )
    end
    log(event: :critic, data: { verdict: :flaw, flaw: flaw.to_s[0, 200] })
    { verdict: :flaw, flaw: flaw, confidence: 0.7 }
  end
rescue StandardError => e
  { verdict: :pass, error: e.message }
end

.help ⇒ Object

Display Usage for this Module



1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
 
# File 'lib/pwn/ai/agent/curriculum.rb', line 1117

public_class_method def self.help
  puts <<~USAGE
    USAGE:
      # Tier 4 — self-play
      PWN::AI::Agent::Curriculum.practice(limit: 3)                     # S1 mistake-driven auto-curriculum
      PWN::AI::Agent::Curriculum.offline_judge(since_hours: 24)         # P3 offline ORM/PRM fill
      PWN::AI::Agent::Curriculum.preference_balance                     # P5 W1 diversity report
      PWN::AI::Agent::Curriculum.counterfactual(request:, name:, args:, error:, hint:)  # S2 A/B → DPO pair
      PWN::AI::Agent::Curriculum.critic(request:, final:)               # S3 tool-armed constitutional critic
      PWN::AI::Agent::Curriculum.red_team_plan(request:, plan:)         # S4 telemetry-grounded plan review
      PWN::AI::Agent::Curriculum.hindsight(request:, final:, session_id:) # C3 HER soft-relabel

      # Tier 5 — close the weight loop
      PWN::AI::Agent::Curriculum.train_and_gate(dry_run: true)          # W2 export-ready; promote only with trainer+dry_run:false
      PWN::AI::Agent::Curriculum.calibrate(predicted: 0.8, actual: 1.0) # W3 Brier → Metrics[:calibration]

      Cron self-improvement (seeded by PWN::Cron.install_defaults):
        curriculum_practice_nightly  0 3 * * *   Curriculum.practice(limit: 3)
        curriculum_offline_judge     30 3 * * *  Curriculum.offline_judge(since_hours: 24, limit: 40)
        curriculum_train_weekly      0 4 * * 0   Curriculum.train_and_gate(dry_run: true)
        learning_consolidate_nightly 0 5 * * *   Learning.consolidate

      Config (PWN::Env[:ai][:agent]) — nil = auto (ON for remote engines, OFF for ollama):
        :critic         - Boolean/nil, run S3 before every note_outcome
        :red_team_plan  - Boolean/nil, run S4 after every plan_first
        :counterfactual - Boolean/nil, run S2 on REPEAT_THRESHOLD
        :hindsight      - Boolean, HER soft-relabel failures (default true)
        :reward_llm     - Boolean/nil, force ORM/PRM LLM teacher (nil=on for remote)

      #{self}.authors
  USAGE
end

.hindsight(opts = {}) ⇒ Object

Supported Method Parameters

PWN::AI::Agent::Curriculum.hindsight( request: 'required - the FAILED goal', final: 'required - the final produced anyway', session_id: 'required - trajectory to relabel' )



389
390
391
392
393
394
395
396
397
398
399
400
401
 
# File 'lib/pwn/ai/agent/curriculum.rb', line 389

public_class_method def self.hindsight(opts = {})
  return nil unless enabled?(key: :hindsight, default: true)
  return nil unless reflect_available?

  req = "The agent FAILED at: #{opts[:request].to_s[0, 300]}\nBut it produced: #{opts[:final].to_s[0, 800]}\n\nIn ≤12 words, what goal DID this trajectory accomplish? Reply with the goal only, or NOTHING if truly nothing."
  achieved = Reflect.on(request: req, suppress_pii_warning: true).to_s.strip
  return nil if achieved.empty? || achieved.upcase == 'NOTHING' || achieved.length > 200

  Learning.note_outcome(task: achieved, success: 'soft', details: "HER-relabelled from failed: #{opts[:request].to_s[0, 100]}", session_id: opts[:session_id], tags: %w[hindsight her soft], score: 0.7) if defined?(Learning)
  { original: opts[:request].to_s[0, 100], achieved: achieved }
rescue StandardError
  nil
end

.offline_judge(opts = {}) ⇒ Object

Priority-fix 3 — Offline ORM/PRM pass over recent sessions so local :failure_only introspect does not starve the reward corpus. Cron this nightly. Never raises.

Supported Method Parameters

r = PWN::AI::Agent::Curriculum.offline_judge( since_hours: 'optional - lookback window (default 24)', limit: 'optional - max sessions to score (default 40)', prm: 'optional - also run Process Reward Model (default true)', commit: 'optional - write scores into learning/sentinel (default true)' )



162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
 
# File 'lib/pwn/ai/agent/curriculum.rb', line 162

public_class_method def self.offline_judge(opts = {})
  return { skipped: 'no Sessions' } unless defined?(PWN::Sessions)
  return { skipped: 'no Reward' } unless defined?(Reward)

  since_h = (opts[:since_hours] || 24).to_i
  limit   = (opts[:limit] || 40).to_i
  do_prm  = opts.key?(:prm) ? opts[:prm] : true
  commit  = opts.key?(:commit) ? opts[:commit] : true
  cutoff  = Time.now.utc - (since_h * 3_600)

  # PWN::Sessions.list is arity-0 (no kwargs). Cap/sort after the call.
  sids = if PWN::Sessions.respond_to?(:list)
           Array(PWN::Sessions.list)
             .sort_by { |s| s.is_a?(Hash) ? s[:mtime].to_s : '' }
             .reverse
             .first(limit * 2)
             .map { |s| s.is_a?(Hash) ? (s[:id] || s[:session_id] || s['id'] || s['session_id']) : s.to_s }
         else
           dir = (PWN::Sessions::SESSIONS_DIR if defined?(PWN::Sessions::SESSIONS_DIR)) || File.join(Dir.home, '.pwn', 'sessions')
           Dir[File.join(dir, '*.jsonl')].sort_by { |f| -File.mtime(f).to_i }.first(limit * 2).map { |f| File.basename(f, '.jsonl') }
         end

  scored = []
  sids.first(limit * 3).each do |sid|
    break if scored.length >= limit

    t = begin
      PWN::Sessions.load(session_id: sid)
    rescue StandardError
      []
    end
    next if t.nil? || t.empty?

    mtime = begin
      path = File.join(
        (PWN::Sessions::SESSIONS_DIR if defined?(PWN::Sessions::SESSIONS_DIR)) || File.join(Dir.home, '.pwn', 'sessions'),
        "#{sid}.jsonl"
      )
      File.exist?(path) ? File.mtime(path).utc : nil
    rescue StandardError
      nil
    end
    next if mtime && mtime < cutoff

    if commit && defined?(Learning)
      prior = Learning.outcomes(limit: 500).find do |o|
        o[:session_id].to_s == sid.to_s && o[:score] && Array(o[:tags]).include?('offline_judge')
      end
      next if prior
    end

    user = t.reverse.find { |e| e[:role].to_s == 'user' }
    final = t.reverse.find { |e| e[:role].to_s == 'assistant' && !e[:content].to_s.start_with?('PLAN:') }
    next unless user && final

    req = user[:content].to_s
    fin = final[:content].to_s
    next if req.strip.empty? || fin.strip.empty?

    v = Reward.judge(request: req, final: fin, session_id: sid, commit: commit)
    Reward.prm(request: req, session_id: sid) if do_prm
    # P7/W3 — offline path must also fill calibration so the controller
    # (force plan_first/critic at n≥8) actually becomes reachable under
    # :failure_only local introspect. Pull p(success)= out of any PLAN.
    if commit
      plan = t.find { |e| e[:role].to_s == 'assistant' && e[:content].to_s.start_with?('PLAN:') }
      pred = plan && plan[:content].to_s[/p\(success\)\s*=\s*([01](?:\.\d+)?)/i, 1]
      if pred
        eng = (PWN::Env.dig(:ai, :active) if defined?(PWN::Env))
        calibrate(predicted: pred.to_f, actual: v[:score].to_f, engine: eng)
      end
    end
    if commit && defined?(Learning)
      Learning.note_outcome(
        task: req[0, 120],
        success: v[:score].to_f >= 0.6,
        score: v[:score],
        details: "offline_judge #{v[:verdict]}(#{v[:score]}) #{v[:rationale]}",
        session_id: sid,
        tags: %w[offline_judge auto]
      )
    end
    scored << { session_id: sid, score: v[:score], verdict: v[:verdict] }
  end

  mean = scored.empty? ? nil : (scored.sum { |r| r[:score].to_f } / scored.length).round(3)
  out = { scored: scored.length, mean: mean, since_hours: since_h, results: scored.first(10) }
  log(event: :offline_judge, data: out)
  out
rescue StandardError => e
  { error: "#{e.class}: #{e.message}" }
end

.practice(opts = {}) ⇒ Object 



81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
 
# File 'lib/pwn/ai/agent/curriculum.rb', line 81

public_class_method def self.practice(opts = {})
  limit   = (opts[:limit] || 3).to_i
  per     = (opts[:prompts_per] || 2).to_i
  dry_run = opts[:dry_run] ? true : false
  return { skipped: 'recursion guard' } if in_curriculum?

  FileUtils.mkdir_p(CURRICULUM_DIR)
  # 2.4 / 2.5 / P1 — skip reward_signal + needs_code_change / parked
  # + cooldown thrash + needs_human. Over-fetch so filters still fill.
  fetch_n = [limit * 4, 12].max
  candidates = if defined?(Mistakes)
                 Mistakes.top(limit: fetch_n, unresolved_only: true, practiceable_only: true)
               else
                 []
               end
  cool = load_cooldown
  targets = candidates.reject { |m| practice_skip?(mistake: m, cooldown: cool) }.first(limit)
  results = []

  with_curriculum_guard do
    targets.each do |m|
      next if practice_skip?(mistake: m, cooldown: cool)

      prompts = generate_reproducers(mistake: m, count: [per, 2].max)
      runs = dry_run ? [] : prompts.map { |p| self_play(prompt: p, tag: "practice:#{m[:signature]}") }
      solved = runs.select { |r| r[:score].to_f >= 0.7 }
      mean = runs.empty? ? 0.0 : (runs.sum { |r| r[:score].to_f } / runs.length)
      resolved = false
      # 2.4 — auto-resolve only with N≥2 holdout successes + store trace
      if solved.length >= 2 && defined?(Mistakes)
        best = solved.max_by { |r| r[:score] }
        fix = best[:final].to_s.lines.first(3).join.strip[0, 400]
        Mistakes.resolve(
          signature: m[:signature],
          fix: "auto-curriculum: #{fix}",
          structured: {
            strategy: 'curriculum_practice',
            tool: m[:tool],
            holdout_tests: solved.map { |r| r[:prompt] || r[:request] }.compact.first(5),
            winning_trace: best[:trace].to_s[0, 2_000]
          }
        )
        Reward.record_preference(prompt: prompts.first.to_s, rejected: m[:snippet].to_s, chosen: fix, source: :curriculum) if defined?(Reward)
        resolved = true
        cool.delete(m[:signature].to_s)
      elsif !dry_run
        # P1 — track zero-progress nights; park after COOLDOWN_FAIL_NIGHTS
        bump_cooldown!(cooldown: cool, signature: m[:signature], mean: mean)
      end
      results << {
        signature: m[:signature], tool: m[:tool], prompts: prompts,
        runs: runs.map { |r| { score: r[:score], verdict: r[:verdict] } },
        resolved: resolved, mean_score: mean.round(3)
      }
    end
  end
  save_cooldown(cooldown: cool)
  log(event: :practice, data: results)
  {
    practiced: results.length,
    resolved: results.count { |r| r[:resolved] },
    skipped_cooldown: cool.count { |_, v| v[:fail_nights].to_i >= COOLDOWN_FAIL_NIGHTS },
    results: results,
    dry_run: dry_run
  }
rescue StandardError => e
  { error: "#{e.class}: #{e.message}" }
end

.preference_balance(opts = {}) ⇒ Object

P5 — W1 diversity report so monoculture is visible before DPO export.



257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
 
# File 'lib/pwn/ai/agent/curriculum.rb', line 257

public_class_method def self.preference_balance(opts = {})
  return { total: 0 } unless defined?(Reward)

  rows = Reward.preferences(limit: opts[:limit] || 10_000)
  by = Hash.new(0)
  rows.each { |r| by[r[:source].to_s] += 1 }
  total = rows.length
  frac = by.transform_values { |n| total.zero? ? 0.0 : (n.to_f / total).round(3) }
  monoculture = total.positive? && (by.values.max.to_f / total) > 0.7
  {
    total: total,
    by_source: by,
    fractions: frac,
    monoculture: monoculture,
    advice: if monoculture
              'W1 monoculture: enable :counterfactual/:critic or loosen S2 gates; DPO will overfit mistakes_resolve prose.'
            else
              'W1 source mix OK'
            end
  }
rescue StandardError => e
  { error: "#{e.class}: #{e.message}" }
end

.red_team_plan(opts = {}) ⇒ Object

Supported Method Parameters

hint = PWN::AI::Agent::Curriculum.red_team_plan( request: 'required - user goal', plan: 'required - numbered plan text from plan_first' )



361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
 
# File 'lib/pwn/ai/agent/curriculum.rb', line 361

public_class_method def self.red_team_plan(opts = {})
  return nil unless enabled?(key: :red_team_plan)
  return nil if in_curriculum?

  ensure_persona(name: RED_TEAM_NAME, role: 'You are pwn-ai\'s adversarial plan reviewer. Given a numbered tool plan and telemetry from THIS host (tool success rates, known mistakes, environment drift), identify the ONE step most likely to fail and say why in ≤2 lines. Cite the metric/mistake/drift. If the plan is sound reply: SOUND.')
  telemetry = build_telemetry
  reply = with_curriculum_guard do
    ask_persona(name: RED_TEAM_NAME, request: "GOAL: #{opts[:request].to_s[0, 300]}\n\nPLAN:\n#{opts[:plan].to_s[0, 1_200]}\n\nHOST TELEMETRY:\n#{telemetry}")
  end
  return nil if reply.to_s.strip.upcase.start_with?('SOUND') || reply.to_s.strip.empty?

  "[pwn-ai/red_team] pre-emptive: #{reply.to_s.strip[0, 400]}"
rescue StandardError => e
  warn "[pwn-ai/curriculum] red_team_plan swallowed: #{e.class}: #{e.message}"
  nil
end

.train_and_gate(opts = {}) ⇒ Object

Supported Method Parameters

r = PWN::AI::Agent::Curriculum.train_and_gate( base_model: 'optional - ollama base tag (default PWN::Env[:ollama][:model])', trainer: 'optional - :unsloth | :axolotl | :auto (default :auto)', dry_run: 'optional - export + build eval set but do not train (default true)' )

Best-effort orchestrator. When a supported trainer is installed it produces ~/.pwn/finetune/pwn-vN/, cuts an ollama Modelfile with the LoRA adapter, then REPLAYS Mistakes.top against vN and vN+1 under Reward.judge. Promotes (writes MODELS_FILE) iff vN+1 resolves more signatures. When no trainer is present it still exports SFT+DPO and emits the exact CLI to run manually — so the pipeline is complete even on a box without GPU.



422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
 
# File 'lib/pwn/ai/agent/curriculum.rb', line 422

public_class_method def self.train_and_gate(opts = {})
  dry_run = opts.key?(:dry_run) ? opts[:dry_run] : true
  FileUtils.mkdir_p(CURRICULUM_DIR)
  sft = defined?(Learning) ? Learning.export_finetune(format: :sharegpt) : nil
  dpo = defined?(Reward) ? Reward.export_dpo : nil
  evalset = build_eval_set

  state = load_models
  version = state[:current].to_i + 1
  base = opts[:base_model] || (PWN::Env.dig(:ai, :ollama, :model) if defined?(PWN::Env)) || 'llama3'
  trainer = detect_trainer(preference: opts[:trainer])

  result = {
    version: version, base: base, trainer: trainer,
    sft: sft, dpo: dpo, eval_prompts: evalset.length, dry_run: dry_run
  }

  if dry_run || trainer.nil?
    result[:export_only] = true
    result[:weight_loop] = :export_ready
    result[:advice] = if trainer.nil?
                        'No trainer found — weight loop is EXPORT-ONLY on this host. ' \
                          'Install unsloth or axolotl on a GPU box, then re-run with dry_run:false. ' \
                          "Datasets ready at #{sft&.[](:path)} + #{dpo&.[](:path)}."
                      else
                        'dry_run — datasets + eval set exported; pass dry_run:false to train+gate+promote.'
                      end
    result[:manual_cli] = manual_train_cli(base: base, sft: sft, dpo: dpo, version: version)
    # P5 — surface W1 monoculture beside the export so operators see it
    result[:preference_balance] = begin
      preference_balance
    rescue StandardError
      nil
    end
    log(event: :train_and_gate, data: result)
    return result
  end

  adapter = run_trainer(trainer: trainer, base: base, sft: sft, dpo: dpo, version: version)
  return result.merge(error: 'trainer produced no adapter') unless adapter

  candidate = ollama_create(base: base, adapter: adapter, version: version)
  baseline  = state[:tag] || base
  gate = ab_gate(baseline: baseline, candidate: candidate, evalset: evalset)
  promoted = gate[:candidate_resolved] > gate[:baseline_resolved]
  if promoted
    state[:previous] = state[:tag]
    state[:tag] = candidate
    state[:current] = version
    state[:promoted_at] = Time.now.utc.iso8601
    state[:gate] = gate
    save_models(state: state)
  end
  result.merge(adapter: adapter, candidate: candidate, gate: gate, promoted: promoted)
rescue StandardError => e
  { error: "#{e.class}: #{e.message}" }
end