Module: PWN::Migrate

Defined in:
lib/pwn/migrate.rb

Overview

PWN::Migrate — the ~/.pwn state doctor and auto-migrator.

PWN persists a growing set of files under ~/.pwn (encrypted config, memory, learning outcomes, metrics, mistakes, extrospection, cron jobs, agents, skills, sessions, swarm buses, …). Each file is owned by a different module and each release can add keys or change shape. A user upgrading gem install pwn between two versions could therefore hit KeyError, NoMethodError for nil, or a silent empty-fallback because their on-disk state predates the new loader.

PWN::Migrate closes that gap. It is called from:

* `pwn setup --migrate`       — explicit doctor + autofix
* `PWN::Setup.check`          — read-only ~/.pwn state section
* `PWN::Config.refresh_env`   — one-line drift warning on every launch

It works by:

1. Stamping `~/.pwn/.schema` with `{ schema:, pwn_version:, at: }`
 the first time it runs.  Any release that changes the on-disk
 shape of a `~/.pwn` file bumps `SCHEMA_VERSION` here and appends
 an idempotent lambda to `MIGRATIONS`.
2. Declaratively verifying every state file against its OWNING
 module's own loader (`STATE_FILES`) — no re-implemented parsers.
 If the raw file has bytes but the owner returned its
 empty-fallback, the file is flagged incompatible.
3. Autofixing:  timestamped backup → ordered schema migrations →
 per-file repair (quarantine corrupt / re-seed missing) →
 `pwn.yaml` deep-key backfill (missing keys from the current
 `PWN::Config.env_template` are merged in-place under the user's
 values, then the vault is re-encrypted with the SAME key/iv).

Everything is idempotent, dry-run capable, and never overwrites a user-set value.

Constant Summary collapse

PWN_ROOT = 
File.join(Dir.home, '.pwn')
SCHEMA_FILE = 
File.join(PWN_ROOT, '.schema')
BACKUP_ROOT = 
File.join(PWN_ROOT, 'backup')
QUARANTINE_DIR = 
File.join(PWN_ROOT, 'quarantine')
SCHEMA_VERSION =

Bump this whenever the shape of any file under ~/.pwn changes in a way that requires a one-time transform. Add the transform as an entry in MIGRATIONS keyed by the NEW schema number.

1
OK = 
"\e[32mok\e[0m"
BAD = 
"\e[31mFAIL\e[0m"
WARN = 
"\e[33mdrift\e[0m"
STATE_FILES =

────────────────────────────────────────────────────────────────── Declarative registry of every persisted ~/.pwn artefact.

Each entry knows how to (a) locate itself, (b) ask its owning module to load it, and (c) decide whether the loaded shape is what this pwn version expects. Verifiers are intentionally SHALLOW — they detect "loader gave up and returned the fallback" rather than re-validating every key, so ownership stays with the module.

:fix values:

:quarantine — move the file aside; owner re-seeds on next write
:vault      — deep-merge missing keys from PWN::Config.env_template
:seed       — call the owner's idempotent seeder (dirs / defaults)
:jsonl      — strip unparsable lines only (keeps good history)
:skills     — PWN::Config.migrate_legacy_skills

──────────────────────────────────────────────────────────────────

{
  'pwn.yaml' => {
    owner: 'PWN::Config',
    kind: :vault,
    fix: :vault,
    verify: lambda { |p, _raw|
      return { ok: false, why: 'missing' } unless File.file?(p)

      enc = PWN::Plugins::Vault.file_encrypted?(file: p)
      # Shape (missing keys vs env_template) is checked separately in
      # .vault_drift because it requires the decryptor.
      { ok: enc, why: enc ? nil : 'not encrypted (run PWN::Plugins::Vault.create)' }
    }
  },
  'memory.json' => {
    owner: 'PWN::Memory',
    kind: :json,
    fix: :quarantine,
    verify: lambda { |_p, raw|
      m = PWN::Memory.load
      bad = m.reject { |_k, v| v.is_a?(Hash) && v.key?(:value) }
      { ok: !(m.empty? && raw.to_s.length > 4) && bad.empty?,
        why: bad.any? ? "#{bad.length} entries missing :value" : nil }
    }
  },
  'memory.idx' => {
    owner: 'PWN::MemoryIndex',
    kind: :blob,
    fix: :quarantine,
    verify: ->(_p, _raw) { { ok: true } }
  },
  'learning.jsonl' => {
    owner: 'PWN::AI::Agent::Learning',
    kind: :jsonl,
    fix: :jsonl,
    verify: :jsonl
  },
  'preferences.jsonl' => {
    owner: 'PWN::AI::Agent::Reward',
    kind: :jsonl,
    fix: :jsonl,
    verify: :jsonl
  },
  'metrics.json' => {
    owner: 'PWN::AI::Agent::Metrics',
    kind: :json,
    fix: :quarantine,
    verify: lambda { |_p, raw|
      m = PWN::AI::Agent::Metrics.load
      fell_back = m == { tools: {}, updated_at: nil } && raw.to_s.length > 4
      { ok: m.is_a?(Hash) && m.key?(:tools) && !fell_back,
        why: fell_back ? 'loader fell back to empty (unparsable / wrong shape)' : nil }
    }
  },
  'mistakes.json' => {
    owner: 'PWN::AI::Agent::Mistakes',
    kind: :json,
    fix: :quarantine,
    verify: lambda { |_p, raw|
      m = PWN::AI::Agent::Mistakes.load
      fell_back = m.empty? && raw.to_s.length > 4
      { ok: m.is_a?(Hash) && !fell_back, why: fell_back ? 'loader fell back to {}' : nil }
    }
  },
  'extrospection.json' => {
    owner: 'PWN::AI::Agent::Extrospection',
    kind: :json,
    fix: :quarantine,
    verify: lambda { |_p, raw|
      m = PWN::AI::Agent::Extrospection.load
      shape = m.is_a?(Hash) && m.key?(:observations) && m[:observations].is_a?(Array)
      fell_back = shape && m[:snapshot].to_h.empty? && m[:observations].empty? && raw.to_s.length > 8
      { ok: shape && !fell_back, why: shape ? nil : 'missing :observations array' }
    }
  },
  'reward_sentinel.json' => {
    owner: 'PWN::AI::Agent::Reward',
    kind: :json,
    fix: :quarantine,
    verify: ->(_p, _raw) { { ok: true } }
  },
  'agents.yml' => {
    owner: 'PWN::AI::Agent::Swarm',
    kind: :yaml,
    fix: :quarantine,
    verify: lambda { |_p, raw|
      m = PWN::AI::Agent::Swarm.personas
      fell_back = m.empty? && raw.to_s.strip.length > 4
      bad = m.reject { |_k, v| v.is_a?(Hash) && v[:role] }
      { ok: !fell_back && bad.empty?,
        why: (fell_back && 'loader fell back to {}') || (bad.any? && "#{bad.length} personas missing :role") || nil }
    }
  },
  'cron/jobs.yml' => {
    owner: 'PWN::Cron',
    kind: :yaml,
    fix: :seed,
    verify: lambda { |_p, raw|
      j = PWN::Cron.list
      fell_back = j.empty? && raw.to_s.strip.length > 8
      bad = j.reject { |_id, v| v.is_a?(Hash) && v[:schedule] }
      { ok: !fell_back && bad.empty?,
        why: (fell_back && 'loader fell back to {}') || (bad.any? && "#{bad.length} jobs missing :schedule") || nil }
    }
  },
  'skills/' => {
    owner: 'PWN::Config',
    kind: :dir,
    fix: :skills,
    verify: lambda { |p, _raw|
      return { ok: true } unless File.directory?(p)

      legacy = Dir.glob(File.join(p, '*.{rb,md,txt,skill,yml,yaml}')).length
      { ok: legacy.zero?, why: legacy.zero? ? nil : "#{legacy} legacy flat skill file(s)" }
    }
  },
  'sessions/' => { owner: 'PWN::Sessions', kind: :dir, fix: :seed, verify: ->(_p, _r) { { ok: true } } },
  'swarm/' => { owner: 'PWN::AI::Agent::Swarm', kind: :dir, fix: :seed, verify: ->(_p, _r) { { ok: true } } },
  'curriculum/' => { owner: 'PWN::AI::Agent::Curriculum', kind: :dir, fix: :seed, verify: ->(_p, _r) { { ok: true } } },
  'finetune/' => { owner: 'PWN::AI::Agent::Curriculum', kind: :dir, fix: :seed, verify: ->(_p, _r) { { ok: true } } },
  'extrospection/' => { owner: 'PWN::AI::Agent::Extrospection', kind: :dir, fix: :seed, verify: ->(_p, _r) { { ok: true } } }
}.freeze
MIGRATIONS =

Ordered, idempotent one-shot transforms. Key == the schema number a ~/.pwn tree is AT after the lambda runs. Add a new entry every time SCHEMA_VERSION is bumped. Each lambda receives (root, io).

{
  1 => lambda { |root, io|
    # v0 → v1 : agentskills.io directory layout + seeded RL cron jobs
    #           + ensure all state subdirs exist.
    %w[skills sessions swarm cron curriculum finetune extrospection].each do |d|
      FileUtils.mkdir_p(File.join(root, d))
    end
    r = begin
      PWN::Config.migrate_legacy_skills
    rescue StandardError
      { migrated: 0 }
    end
    io.puts "    · migrate_legacy_skills → #{r[:migrated]} converted" if r[:migrated].to_i.positive?
    PWN::Cron.install_defaults if defined?(PWN::Cron)
  }
}.freeze

Class Method Summary collapse

Class Method Details

.authors ⇒ Object

Author(s)

0day Inc. support@0dayinc.com



670
671
672
673
674
 
# File 'lib/pwn/migrate.rb', line 670

public_class_method def self.authors
  "AUTHOR(S):
    0day Inc. <support@0dayinc.com>
  "
end

.backfill_vault(opts = {}) ⇒ Object

Supported Method Parameters

r = PWN::Migrate.backfill_vault( key: 'optional', iv: 'optional', dry_run: false, io: $stdout )

Decrypt → deep-merge template UNDER user (never overwrites) → re-encrypt with the SAME key/iv. This is the fix for the #1 upgrade break: NoMethodError: undefined method '[]' for nil when a release added env[:ai][:new_engine][:…] and the user's vault predates it.



414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
 
# File 'lib/pwn/migrate.rb', line 414

public_class_method def self.backfill_vault(opts = {})
  io      = opts[:io] || $stdout
  dry_run = opts[:dry_run] ? true : false
  d = vault_drift(key: opts[:key], iv: opts[:iv])
  unless d
    io.puts '  vault    : skipped (no decryptor available — run with --pwn-dec or set key:/iv:)'
    return nil
  end
  if d[:missing].empty?
    io.puts "  vault    : #{OK} (no missing keys vs PWN::Config.env_template)"
    return { rel: 'pwn.yaml', added: 0, missing: [] }
  end

  io.puts "  vault    : #{WARN} #{d[:missing].length} missing key(s) — backfilling under user values"
  d[:missing].first(8).each { |p| io.puts "             + #{p}" }
  io.puts "             … (+#{d[:missing].length - 8} more)" if d[:missing].length > 8
  return { rel: 'pwn.yaml', added: d[:missing].length, missing: d[:missing], dry_run: true } if dry_run

  merged = deep_defaults(user: d[:user], tmpl: d[:template])
  yaml   = File.join(PWN_ROOT, 'pwn.yaml')
  key, iv = decryptor(opts)
  # Same colon-stripping default_env uses so the file stays diff-clean.
  File.write(yaml, YAML.dump(merged).gsub(/^(\s*):/, '\1'))
  File.chmod(0o600, yaml)
  PWN::Plugins::Vault.encrypt(file: yaml, key: key, iv: iv)

  { rel: 'pwn.yaml', added: d[:missing].length, missing: d[:missing] }
rescue StandardError => e
  io.puts "  vault    : #{BAD} #{e.class}: #{e.message}"
  nil
end

.check(opts = {}) ⇒ Object

Supported Method Parameters

PWN::Migrate.check( io: 'optional - IO to write the report to (default $stdout)' )

Human-readable ~/.pwn state doctor. Read-only. Called from PWN::Setup.check so pwn setup shows this alongside the native-gem / toolchain report.



286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
 
# File 'lib/pwn/migrate.rb', line 286

public_class_method def self.check(opts = {})
  io = opts[:io] || $stdout
  st = status

  io.puts "~/.pwn state         schema #{st[:schema_installed]} → #{st[:schema_current]} " \
          "#{st[:needs_migration] ? "(#{WARN} — run `pwn setup --migrate`)" : "(#{OK})"}"
  st[:files].each do |f|
    mark = f[:ok] ? OK : BAD
    note = f[:ok] ? '' : "→ #{f[:why]}  [fix: #{f[:fix]}]"
    sz   = f[:exists] ? human_size(bytes: f[:size]) : '—'
    io.puts "  #{f[:rel].ljust(22)} #{mark}   #{sz.to_s.ljust(8)} #{f[:owner].to_s.ljust(30)} #{note}"
  end
  unless st[:incompatible].empty?
    io.puts
    io.puts "#{st[:incompatible].length} file(s) incompatible with pwn #{st[:pwn_version]} — " \
            'run `pwn setup --migrate --fix` to autofix (a timestamped backup is taken first).'
  end
  st
rescue StandardError => e
  raise e
end

.env_template ⇒ Object

Supported Method Parameters

tmpl = PWN::Migrate.env_template

The canonical current-release pwn.yaml shape, WITHOUT the I/O side-effects of PWN::Config.default_env. Prefers PWN::Config.env_template when that refactor is present.



453
454
455
456
457
458
459
 
# File 'lib/pwn/migrate.rb', line 453

public_class_method def self.env_template
  return PWN::Config.env_template if PWN::Config.respond_to?(:env_template)

  # Fallback: intercept default_env's write path.  Never reached once
  # PWN::Config.env_template ships, but keeps Migrate self-sufficient.
  {}
end

.help ⇒ Object

Display Usage for this Module



678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
 
# File 'lib/pwn/migrate.rb', line 678

public_class_method def self.help
  puts "USAGE:
    # Is ~/.pwn older than this pwn release's schema?
    #{self}.needed?             # => true/false (cheap; used by refresh_env)

    # Read-only per-file compatibility report.
    #{self}.status              # => { schema_installed:, schema_current:, files:[…], incompatible:[…] }
    #{self}.check               # human-readable to $stdout

    # Apply schema migrations + (optionally) autofix incompatible files.
    # Always takes a timestamped backup under ~/.pwn/backup/<ts>/ first.
    #{self}.run(
      fix:     'optional - also autofix per-file (default false)',
      backup:  'optional - default true',
      dry_run: 'optional - default false',
      key:     'optional - vault key (default: ~/.pwn/pwn.yaml.decryptor)',
      iv:      'optional - vault iv'
    )

    # pwn.yaml specifically — deep-merge missing keys from the
    # current release template UNDER the user's values, re-encrypt.
    #{self}.vault_drift         # => { missing: [dotted.paths], … } or nil
    #{self}.backfill_vault

    # Data:
    #{self}::SCHEMA_VERSION     # bump when a ~/.pwn file shape changes
    #{self}::STATE_FILES        # rel-path → owner → verifier → fix
    #{self}::MIGRATIONS         # ordered idempotent transforms

    # From the shell:
    pwn setup --migrate                # == #{self}.run
    pwn setup --migrate --fix          # == #{self}.run(fix: true)
    pwn setup --migrate --dry-run

    #{self}.authors
  "
end

.installed_schema ⇒ Object

Supported Method Parameters

schema = PWN::Migrate.installed_schema

→ Integer schema number stamped in ~/.pwn/.schema (0 when absent).



226
227
228
229
230
231
232
 
# File 'lib/pwn/migrate.rb', line 226

public_class_method def self.installed_schema
  return 0 unless File.file?(SCHEMA_FILE)

  JSON.parse(File.read(SCHEMA_FILE))['schema'].to_i
rescue StandardError
  0
end

.needed? ⇒ Boolean

Supported Method Parameters

bool = PWN::Migrate.needed?

Cheap predicate for PWN::Config.refresh_env — schema stamp only, no per-file probes, so it adds ~0 to launch time.

Returns:

  • (Boolean) 


240
241
242
243
244
 
# File 'lib/pwn/migrate.rb', line 240

public_class_method def self.needed?
  File.directory?(PWN_ROOT) && installed_schema < SCHEMA_VERSION
rescue StandardError
  false
end

.run(opts = {}) ⇒ Object

Supported Method Parameters

report = PWN::Migrate.run( fix: 'optional - also autofix incompatible files (default: schema-migrations only)', backup: 'optional - take timestamped backup of ~/.pwn first (default true)', dry_run: 'optional - report what WOULD happen, write nothing (default false)', key: 'optional - vault key (default: read ~/.pwn/pwn.yaml.decryptor)', iv: 'optional - vault iv (default: read ~/.pwn/pwn.yaml.decryptor)', io: 'optional - IO to write to (default $stdout)' )

  1. backup 2. schema migrations installed→current 3. per-file autofix (when fix:true) 4. pwn.yaml key backfill (when fix:true and decryptor available) 5. stamp .schema.


322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
 
# File 'lib/pwn/migrate.rb', line 322

public_class_method def self.run(opts = {})
  fix     = opts[:fix] ? true : false
  dry_run = opts[:dry_run] ? true : false
  backup  = opts.fetch(:backup, true)
  io      = opts[:io] || $stdout

  FileUtils.mkdir_p(PWN_ROOT)
  before = status
  io.puts "PWN::Migrate — pwn #{before[:pwn_version]} · ~/.pwn schema " \
          "#{before[:schema_installed]} → #{before[:schema_current]}" \
          "#{'  (dry-run)' if dry_run}"

  bpath = nil
  if backup && !dry_run && File.directory?(PWN_ROOT)
    bpath = backup!
    io.puts "  backup   : #{bpath}"
  end

  # ── ordered schema migrations ────────────────────────────────────
  applied = []
  ((before[:schema_installed] + 1)..SCHEMA_VERSION).each do |n|
    m = MIGRATIONS[n]
    next unless m

    io.puts "  migrate  : v#{n - 1} → v#{n}"
    m.call(PWN_ROOT, io) unless dry_run
    applied << n
  end

  # ── per-file autofix ─────────────────────────────────────────────
  fixed = []
  if fix
    before[:incompatible].each do |f|
      io.puts "  fix      : #{f[:rel].ljust(22)} [#{f[:fix]}]  #{f[:why]}"
      next if dry_run

      fixed << f.merge(result: apply_fix(file: f))
    end

    # pwn.yaml key backfill runs even when the vault verifier passed —
    # "encrypted" ≠ "has every key this release added".
    vd = backfill_vault(key: opts[:key], iv: opts[:iv], dry_run: dry_run, io: io)
    fixed << vd if vd && vd[:added].to_i.positive?
  end

  stamp! unless dry_run

  after = status
  io.puts
  io.puts "  result   : #{after[:incompatible].length} incompatible remaining " \
          "(was #{before[:incompatible].length}) · schema now #{after[:schema_installed]}"

  { backup: bpath, applied_migrations: applied, fixed: fixed,
    before: before, after: after, dry_run: dry_run }
rescue StandardError => e
  raise e
end

.status ⇒ Object

Supported Method Parameters

rows = PWN::Migrate.status

Per-file compatibility report — the machine-readable half of .check. Never writes. Never raises (each verifier is rescued).



252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
 
# File 'lib/pwn/migrate.rb', line 252

public_class_method def self.status
  files = STATE_FILES.map do |rel, meta|
    path = File.join(PWN_ROOT, rel)
    exists = meta[:kind] == :dir ? File.directory?(path) : File.file?(path)
    raw = exists && meta[:kind] != :dir ? safe_read(path: path) : nil
    v = verify_one(path: path, raw: raw, meta: meta)
    {
      rel: rel, path: path, owner: meta[:owner], kind: meta[:kind],
      exists: exists, size: exists ? File.size?(path) : nil,
      ok: v[:ok], why: v[:why], fix: meta[:fix]
    }
  end
  {
    root: PWN_ROOT,
    schema_installed: installed_schema,
    schema_current: SCHEMA_VERSION,
    pwn_version: (defined?(PWN::VERSION) ? PWN::VERSION : nil),
    needs_migration: needed?,
    files: files,
    incompatible: files.reject { |f| f[:ok] }
  }
rescue StandardError => e
  raise e
end

.vault_drift(opts = {}) ⇒ Object

Supported Method Parameters

drift = PWN::Migrate.vault_drift( key: 'optional - vault key', iv: 'optional - vault iv' )

Deep-diff the user's decrypted pwn.yaml against the current PWN::Config.env_template. → { missing: [dotted.paths], user: … } Returns nil when the decryptor is unavailable (never prompts).



389
390
391
392
393
394
395
396
397
398
399
400
401
 
# File 'lib/pwn/migrate.rb', line 389

public_class_method def self.vault_drift(opts = {})
  key, iv = decryptor(opts)
  return nil unless key && iv

  yaml = File.join(PWN_ROOT, 'pwn.yaml')
  return nil unless File.file?(yaml) && PWN::Plugins::Vault.file_encrypted?(file: yaml)

  user = PWN::Plugins::Vault.dump(file: yaml, key: key, iv: iv)
  tmpl = env_template
  { missing: missing_paths(tmpl: tmpl, user: user), user: user, template: tmpl }
rescue StandardError
  nil
end