Class: Iriq::CLI

Inherits:
Object
  • Object
show all
Defined in:
lib/iriq/cli.rb

Overview

Flag-driven CLI. The default action for an input is a combined parse + normalize + explain summary; the -p/-n/-e flags select individual sections. The only subcommand is cluster, which is structurally different (many inputs, not one). Construct with explicit IO so specs can run it without shelling out.

Constant Summary collapse

SECTION_FLAGS = 
%i[parse normalize].freeze
TOP_N_STATS = 
10
LARGE_BATCH_THRESHOLD =

When extraction yields this many or more IRIs, the default pipe output switches from a URL list to clusters — a longer list is easier to read as route-shape groups.

10
USAGE = 
<<~TXT
  iriq — find a URL's shape: the route template behind it (e.g. /users/{id}).

  Usage: iriq [options] <input>
         iriq [options] < text
         iriq cluster [options] [file]

  <input> may be an IRI, a file path (extracted automatically), or piped
  text via stdin.

  Sections (combine freely):
    -n, --normalize       Shape — variable parts become placeholders
    -c, --canonical       Clean form — tidy scheme/host, keep the values
    -p, --parse           Parsed fields
    -e, --explain         Annotated trace — per-segment notes about why
                          each placeholder / canonical value was chosen

  Corpus + stats:
        --corpus PATH     Use a specific corpus file (overrides the default).
                          Extension picks the backend: .db/.sqlite/.sqlite3
                          are SQLite; anything else is JSON.
    -C, --no-corpus       Disable corpus persistence for this invocation.
                          Same as IRIQ_NO_CORPUS=1 in the environment.
        --reset           Delete the corpus database (default path or the
                          one resolved via --corpus / IRIQ_CORPUS) and exit.
        --host MODE       Host-keying strategy for clustering:
                          full (default), registrable (or reg) strips
                          subdomains, none ignores host entirely.
        --stats           Print rolling aggregates
        --reinfer         Replay the source-IRI log through the current
                          classifier + reducers; rebuilds materialized
                          views from scratch.
        --propose-recognizers
                          Scan observed values for shape patterns that
                          recur enough to suggest a new Recognizer.
                          Combine with --json for structured output.
        --cross-host-shapes
                          List route shapes that recur across
                          multiple hosts. Combine with --min-hosts.
        --activate-above F  With --propose-recognizers, promote every
                          proposal at or above CONFIDENCE F into a
                          live Recognizer on the corpus, then
                          reinfer. Confidence integrates coverage
                          and cross-host corroboration.

  Environment:
        IRIQ_CORPUS=PATH    Set the corpus path (overrides the default).
        IRIQ_NO_CORPUS=1    Disable the default corpus (equivalent to -C).

  Thresholds (apply to --propose-recognizers / --cross-host-shapes):
        --min-observations N  proposal noise floor (default 20)
        --min-coverage F      proposal coverage floor (default 0.7)
        --min-hosts N         proposal: minimum hosts (default 1);
                              cross-host-shapes: minimum hosts to
                              list (default 2)

  Other:
    -h, --help            Show this message
    -j, --json            Emit JSON instead of human-readable output
    -J, --ndjson          Newline-delimited JSON (one object per line). Implies --json.
    -N, --no-hints        Use {integer} placeholders instead of {user_id}
        --no-scheme-less  Skip foo.com/path extraction (explicit-scheme only)
    -V, --version         Print version

  Subcommands:
    cluster [file]        Force cluster view (default for ≥10 IRIs anyway)
    completion <shell>    Print shell completion script (bash | zsh)

  Examples:
    iriq foo.com/users/456
    iriq -n https://foo.com/users/123
    iriq ./access.log                     # auto-detect file → extract URLs
    cat README.md | iriq -n               # one normalized URL per line
    tail -f access.log | iriq -J          # live stream → NDJSON per IRI
    cat README.md | iriq --corpus c.json
TXT

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(stdin: $stdin, stdout: $stdout, stderr: $stderr) ⇒ CLI

Returns a new instance of CLI.



100
101
102
103
104
 
# File 'lib/iriq/cli.rb', line 100

def initialize(stdin: $stdin, stdout: $stdout, stderr: $stderr)
  @stdin  = stdin
  @stdout = stdout
  @stderr = stderr
end

Instance Attribute Details

#stderr ⇒ Object (readonly)

Returns the value of attribute stderr.



98
99
100
 
# File 'lib/iriq/cli.rb', line 98

def stderr
  @stderr
end

#stdin ⇒ Object (readonly)

Returns the value of attribute stdin.



98
99
100
 
# File 'lib/iriq/cli.rb', line 98

def stdin
  @stdin
end

#stdout ⇒ Object (readonly)

Returns the value of attribute stdout.



98
99
100
 
# File 'lib/iriq/cli.rb', line 98

def stdout
  @stdout
end

Instance Method Details

#parseable_iri?(input) ⇒ Boolean

Returns:

  • (Boolean) 


170
171
172
173
174
175
 
# File 'lib/iriq/cli.rb', line 170

def parseable_iri?(input)
  Iriq.parse(input)
  true
rescue Iriq::ParseError
  false
end

#run(argv) ⇒ Object

Returns an integer exit code.



107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
 
# File 'lib/iriq/cli.rb', line 107

def run(argv)
  # Pre-scan so an error during option parsing can still honor --json.
  # Re-set authoritatively from opts once parsing succeeds.
  @json = json_requested?(argv)
  args, opts = parse_options(argv)
  @json = opts[:json]

  return print_usage(stdout, 0) if opts[:help]
  return print_version          if opts[:version]

  # `iriq completion <shell>` short-circuits — no corpus, no IRI input,
  # just emit the script bundled with the gem.
  if args.first == "completion"
    args.shift
    return cmd_completion(args)
  end

  explicit_cluster = (args.first == "cluster")
  args.shift if explicit_cluster

  # Auto-detect: a positional argument that isn't parseable as an IRI
  # but IS an existing file gets treated as a file to extract from. This
  # is what makes `iriq ./access.log` and `iriq /var/log/foo.log` Just
  # Work without a separate --extract flag.
  positional_is_file = args.first && File.file?(args.first) && !parseable_iri?(args.first)

  batch_mode = explicit_cluster || positional_is_file ||
               (args.empty? && piped_stdin?)

  # --reset short-circuits: delete the resolved corpus file (+ sidecars)
  # and exit. Resolves through the same precedence chain as the normal
  # path so `--reset --corpus other.db` and `IRIQ_CORPUS=… --reset` Just Work.
  if opts[:reset]
    return cmd_reset(opts)
  end

  return print_usage(stdout, 0) if args.empty? && !batch_mode && !opts[:reinfer] && !opts[:propose] && !opts[:cross_host_shapes]

  corpus_path = resolve_corpus_path(opts)
  corpus = corpus_path ? load_corpus(corpus_path, host_strategy: opts[:host_strategy], announce_create: true) : nil

  code = if opts[:reinfer]
    cmd_reinfer(corpus, opts)
  elsif opts[:propose]
    cmd_propose(corpus, opts)
  elsif opts[:cross_host_shapes]
    cmd_cross_host_shapes(corpus, opts)
  elsif batch_mode
    cmd_batch(args, opts, corpus, explicit_cluster: explicit_cluster)
  elsif opts[:stats]
    cmd_stats(corpus, opts)
  else
    cmd_summary(args, opts, corpus)
  end

  corpus.save(corpus_path) if corpus && corpus_path
  code
rescue Iriq::ParseError => e
  emit_error("parse_error", e.message, 2, human: "iriq: parse error: #{e.message}")
rescue OptionParser::ParseError => e
  emit_error("option_error", e.message, 1)
end