Module: Smith::Doctor::Checks::ModelsRegistry

Defined in:
lib/smith/doctor/checks/models_registry.rb

Overview

Validates that every registered agent's resolved model has either:

(a) an explicit application-side Smith::Models.register override, OR
(b) a matching Smith::Models::Inference rule (library-shipped).

If neither, the model gets safe defaults (no thinking, accepts temp, no tool routing) which may silently degrade behavior. Reports the uncovered models so hosts know to register overrides or rely on the safe defaults knowingly.

Class Method Summary collapse

Class Method Details

.ambiguous_reference_description(reference) ⇒ Object 



62
63
64
 
# File 'lib/smith/doctor/checks/models_registry.rb', line 62

def ambiguous_reference_description(reference)
  "#{reference.model_id} (providers: #{registered_providers_for(reference).join(", ")})"
end

.classified_model_references ⇒ Object

Walk Smith::Agent::Registry. For each agent, extract every static model id Smith can know at boot: the primary model "..." value and any static fallback models. Block-form primary models are skipped because they resolve per-attempt, but their static fallbacks still need coverage checks. Check whether find_or_infer returns a custom (non-default) Profile, meaning either an explicit override or an inference rule matched. Returns [uncovered, ambiguous] reference lists.



80
81
82
83
84
85
86
87
88
89
90
91
92
 
# File 'lib/smith/doctor/checks/models_registry.rb', line 80

def classified_model_references
  return [[], []] unless defined?(Smith::Agent::Registry)

  uncovered = []
  ambiguous = []
  static_model_references.uniq(&:key).each do |reference|
    case model_coverage(reference)
    when :uncovered then uncovered << reference
    when :ambiguous then ambiguous << reference
    end
  end
  [uncovered, ambiguous]
end

.covered_model?(reference) ⇒ Boolean

Returns:

  • (Boolean) 


121
122
123
124
 
# File 'lib/smith/doctor/checks/models_registry.rb', line 121

def covered_model?(reference)
  Smith::Models.find(reference.model_id, provider: reference.provider) ||
    inferred_profile_matches?(reference)
end

.inferred_profile_matches?(reference) ⇒ Boolean

Returns:

  • (Boolean) 


126
127
128
129
130
131
 
# File 'lib/smith/doctor/checks/models_registry.rb', line 126

def inferred_profile_matches?(reference)
  return false unless defined?(Smith::Models::Inference)

  profile = Smith::Models::Inference.profile_for(reference.model_id)
  profile && (reference.provider.nil? || profile.provider.to_sym == reference.provider)
end

.inspectable_agent?(agent) ⇒ Boolean

Returns:

  • (Boolean) 


106
107
108
 
# File 'lib/smith/doctor/checks/models_registry.rb', line 106

def inspectable_agent?(agent)
  agent.is_a?(Class) && agent.respond_to?(:chat_kwargs)
end

.model_coverage(reference) ⇒ Object 



94
95
96
97
98
 
# File 'lib/smith/doctor/checks/models_registry.rb', line 94

def model_coverage(reference)
  covered_model?(reference) ? :covered : :uncovered
rescue Smith::Models::AmbiguousProfileError
  :ambiguous
end

.registered_providers_for(reference) ⇒ Object 



66
67
68
69
70
 
# File 'lib/smith/doctor/checks/models_registry.rb', line 66

def registered_providers_for(reference)
  Smith::Models.all
               .select { |profile| profile.model_id == reference.model_id }
               .map { |profile| profile.provider.to_s }
end

.report_ambiguous_models(report, ambiguous) ⇒ Object

An unqualified agent model id registered under more than one provider cannot resolve to a single profile: chat construction fails closed with AmbiguousProfileError. Doctor must report that configuration instead of crashing on it.



48
49
50
51
52
53
54
55
56
57
58
59
60
 
# File 'lib/smith/doctor/checks/models_registry.rb', line 48

def report_ambiguous_models(report, ambiguous)
  return if ambiguous.empty?

  descriptions = ambiguous.map { |reference| ambiguous_reference_description(reference) }
  report.add(
    name: "models.ambiguity",
    status: :fail,
    message: "#{ambiguous.size} agent model id(s) match profiles from multiple registered providers",
    detail: "Ambiguous: #{descriptions.join("; ")}. Chat construction fails closed for " \
            "these agents until each declares an explicit provider (for example " \
            "model \"gpt-5\", provider: :openai) selecting exactly one registered profile."
  )
end

.report_model_coverage(report, uncovered) ⇒ Object 



23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
 
# File 'lib/smith/doctor/checks/models_registry.rb', line 23

def report_model_coverage(report, uncovered)
  if uncovered.empty?
    report.add(
      name: "models.coverage",
      status: :pass,
      message: "All registered agents have model profiles or matching inference rules"
    )
  else
    report.add(
      name: "models.coverage",
      status: :warn,
      message: "#{uncovered.size} agent model(s) without explicit profile or matching inference rule",
      detail: "Uncovered: #{uncovered.join(", ")}. These models will get safe defaults " \
              "(no thinking, accepts temperature, no tool routing). Either register an " \
              "explicit Smith::Models::Profile via Smith::Models.register, OR add an " \
              "Inference rule via Smith::Models::Inference.prepend_rule if the model " \
              "fits an existing provider pattern."
    )
  end
end

.run(report) ⇒ Object 



17
18
19
20
21
 
# File 'lib/smith/doctor/checks/models_registry.rb', line 17

def run(report)
  uncovered, ambiguous = classified_model_references
  report_ambiguous_models(report, ambiguous)
  report_model_coverage(report, uncovered)
end

.static_model_references ⇒ Object 



100
101
102
103
104
 
# File 'lib/smith/doctor/checks/models_registry.rb', line 100

def static_model_references
  Smith::Agent::Registry.each.with_object([]) do |(_key, agent), references|
    references.concat(static_model_references_for(agent)) if inspectable_agent?(agent)
  end
end

.static_model_references_for(agent) ⇒ Object 



110
111
112
113
114
115
116
117
118
119
 
# File 'lib/smith/doctor/checks/models_registry.rb', line 110

def static_model_references_for(agent)
  primary = agent.chat_kwargs[:model]
  references = Array(agent.respond_to?(:fallback_models) ? agent.fallback_models : nil).dup
  if primary
    references.unshift(
      Smith::Agent::ModelReference.coerce(primary, provider: agent.chat_kwargs[:provider])
    )
  end
  references
end