Module: Metanorma::Standoc::Boilerplate

Includes:
Core::Boilerplate
Included in:
Cleanup
Defined in:
lib/metanorma/cleanup/boilerplate.rb,
lib/metanorma/cleanup/terms_boilerplate.rb,
lib/metanorma/cleanup/boilerplate_liquid.rb

Constant Summary collapse

ADOC_MACRO_PATTERN =

The boilerplate file is in Liquid AsciiDoc format (technically, ‘boilerplate.adoc.liquid`).

This file is processed separately from the main Metanorma document and therefore is oblivious of the ‘concept-mention }` syntax.

Due to historic reasons, the Liquid objects being evaluated in the boilerplate document are XML strings. Notably these are the document metadata, that are extracted from the already generated Metanorma XML.

These XML strings are then passed into the AsciiDoc macros such as ‘span:publisher`.

Here, we need to interpolate the XML strings into the AsciiDoc macros without breaking the AsciiDoc syntax.

EXAMPLE 1: ‘mailto:pub_email }[]`, we need to convert it to: `mailto:pass-format:metanorma[pub_email_xml] }[]`

EXAMPLE 2: ‘link:pub_uri}[pub_address }, pub_uri }]` We need to convert it to: `link:pass-format:metanorma[pub_uri_xml] }[pass-format:metanorma[pub_address_xml] }, pass-format:metanorma[pub_uri_xml] }]`

NOTE: The boilerplate may use macros that contain one or more ‘… }` in the target, and can contain spaces in them.

NOTE: The routine needs to handle cases where the content contains an escaped closing bracket ‘]`.

/\S+:[^\[\n]*\[[^\]\\]*(?:\\.[^\]\\]*)*\]/

Instance Method Summary collapse

Instance Method Details

#boilerplate(xml, conv) ⇒ Object 



129
130
131
132
133
134
135
136
137
138
139
140
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 129

def boilerplate(xml, conv)
  # prevent infinite recursion of asciidoc boilerplate processing
  xml.at("//metanorma-extension/semantic-metadata/" \
         "headless[text() = 'true']") and return nil
  file = boilerplate_file(xml)
  file2 = @boilerplateauthority
  @boilerplateauthority &&
    !(Pathname.new @boilerplateauthority).absolute? and
    file2 = File.join(@localdir, @boilerplateauthority)
  resolve_boilerplate_files(process_boilerplate_file(file, conv),
                            process_boilerplate_file(file2, conv))
end

#boilerplate_cleanup(xmldoc) ⇒ Object 



76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 76

def boilerplate_cleanup(xmldoc)
  isodoc = boilerplate_isodoc(xmldoc) or return
  had_templates = ::Metanorma::Core::Boilerplate
    .docidentifier_templates?(xmldoc)
  docidentifier_boilerplate_isodoc(xmldoc, isodoc)
  had_templates and refresh_isodoc_bibdata(xmldoc, isodoc)
  termdef_boilerplate_cleanup(xmldoc)
  termdef_boilerplate_insert(xmldoc, isodoc)
  unwrap_boilerplate_clauses(xmldoc, self.class::TERM_CLAUSE)
  if f = xmldoc.at(self.class::NORM_REF)
    norm_ref_preface(f, isodoc)
    unwrap_boilerplate_clauses(f, ".")
  end
  initial_boilerplate(xmldoc, isodoc)
end

#boilerplate_file(_xmldoc) ⇒ Object 



124
125
126
127
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 124

def boilerplate_file(_xmldoc)
  ret = File.join(@libdir, "boilerplate.xml")
  File.exist?(ret) ? ret : ""
end

#boilerplate_file_convert(file) ⇒ Object

If Asciidoctor, convert top clauses to tags and wrap in <boilerplate>



201
202
203
204
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 201

def boilerplate_file_convert(file)
  ret = Nokogiri::XML(file).root and return ret
  boilerplate_file_restructure(file)
end

#boilerplate_file_restructure(file) ⇒ Object 



206
207
208
209
210
211
212
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 206

def boilerplate_file_restructure(file)
  ret = adoc2xml(file, @conv.backend&.to_sym)
  boilerplate_xml_cleanup(ret)
  ret.name = "boilerplate"
  boilerplate_top_elements(ret)
  ret
end

#boilerplate_isodoc(xmldoc) ⇒ Object 



48
49
50
51
52
53
54
55
56
57
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 48

def boilerplate_isodoc(xmldoc)
  # prevent infinite recursion of asciidoc boilerplate processing
  # in termdef_boilerplate_insert and initial_boilerplate
  xmldoc.at("//metanorma-extension/semantic-metadata/" \
            "headless[text() = 'true']") and return nil
  @isodoc ||= @conv.isodoc(@lang, @script, @locale)
  @i18n = @isodoc.i18n
  isodoc_bibdata_parse(xmldoc)
  @isodoc
end

#boilerplate_read(file) ⇒ Object

Replace … } with pass-format:metanorma: } to preserve any XML markup provided by Metanorma XML Metadata content, through the ‘pass-format:metanorma` command.

  • If ‘… }` is inside an Asciidoc macro, we have to wrap with pass-format:metanorma:.

  • If this is a macro target (e.g. ‘mailto:{x}[]`, body: mailto:) then do not use pass-format:metanorma.



47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
 
# File 'lib/metanorma/cleanup/boilerplate_liquid.rb', line 47

def boilerplate_read(file)
  ret = File.read(file, encoding: "UTF-8")
  /\.adoc(\.liquid)?$/.match?(file) or return ret

  # Split content into macro and non-macro parts
  parts = ret.split(/(#{ADOC_MACRO_PATTERN})/o)

  parts.map.with_index do |part, index|
    if index.odd? && valid_macro?(part)
      # This is a macro - leave unchanged
      part
    else
      # Not a macro - wrap {{ }} patterns
      part.gsub(/(?<!\{)(\{\{[^{}]+\}\})(?!\})/,
                "pass-format:metanorma[++\\1++]")
    end
  end.join
end

#boilerplate_snippet_cleanup(node) ⇒ Object

Standoc override of the metanorma-core extension hook. Applies namespace cleanup and externally-sourced footnote separation to a Nokogiri node before the snippet is serialised back into the surrounding document.



231
232
233
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 231

def boilerplate_snippet_cleanup(node)
  boilerplate_xml_cleanup(@conv.separate_numbering_footnotes(node))
end

#boilerplate_snippet_convert(adoc, isodoc, **kwargs) ⇒ Object

Standoc-side wrapper around the metanorma-core helper. Existing 2-arg call sites supply no kwargs and inherit them all from instance state; core’s docidentifier_boilerplate_isodoc passes kwargs through and they take precedence.



218
219
220
221
222
223
224
225
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 218

def boilerplate_snippet_convert(adoc, isodoc, **kwargs)
  defaults = {
    lang: @lang, script: @script,
    backend: @conv.backend&.to_sym,
    flush_caches: @flush_caches, localdir: @localdir,
  }
  super(adoc, isodoc, **defaults.merge(kwargs))
end

#docidentifier_boilerplate_isodoc(xmldoc, isodoc) ⇒ Object

Standoc-side wrapper around Core::Boilerplate’s iterator. The core helper owns the loop body (substitute Liquid + Asciidoc, then splice the inner <p> children back); this wrapper supplies standoc-level kwargs from instance state so existing 2-arg callers do not need to thread them through.



109
110
111
112
113
114
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 109

def docidentifier_boilerplate_isodoc(xmldoc, isodoc)
  super(xmldoc, isodoc,
        lang: @lang, script: @script,
        backend: @conv.backend&.to_sym,
        flush_caches: @flush_caches, localdir: @localdir)
end

#dup_with_namespace(elem) ⇒ Object 



42
43
44
45
46
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 42

def dup_with_namespace(elem)
  ret = elem.dup
  ret.add_namespace(nil, @conv.xml_namespace)
  ret
end

#external_terms_boilerplate(sources) ⇒ Object 



4
5
6
7
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 4

def external_terms_boilerplate(sources)
  e = @i18n.external_terms_boilerplate
  e.gsub(/%(?=\p{P}|\p{Z}|$)/, sources || "???")
end

#initial_boilerplate(xml, isodoc) ⇒ Object 



116
117
118
119
120
121
122
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 116

def initial_boilerplate(xml, isodoc)
  xml.at("//boilerplate") and return
  preface = xml.at("//preface | //sections | //annex | //references") or
    return
  b = boilerplate(xml, isodoc) or return
  preface.previous = b
end

#internal_external_terms_boilerplate(sources) ⇒ Object 



9
10
11
12
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 9

def internal_external_terms_boilerplate(sources)
  e = @i18n.internal_external_terms_boilerplate
  e.gsub(/%(?=\p{P}|\p{Z}|$)/, sources || "??")
end

#isodoc_bibdata_parse(xmldoc) ⇒ Object 



59
60
61
62
63
64
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 59

def isodoc_bibdata_parse(xmldoc)
  # initialise @isodoc.xrefs, for @isodoc.xrefs.info
  x = dup_with_namespace(xmldoc.root)
  xml = Nokogiri::XML(x.to_xml)
  @isodoc.bibdata(xml) # do i18n
end

#merge_boilerplate_files(built_in, user_add) ⇒ Object 



161
162
163
164
165
166
167
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 161

def merge_boilerplate_files(built_in, user_add)
  %w(copyright license legal feedback).each do |w|
    resolve_boilerplate_statement(built_in, user_add, w)
    resolve_boilerplate_append(built_in, user_add, w)
  end
  to_xml(built_in)
end

#norm_ref_boilerplate_insert_location(ref) ⇒ Object 



30
31
32
33
34
35
36
37
38
39
40
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 30

def norm_ref_boilerplate_insert_location(ref)
  while (n = ref.parent) && %w(clause references).include?(n.name)
    n.elements.detect do |e|
      !%(title references).include?(e.name) &&
        !e.at("./self::clause[@type = 'boilerplate']") &&
        !e.at("./self::clause[.//references][not(.//clause[not(.//bibitem)])]")
    end and break
    ref = n
  end
  ref.at("./title")
end

#norm_ref_preface(ref, isodoc) ⇒ Object 



9
10
11
12
13
14
15
16
17
18
19
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 9

def norm_ref_preface(ref, isodoc)
  ins = norm_ref_boilerplate_insert_location(ref)
  ins2 = norm_ref_process_boilerplate_note(ref)
  ins2 == :populated and return
  ins2 == :missing or ins = ins2
  refs = ref.elements.select do |e|
    %w(references bibitem).include? e.name
  end
  pref = refs.empty? ? @i18n.norm_empty_pref : @i18n.norm_with_refs_pref
  ins.next = boilerplate_snippet_convert(pref, isodoc)
end

#norm_ref_process_boilerplate_note(ref) ⇒ Object 



21
22
23
24
25
26
27
28
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 21

def norm_ref_process_boilerplate_note(ref)
  ins2 = ref.at("./note[@type = 'boilerplate']") or return :missing
  if ins2 && ins2.text.strip.downcase == "(default)"
    ins2.children = " "
    ins2.children.first
  else :populated
  end
end

#process_boilerplate_file(filename, conv) ⇒ Object 



142
143
144
145
146
147
148
149
150
151
152
153
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 142

def process_boilerplate_file(filename, conv)
  filename.nil? || filename.empty? and return
  filename = filename.strip
  unless File.exist?(filename)
    @log.add("STANDOC_1", nil, params: [filename])
    return
  end

  b = conv.populate_template(boilerplate_read(filename), nil)
    .gsub(/pass-format:metanorma\[\+\+\+\+\]/, "")
  boilerplate_file_convert(b)
end

#refresh_isodoc_bibdata(xmldoc, _isodoc) ⇒ Object

Re-seed isodoc state from the now-resolved xmldoc bibdata after docidentifier_boilerplate_isodoc has substituted any @boilerplate=“true” Liquid templates. Standoc default just re-runs isodoc_bibdata_parse so any cached i18n / meta state picks up the resolved docidentifier values; flavors that populate richer derived state (e.g. metanorma-generic’s bibdata_hash) override to refresh that too. See github.com/metanorma/metanorma/issues/558.



100
101
102
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 100

def refresh_isodoc_bibdata(xmldoc, _isodoc)
  isodoc_bibdata_parse(xmldoc)
end

#resolve_boilerplate_append(built_in, user_add, statement) ⇒ Object 



178
179
180
181
182
183
184
185
186
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 178

def resolve_boilerplate_append(built_in, user_add, statement)
  b = user_add.at("./#{statement}-statement-append") or return
  if a = built_in.at("./#{statement}-statement")
    resolve_boilerplate_append1(a, b, statement)
  else
    b.name = "#{statement}-statement"
    built_in << b
  end
end

#resolve_boilerplate_append1(built_in, user_add, statement) ⇒ Object 



188
189
190
191
192
193
194
195
196
197
198
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 188

def resolve_boilerplate_append1(built_in, user_add, statement)
  if user_add.at("./clause") then built_in << user_add.children
  else
    user_add.name = "clause"
    if user_add["id"].nil? || Metanorma::Utils::guid_anchor?(user_add["id"])
      user_add["anchor"] = "_boilerplate-#{statement}-statement-append"
      add_id(user_add)
    end
    built_in << user_add
  end
end

#resolve_boilerplate_files(built_in, user_add) ⇒ Object 



155
156
157
158
159
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 155

def resolve_boilerplate_files(built_in, user_add)
  built_in || user_add or return
  built_in && user_add or return to_xml(built_in || user_add)
  merge_boilerplate_files(built_in, user_add)
end

#resolve_boilerplate_statement(built_in, user_add, statement) ⇒ Object 



169
170
171
172
173
174
175
176
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 169

def resolve_boilerplate_statement(built_in, user_add, statement)
  b = user_add.at("./#{statement}-statement") or return
  if a = built_in.at("./#{statement}-statement")
    b.text.strip.empty? and a.remove or a.replace(b)
  else
    built_in << b
  end
end

#term_defs_boilerplate(div, source, term, _preface, isodoc) ⇒ Object 



14
15
16
17
18
19
20
21
22
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 14

def term_defs_boilerplate(div, source, term, _preface, isodoc)
  verify_term_defs_source(source)
  a = @i18n.term_def_boilerplate and
    div.next = boilerplate_snippet_convert(a, isodoc)
  a = if source.empty? && term.nil? then @i18n.no_terms_boilerplate
      else term_defs_boilerplate_cont(source, term, isodoc)
      end
  a and div.next = boilerplate_snippet_convert(a, isodoc)
end

#term_defs_boilerplate_cont(src, term, isodoc) ⇒ Object 



31
32
33
34
35
36
37
38
39
40
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 31

def term_defs_boilerplate_cont(src, term, isodoc)
  sources = isodoc.sentence_join(src.map do |s|
    %{&lt;&lt;#{s['bibitemid']}&gt;&gt;}
  end).gsub("&lt;", "<").gsub("&gt;", ">")
  if src.empty? then @i18n.internal_terms_boilerplate
  elsif term.nil? then external_terms_boilerplate(sources)
  else
    internal_external_terms_boilerplate(sources)
  end
end

#termdef_boilerplate_cleanup(xmldoc) ⇒ Object 



42
43
44
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 42

def termdef_boilerplate_cleanup(xmldoc)
  # termdef_remove_initial_paras(xmldoc)
end

#termdef_boilerplate_climb_up(clause, container) ⇒ Object 



71
72
73
74
75
76
77
78
79
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 71

def termdef_boilerplate_climb_up(clause, container)
  container.at(".//*[@id = '#{clause['id']}']") or return clause
  while (n = clause.parent)
    n.at(".//definitions") and break
    clause = n
    n["id"] == container["id"] and break
  end
  clause
end

#termdef_boilerplate_insert(xmldoc, isodoc, once = false) ⇒ Object 



50
51
52
53
54
55
56
57
58
59
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 50

def termdef_boilerplate_insert(xmldoc, isodoc, once = false)
  if once
    f = termdef_boilerplate_insert_location(xmldoc) and
      termdef_boilerplate_insert1(f, xmldoc, isodoc)
  else
    xmldoc.xpath(self.class::TERM_CLAUSE).each do |f|
      termdef_boilerplate_insert1(f, xmldoc, isodoc)
    end
  end
end

#termdef_boilerplate_insert1(sect, xmldoc, isodoc) ⇒ Object 



81
82
83
84
85
86
87
88
89
90
91
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 81

def termdef_boilerplate_insert1(sect, xmldoc, isodoc)
  ins = sect.at("./title")
  if (ins2 = sect.at("./clause[@type = 'boilerplate'] | " \
          "./note[@type = 'boilerplate']"))
    ins2.text.strip.downcase == "(default)" or return
    ins2.children = " "
    ins = ins2.children.first
  end
  term_defs_boilerplate(ins, xmldoc.xpath(".//termdocsource"),
                        sect.at(".//term"), sect.at(".//p"), isodoc)
end

#termdef_boilerplate_insert_location(xmldoc) ⇒ Object 



61
62
63
64
65
66
67
68
69
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 61

def termdef_boilerplate_insert_location(xmldoc)
  f = xmldoc.at(self.class::TERM_CLAUSE)
  root = xmldoc.at("//sections/terms | //sections/clause[@type = 'terms']")
  if f && root && f["id"] != root["id"]
    f = termdef_boilerplate_climb_up(f, root)
  elsif !f && root then f = root
  end
  f
end

#termdef_remove_initial_paras(xmldoc) ⇒ Object 



46
47
48
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 46

def termdef_remove_initial_paras(xmldoc)
  xmldoc.xpath("//terms/p | //terms/ul").each(&:remove)
end

#unwrap_boilerplate_clauses(xmldoc, xpath) ⇒ Object 



66
67
68
69
70
71
72
73
74
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 66

def unwrap_boilerplate_clauses(xmldoc, xpath)
  xmldoc.xpath(xpath).each do |f|
    f.xpath(".//clause[@type = 'boilerplate'] | " \
            ".//note[@type = 'boilerplate']").each do |c|
      c.at("./title")&.remove
      c.replace(c.children)
    end
  end
end

#verify_term_defs_source(source) ⇒ Object 



24
25
26
27
28
29
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 24

def verify_term_defs_source(source)
  source.each do |s|
    @anchors[s["bibitemid"]] or
      @log.add("STANDOC_28", nil, params: [s["bibitemid"]])
  end
end