Module: Metanorma::Standoc::Boilerplate

Included in:
Cleanup
Defined in:
lib/metanorma/cleanup/boilerplate.rb,
lib/metanorma/cleanup/terms_boilerplate.rb,
lib/metanorma/cleanup/boilerplate_liquid.rb

Constant Summary collapse

ADOC_MACRO_PATTERN =

The boilerplate file is in Liquid AsciiDoc format (technically, ‘boilerplate.adoc.liquid`).

This file is processed separately from the main Metanorma document and therefore is oblivious of the ‘concept-mention }` syntax.

Due to historic reasons, the Liquid objects being evaluated in the boilerplate document are XML strings. Notably these are the document metadata, that are extracted from the already generated Metanorma XML.

These XML strings are then passed into the AsciiDoc macros such as ‘span:publisher`.

Here, we need to interpolate the XML strings into the AsciiDoc macros without breaking the AsciiDoc syntax.

EXAMPLE 1: ‘mailto:pub_email }[]`, we need to convert it to: `mailto:pass-format:metanorma[pub_email_xml] }[]`

EXAMPLE 2: ‘link:pub_uri}[pub_address }, pub_uri }]` We need to convert it to: `link:pass-format:metanorma[pub_uri_xml] }[pass-format:metanorma[pub_address_xml] }, pass-format:metanorma[pub_uri_xml] }]`

NOTE: The boilerplate may use macros that contain one or more ‘… }` in the target, and can contain spaces in them.

NOTE: The routine needs to handle cases where the content contains an escaped closing bracket ‘]`.

/\S+:[^\[\n]*\[[^\]\\]*(?:\\.[^\]\\]*)*\]/

Instance Method Summary collapse

Instance Method Details

#boilerplate(xml, conv) ⇒ Object 



111
112
113
114
115
116
117
118
119
120
121
122
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 111

def boilerplate(xml, conv)
  # prevent infinite recursion of asciidoc boilerplate processing
  xml.at("//metanorma-extension/semantic-metadata/" \
         "headless[text() = 'true']") and return nil
  file = boilerplate_file(xml)
  file2 = @boilerplateauthority
  @boilerplateauthority &&
    !(Pathname.new @boilerplateauthority).absolute? and
    file2 = File.join(@localdir, @boilerplateauthority)
  resolve_boilerplate_files(process_boilerplate_file(file, conv),
                            process_boilerplate_file(file2, conv))
end

#boilerplate_cleanup(xmldoc) ⇒ Object 



74
75
76
77
78
79
80
81
82
83
84
85
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 74

def boilerplate_cleanup(xmldoc)
  isodoc = boilerplate_isodoc(xmldoc) or return
  docidentifier_boilerplate_isodoc(xmldoc, isodoc)
  termdef_boilerplate_cleanup(xmldoc)
  termdef_boilerplate_insert(xmldoc, isodoc)
  unwrap_boilerplate_clauses(xmldoc, self.class::TERM_CLAUSE)
  if f = xmldoc.at(self.class::NORM_REF)
    norm_ref_preface(f, isodoc)
    unwrap_boilerplate_clauses(f, ".")
  end
  initial_boilerplate(xmldoc, isodoc)
end

#boilerplate_file(_xmldoc) ⇒ Object 



106
107
108
109
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 106

def boilerplate_file(_xmldoc)
  ret = File.join(@libdir, "boilerplate.xml")
  File.exist?(ret) ? ret : ""
end

#boilerplate_file_convert(file) ⇒ Object

If Asciidoctor, convert top clauses to tags and wrap in <boilerplate>



183
184
185
186
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 183

def boilerplate_file_convert(file)
  ret = Nokogiri::XML(file).root and return ret
  boilerplate_file_restructure(file)
end

#boilerplate_file_restructure(file) ⇒ Object 



188
189
190
191
192
193
194
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 188

def boilerplate_file_restructure(file)
  ret = adoc2xml(file, @conv.backend.to_sym)
  boilerplate_xml_cleanup(ret)
  ret.name = "boilerplate"
  boilerplate_top_elements(ret)
  ret
end

#boilerplate_isodoc(xmldoc) ⇒ Object 



46
47
48
49
50
51
52
53
54
55
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 46

def boilerplate_isodoc(xmldoc)
  # prevent infinite recursion of asciidoc boilerplate processing
  # in termdef_boilerplate_insert and initial_boilerplate
  xmldoc.at("//metanorma-extension/semantic-metadata/" \
            "headless[text() = 'true']") and return nil
  @isodoc ||= @conv.isodoc(@lang, @script, @locale)
  @i18n = @isodoc.i18n
  isodoc_bibdata_parse(xmldoc)
  @isodoc
end

#boilerplate_read(file) ⇒ Object

Replace … } with pass-format:metanorma: } to preserve any XML markup provided by Metanorma XML Metadata content, through the ‘pass-format:metanorma` command.

  • If ‘… }` is inside an Asciidoc macro, we have to wrap with pass-format:metanorma:.

  • If this is a macro target (e.g. ‘mailto:{x}[]`, body: mailto:) then do not use pass-format:metanorma.



47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
 
# File 'lib/metanorma/cleanup/boilerplate_liquid.rb', line 47

def boilerplate_read(file)
  ret = File.read(file, encoding: "UTF-8")
  /\.adoc(\.liquid)?$/.match?(file) or return ret

  # Split content into macro and non-macro parts
  parts = ret.split(/(#{ADOC_MACRO_PATTERN})/o)

  parts.map.with_index do |part, index|
    if index.odd? && valid_macro?(part)
      # This is a macro - leave unchanged
      part
    else
      # Not a macro - wrap {{ }} patterns
      part.gsub(/(?<!\{)(\{\{[^{}]+\}\})(?!\})/,
                "pass-format:metanorma[++\\1++]")
    end
  end.join
end

#boilerplate_snippet_convert(adoc, isodoc) ⇒ Object 



196
197
198
199
200
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 196

def boilerplate_snippet_convert(adoc, isodoc)
  b = isodoc.populate_template(adoc, nil)
  ret = boilerplate_xml_cleanup(adoc2xml(b, @conv.backend.to_sym))
  @i18n.l10n(ret.children.to_xml, @lang, @script).strip
end

#docidentifier_boilerplate_isodoc(xmldoc, isodoc) ⇒ Object 



87
88
89
90
91
92
93
94
95
96
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 87

def docidentifier_boilerplate_isodoc(xmldoc, isodoc)
  xmldoc.xpath("//docidentifier[@boilerplate]").each do |d|
    b = d["boilerplate"] == "true"
    d.delete("boilerplate")
    b or next
    id = boilerplate_snippet_convert(to_xml(d.children), isodoc)
    p = Nokogiri::XML(id).at("//p")
    d.children = p ? to_xml(p&.children) : id
  end
end

#dup_with_namespace(elem) ⇒ Object 



40
41
42
43
44
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 40

def dup_with_namespace(elem)
  ret = elem.dup
  ret.add_namespace(nil, @conv.xml_namespace)
  ret
end

#external_terms_boilerplate(sources) ⇒ Object 



4
5
6
7
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 4

def external_terms_boilerplate(sources)
  e = @i18n.external_terms_boilerplate
  e.gsub(/%(?=\p{P}|\p{Z}|$)/, sources || "???")
end

#initial_boilerplate(xml, isodoc) ⇒ Object 



98
99
100
101
102
103
104
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 98

def initial_boilerplate(xml, isodoc)
  xml.at("//boilerplate") and return
  preface = xml.at("//preface | //sections | //annex | //references") or
    return
  b = boilerplate(xml, isodoc) or return
  preface.previous = b
end

#internal_external_terms_boilerplate(sources) ⇒ Object 



9
10
11
12
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 9

def internal_external_terms_boilerplate(sources)
  e = @i18n.internal_external_terms_boilerplate
  e.gsub(/%(?=\p{P}|\p{Z}|$)/, sources || "??")
end

#isodoc_bibdata_parse(xmldoc) ⇒ Object 



57
58
59
60
61
62
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 57

def isodoc_bibdata_parse(xmldoc)
  # initialise @isodoc.xrefs, for @isodoc.xrefs.info
  x = dup_with_namespace(xmldoc.root)
  xml = Nokogiri::XML(x.to_xml)
  @isodoc.bibdata(xml) # do i18n
end

#merge_boilerplate_files(built_in, user_add) ⇒ Object 



143
144
145
146
147
148
149
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 143

def merge_boilerplate_files(built_in, user_add)
  %w(copyright license legal feedback).each do |w|
    resolve_boilerplate_statement(built_in, user_add, w)
    resolve_boilerplate_append(built_in, user_add, w)
  end
  to_xml(built_in)
end

#norm_ref_boilerplate_insert_location(ref) ⇒ Object 



28
29
30
31
32
33
34
35
36
37
38
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 28

def norm_ref_boilerplate_insert_location(ref)
  while (n = ref.parent) && %w(clause references).include?(n.name)
    n.elements.detect do |e|
      !%(title references).include?(e.name) &&
        !e.at("./self::clause[@type = 'boilerplate']") &&
        !e.at("./self::clause[.//references][not(.//clause[not(.//bibitem)])]")
    end and break
    ref = n
  end
  ref.at("./title")
end

#norm_ref_preface(ref, isodoc) ⇒ Object 



7
8
9
10
11
12
13
14
15
16
17
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 7

def norm_ref_preface(ref, isodoc)
  ins = norm_ref_boilerplate_insert_location(ref)
  ins2 = norm_ref_process_boilerplate_note(ref)
  ins2 == :populated and return
  ins2 == :missing or ins = ins2
  refs = ref.elements.select do |e|
    %w(references bibitem).include? e.name
  end
  pref = refs.empty? ? @i18n.norm_empty_pref : @i18n.norm_with_refs_pref
  ins.next = boilerplate_snippet_convert(pref, isodoc)
end

#norm_ref_process_boilerplate_note(ref) ⇒ Object 



19
20
21
22
23
24
25
26
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 19

def norm_ref_process_boilerplate_note(ref)
  ins2 = ref.at("./note[@type = 'boilerplate']") or return :missing
  if ins2 && ins2.text.strip.downcase == "(default)"
    ins2.children = " "
    ins2.children.first
  else :populated
  end
end

#process_boilerplate_file(filename, conv) ⇒ Object 



124
125
126
127
128
129
130
131
132
133
134
135
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 124

def process_boilerplate_file(filename, conv)
  filename.nil? || filename.empty? and return
  filename = filename.strip
  unless File.exist?(filename)
    @log.add("STANDOC_1", nil, params: [filename])
    return
  end

  b = conv.populate_template(boilerplate_read(filename), nil)
    .gsub(/pass-format:metanorma\[\+\+\+\+\]/, "")
  boilerplate_file_convert(b)
end

#resolve_boilerplate_append(built_in, user_add, statement) ⇒ Object 



160
161
162
163
164
165
166
167
168
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 160

def resolve_boilerplate_append(built_in, user_add, statement)
  b = user_add.at("./#{statement}-statement-append") or return
  if a = built_in.at("./#{statement}-statement")
    resolve_boilerplate_append1(a, b, statement)
  else
    b.name = "#{statement}-statement"
    built_in << b
  end
end

#resolve_boilerplate_append1(built_in, user_add, statement) ⇒ Object 



170
171
172
173
174
175
176
177
178
179
180
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 170

def resolve_boilerplate_append1(built_in, user_add, statement)
  if user_add.at("./clause") then built_in << user_add.children
  else
    user_add.name = "clause"
    if user_add["id"].nil? || Metanorma::Utils::guid_anchor?(user_add["id"])
      user_add["anchor"] = "_boilerplate-#{statement}-statement-append"
      add_id(user_add)
    end
    built_in << user_add
  end
end

#resolve_boilerplate_files(built_in, user_add) ⇒ Object 



137
138
139
140
141
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 137

def resolve_boilerplate_files(built_in, user_add)
  built_in || user_add or return
  built_in && user_add or return to_xml(built_in || user_add)
  merge_boilerplate_files(built_in, user_add)
end

#resolve_boilerplate_statement(built_in, user_add, statement) ⇒ Object 



151
152
153
154
155
156
157
158
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 151

def resolve_boilerplate_statement(built_in, user_add, statement)
  b = user_add.at("./#{statement}-statement") or return
  if a = built_in.at("./#{statement}-statement")
    b.text.strip.empty? and a.remove or a.replace(b)
  else
    built_in << b
  end
end

#term_defs_boilerplate(div, source, term, _preface, isodoc) ⇒ Object 



14
15
16
17
18
19
20
21
22
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 14

def term_defs_boilerplate(div, source, term, _preface, isodoc)
  verify_term_defs_source(source)
  a = @i18n.term_def_boilerplate and
    div.next = boilerplate_snippet_convert(a, isodoc)
  a = if source.empty? && term.nil? then @i18n.no_terms_boilerplate
      else term_defs_boilerplate_cont(source, term, isodoc)
      end
  a and div.next = boilerplate_snippet_convert(a, isodoc)
end

#term_defs_boilerplate_cont(src, term, isodoc) ⇒ Object 



31
32
33
34
35
36
37
38
39
40
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 31

def term_defs_boilerplate_cont(src, term, isodoc)
  sources = isodoc.sentence_join(src.map do |s|
    %{&lt;&lt;#{s['bibitemid']}&gt;&gt;}
  end).gsub("&lt;", "<").gsub("&gt;", ">")
  if src.empty? then @i18n.internal_terms_boilerplate
  elsif term.nil? then external_terms_boilerplate(sources)
  else
    internal_external_terms_boilerplate(sources)
  end
end

#termdef_boilerplate_cleanup(xmldoc) ⇒ Object 



42
43
44
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 42

def termdef_boilerplate_cleanup(xmldoc)
  # termdef_remove_initial_paras(xmldoc)
end

#termdef_boilerplate_climb_up(clause, container) ⇒ Object 



71
72
73
74
75
76
77
78
79
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 71

def termdef_boilerplate_climb_up(clause, container)
  container.at(".//*[@id = '#{clause['id']}']") or return clause
  while (n = clause.parent)
    n.at(".//definitions") and break
    clause = n
    n["id"] == container["id"] and break
  end
  clause
end

#termdef_boilerplate_insert(xmldoc, isodoc, once = false) ⇒ Object 



50
51
52
53
54
55
56
57
58
59
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 50

def termdef_boilerplate_insert(xmldoc, isodoc, once = false)
  if once
    f = termdef_boilerplate_insert_location(xmldoc) and
      termdef_boilerplate_insert1(f, xmldoc, isodoc)
  else
    xmldoc.xpath(self.class::TERM_CLAUSE).each do |f|
      termdef_boilerplate_insert1(f, xmldoc, isodoc)
    end
  end
end

#termdef_boilerplate_insert1(sect, xmldoc, isodoc) ⇒ Object 



81
82
83
84
85
86
87
88
89
90
91
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 81

def termdef_boilerplate_insert1(sect, xmldoc, isodoc)
  ins = sect.at("./title")
  if (ins2 = sect.at("./clause[@type = 'boilerplate'] | " \
          "./note[@type = 'boilerplate']"))
    ins2.text.strip.downcase == "(default)" or return
    ins2.children = " "
    ins = ins2.children.first
  end
  term_defs_boilerplate(ins, xmldoc.xpath(".//termdocsource"),
                        sect.at(".//term"), sect.at(".//p"), isodoc)
end

#termdef_boilerplate_insert_location(xmldoc) ⇒ Object 



61
62
63
64
65
66
67
68
69
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 61

def termdef_boilerplate_insert_location(xmldoc)
  f = xmldoc.at(self.class::TERM_CLAUSE)
  root = xmldoc.at("//sections/terms | //sections/clause[@type = 'terms']")
  if f && root && f["id"] != root["id"]
    f = termdef_boilerplate_climb_up(f, root)
  elsif !f && root then f = root
  end
  f
end

#termdef_remove_initial_paras(xmldoc) ⇒ Object 



46
47
48
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 46

def termdef_remove_initial_paras(xmldoc)
  xmldoc.xpath("//terms/p | //terms/ul").each(&:remove)
end

#unwrap_boilerplate_clauses(xmldoc, xpath) ⇒ Object 



64
65
66
67
68
69
70
71
72
 
# File 'lib/metanorma/cleanup/boilerplate.rb', line 64

def unwrap_boilerplate_clauses(xmldoc, xpath)
  xmldoc.xpath(xpath).each do |f|
    f.xpath(".//clause[@type = 'boilerplate'] | " \
            ".//note[@type = 'boilerplate']").each do |c|
      c.at("./title")&.remove
      c.replace(c.children)
    end
  end
end

#verify_term_defs_source(source) ⇒ Object 



24
25
26
27
28
29
 
# File 'lib/metanorma/cleanup/terms_boilerplate.rb', line 24

def verify_term_defs_source(source)
  source.each do |s|
    @anchors[s["bibitemid"]] or
      @log.add("STANDOC_28", nil, params: [s["bibitemid"]])
  end
end