Class: Html2rss::Selectors::PostProcessors::SanitizeHtml

Inherits:
Base
  • Object
show all
Defined in:
lib/html2rss/selectors/post_processors/sanitize_html.rb

Overview

Returns sanitized HTML code as String.

It sanitizes by using the sanitize gem with Sanitize::Config::RELAXED.

Furthermore, it adds:

Imagine this HTML structure:

<section>
  Lorem <b>ipsum</b> dolor...
  <iframe src="https://evil.corp/miner"></iframe>
  <script>alert();</script>
</section>

YAML usage example:

selectors:
 description:
   selector: '.section'
   extractor: html
   post_process:
     name: sanitize_html

Would return:

'<p>Lorem <b>ipsum</b> dolor ...</p>'

Constant Summary collapse

DESCRIPTION =

JSON Schema description exported via schema_doc.

'Sanitize HTML (sanitize gem RELAXED plus html2rss defaults: absolute URLs, ' \
'safe link/img attributes, wrap lone images in anchors).'
EXAMPLES =

Example post-process objects for JSON Schema examples.

[
  { 'name' => 'sanitize_html' }
].freeze
TAG_ATTRIBUTES =

See Also:

{
  'a' => {
    'rel' => 'nofollow noopener noreferrer',
    'target' => '_blank'
  },

  'area' => {
    'rel' => 'nofollow noopener noreferrer',
    'target' => '_blank'
  },

  'img' => {
    'referrerpolicy' => 'no-referrer',
    'crossorigin' => 'anonymous',
    'loading' => 'lazy',
    'decoding' => 'async'
  },

  'iframe' => {
    'referrerpolicy' => 'no-referrer',
    'crossorigin' => 'anonymous',
    'loading' => 'lazy',
    'sandbox' => 'allow-same-origin',
    'src' => true,
    'width' => true,
    'height' => true
  },

  'video' => {
    'referrerpolicy' => 'no-referrer',
    'crossorigin' => 'anonymous',
    'preload' => 'none',
    'playsinline' => 'true',
    'controls' => 'true'
  },

  'audio' => {
    'referrerpolicy' => 'no-referrer',
    'crossorigin' => 'anonymous',
    'preload' => 'none'
  }
}.freeze

Instance Attribute Summary

Attributes inherited from Base

#context, #value

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from Base

assert_type, expect_options, #initialize

Constructor Details

This class inherits a constructor from Html2rss::Selectors::PostProcessors::Base

Class Method Details

.get(html, url) ⇒ String?

rubocop:disable ThreadSafety/ClassInstanceVariable

Parameters:

Returns:

  • (String, nil) 


109
110
111
112
113
114
115
116
117
118
119
 
# File 'lib/html2rss/selectors/post_processors/sanitize_html.rb', line 109

def self.get(html, url)
  return nil if String(html).empty?

  @fragment_cache ||= {}
  key = [html, url.to_s]
  return @fragment_cache[key] if @fragment_cache.key?(key)

  @fragment_cache.clear if @fragment_cache.size > 256
  context = Selectors::Context.new(config: { channel: { url: } }, options: {})
  @fragment_cache[key] = new(html, context).get
end

.sanitize_config(channel_url) ⇒ Hash

rubocop:disable Metrics/MethodLength, ThreadSafety/ClassInstanceVariable

Parameters:

Returns:

  • (Hash) —

    the memoized sanitize configuration



126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
 
# File 'lib/html2rss/selectors/post_processors/sanitize_html.rb', line 126

def self.sanitize_config(channel_url)
  @sanitize_configs ||= {}
  @sanitize_configs[channel_url] ||= begin
    config = Sanitize::Config.merge(
      Sanitize::Config::RELAXED,
      attributes: { all: %w[dir lang alt title translate] },
      add_attributes: TAG_ATTRIBUTES,
      transformers: [
        lambda { |env|
          HtmlTransformers::TransformUrlsToAbsoluteOnes.new(channel_url).call(**env)
        },
        ->(env) { HtmlTransformers::WrapImgInA.new.call(**env) }
      ]
    )
    config[:elements].delete('style')
    config.delete(:css)
    config[:elements].push('audio', 'video', 'source')
    config.freeze
  end
end

.schema_doc ⇒ Hash{Symbol => Object}

Returns JSON Schema fragment for this post-processor.

Returns:

  • (Hash{Symbol => Object}) —

    JSON Schema fragment for this post-processor



95
 
# File 'lib/html2rss/selectors/post_processors/sanitize_html.rb', line 95

def self.schema_doc = SchemaDoc.for_post_processor(name: :sanitize_html, klass: self)

.validate_args!(value, context) ⇒ void

This method returns an undefined value.

Parameters:

  • value (String) —

    extracted selector value

  • context (Selectors::Context) —

    post-processor context



100
101
102
 
# File 'lib/html2rss/selectors/post_processors/sanitize_html.rb', line 100

def self.validate_args!(value, context)
  assert_type value, String, :value, context:
end

Instance Method Details

#get ⇒ String?

Returns:

  • (String, nil) 


150
151
152
153
154
155
156
157
158
159
160
161
162
 
# File 'lib/html2rss/selectors/post_processors/sanitize_html.rb', line 150

def get
  # Temporarily replace newlines with a placeholder to preserve them during space collapsing
  temp_value = value.to_s.gsub("\n", ' __NEWLINE_PLACEHOLDER__ ')
  sanitized_html = Sanitize.fragment(temp_value, self.class.sanitize_config(channel_url)).to_s
  sanitized_html.gsub!(/\s+/, ' ')

  # Restore newlines and clean up surrounding whitespace
  sanitized_html.gsub!(/[ \t\r]*__NEWLINE_PLACEHOLDER__[ \t\r]*/, "\n")
  sanitized_html.gsub!(/\n{3,}/, "\n\n")

  sanitized_html.strip!
  sanitized_html.empty? ? nil : sanitized_html
end