Class: CCommentScanner

Inherits:
Object show all
Defined in:
lib/ceedling/preprocess/c_comment_scanner.rb

Defined Under Namespace

Classes: CommentInfo

Constant Summary collapse

PRESERVE_LINES = 
:preserve_lines
COMPACT = 
:compact

Instance Method Summary collapse

Instance Method Details

#remove(content, comment_infos, mode: COMPACT) ⇒ Object

Given the Array from scan, return a copy of content with every comment replaced according to mode:

:compact (default) — every comment replaced by a single space character.
:preserve_lines    — single-line comments (lines_removed == 0) replaced by a
                   single space; multi-line comments replaced by
                   lines_removed newlines so the total line count is unchanged.

Array is processed in descending position order (i.e. backwards) to ensure each comment removal does not disturb earlier comments in the content with respect to CommentInfo details.



42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
 
# File 'lib/ceedling/preprocess/c_comment_scanner.rb', line 42

def remove(content, comment_infos, mode: COMPACT)
  # comment_infos positions and lengths are byte offsets (recorded by scan using
  # StringScanner#pos).  String#[] on a text-encoded string uses character
  # indexing, which diverges from byte indexing when non-ASCII characters appear
  # before a comment.  Working on a binary (ASCII-8BIT) copy keeps indexing
  # byte-accurate throughout.
  orig_encoding = content.encoding
  result = content.b

  # Process in descending position order so earlier byte positions remain valid
  comment_infos.sort_by { |info| -info.position }.each do |info|
    replacement = ' '
    if (mode == PRESERVE_LINES && info.lines_removed > 0)
      replacement = "\n" * info.lines_removed
    end

    result[info.position, info.length] = replacement
  end

  # Re-tag the encoding without converting bytes.  Only ASCII delimiters were
  # removed and only ASCII characters were inserted, so all remaining multi-byte
  # sequences are intact and the byte content is valid in the original encoding.
  return result.force_encoding(orig_encoding)
end

#scan(io:) ⇒ Object

Scan an IO stream (File or StringIO) and return all C comments found. Returns an Array in ascending position order.

Uses StringScanner for a single-pass scan. Recognises string/character literals and prevents comment detection inside them.

Records:

- // single-line comments (with optional backslash continuation lines)
- /* ... */ block comments (including multiline; unterminated at EOF)


76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
 
# File 'lib/ceedling/preprocess/c_comment_scanner.rb', line 76

def scan(io:)
  content = io.read
  return [] if content.nil? || content.empty?

  # StringScanner#pos returns byte offsets, but String#[index, length] uses
  # character-based indexing for text-encoded (e.g. UTF-8) strings.  Non-ASCII
  # characters such as © (2 bytes in UTF-8) cause the two to diverge: a byte
  # offset N maps to character N-1 after one such character, so comment
  # boundaries and newline counts are wrong.  Forcing binary encoding makes
  # String#[] byte-indexed throughout, keeping it in sync with StringScanner.
  content = content.b

  scanner  = StringScanner.new(content)
  comments = []

  until scanner.eos?
    ch = scanner.peek(1)

    case ch
    when '"', "'"
      # Skip string or character literal -- any // or /* */ inside is not a comment
      skip_string_literal(scanner, ch)

    when '/'
      two = scanner.peek(2)

      if two == '//'
        start = scanner.pos
        scan_line_comment(scanner)
        len = scanner.pos - start
        comments << CommentInfo.new(
          position:      start,
          length:        len,
          lines_removed: content[start, len].count("\n")
        )

      elsif two == '/*'
        start = scanner.pos
        scan_block_comment(scanner)
        len = scanner.pos - start
        comments << CommentInfo.new(
          position:      start,
          length:        len,
          lines_removed: content[start, len].count("\n")
        )

      else
        scanner.getch
      end

    else
      scanner.getch
    end
  end

  return comments
end