Class: CExtractorCodeText

Inherits:
Object show all
Includes:
CExtractorConstants
Defined in:
lib/ceedling/c_extractor/c_extractor_code_text.rb

Constant Summary

Constants included from CExtractorConstants

CExtractorConstants::C11_SPECIFIER_KEYWORDS, CExtractorConstants::DECORATOR_KEYWORDS, CExtractorConstants::DEFAULT_CHUNK_SIZE, CExtractorConstants::DEFAULT_MAX_FUNCTION_LENGTH, CExtractorConstants::DEFAULT_MAX_LINE_LENGTH, CExtractorConstants::MODIFIER_KEYWORDS, CExtractorConstants::MSVC_CALLING_CONVENTIONS, CExtractorConstants::PRIVATE_KEYWORDS, CExtractorConstants::TYPE_KEYWORDS, CExtractorConstants::TYPE_QUALIFIER_KEYWORDS

Instance Method Summary collapse

Instance Method Details

#collect_balanced(scanner, open_char, close_char) ⇒ Array(Boolean, String|nil)

Collect the full text of a balanced delimiter pair starting AT open_char. Nested pairs, string literals (verbatim), and comments (replaced with a single space) are handled correctly. Returns [true, text_including_delimiters] or [false, nil] on unbalanced input or EOS before the matching close delimiter is found.

Works for any single-character delimiter pair: '{}', '()', or '[]'.

Parameters:

  • scanner (StringScanner) —

    positioned at open_char

  • open_char (String) —

    single-character opening delimiter

  • close_char (String) —

    single-character closing delimiter

Returns:

  • (Array(Boolean, String|nil)) 


27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
 
# File 'lib/ceedling/c_extractor/c_extractor_code_text.rb', line 27

def collect_balanced(scanner, open_char, close_char)
  return [false, nil] unless scanner.peek(1) == open_char

  text  = +scanner.getch  # consume and record open_char
  depth = 1

  until scanner.eos?
    ch = scanner.peek(1)

    if ch == '"' || ch == "'"
      before = scanner.pos
      skip_c_string(scanner, ch)
      text << scanner.string[before...scanner.pos]
    elsif scanner.check(%r{/[/*]})
      skip_comment(scanner)
      text << ' '
    elsif ch == open_char
      depth += 1
      text  << scanner.getch
    elsif ch == close_char
      depth -= 1
      text  << scanner.getch
      return [true, text] if depth == 0
    else
      text << scanner.getch
    end
  end

  [false, nil]
end

#extract_balanced_braces(scanner) ⇒ Array(Boolean, String|nil)

Extract a balanced block of braces from the scanner. Delegates to collect_balanced() — see its documentation for full details.

Parameters:

  • scanner (StringScanner) —

    positioned at the opening '{'

Returns:

  • (Array(Boolean, String|nil)) 


63
64
65
 
# File 'lib/ceedling/c_extractor/c_extractor_code_text.rb', line 63

def extract_balanced_braces(scanner)
  collect_balanced(scanner, '{', '}')
end

#skip_c_string(scanner, quote) ⇒ Object

Skip a C string or character literal Handles escape sequences to avoid false termination on escaped quotes

Parameters:

scanner: StringScanner positioned at the opening quote
quote: The quote character (either '"' for strings or "'" for characters)

Returns: Number of bytes skipped (including opening and closing quotes)

Side effects: Advances scanner position past the closing quote (or to end of string if unterminated)

Examples:

"hello"       -> skips 7 bytes
'a'           -> skips 3 bytes
"say \"hi\""  -> skips 11 bytes (handles escaped quotes)
"path\\file"  -> skips 11 bytes (handles escaped backslashes)


83
84
85
86
87
88
89
90
91
92
93
94
95
96
 
# File 'lib/ceedling/c_extractor/c_extractor_code_text.rb', line 83

def skip_c_string(scanner, quote)
  start_pos = scanner.pos
  scanner.getch # Opening quote
  
  until scanner.eos?
    if scanner.scan(/\\/)
      scanner.getch
    elsif scanner.getch == quote
      break
    end
  end
  
  return (scanner.pos - start_pos)
end

#skip_comment(scanner) ⇒ Object 



184
185
186
187
188
189
190
191
192
 
# File 'lib/ceedling/c_extractor/c_extractor_code_text.rb', line 184

def skip_comment(scanner)
  # Single line comment
  if scanner.scan(%r{//})
    scanner.skip_until(/\n/) || scanner.terminate
  # Multiline comment
  elsif scanner.scan(%r{/\*})
    scanner.skip_until(%r{\*/}) || scanner.terminate
  end
end

#skip_compiler_extension(scanner) ⇒ Object

Skip a single compiler extension at the current scanner position. Uses collect_balanced() for paren matching so all nesting depths are handled. Returns true and advances the scanner if an extension is found; returns false without moving if the current position is not the start of a known compiler extension.

Handles:

__word__(…) — any double-underscore attribute form, e.g. __attribute__((…))
__declspec(…) — MSVC declaration specifier, including nested forms
Bare MSVC calling-convention keywords (__cdecl, __stdcall, etc.)

Does NOT skip __int64, __int32, or any non-extension __ identifiers.



205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
 
# File 'lib/ceedling/c_extractor/c_extractor_code_text.rb', line 205

def skip_compiler_extension(scanner)
  if scanner.check(/__\w+__\s*\(/)
    scanner.skip(/__\w+__/)
    skip_deadspace(scanner)
    collect_balanced(scanner, '(', ')') if scanner.peek(1) == '('
    return true
  end

  if scanner.check(/__declspec\s*\(/)
    scanner.skip(/__declspec/)
    skip_deadspace(scanner)
    collect_balanced(scanner, '(', ')') if scanner.peek(1) == '('
    return true
  end

  MSVC_CALLING_CONVENTIONS.each do |kw|
    if scanner.check(/#{Regexp.escape(kw)}\b/)
      scanner.skip(/#{Regexp.escape(kw)}/)
      return true
    end
  end

  false
end

#skip_deadspace(scanner) ⇒ Object

Skip "deadspace" - non-code elements that should be ignored during extraction Deadspace includes:

- Whitespace (spaces, tabs, newlines, carriage returns)
- Comments (both single-line // and multi-line /* */)

NOTE: Preprocessing directives (lines starting with #) are NOT deadspace — they are first-class features handled by CExtractorPreprocessing.

This method repeatedly scans for and skips these elements until no more are found, ensuring all consecutive deadspace is consumed in a single call.

Parameters:

scanner: StringScanner positioned at potential deadspace

Returns: Number of bytes skipped

Side effects: Advances scanner position past all consecutive deadspace

Examples:

"   \n// comment\ncode" -> skips to "code"
"/* block */  \t\ncode" -> skips to "code"
"code"                  -> skips 0 bytes (no deadspace)


165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
 
# File 'lib/ceedling/c_extractor/c_extractor_code_text.rb', line 165

def skip_deadspace(scanner)
  start_pos = scanner.pos

  loop do
    initial = scanner.pos

    # Skip whitespace
    scanner.skip(/\s+/)

    # Skip comments
    skip_comment(scanner) if scanner.check(%r{/[/*]})

    # If nothing was skipped, we're done
    break if scanner.pos == initial
  end

  return (scanner.pos - start_pos)
end

#skip_semicolons(scanner) ⇒ Object

Skip consecutive semicolons and any intervening deadspace

This method handles cases where multiple semicolons appear in sequence, potentially separated by whitespace or comments. This is valid C syntax (null statements) and can occur due to:

- Macro expansions
- Code generation
- Coding mistakes that don't cause compilation errors

The method repeatedly:

1. Skips any deadspace (whitespace and comments)
2. Checks for a semicolon
3. If found, consumes it and continues
4. If not found, restores position and exits

Parameters:

scanner: StringScanner positioned at potential semicolons/deadspace

Returns: Nothing (void method)

Side effects: Advances scanner position past all consecutive semicolons and deadspace

Examples:

";;;"               -> skips all three semicolons
"; ; ;"             -> skips semicolons and spaces
"; /* comment */ ;" -> skips semicolons and comment
"; code"            -> skips first semicolon, stops at "code"
"code"              -> skips nothing


126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
 
# File 'lib/ceedling/c_extractor/c_extractor_code_text.rb', line 126

def skip_semicolons(scanner)
  while !scanner.eos?
    start_pos = scanner.pos
    skip_deadspace(scanner)
    break if scanner.eos?
    
    # If we find a semicolon, consume it and continue
    if scanner.scan(/;/)
      next
    else
      # Not a semicolon, restore position and break
      scanner.pos = start_pos
      break
    end
  end
end

#strip_compiler_extensions(text) ⇒ Object

Strip all compiler extensions from a string and return the cleaned result. Uses an internal StringScanner plus collect_balanced() so all paren nesting depths are handled correctly (e.g. __declspec(align(8)), attribute((format(printf,1,2)))). Whitelist-based: only known forms are stripped; __int64, __int32, and any other non-extension __ identifiers are preserved verbatim. Whitespace is normalized to single spaces and the result is stripped.

Handles:

__word__(…) — any double-underscore attribute form
__declspec(…) — MSVC declaration specifier (including nested parens)
Bare MSVC calling conventions, MSVC inline hints, C11 specifier keywords


241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
 
# File 'lib/ceedling/c_extractor/c_extractor_code_text.rb', line 241

def strip_compiler_extensions(text)
  scanner = StringScanner.new(text)
  result  = +""

  bare_strip = MSVC_CALLING_CONVENTIONS +
               ['__forceinline', '__inline__', '__inline'] +
               C11_SPECIFIER_KEYWORDS

  until scanner.eos?
    # __word__(…) — any double-underscore attribute form including __attribute__((…))
    if scanner.check(/__\w+__\s*\(/)
      scanner.skip(/__\w+__/)
      skip_deadspace(scanner)
      collect_balanced(scanner, '(', ')') if scanner.peek(1) == '('
      next
    end

    # __declspec(…) — handles nested forms like __declspec(align(8))
    if scanner.check(/__declspec\s*\(/)
      scanner.skip(/__declspec/)
      skip_deadspace(scanner)
      collect_balanced(scanner, '(', ')') if scanner.peek(1) == '('
      next
    end

    # Whitelisted bare keywords (calling conventions, inline hints, C11 specifiers)
    stripped = bare_strip.any? do |kw|
      if scanner.check(/#{Regexp.escape(kw)}\b/)
        scanner.skip(/#{Regexp.escape(kw)}/)
        true
      end
    end
    next if stripped

    result << scanner.getch
  end

  result.gsub!(/\s+/, ' ')
  result.strip!
  result
end