Module: Documentrix::Documents::Splitters::Common

Included in:

Character, RecursiveCharacter, Semantic

Defined in:

lib/documentrix/documents/splitters/common.rb

Overview

A shared utility module for text splitters that provides consistent handling of chunk size constraints.

This module is intended to be included in splitter classes that implement a maximum chunk size limit. It expects the including class to provide the following attributes:

• force [Boolean]: Whether to hard-split chunks that exceed the limit.
• chunk_size [Integer]: The maximum allowed size for a single chunk.