Class ChunkingEngine

Splits raw document text into an ordered array of DocumentChunk objects suitable for embedding and storage in a vector index.

Example

const engine = new ChunkingEngine();
const chunks = await engine.chunk(content, { strategy: 'fixed', chunkSize: 512 });

Index

Constructors

Methods

Constructors

constructor

new ChunkingEngine(): ChunkingEngine
Returns ChunkingEngine

Methods

chunk

chunk(content, options): Promise<DocumentChunk[]>
Chunks the provided content string according to the given options.

All strategy implementations are async to accommodate the optional embedFn used by the semantic strategy.
Parameters
- content: string
  Full document text to split.
- options: ChunkOptions
  Chunking strategy and tuning parameters.
Returns Promise<DocumentChunk[]>
Ordered array of DocumentChunk objects with sequential indices.
- Defined in src/memory/io/ingestion/ChunkingEngine.ts:229