Class FolderScanner

Recursively scans a directory and loads every file whose extension has a registered loader in the supplied LoaderRegistry.

Example

const registry = new LoaderRegistry();
const scanner  = new FolderScanner(registry);

const result = await scanner.scan('/knowledge-base', {
  recursive:  true,
  include:    ['**/*.md', '**/*.pdf'],
  exclude:    ['**/node_modules/**'],
  onProgress: (file, i, total) => console.log(`${i}/${total} ${file}`),
});

console.log(`Loaded ${result.documents.length} documents`);
console.log(`Failed: ${result.failed.length}`);
Copy