TextSearch: App-side full-text index
Build a local index over declared fields; every query must supply an application filter.
Constructor options include indexRevision, fields, and storedFields, plus optional tokenize, boost, and combineWith. upsert adds or replaces by document id. search({query,limit,filter}) returns {id,score,terms,document}[]; document contains only storedFields and is not a full copy of the original. query cannot be empty, limit is 1–1000, and filter must synchronously return boolean.
combineWith defaults to 'OR'; tokenization comes from MiniSearch by default, and languages such as Chinese can supply a tokenizer explicitly. remove waits asynchronously for vacuum, and clear empties the current index. snapshot/restore checks the engine identity and configuration fingerprint; indexRevision is how the App expresses the format/tokenization semantics version, and it should also be updated when custom tokenization behavior changes. A snapshot is not a persistence service; saving, expiry, deletion, and access policy belong to the App.
Progressive examples for the same business case
First index two interface documents
Beginner · Node.js
import { TextSearch } from '@jangts/agent-core/kits/search';
type Document = { id: string; title: string; body: string };
const index = new TextSearch<Document>({ indexRevision: 'manual-v1', fields: ['title', 'body'], storedFields: ['title'] });
index.upsert({ id: 'workflow', title: 'Workflow', body: 'Define reusable agent nodes.' });
index.upsert({ id: 'model', title: 'Model', body: 'Bind a model provider.' });
console.log(index.search({ query: 'agent', limit: 5, filter: () => true }));import { TextSearch } from '@jangts/agent-core/kits/search';
const index = new TextSearch({ indexRevision: 'manual-v1', fields: ['title', 'body'], storedFields: ['title'] });
index.upsert({ id: 'workflow', title: 'Workflow', body: 'Define reusable agent nodes.' });
index.upsert({ id: 'model', title: 'Model', body: 'Bind a model provider.' });
console.log(index.search({ query: 'agent', limit: 5, filter: () => true }));Add access filtering, weighting, and a snapshot to the same search
Configuration · Node.js
Metadata allowed for filtering must be registered as storedFields; field filtering must rely on a trusted App identity and must not trust an identity the model claims.
import { TextSearch } from '@jangts/agent-core/kits/search';
type Document = { id: string; title: string; body: string; audience: string };
const options = { indexRevision: 'manual-v2', fields: ['title', 'body'] as const, storedFields: ['title', 'audience'] as const, boost: { title: 3 }, combineWith: 'AND' as const };
const index = new TextSearch<Document>(options);
index.upsert({ id: 'workflow', title: 'Agent Workflow', body: 'Define reusable nodes.', audience: 'public' });
index.upsert({ id: 'private', title: 'Agent internals', body: 'Internal notes.', audience: 'owner' });
const restored = TextSearch.restore<Document>(options, index.snapshot());
console.log(restored.search({ query: 'agent', limit: 5, filter: document => document.audience === 'public' }));
await restored.remove('workflow');import { TextSearch } from '@jangts/agent-core/kits/search';
const options = { indexRevision: 'manual-v2', fields: ['title', 'body'], storedFields: ['title', 'audience'], boost: { title: 3 }, combineWith: 'AND' };
const index = new TextSearch(options);
index.upsert({ id: 'workflow', title: 'Agent Workflow', body: 'Define reusable nodes.', audience: 'public' });
index.upsert({ id: 'private', title: 'Agent internals', body: 'Internal notes.', audience: 'owner' });
const restored = TextSearch.restore(options, index.snapshot());
console.log(restored.search({ query: 'agent', limit: 5, filter: document => document.audience === 'public' }));
await restored.remove('workflow');