sources

KIT · constant · Runtime value

Import

TypeScript

import { sources } from '@jangts/agent-core/mock/web';

JavaScript

import { sources } from '@jangts/agent-core/mock/web';

Other public entry points:@jangts/agent-core/mock/web.

Definition

Member index

Member	Purpose
web	Retrieve a page and return its readable text.

Member details

web

Retrieve a page and return its readable text.

sources.web(options?: WebOptions): Exclude<TextSource, string>

Parameter	Type	Passing	Description
options	WebOptions	Optional	—

Fields of options

Field	Type	Required	Description
url	string | ((context: GenerationContext) => string) | undefined	Optional	Fixed URL or synchronous selector; when omitted the prompt must contain exactly one URL.
profile	"browser" | "crawler" | undefined	Optional	Header profile; only browser (alias crawler) exists today. Default browser.
headers	Record<string, string> | undefined	Optional	Content negotiation only; never copies browser cookies or model credentials.
timeoutMs	number | undefined	Optional	Total retrieval deadline in milliseconds (1-120 000); default 20 000.
maxBytes	number | undefined	Optional	Maximum page size in bytes (1-20 971 520); default 2 MiB, applied before and after decoding.
maxChars	number | undefined	Optional	Maximum characters of extracted text kept for analysis (1-1 000 000); default 30 000.
maxRedirects	number | undefined	Optional	Redirect budget (0-10); default 3. Each target is re-validated like the original URL.
fallback	string | undefined	Optional	Explicit replacement source text on retrieval failure; request cancellation still throws.
load	((url: URL, signal: AbortSignal, maxBytes: number) => Promise<LoadedPage>) | undefined	Optional	Trusted offline/custom transport. It owns its destination policy; not supplied by remote clients.

Returns: (context: GenerationContext) => string \| Promise<string>.

Examples

• MockGPT Web: trusted sources and local summaries: Get body text from a real web page or a trusted load, then hand it to input-dependent generators.

A trusted load supplies offline body text

Beginner · Node.js

TypeScript

import { MockGPT, generators } from '@jangts/agent-core/mock';
import { sources, isPublicAddress } from '@jangts/agent-core/mock/web';
const source = sources.web({ url: 'https://example.com/article', async load(url, signal, maxBytes) {
  signal.throwIfAborted();
  const bytes = new TextEncoder().encode('服务维护完成。健康检查通过。');
  if (bytes.byteLength > maxBytes) throw new Error('来源超过预算');
  return { bytes, url: url.href, contentType: 'text/plain; charset=utf-8' };
} });
const mock = new MockGPT({ rules: [{ generator: generators.summary({ source }) }] });
console.log(await mock.generate({ model: 'demo', messages: [{ role: 'user', content: '概括正文' }] }));
console.log(isPublicAddress('127.0.0.1')); // false

JavaScript

import { MockGPT, generators } from '@jangts/agent-core/mock';
import { sources, isPublicAddress } from '@jangts/agent-core/mock/web';
const source = sources.web({ url: 'https://example.com/article', async load(url, signal, maxBytes) {
        signal.throwIfAborted();
        const bytes = new TextEncoder().encode('服务维护完成。健康检查通过。');
        if (bytes.byteLength > maxBytes)
            throw new Error('来源超过预算');
        return { bytes, url: url.href, contentType: 'text/plain; charset=utf-8' };
    } });
const mock = new MockGPT({ rules: [{ generator: generators.summary({ source }) }] });
console.log(await mock.generate({ model: 'demo', messages: [{ role: 'user', content: '概括正文' }] }));
console.log(isPublicAddress('127.0.0.1')); // false

Explicitly enable a web source and a failure replacement text

Configuration · Node.js

TypeScript

import { MockGPT, generators } from '@jangts/agent-core/mock';
import { sources } from '@jangts/agent-core/mock/web';
const mock = new MockGPT({ rules: [{ generator: generators.summary({ source: sources.web({
  url: 'https://example.com/article', timeoutMs: 5000,
  fallback: '网页不可读取，请提供正文。',
}) }) }] });
console.log(await mock.generate({ model: 'demo', messages: [{ role: 'user', content: '总结文章' }] }));
console.log(mock.records); // 检查 sourceFallback/sourceError，不把替代文本当原文

JavaScript

import { MockGPT, generators } from '@jangts/agent-core/mock';
import { sources } from '@jangts/agent-core/mock/web';
const mock = new MockGPT({ rules: [{ generator: generators.summary({ source: sources.web({
                    url: 'https://example.com/article', timeoutMs: 5000,
                    fallback: '网页不可读取，请提供正文。',
                }) }) }] });
console.log(await mock.generate({ model: 'demo', messages: [{ role: 'user', content: '总结文章' }] }));
console.log(mock.records); // 检查 sourceFallback/sourceError，不把替代文本当原文

Related definitions

• TextSource
• WebOptions