# @nb-corp/nb-search v0.5.2 — Complete Documentation Corpus Canonical documentation index: https://nb-corp.github.io/nb-search/llms.txt Package: @nb-corp/nb-search (TypeScript SDK, CLI, 3-tool MCP server, and Agent Skill) Repository: https://github.com/NB-Corp/nb-search --- Source: https://nb-corp.github.io/nb-search/en/ ## 30-Second Zero-Key Quick Start Requires **Node.js >= 24.15**. Install the CLI and try web search, library docs, and URL-to-Markdown extraction: ```bash npm install -g @nb-corp/nb-search nb-search search "WebAssembly component model" nb-search search "react useOptimistic" --lane context7.docs nb-search fetch "https://nodejs.org/en/about/releases/" --pipeline direct.fetch ``` For Claude Code, Codex, Claude Desktop, or Cursor, add this MCP config. Omit `env` when using only zero-key sources: ```json { "mcpServers": { "nb-search": { "command": "npx", "args": ["-y", "--package=@nb-corp/nb-search", "nb-search-mcp"], "env": { "NB_SEARCH_TAVILY_API_KEY": "YOUR_API_KEY", "NB_SEARCH_EXA_API_KEY": "YOUR_API_KEY" } } } } ``` ## Why Not Stack Separate Vendor MCPs? | Capability | 4–5 vendor MCPs | BYOK wrapper (`mcp-omnisearch`) | Keyless scraper (`open-webSearch`) | `@nb-corp/nb-search` | | --- | --- | --- | --- | --- | | **Context overhead** | ~15k–35k tokens, 20+ tools | ~4k–8k tokens | ~1k–2k tokens | **~1.4k MCP / ~110 Skill tokens** | | **Zero-key start** | Keys required | Keys required | Public HTML only | **Six keyless source types + optional BYOK** | | **Multi-engine fusion** | Manual merging | One engine per call | No API provenance | **URL deduplication + RRF + provenance** | | **Multi-key failover** | None | One key per source | IP rate limits | **32 keys/provider, `429` cooldown, `402` skip** | | **URL credit protection** | Paid scraper per fetch | Vendor scrape API | Basic HTML extraction | **Defuddle first + ordered routing + challenge detection** | | **Research beyond 60s** | stdio timeout | Blocking calls time out | Not supported | **Detached `run` / `get` / `read` / `cancel` jobs** | ## Documentation & AI Index - **Get started**: [Quick Start](/en/guide/quickstart) · [Configuration & Key Rotation](/en/guide/configuration) · [SDK / CLI / MCP / Skill](/en/guide/integrations) - **Explore capabilities**: [Sources Catalog](/en/sources/) · [Web & Regional Search](/en/sources/web-search) · [URL Fetch & Routing](/en/guide/fetch) · [Async Jobs](/en/guide/jobs) - **Reference & machine-readable docs**: [CLI](/en/reference/cli) · [Runtime Contract](/en/reference/runtime) · [Remote Protocol](/en/reference/remote-protocol) · [llms.txt](/llms.txt) · [llms-full.txt](/llms-full.txt) --- Source: https://nb-corp.github.io/nb-search/en/guide/quickstart # Quickstart This guide shows how to prepare a local runtime, inspect the available capability catalog, and verify your first search and fetch operations. > **Version range** This documentation applies to `@nb-corp/nb-search` `0.5.0` and later (current version: `0.5.2`). ## Requirements - **Node.js**: `>=24.15.0` - **Source build tool**: `pnpm@10.33.0` (required only when building from the source repository) ## Get and build ```bash [Global CLI] npm install -g @nb-corp/nb-search nb-search search "WebAssembly component model" nb-search fetch "https://nodejs.org/en/about/releases/" --pipeline direct.fetch ``` ```bash [Project SDK] npm install @nb-corp/nb-search ``` ```bash [Build from source] git clone https://github.com/NB-Corp/nb-search.git cd nb-search pnpm install --frozen-lockfile pnpm build ``` The installed package and source build both provide the official launcher. When using the package as a Skill, keep the package root and the complete `SKILL.md`, `scripts/`, and `dist/` structure; copying only part of the directory causes the launcher to return `ENTRY_UNAVAILABLE`. ### Load the official Skill A host agent or workflow environment can mount the npm package or source directory directly as a Skill: - **Global npm install**: After running `npm install -g @nb-corp/nb-search`, use `$(npm root -g)/@nb-corp/nb-search/skills/nb-search` (or the package root, `$(npm root -g)/@nb-corp/nb-search`); for a project-local installation, use `node_modules/@nb-corp/nb-search/skills/nb-search`. - **Source build**: Set the repository root as ``. - **Invocation contract**: The Skill dispatch script is fixed at `node "/scripts/nb-search.mjs"`. The launcher locates and loads the packaged `dist/cli.mjs` automatically, so the host does not need any additional system PATH configuration. - **Nested layout**: The package also includes a compatible `skills/nb-search` entry point for hosts with a particular layout. It is not an independently movable package and still depends on `scripts/` and the build output in the root directory. Preserve the relative paths to the package root. ### CLI debugging entry points - When installed as an npm dependency: use `node node_modules/@nb-corp/nb-search/scripts/nb-search.mjs`, or invoke the local executable with `npx nb-search`. - From a source checkout: run `node scripts/nb-search.mjs` from the repository root. The rest of this guide uses `nb-search` for the entry command. When debugging from the source root, replace `nb-search` with `node scripts/nb-search.mjs`; when debugging in an application project, replace it with `npx nb-search`. ## Step 1: View the capabilities catalog The `capabilities` command prints the list of Providers, Lanes, and Pipelines supported by the current runtime. It does not require any Provider keys and does not probe external services. ```bash [POSIX (Linux / macOS)] nb-search capabilities ``` ```powershell [Windows PowerShell] nb-search capabilities ``` The output is a JSON object that conforms to the global `3.0` Schema Envelope. Before any third-party keys are configured, most Lanes that require external credentials are marked as not ready, but the static catalog remains fully available. > **Status note** `ready` in the output means only that the local configuration and runtime ports have been initialized. It does not indicate upstream Provider network connectivity, quota, or service health. ## Step 2: Verify keyless fetching Verify that the local runtime can use a public web page with body content, or test with `direct.local` inline content without accessing the network: ```bash [POSIX (public URL fetch)] nb-search fetch "https://nodejs.org/en/about/releases/" --pipeline direct.fetch ``` ```powershell [Windows PowerShell (public URL fetch)] nb-search fetch "https://nodejs.org/en/about/releases/" --pipeline direct.fetch ``` ```bash [POSIX (offline local inline-data verification)] nb-search fetch --stdin << 'EOF' {"action":"run","source":{"kind":"inline_text","content":"Hello, nb-search local fetch verification.","media_type":"text/plain"},"pipeline":"direct.local"} EOF ``` After a successful fetch, the returned Envelope object has a top-level `schema_version: "3.0"` property, and the extracted documents are in the top-level `documents` array. The fetch operation contract for an individual pipeline follows `nb-search.fetch@1`. ## Step 3: Keyless search and BYOK multi-source search In version `0.5.0` and later, general web search defaults to the keyless `duckduckgo.search` (and also includes the keyless `github.repositories` and `context7.docs`), so you can run your first search without configuring an API key: ```bash nb-search search "VitePress sitemap generation" nb-search search "react useOptimistic" --lane context7.docs nb-search search "language:typescript model context protocol" --lane github.repositories ``` > **Keyless-source error boundary** If the public HTML endpoint for `duckduckgo.search` triggers human verification or `429` rate limiting, the CLI returns an explicit error code. It **never fakes an empty result and never silently switches to a paid API**. `DEFAULT_NOT_CONFIGURED` is returned only when the default source has been explicitly cleared (`"defaults": { "search_lane": null }`) and the command does not pass `--lane`. After adding your own commercial API keys (BYOK), you can select one high-precision engine or use `--lanes` for multi-source RRF deduplication and fusion: ```bash [POSIX (Linux / macOS)] export NB_SEARCH_TAVILY_API_KEY="YOUR_API_KEY" export NB_SEARCH_EXA_API_KEY="YOUR_API_KEY" nb-search search "site:github.com node" --lanes exa.search,tavily.search ``` ```powershell [Windows PowerShell] $env:NB_SEARCH_TAVILY_API_KEY = "YOUR_API_KEY" $env:NB_SEARCH_EXA_API_KEY = "YOUR_API_KEY" nb-search search "site:github.com node" --lanes exa.search,tavily.search ``` After execution, `stdout` returns an execution Envelope whose top level includes `schema_version: "3.0"`. Its `output` contains an `nb-search.results@1` object (or a single-source Typed Schema such as `context7.docs`). To change the default source, configure rotation for up to 32 keys, or set URL routing, see the [Configuration System Guide](/en/guide/configuration). --- Source: https://nb-corp.github.io/nb-search/en/guide/configuration # Configuration System `nb-search` uses Configuration Schema Version 4 (`schema_version: "4"`). Configuration declares Provider instances, custom Lanes/Pipelines, credential-slot bindings, and default behavior. ## Configuration sources and merge priority When the runtime resolves configuration, it merges sources strictly from lowest to highest priority. Higher-priority values override lower-priority values: 1. **Built-in default configuration** (Built-in structure) 2. **Local configuration file**: `$NB_SEARCH_HOME/config.json`, or the absolute or relative path specified by `NB_SEARCH_CONFIG` (if the variable is set but the target file does not exist, resolution fails immediately) 3. **Standard environment variables**: environment variables beginning with `NB_SEARCH_*` 4. **In-process configuration** (`config`) 5. **In-process overrides** (`overrides`) When applying a patch: - A field value of `null` explicitly deletes the field. - Arrays are replaced as a whole; they are not merged by appending. ## Standard environment variables Standard environment variables cover the runtime base environment and Provider credentials or endpoint configuration. ### Runtime base environment | Environment variable | Description | Default | |---|---|---| | `NB_SEARCH_HOME` | Configuration and local state directory | `.nb-search` in the user's home directory (`~/.nb-search`) | | `NB_SEARCH_CONFIG` | Explicit path to the configuration JSON file | None (reads `$NB_SEARCH_HOME/config.json` by default) | | `NB_SEARCH_JOBS_ROOT` | Root directory for asynchronous job artifacts | `$NB_SEARCH_HOME/jobs` | | `NB_SEARCH_RETENTION_HOURS` | Retention period for asynchronous artifacts, in hours | `72` | | `NB_SEARCH_LOG_LEVEL` | Log level (`error`, `warn`, `info`, `debug`) | `warn` | ### Core Provider environment variables Credentials are bound only when an environment variable is non-empty. An empty string means “not configured” and overrides a lower-priority credential. Any API key environment variable accepts either one key or up to **32 unique keys** separated by commas (for example, `NB_SEARCH_TAVILY_API_KEY="key-one,key-two"`). ```bash [General and aggregate search] NB_SEARCH_EXA_API_KEY="" NB_SEARCH_EXA_BASE_URL="" NB_SEARCH_TAVILY_API_KEY="" NB_SEARCH_TAVILY_BASE_URL="" NB_SEARCH_BRAVE_API_KEY="" NB_SEARCH_BRAVE_BASE_URL="" NB_SEARCH_FIRECRAWL_API_KEY="" NB_SEARCH_FIRECRAWL_BASE_URL="" NB_SEARCH_SEARXNG_BASE_URL="" ``` ```bash [Fetch and specialized sources] NB_SEARCH_JINA_API_KEY="" NB_SEARCH_JINA_BASE_URL="" NB_SEARCH_GITHUB_TOKEN="" NB_SEARCH_CONTEXT7_API_KEY="" NB_SEARCH_ZHIPU_API_KEY="" NB_SEARCH_ZHIPU_BASE_URL="" NB_SEARCH_PARALLEL_API_KEY="" # The built-in Zhihu source is opt-in. In addition to this variable, declare the instance, slot, and Lane explicitly in config.json. NB_SEARCH_ZHIHU_API_KEY="" ``` ```bash [Models and advanced synthesis sources] NB_SEARCH_GROK_API_KEY="" NB_SEARCH_GROK_BASE_URL="" NB_SEARCH_GROK_MODEL="" # The built-in GMA instance and slot (grok-multi-agent.default); the default slot environment variable points to NB_SEARCH_GROK_API_KEY. NB_SEARCH_GROK_MULTI_AGENT_BASE_URL="" NB_SEARCH_GROK_MULTI_AGENT_MODEL="" # OpenAI-Compatible (OAC) NB_SEARCH_OAC_API_KEY="" NB_SEARCH_OAC_BASE_URL="" NB_SEARCH_OAC_MODEL="" ``` ## Configuration file structure (`config.json`) The following valid `config.json` example configures a default search source, a URL fetch chain, a local-file authorization scope, and a fetch byte limit: ```json { "schema_version": "4", "defaults": { "search_lane": "tavily.search", "fetch_chain": [ { "input_kind": "url", "representation": "markdown", "pipelines": ["direct.fetch", "jina.reader"] } ] }, "fetch": { "file_scopes": [ { "id": "workspace", "root": "/path/to/project/docs", "media_types": ["text/plain", "text/markdown"] } ] }, "execution": { "fetch": { "max_source_bytes": 10485760 } } } ``` > **Field-name constraints** - The configuration Schema version field is named `schema_version` and is a string literal with the value `"4"`. - The pipeline list in a fetch chain is named `pipelines` (an array), not `pipeline_ids`. - The local-fetch source-size limit is declared in `execution.fetch.max_source_bytes`; each `file_scopes` entry supports only `id`, `root`, and optional `media_types`. ## GMA custom credential-slot example The built-in GMA instance is `grok-multi-agent.default`, and its default credential slot points to the `NB_SEARCH_GROK_API_KEY` environment variable, just like Grok. To bind GMA to a separate dedicated environment variable (such as `MY_GMA_KEY`) and specify the communication protocol, declare a separate credential slot in the configuration: ```json { "schema_version": "4", "provider_instances": { "grok-multi-agent.default": { "provider_id": "grok-multi-agent", "enabled": true, "credential_slot_id": "gma.custom", "base_url": "https://relay.example.test/gma", "options": { "api_mode": "messages", "reasoning_effort": "high" } } }, "credential_slots": { "gma.custom": { "provider_id": "grok-multi-agent", "env": "MY_GMA_KEY" } } } ``` > **Slot constraint** When declaring `credential_slots`, the slot configuration object's `provider_id` must exactly match the `provider_id` of the `provider_instance` that references the slot (both are `"grok-multi-agent"` here). Otherwise, configuration validation fails. ## Multi-key rotation and local quota inspection (`keyUsage()`) The local runtime supports **up to 32 unique, non-empty keys** for any built-in or custom `credential_slots.*.env` setting (separated by English commas in priority order, for example, `NB_SEARCH_TAVILY_API_KEY="key-one,key-two"`). A single key remains fully compatible. Key plaintext is never written to `config_fingerprint`, job snapshots, or usage response bodies. Configure the scheduling strategy and quota-cache period per Provider instance in `config.json` (the Schema version remains `"4"`): ```json { "schema_version": "4", "provider_instances": { "tavily.default": { "key_strategy": "priority", "balance_ttl_ms": 600000 } } } ``` ### Scheduling and failover rules - **Rotation strategy (`key_strategy`)**: The default is `"round-robin"` (balanced rotation in order). With `"priority"`, the first available key is always preferred, and the runtime moves forward only when that key is limited or fails. - **State machine and error recovery**: - `401` / `403`: quarantine the key for the lifetime of the current runtime instance; - `429`: cool the key down according to the `Retry-After` response header (60 seconds by default), then immediately switch to the next available key; - `402`: mark the key as quota-exhausted (`exhausted`) until the `balance_ttl_ms` cache expires or an explicit usage refresh restores it; - `5xx` and connection failures: perform at most 2 bounded exponential-backoff retries on the current key, then switch to the next key. - **Process boundary**: Key cooldown and health state live in the current process memory (detached `async` jobs run in an independent worker subprocess). When all available keys have been tried and fail, the outer execution-level retry strategy (`execution.retry_count`) still applies under the original rules. ### Local SDK `keyUsage()` and provider metering differences The local SDK runtime exposes `runtime.keyUsage({ provider_instance_id, refresh })`, which returns redacted key indexes, health status, and an upstream quota summary (`KeyUsage[]`, without key plaintext): - **Caching and rate limits**: When `refresh` is omitted (or set to `false`), the runtime prefers the in-memory cache, which is valid for 10 minutes by default (`balance_ttl_ms` ranges from `60000` to `86400000` milliseconds). Tavily rate-limits its official `/usage` endpoint to at most 10 requests every 10 minutes. For routine monitoring, prefer cached reads and avoid passing `refresh: true` frequently. - **Source behavior**: - **Tavily**: Calls the key-level `/usage` endpoint (retaining the `base_url` prefix) and returns usage and limits for the current billing period at key level. Ordinary `search`/`fetch` dispatches do not make an additional balance request. - **Firecrawl**: Calls `/v1/team/credit-usage` (retaining the `base_url` prefix) and returns the **team-wide** remaining credits. Multiple keys in the same team may show the same remaining balance. Ordinary requests do not pre-check the balance. - **Zhihu (`zhihu`)**: Calls `GET /api/v1/quota` and returns API-specific request counters in `KeyUsage.usage.meters`. When `allow_paid: false` (the default), dispatch also performs a free-quota gate check before the request; see [Explicitly enable the Zhihu source](/en/sources/web-search#zhihu). - **Exa and other sources**: Exa's historical usage endpoint requires a separate team-management key and does not return remaining available quota. Sources without a single-key remaining-quota endpoint do not return a usage entry from `keyUsage()`. - **Protocol boundary**: `keyUsage()` is exposed only by the local SDK (`LocalNbSearchRuntime`), not by the CLI, MCP, or Remote HTTP protocol clients. ## Local and remote credential storage (`secrets.json` / `remote-secrets.json`) To avoid writing plaintext keys to shell history or configuration files, the CLI provides protected file-based credential storage: ### 1. Local credential storage (`$NB_SEARCH_HOME/secrets.json`) ```json { "schema_version": "1", "values": { "NB_SEARCH_TAVILY_API_KEY": "YOUR_TAVILY_KEY", "NB_SEARCH_BRAVE_API_KEY": "YOUR_BRAVE_KEY" } } ``` - `secrets.json` fills values that are not set in the process environment. If the current shell already contains the variable (even as an empty string), the environment variable takes precedence. - Only the CLI reads this file by default and passes its values to local child processes. The SDK and MCP do not automatically read the CLI's `secrets.json`. ### 2. Remote client credentials and configuration files (`profiles.json` / `remote-secrets.json`) Configure remote service endpoints in `$NB_SEARCH_HOME/profiles.json`: ```json { "schema_version": "1", "profiles": { "cloud": { "kind": "remote", "base_url": "https://search.example.test/api/", "token_env": "NB_SEARCH_CLOUD_TOKEN", "timeout_ms": 120000 } } } ``` Provide the remote service Access Key through the `NB_SEARCH_CLOUD_TOKEN` environment variable or the protected `$NB_SEARCH_HOME/remote-secrets.json` file: ```json { "schema_version": "1", "values": { "NB_SEARCH_CLOUD_TOKEN": "YOUR_SERVICE_KEY" } } ``` Remote configuration and local Provider credentials are completely separate: selecting a remote profile does not load local search-source keys. ### 3. File storage and safe recovery - **Storage and link support**: Configuration files and directories use the operating system's standard creation permissions (files `0600`, directories `0700`). Ordinary configuration-file symlinks and parent-directory junctions are fully supported, with no restriction on same-name directories or near-matching ordinary filenames. The metadata leaf names (`.config-access.lock`, `.config-revision.json`, `..nb-search.lock`, `..nb-search-revision.json`) cannot be used as configuration or key files, and the metadata leaf itself cannot be redirected through a symlink. The system does not force permission changes on existing files or rewrite user data links. - **Concurrent snapshots and transaction locks**: Concurrent local reads (`capabilities`, `search`, `fetch`, and `jobs`) do not need locks and run safely against versioned snapshots. Transaction locks and version markers are created only for configuration writes or migrations. When multiple homes share a custom configuration path, each real data target gets a dedicated adjacent lock (`..nb-search.lock`) and revision file (`..nb-search-revision.json`) to prevent cross-home overwrites. - **Configuration writes and safe recovery**: Ordinary manual edits to configuration files take effect directly; you do not need to manage revision files manually. The CLI ensures atomic pairing and transaction coordination only while writing configuration or running a migration. If the CLI reports that configuration is busy or needs recovery (with a specific marker path), do not blindly delete a single lock file or the revision file. First confirm that the preceding write or migration process has stopped. Then use the backup files under the home directory (`.migration-*-backup.json`) to restore the complete configuration and credential pair, publish a new revision for the targets associated with the transaction, and remove the corresponding locks before resuming. Normal users do not need to create or manage any lock files manually. ## Offline configuration health check (`--doctor`) The CLI provides a `--doctor` option to validate the current configuration format and credential slots while offline: ```bash node scripts/nb-search.mjs --doctor ``` This command summarizes only local JSON syntax and the presence of credential environment variables. A valid keyless configuration also returns success. It does not build runtime ports or prove that upstream Providers are reachable. To check source availability and runtime-port readiness, use `capabilities`. --- Source: https://nb-corp.github.io/nb-search/en/guide/integrations # Integration Methods `nb-search` provides five common integration forms: the Node.js / TypeScript SDK, the CLI, an MCP `stdio` server, an Agent Skill, and a Remote HTTP protocol client. All forms share the same Schema v4 configuration and `3.0` result-envelope contract. ## 1. Node.js / TypeScript SDK The SDK is suitable for embedding directly in a Node.js backend service or Agent host process. It requires Node.js `>=24.15.0`. ### Search, capability catalog, and key-quota inspection ```typescript import { createNbSearchRuntime } from '@nb-corp/nb-search'; const runtime = createNbSearchRuntime({ config: { schema_version: '4', defaults: { search_lane: 'brave.search' } } }); // 1. Run a synchronous search const searchResponse = await runtime.search({ action: 'run', query: 'node 24 LTS release notes', execution: 'sync' }); // 2. Get static capability-catalog metadata (does not probe the network) const catalog = await runtime.capabilities(); // 3. Inspect redacted multi-key health status and remaining quota (local SDK only) const usage = await runtime.keyUsage(); ``` ### Read URL content Reading web-page content is the primary fetch scenario for `nb-search`: ```typescript const fetchResponse = await runtime.fetch({ action: 'run', source: { kind: 'url', url: 'https://nodejs.org/en/about/releases/' }, representation: 'markdown', execution: 'sync' }); if (fetchResponse.action === 'run' && fetchResponse.execution === 'sync' && fetchResponse.status === 'succeeded' && fetchResponse.documents.length > 0) { console.log(fetchResponse.documents[0]!.content); } ``` ### Compatibility: offline, network-free content fetching `direct.local` and its inline/file inputs remain available for cleaning and extracting authorized local text or HTML within the process without sending data over the network. To process local files, HTML, PDF, or Office documents (including OCR) and save complete Markdown and assets, use the separate [nb-extract](https://github.com/NB-Corp/nb-extract) tool: ```typescript import { createNbSearchRuntime } from '@nb-corp/nb-search'; const runtime = await createNbSearchRuntime(); const fetchResponse = await runtime.fetch({ action: 'run', source: { kind: 'inline_text', content: 'Hello, nb-search offline fetch.', media_type: 'text/plain' }, pipeline: 'direct.local', execution: 'sync' }); if (fetchResponse.action === 'run' && fetchResponse.execution === 'sync' && fetchResponse.status === 'succeeded' && fetchResponse.documents.length > 0) { const doc = fetchResponse.documents[0]!; console.log('Document fetched successfully, content:', doc.content); } ``` The local SDK runtime (`LocalNbSearchRuntime`) exposes **four core methods**: `search`, `fetch`, `capabilities`, and the machine-local-only `keyUsage` (see [Multi-key rotation and local quota inspection](/en/guide/configuration#multi-key-rotation-and-local-quota-inspection-keyusage)). The general `NbSearchRuntime` and remote-client interfaces include the first three transport-independent methods. ### TypeScript consumer compatibility note The verified SDK consumer build settings use `skipLibCheck: true`. In a TypeScript 6.0.2 environment, enabling complete third-party declaration checking (`skipLibCheck: false`) causes the compiler to report variance-check errors (TS2636) in the bundled Zod type declarations. Setting `skipLibCheck: true` skips only rechecking declaration files in external dependency packages; it does not lower the strict type-checking level for your application's own code. Projects that require exhaustive dependency-declaration checking should evaluate this compatibility characteristic before integration. ### Advanced host injection: Trusted HTTP Transport For environments that need centralized auditing or proxying of host-process network traffic, `createNbSearchRuntime({ http_transport })` accepts an object implementing the root-exported `HttpTransport` interface (and also exports `HttpRequest`, `HttpResponse`, and `ResponseLimitError`). The underlying Provider adapters receive this custom transport and use it for their outbound HTTP requests. Note: - Injecting a transport disables detached asynchronous jobs for the current runtime instance (they return `LANE_EXECUTION_UNSUPPORTED`); only synchronous (`sync`) execution is available. - `direct.fetch`, `wayback.fetch`, `browser.render`, and custom Providers may use independent network channels and are not guaranteed to be globally controlled by this injected object. See the [model runtime specification](/en/reference/runtime) for the detailed advanced contract. ## 2. Command-line tool (CLI) The CLI is suitable for scripted batch processing, local operations and debugging, or invocation as an independent child process. ```bash # Run a search node scripts/nb-search.mjs search "site:github.com vitest" --lane exa.search # Fetch a page node scripts/nb-search.mjs fetch "https://nodejs.org/en/about/releases/" --pipeline direct.fetch # View the capability catalog node scripts/nb-search.mjs capabilities ``` The CLI also accepts a complete JSON parameter object through standard input, which is useful for complex structured parameters or Agent-host dispatch: ```bash echo '{"action":"run","query":"typescript","lane":"tavily.search","execution":"sync"}' | \ node scripts/nb-search.mjs search --stdin ``` > **Stdin rules** When using `--stdin`, the input must be one complete UTF-8 JSON object and must not exceed 4 MiB. Do not mix positional arguments on the command line. ## 3. MCP server (`stdio`) and Agent Skill ### MCP server (approximately 1.4k context tokens) `nb-search` natively implements a Model Context Protocol (MCP) `stdio` service. It registers exactly **three structured tools**—`search`, `fetch`, and `capabilities`—which can save more than 90% of the tool-schema context overhead compared with mounting four or five vendor MCP servers at once. Connect through `npx` in `Claude Code`, `Claude Desktop`, `Codex`, or `Cursor` (the `env` block can be omitted when using only keyless default sources): ```json { "mcpServers": { "nb-search": { "command": "npx", "args": ["-y", "--package=@nb-corp/nb-search", "nb-search-mcp"], "env": { "NB_SEARCH_TAVILY_API_KEY": "YOUR_API_KEY", "NB_SEARCH_EXA_API_KEY": "YOUR_API_KEY" } } } } ``` If you have built the source locally or installed dependencies, you can also run `node dist/mcp.mjs` directly. All MCP tool parameters use strict JSON Schema field names (`action: "run" | "get" | "read" | "cancel"`). ### Agent Skill (approximately 110 idle tokens) In CLI-capable coding agents such as `Claude Code`, `Codex`, or `OpenCode`, mounting the built-in Agent Skill keeps idle context to approximately 110 tokens and loads the full protocol only when a retrieval task is triggered: ```bash npm install -g @nb-corp/nb-search # Point the host's Skill directory at skills/nb-search inside the global package (or at the package root). echo "$(npm root -g)/@nb-corp/nb-search/skills/nb-search" ``` ## 4. Remote protocol client (Remote Client) The TypeScript remote client also runs in a Node.js environment (`createNbSearchRemoteClient` is a synchronous factory function and does not require `await`). If another language or service needs to integrate without a Node dependency, it can implement requests directly according to [HTTP Protocol v1](/en/reference/remote-protocol). ```typescript import { createNbSearchRemoteClient } from '@nb-corp/nb-search'; const accessKey = process.env.NB_SEARCH_REMOTE_KEY; if (!accessKey) { throw new Error('Missing NB_SEARCH_REMOTE_KEY environment variable'); } // Create the client synchronously; no await is needed. const client = createNbSearchRemoteClient({ base_url: 'https://search.example.test/api/', access_key: accessKey }); const result = await client.search({ action: 'run', query: 'distributed systems', lane: 'searxng.search', execution: 'sync' }); ``` ### Important remote-protocol constraints - **URL-only fetching**: A remote client's `fetch` request supports only a public `url` input. The client rejects `inline_text`, `inline_bytes`, and `file` inputs before making a network request. - **No silent redirects or retries**: The remote client treats 3xx status codes as protocol errors and never forwards authentication headers to a redirect target. The caller must handle network errors explicitly; the client does not retry implicitly. - **Service boundary**: No public cloud-hosted service is provided officially. This remote-protocol specification is for communication with a self-hosted or internally deployed compatible gateway. --- Source: https://nb-corp.github.io/nb-search/en/guide/search # Search Capabilities (Search) The `nb-search` search interface supports general web-link discovery, structured knowledge-base queries, and specialized LLM-based synthesis and analysis. ## Architectural distinction: Envelope and Output Schema When a search runs, the outermost response envelope uses one common specification version: - Global envelope version: `schema_version: "3.0"` - Core payload field: `output` (or `error` on failure) Depending on the operation, `output` uses one of two channels: 1. **Results channel** (`channel: "results"`): - The payload follows `schema_id: "nb-search.results@1"`. - It contains a standardized `results` array with titles, canonical URLs, summaries, evidence groups (`evidence_groups`), and provenance (`provenance`). - It supports a single-Lane query, multi-Lane aggregation, or a preset that contains only Results operations. - Example sources: `tavily.search`, `brave.search`, `exa.search`, `github.repositories`. 2. **Typed channel** (`channel: "typed"`): - The payload follows the declared Typed Schema, such as `nb-search.docs-context@1`, `nb-search.synthesis@1`, or `nb-search.multi-agent-research@1`. - It is intended for structured research reports or knowledge-document context. - **Constraint**: Typed operations can target **only one Lane** at a time. They do not support multi-Lane aggregation and cannot be configured in a multi-source Results Preset. - Example sources: `context7.docs`, `exa.synthesis`, `grok.synthesis`, `gma.research`, `oac.synthesis`. ## Single-source queries and multi-source aggregation ### Single-source query For a single-query × single-results-Lane query, the system preserves the relative order of the source's entries after canonicalization and deduplication (for a single source, `rrf_score` is not emitted and the original upstream floating-point score is not retained): ```bash node scripts/nb-search.mjs search "fastify vs express" --lane brave.search ``` If one request sends multiple queries to the same Lane (for example, a query array sent through stdin), the system creates a separate candidate list for each query and merges them. ### Multi-source aggregation and deduplication (RRF) In the CLI, specify a multi-source query with comma-separated `--lanes` or by passing `--lane` repeatedly. Both forms are mutually exclusive with `--preset`: ```bash [CLI comma-separated] node scripts/nb-search.mjs search "postgresql replication" --lanes brave.search,tavily.search ``` ```bash [CLI repeated flag] node scripts/nb-search.mjs search "postgresql replication" --lane brave.search --lane tavily.search ``` ```json [Standard input JSON] { "action": "run", "query": "postgresql replication", "lanes": ["brave.search", "tavily.search"], "execution": "sync" } ``` During multi-source execution, the system merges results deterministically: 1. **URL canonicalization and deduplication**: Remove equivalent tracking parameters and suffixes. 2. **Reciprocal Rank Fusion (RRF)**: Calculate a global combined score from each source's relative rank. 3. **Evidence-group clustering**: Preserve independent evidence fragments captured by different sources. 4. **Deterministic arbitration**: Apply a deterministic fallback ordering for tied scores so identical inputs produce stable output. ## Synchronous and asynchronous execution - **Synchronous execution (`execution: "sync"`)**: Waits for the search to finish and returns data in the response. If the payload exceeds `execution.max_inline_bytes` (16 MiB by default), the system returns `OUTPUT_TOO_LARGE`; it never truncates implicitly or silently switches to asynchronous execution. - **Asynchronous execution (`execution: "async"`)**: Returns a job receipt immediately. Asynchronous calls must provide an `idempotency_key` for idempotency. See the [Async Jobs Guide](/en/guide/jobs) for job polling and reconstruction of complete artifacts. --- Source: https://nb-corp.github.io/nb-search/en/guide/fetch # Content Fetching (Fetch) `nb-search` primarily uses `fetch` to read the content of public web URLs after search. Existing entry points such as `fetch`, `direct.local`, and `browser.render` remain available for compatibility; the contract for controlled local files and inline inputs appears below. To process local files, HTML, PDF, or Office documents (including OCR) and save complete Markdown and assets, use the separate [nb-extract](https://github.com/NB-Corp/nb-extract) tool. ## Architectural distinction: Envelope and Document Schema A fetch call returns an outer envelope with `schema_version: "3.0"`. Successfully extracted content is stored in the top-level `documents` array. Each document follows the fetch pipeline's `nb-search.fetch@1` operation contract. ## Input types and representations ### Input source (Source Kind) - `url`: A public HTTP(S) address. - `file`: A local file. Only relative paths are allowed, and the path must be inside a configured `file_scopes` authorization directory. - `inline_text` / `inline_bytes`: A string or Base64-encoded data passed directly in memory. ### Output representation (Representation) - `markdown` (default): Clean Markdown text. `direct.fetch` uses **Defuddle + linkedom** for static HTML extraction of body text, links, mathematical formulas, and code blocks (it does not execute page scripts or make additional network requests), preserves plain-text whitespace, formats JSON, and locally decodes Base64 UTF-8 responses from `api.github.com` `contents` / `readme` endpoints within the byte limit. - `text`: Plain text. ## Ordered URL routing (`fetch.routing`) and fetch chains For public `url` requests, the runtime determines the pipeline sequence to execute using the following fixed priority, and **strictly tries only the pipelines in the selected sequence**: 1. **Explicit `--pipeline `**: When the caller names a pipeline, run only that pipeline. 2. **The first matching enabled user rule (`fetch.routing.rules`)**: Run the ordered `pipelines` declared by the rule, or restore the default chain with `"use": "default"`. 3. **The enabled built-in rule package (`kiki-common-direct@1`, enabled by default)**: Route known plain-text `raw.githubusercontent.com`, `api.github.com` repository metadata/directories/README/Release data, `registry.npmjs.org` package metadata, and `pypi.org` JSON to **`direct.fetch` only**. A failure is reported directly; Jina or commercial paid scrapers are never appended silently. 4. **The default `defaults.fetch_chain`**: - **Default chain for URL input**: Use keyless `direct.fetch` first (supports `sync` and detached `async`), then try the keyless or optionally keyed `jina.reader` (`sync`) if a fallback is needed. - **Default chain for local files and inline data**: Use only `direct.local` (no external egress, `egress: "none"`, and not part of URL routing). You can customize domain and path glob rules in `config.json`. The routing decision and every attempt are recorded in the returned receipt's top-level `routing` and `lane_outcomes[].attempts` fields: ```json { "schema_version": "4", "fetch": { "routing": { "rules": [ { "id": "docs-direct-then-jina", "match": { "origin": "https://raw.githubusercontent.com", "path_globs": ["/**/*.md"] }, "action": { "pipelines": ["direct.fetch", "jina.reader"] } }, { "id": "npm-restore-default-chain", "match": { "origin": "https://registry.npmjs.org" }, "action": { "use": "default" } } ] } } } ``` ### Error termination and failover boundaries (Fallback) In a fetch chain, **fallback is not the same as a network retry**; it means trying the next pipeline in the chain in order. When the default chain tries multiple pipelines, a fallback service such as `jina.reader` may receive the target URL. To call exactly one pipeline, specify `--pipeline ` explicitly. Not every failure triggers fallback: - **Terminal errors**: A `404` or `410` status, SSRF blocking (`FETCH_BLOCKED`), a content type outside the allowlist (`FETCH_CONTENT_TYPE_REJECTED`), caller cancellation (`CANCELLED`), a timeout (`DEADLINE_EXCEEDED`), or an exceeded budget (`BUDGET_EXCEEDED`) terminates the entire fetch immediately; the next pipeline is not tried. - **Fallback-triggering errors**: A failed quality gate (`QUALITY_GATE_FAILED`), response bytes over the limit (`FETCH_BYTES_LIMIT`), upstream authentication failure (`PROVIDER_AUTH`), rate limiting (`PROVIDER_RATE_LIMIT`), service unavailability (`PROVIDER_UNAVAILABLE`), an internal error (`INTERNAL`), or an HTTP error other than `404`/`410` records the current outcome and moves to the next candidate pipeline. ## Pipeline selection: default chain vs. explicit selection The built-in default chain contains only `direct.fetch` and `jina.reader`. Other fetch pipelines—including `exa.contents`, `tavily.extract`, `firecrawl.scrape`, `wayback.fetch`, `browser.render`, `oac.fetch`, and local `direct.local`—are not enabled in the default URL chain. You can: 1. Specify `--pipeline ` when making the call. 2. Define your own default pipeline chain in the host configuration's `defaults.fetch_chain`. Example of explicit selection: ```bash # Fetch a historical Wayback snapshot of the page node scripts/nb-search.mjs fetch "https://nodejs.org/en/about/releases/" --pipeline wayback.fetch ``` ## Security limits and network protection The fetch system enforces the following security policies and boundary controls: 1. **URL validation and SSRF protection**: - General preflight (`parsePublicUrl`) checks URL syntax and the literal hostname/IP, rejecting URLs with usernames or passwords, invalid protocols, or literal private/loopback IP addresses. - Built-in direct fetching (`direct.fetch`) applies strict local-network protection: before connecting, it resolves DNS, validates every returned IP, and pins the connection. It repeats resolution and strict checks for every HTTP redirect, blocking redirects to private-network or metadata addresses. - Third-party proxy-fetch pipelines such as `jina.reader`, `exa.contents`, and `tavily.extract` send requests from their cloud services; this machine cannot control internal DNS resolution or redirects at the target site. 2. **Byte and character limit handling**: - During direct HTTP fetching, if a `direct.fetch` response exceeds `max_response_bytes`, the system may truncate it within bounds, records `FETCH_BYTES_LIMIT` in `warnings`, and sets `documents[].truncated: true`. - If the extracted body exceeds `max_content_chars`, the body is truncated and `FETCH_CONTENT_CHARS_LIMIT` is recorded in `warnings`. - If local `inline_text`, `inline_bytes`, or `file` input exceeds `max_source_bytes`, it is rejected before entering the pipeline with `FETCH_BYTES_LIMIT`. - If an API response from a third-party proxy-fetch pipeline (Jina, Exa, Tavily, Firecrawl, and others) exceeds the client's transport limit (`max_response_bytes`), that pipeline fails with `FETCH_BYTES_LIMIT`; the incomplete API response is not treated as successful content. - The default quality rules set no minimum body-length gate (`min_content_chars: 0`) and no default blocking markers (`blocked_markers: []`). If the host explicitly configures a custom quality gate, truncated content must still pass that gate and the final envelope inline-size limit (`max_inline_bytes`) before it can be returned as a success. - Even when the complete JSON Artifact is read with the CLI's `read --all`, that means only that the job artifact file was read in full; it does not mean that the fetched upstream web page was not truncated. 3. **MIME-type allowlist**: Non-text/HTML types are rejected with `FETCH_CONTENT_TYPE_REJECTED`. 4. **Envelope inline-size limit**: If the complete synchronous result envelope exceeds `execution.max_inline_bytes` (16 MiB by default), the system returns `OUTPUT_TOO_LARGE`. --- Source: https://nb-corp.github.io/nb-search/en/guide/jobs # Async Jobs and Artifacts (Jobs) For longer-running research sources with multiple reasoning steps (such as `gma.research` and `browser.render`) or high-throughput batch tasks, `nb-search` provides a unified asynchronous job-management mechanism. ## Job lifecycle and states When submitting an asynchronous request (`execution: "async"`), you must provide an `idempotency_key`. After accepting the request, the system returns a job receipt containing a valid UUID string in `job.job_id` for follow-up operations. An asynchronous job has the following public states: - `queued`: Entered the task queue and waiting to be processed. - `running`: Currently executing. - `succeeded`: Execution finished; an immutable artifact was successfully generated and published. - `failed`: Execution failed, for example because an upstream service failed or the `DEADLINE_EXCEEDED` budget was exhausted. - `cancelled`: Explicitly cancelled by the caller. > **Lifecycle state and business result are separate** The job lifecycle state `succeeded` means only that the job completed without crashing. It does not guarantee that the returned business data is rich. A partial business result (`partial`) or an empty result (`empty`) is reported explicitly in the artifact status field and the CLI exit code. ## Artifact immutability and retention (TTL) - After an asynchronous job finishes successfully, the system publishes an immutable JSON artifact file. - Retention is measured from the time of publication. The default retention period is 72 hours (`NB_SEARCH_RETENTION_HOURS=72`). - After the retention period, attempts to read the job return `JOB_NOT_FOUND`. The system does not automatically rerun the job or regenerate an expired artifact. ## CLI asynchronous workflow The CLI provides complete asynchronous subcommands: `run`, `get`, `read`, and `cancel`. Local tasks run directly; add `--profile ` explicitly for remote tasks. ### 1. Submit an asynchronous job and set a wait budget (`--wait`) An asynchronous submission must include `--idempotency-key`. Use `--wait` to set the number of milliseconds to wait synchronously in the current process (from 1 to 3,600,000 ms): ```bash node scripts/nb-search.mjs search "deep research brief" \ --lane gma.research \ --execution async \ --idempotency-key "brief-20260907-001" \ --wait 120000 ``` - **Completes during the wait**: The CLI prints the final reconstructed logical result. The exit code depends on the logical data state: `0` for complete success, `3` for a partial result, `4` for an empty result, and `2` for failure. - **Wait budget expires**: If the specified time elapses while the job continues in the background, the CLI prints timeout-status JSON and exits with **`5`** (not 7). The background job continues running; upstream computation is not cancelled automatically. Get the actual generated `job_id` from the submission receipt (for example, `3f1e9a2b-7c4d-4e1a-8f23-9b5d6e7f8a10`) for subsequent operations. ### 2. Query job status (`get`) ```bash node scripts/nb-search.mjs search get "3f1e9a2b-7c4d-4e1a-8f23-9b5d6e7f8a10" ``` If the job is still queued or running, the CLI exits with **`7`** (Pending). When metadata is retrieved successfully, it exits with `0` (which means only that the status was retrieved, not that the business data is complete). For a remote-service job, add `--profile ` explicitly. ### 3. Read the complete artifact (`read --all`) For a successfully published job, the underlying artifact is stored through pagination and Base64 slices. The caller does not need to join Base64 slices manually; use `--all` to read and reconstruct the standard logical object directly: ```bash node scripts/nb-search.mjs search read "3f1e9a2b-7c4d-4e1a-8f23-9b5d6e7f8a10" --all ``` `--all` automatically verifies slice continuity, offsets, SHA-256 integrity, UTF-8 encoding, and the corresponding Schema, then prints the complete logical JSON (search returns top-level `results` or a Typed payload; fetch returns top-level `documents`). > **Note** `--all` is a helper for reconstructing a complete artifact in one operation. Do not combine it with pagination parameters such as `--cursor` or `--page-size`. ### 4. Cancel a job (`cancel`) ```bash node scripts/nb-search.mjs search cancel "3f1e9a2b-7c4d-4e1a-8f23-9b5d6e7f8a10" ``` - Exit code `0` means only that the cancellation request was acknowledged. It does not guarantee that the upstream operation has fully stopped and does not promise any refund or recovery of incurred costs. - After cancelling, use `get` to check whether the job has actually transitioned to the terminal `cancelled` state. --- Source: https://nb-corp.github.io/nb-search/en/guide/troubleshooting # Troubleshooting This page summarizes common `nb-search` error codes, their causes, and the corresponding fixes. ## CLI exit-code quick reference | Exit code | Meaning | Description | |---:|---|---| | `0` | Success | The operation succeeded, including an offline doctor check, capability output, or an acknowledged cancellation request | | `2` | Error | Invalid call parameters, configuration errors, network-transfer errors, protocol-validation failures, job failures, or a migration block | | `3` | Partial success | The logical result is `partial`, or the migration report is partially complete | | `4` | Empty result | Search or fetch completed successfully but matched no results | | `5` | Wait timeout | The `--wait` budget was exhausted, or the underlying task reached its timeout threshold | | `6` | Cancelled | The job was explicitly cancelled by the caller | | `7` | Job in progress | The asynchronous job is still in `queued` or `running` state | ## Diagnose core error codes ### `DEFAULT_NOT_CONFIGURED` - **Cause**: In version `0.5.0` and later, `defaults.search_lane = "duckduckgo.search"` is configured by default. This error occurs only when the configuration file or a host override explicitly clears `defaults.search_lane` (sets it to `null`) and the search call does not pass `--lane`, `--lanes`, or `--preset`. - **Fix**: Pass an explicit `--lane ` on the command line, or restore `"defaults": { "search_lane": "duckduckgo.search" }` (or another configured source such as `"tavily.search"`) in `config.json`. ### `CREDENTIAL_NOT_CONFIGURED` / `LANE_NOT_CONFIGURED` - **Cause**: The environment-variable key required by the target Lane is missing or an empty string, or the Provider instance is disabled. - **Fix**: Run `node scripts/nb-search.mjs capabilities` to inspect what is missing for the Lane, then set the corresponding `NB_SEARCH_*_API_KEY` in the environment or in a protected credential file. ### `ENDPOINT_NOT_CONFIGURED` - **Cause**: A source that requires a custom base endpoint (such as `searxng.search`, `gma.research`, or `oac.synthesis`) was used without configuring its BASE_URL environment variable. - **Fix**: Check that variables such as `NB_SEARCH_SEARXNG_BASE_URL` or `NB_SEARCH_GROK_MULTI_AGENT_BASE_URL` are set. ### `LANE_EXECUTION_UNSUPPORTED` - **Cause**: The requested execution mode is not supported by the source. For example, a synchronous request was sent to the asynchronous-only `browser.render`, or an asynchronous request was sent to a custom query that supports only synchronous execution. - **Fix**: Use the static capability catalog to select a supported `sync` or `async` `execution` mode. ### `FETCH_CHAIN_UNAVAILABLE` - **Cause**: No Pipeline was specified, and the built-in default fetch chain cannot satisfy the current input source, or none of the Pipelines in the chain is ready. - **Fix**: Specify the concrete Pipeline explicitly with `--pipeline`, or complete the `fetch_chain` configuration for the relevant input type. ### `FETCH_BLOCKED` - **Cause**: The target URL violates the SSRF security policy, including a private IP, loopback address, cloud metadata address, or a redirect to a private-network address. - **Fix**: Ensure that the target is a publicly resolvable public HTTP(S) address. SSRF protection is a hard policy and cannot be bypassed. ### `FETCH_BYTES_LIMIT` - **Cause**: A fetch input source (local text or file) exceeds `execution.fetch.max_source_bytes`, or an HTTP response body exceeds `max_response_bytes`. - **Handling**: Distinguish the input and response paths. An oversized local input is rejected before entering the Pipeline; an oversized third-party proxy-fetch API response fails as a transport error; the `direct.fetch` Pipeline may truncate an oversized response within bounds and return it. See [Content Fetching: Security Limits](/en/guide/fetch#security-limits-and-network-protection) for the exact mechanism. Optimize the target content or adjust configuration for the actual scenario; do not blindly increase the limit. ### `FETCH_CONTENT_TYPE_REJECTED` - **Cause**: The response MIME type is not in the allowlist. - **Fix**: Confirm that the target content type is `text/html`, `text/plain`, or `text/markdown`. ### `QUALITY_GATE_FAILED` - **Cause**: The fetched body has fewer characters than the explicitly configured lower gate (`min_content_chars`), or it contains a blocking marker. - **Fix**: Check whether the target site requires JavaScript rendering. If so, use `browser.render` asynchronously, or adjust `quality.min_content_chars` in the configuration. ### `OUTPUT_TOO_LARGE` - **Cause**: In synchronous mode, the response envelope exceeds `execution.max_inline_bytes` (16 MiB by default). - **Handling**: The system does not automatically truncate a synchronous envelope. In fetch pipelines, `direct.fetch` and `direct.local` support both `sync` and detached `async`; `browser.render` supports only `async`; the other six third-party cloud pipelines (`jina.reader`, `exa.contents`, `tavily.extract`, `firecrawl.scrape`, `wayback.fetch`, and `oac.fetch`) support only `sync`. When the limit is exceeded, fetch can reduce `max_content_chars` (or explicitly use `--execution async` on a pipeline that supports asynchronous execution), while search can reduce `max_results` or switch to `--execution async`. Rerunning a paid-source request may incur additional charges; check the budget first. ### `ENTRY_UNAVAILABLE` - **Cause**: The build artifact `dist/cli.mjs` cannot be found when starting through `scripts/nb-search.mjs`. - **Fix**: When running from source, first run `pnpm build` from the repository root. When bringing it in as a Skill dependency, make sure the entire `dist/` directory is present. ## Offline configuration validation and capability inspection When diagnosing configuration or credential-binding problems, start with the offline diagnostic command: ```bash node scripts/nb-search.mjs --doctor ``` This command validates only local configuration syntax and the presence of credential environment variables. It reports syntax and slot summaries, but it does not prove that runtime ports are ready and does not make an external network request to verify that credentials are actually usable. To view source readiness and supported execution modes: ```bash node scripts/nb-search.mjs capabilities ``` In the returned catalog, inspect each source's `availability` (`ready` or `unavailable`), `issues` list, and supported `execution_modes`. Even `ready` means only that local initialization completed; it does not indicate the upstream service's current network connectivity or quota health. --- Source: https://nb-corp.github.io/nb-search/en/guide/upgrading # Upgrading This page explains version rules, important protocol-evolution notes, and compatibility boundaries when upgrading from an older release to the current version. ## Version coverage This page covers important compatibility boundaries for `@nb-corp/nb-search` as it evolves from `0.3.x` / `0.4.x` to `0.5.2`. Before upgrading or investigating compatibility, check the locally installed version and read the [CHANGELOG](/en/reference/changelog). Update the CLI or project SDK from the official npm registry: ```bash # Global CLI npm install -g @nb-corp/nb-search # Project SDK npm install @nb-corp/nb-search ``` ## Upgrade notes ### 0.5.1–0.5.2: Explicit Zhihu opt-in, multi-key rotation, and `keyUsage()` - **Opt-in Zhihu adapter (`zhihu`)**: Adds `zhihu.search`, `zhihu.global`, and `zhihu.synthesis`, but does not inject any instance, credential slot, or Lane into the built-in default configuration. Upgrading does not affect existing default configuration. See [Web search sources: Explicitly enable Zhihu](/en/sources/web-search#zhihu) for the enablement method and free-quota gate. - **Up to 32-key pool and `keyUsage()`**: Existing `NB_SEARCH_*_API_KEY` environment variables remain compatible with both a single key and a comma-separated multi-key list. The local SDK adds `runtime.keyUsage()` (`LocalNbSearchRuntime`); the remote client and MCP tool list remain unchanged. ### 0.5.0: Keyless default search, Defuddle direct fetch, and ordered URL routing - **Default search source (`duckduckgo.search`)**: When `defaults.search_lane` is not explicitly configured, general search uses keyless `duckduckgo.search` by default. If an upstream host depends on the old behavior of returning `DEFAULT_NOT_CONFIGURED` when no default source exists, explicitly set `"defaults": { "search_lane": null }` in `config.json`. - **Defuddle direct fetch and URL routing (`kiki-common-direct@1`)**: `direct.fetch` now uses Defuddle 0.19.3 + linkedom for static extraction and supports asynchronous jobs. Built-in URL routing sends known GitHub Raw/API, npm, and PyPI JSON targets strictly to `direct.fetch` (you can override or disable this through `fetch.routing`). - **In-progress job fingerprint changes**: The `duckduckgo` (v2), `direct-http`, and the new `crawl`/`research` operations in the `tavily` adapter update operation fingerprints. Before upgrading, wait for in-progress jobs from the old version to finish; persisted old snapshots are not silently rebuilt. ### 0.4.0: Script provider/Lane `0.4.0` added the `script` provider/Lane. It can reuse the existing CLI, SDK, and detached asynchronous job entry points without changing the existing API. Historical notes for the 0.2→0.3 upgrade still apply as written. ### 1. GMA protocol evolution and mode matching In early development snapshots, the GMA (Grok Multi-Agent) adapter had a different structure from the current version: - After the GMA adapter upgraded from Version 1 to Version 2, the new adapter could not deserialize old in-progress job snapshots. Before upgrading, make sure long-running old tasks have finished. - The new version supports an explicit `options.api_mode`: - `chat_completions`: The endpoint appends `/chat/completions`. - `messages`: The endpoint appends `/messages` and uses the structured relay contract. - **Endpoint-conflict protection**: If the configured Base URL already contains a suffix that does not match the mode (for example, putting Grok's `/responses` in a GMA endpoint), the system reports an error and disables the instance. It does not silently fall back. ### 2. Grok Responses independence `grok.synthesis` and `grok.x-synthesis` are built on the independent Grok Responses API and call the `/v1/responses` endpoint directly. They are not interchangeable with the traditional Chat Completions interface. If you migrate an old configuration, credentials that supported only Chat Completions cannot be used automatically for the new Responses operations. ### 3. Configuration Schema evolution and search-layer migration The current runtime uses **Configuration Schema Version 4** (`schema_version: "4"`). - If you maintain credentials from an early search layer (such as `SEARCH_LAYER_CREDENTIALS` or `~/.openclaw/credentials/search.json`), use the CLI's explicit import feature (`nb-search --import-search-layer --dry-run`) to preview a mapping report. After confirming that there are no conflicts, apply it with `--apply`. This tool is not a general-purpose upgrader for arbitrary old versions; see the [CLI migration guide](/en/reference/cli) for the exact rules. - Some private environment variables from early versions (such as `NB_SEARCH_GROK_MULTI_AGENT_API_KEY`) have been normalized in standard Schema v4. The default GMA slot environment variable is now unified with Grok as `NB_SEARCH_GROK_API_KEY`. ### 4. Concurrency control and state recovery In the execution configuration, `max_concurrency` (default `8`) and `max_provider_calls` (default `64`) control concurrency and call budgets inside one query execution. They are not a global queue shared by multiple independent CLI processes. - When multiple tasks or processes run and encounter a configuration-write conflict, wait for the preceding write or migration to finish. - If an abnormal process termination reports unreleased state, do not blindly delete the `jobs` artifact directory, backup files, or unknown locks. Follow the CLI diagnostics to confirm configuration and credential consistency before recovering. --- Source: https://nb-corp.github.io/nb-search/en/sources/ # Sources Catalog `nb-search` ships with 18 search lanes and 9 fetch pipelines. Use the standalone [nb-extract](https://github.com/NB-Corp/nb-extract) tool for local document conversion and Markdown/resource saving. These sources are grouped by use case, cost structure, latency characteristics, and output channel. > **About the dimensions** The relative cost (`free` / `cheap` / `expensive`) and relative latency (`fast` / `medium` / `slow`) in the tables are qualitative metadata labels declared strictly by the runtime configuration. They help an upper-layer orchestrator make coarse-grained choices; they are not actual billing data or benchmark scores. ## Use-case groups and configuration guidance The official built-in usage guidance (`src/lane-guidance.ts`) divides capabilities into three main groups. **These groups describe use cases and integration paths, not a quality ranking**: 1. **Onboarding: general discovery and reading (`onboarding`)**: - **Configuration guidance**: In `0.5.0` and later, the default is the keyless `duckduckgo.search` lane for general web search. Public HTML pages may trigger challenges or rate limits; the tool reports those errors as they are instead of pretending the result set is empty. Repository search and documentation context can be selected explicitly with `github.repositories` and `context7.docs`. You can also configure `tavily.search`, `brave.search`, `exa.search`, and other sources and set one as `defaults.search_lane`; an explicit default or selection takes precedence, and a failure does not silently switch sources. Fetching uses the built-in ordered URL routing package `kiki-common-direct@1` (known GitHub Raw/API, npm, and PyPI metadata use `direct.fetch` only). Other public URLs try `direct.fetch` (Defuddle) first and then the keyless `jina.reader` if that fails. - **Typical sources**: `duckduckgo.search`, `brave.search`, `exa.search`, `tavily.search`, `parallel.search`, `searxng.search`, and `zhipu.search`; fetch pipelines include `direct.fetch` and `jina.reader`. 2. **Specialist capabilities (`specialist`)**: - **Configuration guidance**: Select these explicitly for particular data types, page extraction, or controlled environments. Usually specify them directly with `--lane`, `--lanes`, or `--pipeline` rather than using them as a fallback for the entire web. - **Typical sources**: - Code and engineering: `github.repositories` (repository search), `context7.docs` (framework documentation context). - Deep web/social search: `firecrawl.search` (search with extracted content), `grok.x-synthesis` (synthesis of discussions on X). - Dedicated fetchers: `exa.contents`, `tavily.extract`, `firecrawl.scrape`, `direct.local` (compatible local files and inline content without network egress), `wayback.fetch` (historical snapshots), `browser.render` (headless Chromium rendering), and `oac.fetch` (model-based extraction and summarization). 3. **Synthesis and deep research (`research`)**: - **Configuration guidance**: Use these when a model must perform multi-step reasoning, produce a synthesis, or generate a structured report. They have comparatively higher cost and latency and must be invoked as a single explicit lane; they cannot be placed in a multi-source Results preset. - **Typical sources**: `exa.synthesis`, `tavily.synthesis`, `grok.synthesis`, `gma.research`, and `oac.synthesis`. > **Diagnostics and capability inspection** Run `node scripts/nb-search.mjs --doctor` locally to print a readiness summary with the use-case groups. Call `capabilities` to inspect each source's live port activation and supported modes. The "supported modes" listed below apply to the standard local runtime when no custom `http_transport` is injected; an explicitly injected custom transport disables all async modes. `readiness` reflects local configuration completeness only, not whether an external upstream is online. ## 18 search lanes The 18 built-in search lanes use different underlying providers and operations: | Lane ID | Provider / operation | Output channel and schema | Supported modes (when ready) | Required credentials and endpoint configuration | Relative cost / latency | Characteristics and endpoint conventions | |---|---|---|---|---|---|---| | `brave.search` | `brave/search` | Results (`nb-search.results@1`) | sync, async | `NB_SEARCH_BRAVE_API_KEY` required; Base URL optional | cheap / fast | Independent web search index; appends `/res/v1/web/search` to the endpoint | | `exa.search` | `exa/search` | Results (`nb-search.results@1`) | sync, async | `NB_SEARCH_EXA_API_KEY` required; Base URL optional | cheap / fast | Neural search; appends `/search` to the endpoint | | `exa.synthesis` | `exa/synthesis` | Typed (`nb-search.synthesis@1`) | sync, async | `NB_SEARCH_EXA_API_KEY` required; Base URL optional | expensive / medium | Synthesis-oriented search that returns paragraphs and citations | | `tavily.search` | `tavily/search` | Results (`nb-search.results@1`) | sync, async | `NB_SEARCH_TAVILY_API_KEY` required; Base URL optional | cheap / fast | Agent-optimized search; appends `/search` to the endpoint | | `tavily.synthesis` | `tavily/synthesis` | Typed (`nb-search.synthesis@1`) | sync, async | `NB_SEARCH_TAVILY_API_KEY` required; Base URL optional | cheap / medium | Synthesis-oriented results with a direct answer | | `tavily.crawl` | `tavily/crawl` | Typed (`nb-search.crawl@1`) | sync, async | `NB_SEARCH_TAVILY_API_KEY` required; Base URL optional | cheap / slow | Official Crawl POST; returns a bounded Markdown document set | | `tavily.research` | `tavily/research` | Typed (`nb-search.research@1`) | sync, async | `NB_SEARCH_TAVILY_API_KEY` required; Base URL optional | expensive / slow | Official Research Task POST/GET polling; returns content and sources | | `firecrawl.search` | `firecrawl/search` | Results (`nb-search.results@1`) | sync, async | `NB_SEARCH_FIRECRAWL_API_KEY` required; Base URL optional | cheap / fast | Search endpoint; appends `/v2/search` to the endpoint | | `searxng.search` | `searxng/search` | Results (`nb-search.results@1`) | sync, async | No key; `NB_SEARCH_SEARXNG_BASE_URL` required | free / medium | Self-hosted or private aggregated search instance; appends `/search` to the endpoint | | `zhipu.search` | `zhipu/search` | Results (`nb-search.results@1`) | sync, async | `NB_SEARCH_ZHIPU_API_KEY` required; Base URL optional | cheap / fast | Zhipu Web Search; appends `/paas/v4/web_search` to the endpoint | | `parallel.search` | `parallel/search` | Results (`nb-search.results@1`) | sync, async | `NB_SEARCH_PARALLEL_API_KEY` required | cheap / fast | Fixed endpoint: `https://api.parallel.ai/v1/search` | | `github.repositories` | `github/repositories` | Results (`nb-search.results@1`) | sync, async | `NB_SEARCH_GITHUB_TOKEN` optional | free / fast | On-demand repository search; anonymous requests are more vulnerable to rate limits | | `duckduckgo.search` | `duckduckgo/search` | Results (`nb-search.results@1`) | sync, async | No credentials or self-hosted endpoint required | free / medium | **Zero-key default general web search**; public HTML search reports challenges or rate limits directly | | `context7.docs` | `context7/docs` | Typed (`nb-search.docs-context@1`) | sync, async | `NB_SEARCH_CONTEXT7_API_KEY` optional | cheap / medium | Framework and code-library documentation context extraction; uses public mode without a key | | `grok.synthesis` | `grok/synthesis` | Typed (`nb-search.synthesis@1`) | sync, async | `NB_SEARCH_GROK_API_KEY` required; Base/Model optional | expensive / slow | Based on the xAI Grok Responses API with the `web_search` tool enabled | | `grok.x-synthesis` | `grok/x-synthesis` | Typed (`nb-search.synthesis@1`) | sync, async | `NB_SEARCH_GROK_API_KEY` required; Base/Model optional | expensive / slow | Based on the xAI Grok Responses API with the `x_search` tool enabled | | `gma.research` | `grok-multi-agent/research` | Typed (`nb-search.multi-agent-research@1`) | sync, async | `NB_SEARCH_GROK_API_KEY` and `NB_SEARCH_GROK_MULTI_AGENT_BASE_URL` required | expensive / slow | Deep multi-agent reasoning; supports `chat_completions` and `messages` modes | | `oac.synthesis` | `openai-compatible/synthesis` | Typed (`nb-search.synthesis@1`) | sync, async | `NB_SEARCH_OAC_API_KEY`, `NB_SEARCH_OAC_BASE_URL`, and `NB_SEARCH_OAC_MODEL` required | expensive / slow | Structured synthesis through an OpenAI-compatible Chat Completions endpoint | ## 9 fetch pipelines | Pipeline ID | Provider / operation | Supported input and output | Supported modes (when ready) | Credential and dependency requirements | Relative cost / latency | Default fetch-chain position and description | |---|---|---|---|---|---|---| | `direct.fetch` | `direct-http/fetch` | URL -> `nb-search.fetch@1` | sync, async | No credentials; direct HTTP request | free / fast | **First in the default URL fetch chain**; Defuddle extracts static HTML, JSON, and GitHub API text | | `jina.reader` | `jina-reader/reader` | URL -> `nb-search.fetch@1` | sync | `NB_SEARCH_JINA_API_KEY` optional | free / medium | **Second in the default URL fetch chain**; extracts page text and converts it to Markdown | | `direct.local` | `direct-http/local` | inline / file -> `nb-search.fetch@1` | sync, async | No credentials; `file_scopes` required for files | free / fast | **Only item in the default inline/file fetch chain**; no network egress (`egress: none`) | | `exa.contents` | `exa/contents` | URL -> `nb-search.fetch@1` | sync | `NB_SEARCH_EXA_API_KEY` required | cheap / fast | Requires explicit selection or a custom chain; calls Exa's `/contents` extraction endpoint | | `tavily.extract` | `tavily/extract` | URL -> `nb-search.fetch@1` | sync | `NB_SEARCH_TAVILY_API_KEY` required | cheap / fast | Requires explicit selection or a custom chain; calls Tavily's `/extract` page-extraction endpoint | | `firecrawl.scrape` | `firecrawl/scrape` | URL -> `nb-search.fetch@1` | sync | `NB_SEARCH_FIRECRAWL_API_KEY` required | cheap / medium | Requires explicit selection or a custom chain; calls Firecrawl's `/v2/scrape` endpoint | | `wayback.fetch` | `wayback/fetch` | URL -> `nb-search.fetch@1` | sync | No credentials; automatically queries Wayback archive snapshots | free / medium | Requires explicit selection or a custom chain; useful for pages that are offline or require historical snapshots | | `browser.render` | `browser-render/render` | URL -> `nb-search.fetch@1` | async | Playwright + Chromium required; async only | free / slow | Requires explicit selection or a custom chain; renders client-side JavaScript pages through a browser engine | | `oac.fetch` | `openai-compatible/fetch` | URL -> `nb-search.fetch@1` | sync | OAC key, Base URL, and Model required | expensive / slow | Requires explicit selection or a custom chain; uses a compatible model to summarize and extract page content | ## Explicit opt-in and custom sources The built-in adapters below do not create default instances or lanes automatically. Declare them explicitly in `config.json` or host code before enabling them: - **Built-in Zhihu source (`provider_id: "zhihu"`)**: Supports community Q&A and column search (`zhihu.search`), Chinese web-wide search (`zhihu.global`), and direct synthesis (`zhihu.synthesis`). It includes a free-quota gate through `GET /api/v1/quota` (default `allow_paid: false`) and a local `keyUsage()` metric. See the complete Schema v4 configuration in [Web & Regional Search: opt-in Zhihu](/en/sources/web-search#zhihu). - **Local script source (`provider_id: "script"`) and SDK custom registration**: Bind trusted local `.js`/`.mjs`/`.ts`/`.mts` modules to standard search lanes that support `sync` and detached `async`, or inject a custom adapter into the host process through `ProviderRegistration`. See [Custom Sources and Script Lanes](/en/sources/custom). --- Source: https://nb-corp.github.io/nb-search/en/sources/web-search # Web & Regional Search Web-search lanes focus on returning normalized web links, page summaries, and relevance ranking. The standard Results channel is `nb-search.results@1`; direct Zhihu synthesis through `zhihu.synthesis` returns `nb-search.synthesis@1`. ## Keyless default search source ### DuckDuckGo (`duckduckgo.search`) - **Operation contract**: `duckduckgo/search`; outputs `nb-search.results@1`; supports `sync` and `async`. - **Default behavior**: In version `0.5.0` and later, this is the built-in default search source (`defaults.search_lane = "duckduckgo.search"`). You can search the general web without registering an account, configuring an API key, or passing `--lane` manually. - **Errors and rate limits**: Public HTML search endpoints may trigger a CAPTCHA or `429` rate limiting (the `Retry-After` metadata is retained). The runtime returns challenge pages and unrecognized pages as explicit errors, never disguises them as empty results (`empty`), and never silently switches to a paid search source. - **Metadata labels**: Cost `free`; latency `medium`. ## BYOK general and regional search sources ### 1. Tavily Search (`tavily.search`) - **Operation contract**: `tavily/search`; outputs `nb-search.results@1`; supports `sync` and `async`. - **Required configuration**: `NB_SEARCH_TAVILY_API_KEY` (supports comma-separated rotation of up to 32 keys; the local SDK exposes `keyUsage()` for periodic usage inspection) - **Optional configuration**: `NB_SEARCH_TAVILY_BASE_URL` (retains the URL prefix and appends `/search`) - **Metadata labels**: Cost `cheap`; latency `fast`. - **Advanced lanes from the same source**: The same instance also provides `tavily.synthesis` (`nb-search.synthesis@1`), `tavily.crawl` (`nb-search.crawl@1`, default timeout 180s), and `tavily.research` (`nb-search.research@1`, default timeout 900s). ### 2. Exa Search (`exa.search`) - **Operation contract**: `exa/search`; outputs `nb-search.results@1`; supports `sync` and `async`. - **Required configuration**: `NB_SEARCH_EXA_API_KEY` - **Optional configuration**: `NB_SEARCH_EXA_BASE_URL` (appends `/search` to the endpoint) - **Metadata labels**: Cost `cheap`; latency `fast`. ### 3. Brave Search (`brave.search`) - **Operation contract**: `brave/search`; outputs `nb-search.results@1`; supports `sync` and `async`. - **Required configuration**: `NB_SEARCH_BRAVE_API_KEY` - **Optional configuration**: `NB_SEARCH_BRAVE_BASE_URL` (appends `/res/v1/web/search` to the endpoint) - **Metadata labels**: Cost `cheap`; latency `fast`. ### 4. Firecrawl Search (`firecrawl.search`) - **Operation contract**: `firecrawl/search`; outputs `nb-search.results@1` with page-content excerpts; supports `sync` and `async`. - **Required configuration**: `NB_SEARCH_FIRECRAWL_API_KEY` (the local SDK exposes `keyUsage()` to inspect remaining team credits) - **Optional configuration**: `NB_SEARCH_FIRECRAWL_BASE_URL` (retains the URL prefix and appends `/v2/search`) - **Metadata labels**: Cost `cheap`; latency `fast`. ### 5. SearXNG (`searxng.search`) - **Operation contract**: `searxng/search`; outputs `nb-search.results@1`; supports `sync` and `async`. - **Required configuration**: `NB_SEARCH_SEARXNG_BASE_URL` (self-hosted or private aggregated instance; no API key; appends `/search` to the endpoint) - **Metadata labels**: Cost `free`; latency `medium`. ### 6. Zhipu Search (`zhipu.search`) - **Operation contract**: `zhipu/search`; outputs `nb-search.results@1`; supports `sync` and `async`. - **Required configuration**: `NB_SEARCH_ZHIPU_API_KEY` - **Optional configuration**: `NB_SEARCH_ZHIPU_BASE_URL` (appends `/paas/v4/web_search` to the endpoint) - **Metadata labels**: Cost `cheap`; latency `fast`. ### 7. Parallel Search (`parallel.search`) - **Operation contract**: `parallel/search`; outputs `nb-search.results@1`; supports `sync` and `async`. - **Required configuration**: `NB_SEARCH_PARALLEL_API_KEY` - **Endpoint**: Always calls `https://api.parallel.ai/v1/search` - **Metadata labels**: Cost `cheap`; latency `fast`. ## Opt-in Zhihu source {#zhihu} The built-in `zhihu` adapter (`provider_id: "zhihu"`) is opt-in. The default configuration does not inject a `provider_instance`, `credential_slot`, or `lane`. To use it, declare all three in a Schema v4 `config.json` (the `latency` and `cost` values below are orchestration metadata labels, not request quota limits): ```json { "schema_version": "4", "provider_instances": { "zhihu.manual": { "provider_id": "zhihu", "enabled": true, "credential_slot_id": "zhihu.manual", "base_url": "https://developer.zhihu.com", "options": { "allow_paid": false, "search_db": "all", "zhida_model": "zhida-fast-1p5", "search_path": "/api/v1/content/zhihu_search", "global_search_path": "/api/v1/content/global_search", "synthesis_path": "/v1/chat/completions", "quota_path": "/api/v1/quota" } } }, "credential_slots": { "zhihu.manual": { "provider_id": "zhihu", "env": "NB_SEARCH_ZHIHU_API_KEY" } }, "lanes": { "zhihu.search": { "provider_instance_id": "zhihu.manual", "operation_id": "search", "latency": "fast", "cost": "cheap" }, "zhihu.global": { "provider_instance_id": "zhihu.manual", "operation_id": "global", "latency": "fast", "cost": "cheap" }, "zhihu.synthesis": { "provider_instance_id": "zhihu.manual", "operation_id": "synthesis", "latency": "medium", "cost": "cheap" } } } ``` Set `NB_SEARCH_ZHIHU_API_KEY="YOUR_API_KEY"`, then select the required lane explicitly: ```bash nb-search search "人工智能架构演进" --lane zhihu.search nb-search search "人工智能架构演进" --lane zhihu.global nb-search search "比较近期端侧模型推理框架的工程取舍" --lane zhihu.synthesis ``` ### Operation contracts and path parameters - **`zhihu.search`**: Calls `GET /api/v1/content/zhihu_search` and outputs `nb-search.results@1`. After trimming leading and trailing whitespace, the query must be **2–100 characters** long. The requested count is clamped to **1–10**. The summary prefix retains labels for content type, author, authority level, upvotes, and comments. - **`zhihu.global`**: Calls `GET /api/v1/content/global_search` and outputs `nb-search.results@1`; the requested count is clamped to **1–20**. `options.search_db` defaults to `"all"` and also accepts `"realtime"` or `"static"`. - **`zhihu.synthesis`**: Calls `POST /v1/chat/completions` and outputs a single `typed` channel with schema `nb-search.synthesis@1`; it cannot be mixed with the `results` channel in `--lanes`. `options.zhida_model` defaults to `"zhida-fast-1p5"` and also accepts `"zhida-thinking-1p5"`, `"zhida-agent"`, or any other non-empty custom model name. - **URL prefix and path overrides**: All three lanes support `sync` and `async`. `base_url` defaults to `https://developer.zhihu.com`; endpoint paths are joined according to URL-prefix semantics for `search_path`, `global_search_path`, `synthesis_path`, and `quota_path`. Reverse-proxy prefixes are preserved, such as `https://relay.example.test/zhihu`. ### Free-quota gate (`allow_paid`) and `keyUsage()` metrics - **Block over-quota billing by default (`allow_paid: false`)**: Before dispatching a request, the runtime verifies free quota through `GET /api/v1/quota`. If free quota is exhausted or verification cannot complete, it rejects the request directly (fail-closed). Instances that point to the same quota endpoint and credentials share a reservation ledger within one runtime. If a request is interrupted and its result is undecided, the reservation remains until upstream `TotalUsed` advances or the daily quota resets at `00:00` Beijing time (`Asia/Shanghai`, `UTC+8`). A quota snapshot is cached for at most 60 seconds only when more than 32 requests remain. Set `"allow_paid": true` explicitly only when wallet billing is intentionally allowed. - **Local SDK `keyUsage()` metric**: `runtime.keyUsage()` also reads `GET /api/v1/quota` and returns per-API request counters in `KeyUsage.usage.meters`. For the compatible credits summary, it prefers `zhihu_search` and falls back to the first meter when absent. The key is marked `exhausted` only when both `zhihu_search` and `global_search` have zero remaining quota. ## Multi-source RRF queries All web-search sources that output `nb-search.results@1`, including enabled `zhihu.search` and `zhihu.global`, can run in parallel through comma-separated `--lanes`, repeated `--lane` options, or configured presets: ```bash [Comma-separated form] nb-search search "web standard fetch streaming spec" \ --lanes exa.search,tavily.search,brave.search ``` ```bash [Repeated-option form] nb-search search "web standard fetch streaming spec" \ --lane exa.search --lane tavily.search ``` During aggregation, the runtime canonicalizes and deduplicates URLs, then calculates global rankings with Reciprocal Rank Fusion (RRF), while retaining each source's `evidence_groups` and `provenance`. --- Source: https://nb-corp.github.io/nb-search/en/sources/developer # Developer and Specialist Sources In addition to general web search engines, `nb-search` provides specialist sources for software engineering and technical documentation. ## GitHub repository search (`github.repositories`) - **Operation contract**: `github/repositories`; outputs `nb-search.results@1`; supports `sync` and `async`. - **Metadata labels**: Cost `free`; latency `fast`. - **Configuration and authentication**: - Optional environment variable: `NB_SEARCH_GITHUB_TOKEN` - Always calls the official endpoint: `https://api.github.com/search/repositories` > **Anonymous rate limits** This source still works without a GitHub Token (keyless mode), but GitHub applies very low rate limits to unauthenticated IP requests. Configure a Token in normal environments to avoid rate-limit errors. ### Query example ```bash node scripts/nb-search.mjs search "language:typescript stars:>1000 web search" \ --lane github.repositories ``` ## Context7 documentation search (`context7.docs`) - **Operation contract**: `context7/docs`; outputs `nb-search.docs-context@1`; supports `sync` and `async`. - **Metadata labels**: Cost `cheap`; latency `medium`. - **Configuration and authentication**: - Optional environment variable: `NB_SEARCH_CONTEXT7_API_KEY` - Default endpoint: `https://context7.com` (calls `/api/v2/libs/search` and `/api/v2/context`) When no API Key is configured, the source uses its public free mode. It returns documentation-context slices suitable for a coding assistant consulting a library manual. ### Query example ```bash node scripts/nb-search.mjs search "fastify register plugins hook lifecycle" \ --lane context7.docs ``` --- Source: https://nb-corp.github.io/nb-search/en/sources/grok # Grok, Multi-Agent, and Model Sources For tasks that need more than a list of links—such as having a large model combine live retrieval with a structured report—`nb-search` provides dedicated typed search sources. ## Grok synthesis sources (`grok.synthesis` / `grok.x-synthesis`) These two sources are built on xAI's **Grok Responses API** and output through `nb-search.synthesis@1`. - Metadata labels: cost `expensive`; latency `slow`. - `grok.synthesis`: enables the `web_search` tool for multi-step synthesis and reasoning over public information across the web. - `grok.x-synthesis`: enables the `x_search` tool to search and summarize current discussions and context on X. ### Configuration requirements - Required environment variable: `NB_SEARCH_GROK_API_KEY` - Optional environment variables: `NB_SEARCH_GROK_BASE_URL` (the system resolves it and appends `/v1/responses`), `NB_SEARCH_GROK_MODEL` > **Different from Chat Completions** The Grok Responses API has a separate endpoint and request format (using the `input` field and dedicated tool declarations), so it is not interchangeable with the traditional `/chat/completions` endpoint. The system never implicitly falls back from Grok Responses to the legacy chat interface. ## GMA deep multi-agent research (`gma.research`) `gma.research` is designed for research tasks that require multi-step reasoning and outputs through `nb-search.multi-agent-research@1`. - Metadata labels: cost `expensive`; latency `slow`. ### Protocol modes and endpoint conventions GMA selects its communication protocol through `options.api_mode` and supports two modes: 1. `chat_completions` (default): appends `/chat/completions` to the Base URL. 2. `messages`: appends `/messages` to the Base URL and follows a specific relay contract (POST `/messages`, a top-level `system` field, `x-api-key` and `anthropic-version` request headers, and `reasoning: { effort }` in the JSON request body). Provider requests in both GMA modes use `stream: true`, so the gateway must support SSE. This streaming is part of the provider request contract; `execution: sync|async` still describes the local job lifecycle, and the two concepts are independent. The `public SDK` and CLI still return complete typed JSON and do not expose a token stream to users. Configuration example: ```json { "schema_version": "4", "provider_instances": { "grok-multi-agent.default": { "provider_id": "grok-multi-agent", "enabled": true, "base_url": "https://relay.example.test/v1", "options": { "api_mode": "messages", "reasoning_effort": "high" } } } } ``` > **Do not mix endpoint suffixes or follow redirects** The Base URL suffix must match the selected mode exactly. For example, when the mode is `messages`, the Base URL must not already include `/chat/completions` or `/responses`, and vice versa. If an endpoint conflict is detected, the instance is marked unavailable and fails directly. In addition, both GMA modes reject HTTP 3xx redirects to prevent credential disclosure. ### Required configuration - `NB_SEARCH_GROK_API_KEY`: GMA reuses this credential slot by default. - `NB_SEARCH_GROK_MULTI_AGENT_BASE_URL`: the service entry point for the agent cluster. ### Parallel topics and execution deadlines GMA calls run under the host's existing authorization and budget. Independent research topics can be processed in parallel in one `query` array or submitted as separate asynchronous jobs: 1. **Submit multiple topics**: - **Multiple queries in one job**: Provide several complete briefs in the `query` array of one request. The queries share that job's timeout, cancellation, and retry state. - **Independent async jobs**: Submit separate asynchronous tasks when each topic needs exact tracking, cancellation, or independent retry. Give each task its own stable `idempotency_key`. When a typed batch result is `partial`, it does not include an index mapping to the original `query` array, so topics cannot be associated by array position alone. 2. **Execution timeout mechanism**: - Timeout precedence is request-level `timeout_ms` > explicit `execution.search_timeout_ms` > the default. When neither of the first two is provided, `grok-multi-agent/research` uses 600000 ms (10 minutes) by default; ordinary search remains 30000 ms. - CLI `--wait` is only the local client's waiting budget. It does not change the server or provider execution deadline. - The local default maximum concurrency per job is 8 (`max_concurrency: 8`), and the planned call budget is capped at 64. Cloud execution applies corresponding bounded limits per job and tenant. ```json { "action": "run", "query": [ "Analyze cold-start optimization approaches for WebAssembly in server-side runtimes and compare mainstream approaches", "Survey the current state and typical challenges of eBPF-based cloud-native network observability in 2026" ], "lane": "gma.research", "execution": "async", "idempotency_key": "gma-research-batch-2026-09-08", "timeout_ms": 600000 } ``` ## OpenAI-compatible synthesis source (`oac.synthesis`) Use this source to connect any self-hosted model service or gateway that follows the OpenAI Chat Completions specification. It outputs through `nb-search.synthesis@1`. - Metadata labels: cost `expensive`; latency `slow`. ### Configuration requirements - Required environment variables: - `NB_SEARCH_OAC_API_KEY`: access key. - `NB_SEARCH_OAC_BASE_URL`: service endpoint (the system appends `/chat/completions`). - `NB_SEARCH_OAC_MODEL`: model name. - Optional JSON configuration: `options.fallback_models` (a list of fallback models). Configuration example: ```json { "schema_version": "4", "provider_instances": { "openai-compatible.default": { "provider_id": "openai-compatible", "enabled": true, "base_url": "https://llm.example.test/v1", "options": { "model": "qwen-max", "fallback_models": ["qwen-plus"] } } } } ``` > **Compatibility note** OAC provides a standard HTTP JSON adapter, but the exact response structure depends on how well the upstream model follows the prompt and JSON Schema. The runtime itself does not guarantee the semantic accuracy of a self-hosted upstream model. --- Source: https://nb-corp.github.io/nb-search/en/sources/custom # Custom Sources and Script Lanes Bind trusted JS/TS modules directly in static configuration so the standard CLI, MCP, and SDK can use the same custom Results lane. You can also continue to inject a complete `ProviderRegistration` through the SDK. ## Local scripts: available to the CLI and SDK The built-in `script` adapter provides a `search` operation (`nb-search.results@1`), but it does not create or enable an instance or lane automatically: ```json { "provider_instances": { "local-script": { "provider_id": "script", "enabled": true, "options": { "module": "./search.mjs", "params": { "label": "local" } } } }, "lanes": { "local.search": { "provider_instance_id": "local-script", "operation_id": "search", "latency": "fast", "cost": "free" } } } ``` ```js // search.mjs — no network needed export async function execute(request, context) { context.signal.throwIfAborted(); context.logger.info('Local search started'); // stderr, not stdout return [{ title: request.query, url: 'https://example.com/', snippet: context.options.label }]; } ``` Run `nb-search search "hello" --lane local.search`. Async calls use the existing `execution:"async"`, `idempotency_key`, and get/read/cancel contracts. The SDK can receive the same configuration structure directly through `config`; no `provider_registrations` field is required. A complete no-network example is included in `examples/script-lane/` in the package. ### Module contract - `options.module`: a local `.js`, `.mjs`, `.ts`, or `.mts` file. Relative paths in the canonical configuration file are resolved from the configuration directory; relative paths in an SDK inline `config`/`overrides` are resolved from the cwd used to create the runtime, so SDK callers should prefer absolute paths. The normalized absolute path is stored in the detached snapshot; the worker does not resolve it again from cwd. - `options.params`: an optional JSON object passed as `context.options`; it is copied independently for each call. Do not put secrets in `params`, because they are included in configuration and job snapshots. - The named export `execute(request, context)` takes precedence. `search(query, context)` is also supported, with a `context` that additionally contains the complete `request`. Return `ProviderResult[]` or the corresponding Promise. Each result must include string `title` and `url` fields and may include standard fields such as `snippet`. Typed objects and fetch output are not accepted in the first release. - `request` contains `query`, `limit`, optional `freshness`, `request_time_utc`, and `signal`. `context` contains `signal`, `options`, an optional single `credential` string, `transport: HttpTransport`, and `logger.info/warn/error`. - When credentials are needed, bind them through existing `credential_slots` and the instance's `credential_slot_id`; `provider_id` must be `script`. Credentials from other slots are not placed in the context. - Root-exported types are `ScriptRequest`, `ScriptContext`, `ScriptResults`, and `ScriptModule`. Use `import type` in TypeScript. Node ≥24.15 performs native type erasure: no TS loader is installed and tsconfig path mappings are not executed. Local imports must include their full extension; enums, parameter properties, syntax requiring code transformation, and TSX are unsupported. Ordinary dependencies follow Node package resolution rules based on the module location; install dependencies separately, and TypeScript source dependencies in `node_modules` remain subject to Node's own restrictions. ### Execution and trust boundary Modules are lazy-imported; capabilities and configuration validation do not execute code. A missing module or missing export becomes a safe provider error only when the module is actually called. Node caches modules, so multiple queries can enter the same module concurrently; do not use shared mutable state to store request data. The snapshot freezes the path and parameters but not the script file contents; keep the module and its dependencies available and version-stable until the job ends. **This is user-authorized local code, not a sandbox**: the module runs with the same permissions as the process and can read files, environment variables, and the network, or start programs. Passing one credential in `context` does not isolate access to other processes. Cancellation and timeouts require the module to cooperate with `AbortSignal`; a synchronous infinite loop blocks the process, and asynchronous side effects that do not cooperate are not guaranteed to stop. Do not write to `console.log`/stdout from a script, because that would corrupt CLI JSON; use the provided stderr logger. The SDK does not globally replace `console`. Logs automatically mask only the bound credentials and module path; arbitrary data printed by the module is not guaranteed to be safe. Cloud hosts may install trusted modules themselves and map lanes through a deployment manifest/allowlist, but should not accept arbitrary module paths from tenant requests. `context.transport` is a reusable host transport, not an interceptor for all network I/O by the module. Explicit SDK `http_transport` injection still disables detached async; a normal runtime without injected transport supports script `sync`/`async`. ## SDK manual registration (existing contract) The following rules apply only to code registrations injected through SDK `provider_registrations`; they do not restrict the built-in `script` adapter above. Manual registration declares a `ProviderRegistration` interface exported by the SDK: 1. **Unique identifier**: `provider_id` must not conflict with a built-in Provider or another custom Provider. 2. **Strict descriptor**: - `activation.credential` accepts `'required' | 'none'` (built-in types accept `'required' | 'optional' | 'none'`). - `activation.endpoint` accepts `'required' | 'optional' | 'none'`. - General search output must use `channel: 'results'` with the fixed schema `nb-search.results@1`; specialist structured search must use `channel: 'typed'` and declare a stable `schema_id`. - Fetch output must use `nb-search.fetch@1`. 3. **Execution-mode constraints**: - Custom query operations are synchronous only in the current version. Even when a descriptor declares `built_in_async: true`, the runtime does not expose asynchronous execution for it. - A custom fetch pipeline supports synchronous calls only when `sync` is included in its declaration. 4. **Configuration binding**: - `lanes` and `provider_instances` in configuration are Record maps (key-value objects, not arrays). - Declare the corresponding Provider instance in `provider_instances`, then bind the concrete lane to that instance and operation in `lanes`. ## Complete SDK registration example The following example registers a minimal synchronous in-process search source. The code follows the real type system and passes strict type checking: ```typescript import { createNbSearchRuntime, type ProviderRegistration } from '@nb-corp/nb-search'; const hostQueryRegistration: ProviderRegistration = { descriptor: { provider_id: 'host-query', adapter_version: 'host-1', query_operations: [ { operation_id: 'lookup', output: { channel: 'results', schema_id: 'nb-search.results@1' }, built_in_async: false } ], fetch_operations: [], activation: { credential: 'none', endpoint: 'none' }, option_keys: [] }, create() { return { query: { lookup: { name: 'host-query', async execute(request) { return { channel: 'results', value: { results: [ { title: 'Internal Docs', url: 'https://wiki.example.test/doc/1', snippet: request.query } ] } }; } } }, fetch: {} }; } }; const runtime = await createNbSearchRuntime({ provider_registrations: [hostQueryRegistration], config: { schema_version: '4', provider_instances: { 'host-query.default': { provider_id: 'host-query', enabled: true, options: {} } }, lanes: { 'host.lookup': { provider_instance_id: 'host-query.default', operation_id: 'lookup', latency: 'fast', cost: 'free' } }, defaults: { search_lane: 'host.lookup' } } }); // Run the query const response = await runtime.search({ action: 'run', query: 'microservice architecture', execution: 'sync' }); ``` After registration, `host.lookup` can be called through `runtime.search` in the current host process. The standard CLI cannot reconstruct the closure registration shown here; when CLI and detached async support are required, prefer the script-module configuration at the beginning of this page. Manual registration remains suitable for custom typed output, fetch, or host-closure dependencies. --- Source: https://nb-corp.github.io/nb-search/en/reference/cli # Official CLI Use Node 24.15+ and a built package. The packaged launcher needs no global installation: ```sh node "/scripts/nb-search.mjs" --help node "/scripts/nb-search.mjs" --version ``` In a checkout, run `pnpm build` first and use `node scripts/nb-search.mjs`. An installed `nb-search` command accepts identical arguments. The only business commands are search, fetch, and capabilities. All examples below abbreviate the launcher as `nb-search`. ## Local and remote profiles Local is the default. Global options precede the command: ```sh nb-search --profile local capabilities nb-search --profile cloud capabilities nb-search --profile cloud --doctor ``` `--doctor` is an offline configuration/credential-presence and semantic check, never a provider probe or remote health check. Its local `guide` groups configured lanes into onboarding/general, specialist/on-demand, research/synthesis or unclassified custom operations, using actual provider/operation/output identity rather than lane spelling. Groups are not quality rankings. Cost/latency remain relative; guide readiness is not_evaluated unless supplied from a real runtime catalog, and live_verified is false. Start with one configured general results source instead of enabling every research lane. Local capabilities also performs no provider probe. Remote capabilities sends one authenticated catalog request, not a search/provider request. The CLI reads `profiles.json` under `NB_SEARCH_HOME` (default OS home `.nb-search`): ```json {"schema_version":"1","profiles":{"cloud":{"kind":"remote","base_url":"https://search.example/","token_env":"NB_SEARCH_CLOUD_TOKEN","timeout_ms":120000}}} ``` Supply the service key through that env reference or protected `remote-secrets.json`: ```json {"schema_version":"1","values":{"NB_SEARCH_CLOUD_TOKEN":""}} ``` Never put a token in argv, a URL, or profiles.json. Remote selection does not open canonical provider config or local provider secrets. `local` is reserved; no remote default is inferred. Unknown profiles fail. HTTPS is required; `allow_loopback_http:true` permits HTTP only for literal localhost/127.0.0.1/::1 development services. Optional profile limits mirror the remote factory: `timeout_ms`, `max_request_bytes`, `max_response_bytes`. No redirects, automatic retries, uploads, or silent local fallback. Local canonical configuration follows the SDK's normal precedence. Protected `secrets.json` has the same versioned values-map format, optionally with private imported-key binding metadata generated by migration. File values fill only **absent** process-env names; empty existing env entries intentionally disable a key. The CLI does not modify process.env, global environment, or shell startup files. Its runtime and detached worker receive the same private env copy; snapshots retain only grants, not secret values. SDK/MCP hosts remain responsible for providing their own env. ## Queries and complete JSON input ```sh nb-search search "query" --lane exa.search nb-search search "facet one" --query "facet two" --lanes exa.search,tavily.search nb-search search "research brief" --lane gma.research nb-search search --query=--leading-dashes nb-search search -- --help nb-search fetch "https://example.com/" --pipeline direct.fetch ``` `--lane` can repeat; lane, lanes, and preset forms are mutually exclusive. Search also supports freshness (pd/pw/pm/py), max-results, timeout-ms, execution, and idempotency-key. Fetch supports pipeline, representation (markdown/text), timeout-ms, max-content-chars, execution, and idempotency-key. Numbers are decimal safe integers within input-schema bounds. No hidden query-based routing, automatic full-source expansion, or implicit GMA replacement occurs. Timeout resolution is request-level `timeout_ms` > explicit `execution.search_timeout_ms` > default; when neither is supplied, the `grok-multi-agent/research` provider defaults to 600000 ms (10 minutes), while ordinary search remains 30000 ms. For portable Unicode, quotes, and local file/inline input, pipe **one UTF-8 JSON document** to `search --stdin` or `fetch --stdin`. For example, the stdin bytes may be: ```json {"action":"run","query":"Compare A and B — one complete brief","lane":"gma.research","execution":"async","idempotency_key":"brief-1"} ``` ```json {"action":"run","source":{"kind":"inline_text","content":"Local evidence to normalize.","media_type":"text/plain"},"pipeline":"direct.local"} ``` ```json {"action":"run","source":{"kind":"file","scope":"papers","path":"paper.txt"},"pipeline":"direct.local"} ``` File scope `papers` must already be configured; input JSON grants no arbitrary file access. Remote v1 rejects all inline/file sources **before HTTP IO**, even if a catalog advertises them. `capabilities --stdin` accepts exactly `{}`. Do not combine stdin with positional/shorthand request fields. Profile and applicable helpers (`--wait`, `--all`) may accompany it. A leading BOM/whitespace is allowed; malformed UTF-8/JSON, trailing documents, unknown schema fields, or input exceeding 4 MiB is rejected before runtime construction. The CLI bound does not change the local SDK contract. ## Async receipt, wait, and complete reads ```sh nb-search search "brief" --lane gma.research --execution async --idempotency-key brief-1 --wait 240000 nb-search search get --wait 240000 nb-search search read --all nb-search search read --page-size 8 nb-search search cancel ``` The same get/read/cancel forms apply to fetch. Follow-up job actions default to the local connection; remote follow-ups require an explicit `--profile `. The CLI records async receipt connection identities in immutable `job-connections/.json` files and still checks legacy `job-connections.json`; known mismatches fail before contacting another endpoint. Unknown SDK job IDs require a deliberately chosen profile when remote, with no cross-profile discovery. Service ACLs remain authoritative, including after key rotation. `--wait MS` accepts 1–3,600,000 ms only for async run or get; it never converts sync to async. It controls client-side command polling duration without altering server or provider execution timeouts. It respects poll_after_ms, bounds HTTP and sleep by remaining budget, and does not call cancel on timeout. Successful completion reads the entire artifact. Failed/cancelled completion returns its receipt/metadata. If a start succeeds but local connection recording fails, stdout retains the safe receipt and stderr explains the recording failure; preserve that receipt and original profile. Raw read still returns the original page envelope (`artifact`, `chunks`, optional `next_cursor`). `read --all` cannot combine with cursor/page-size. It checks canonical base64, contiguous chunk indices/offsets and lengths, stable artifact metadata, cursor progress, total byte length, SHA-256, fatal UTF-8, JSON and logical output schema. Limits: 64 MiB reconstructed artifact and 10,000 pages. Nothing is silently truncated. Never ask a model to assemble base64 manually. A complete **search** read/wait prints the stored logical output: results-channel rows or typed-channel data, status and lane outcomes. A complete **fetch** read/wait prints its stored sync fetch envelope. These are intentionally not raw page envelopes. Expired artifacts are not regenerated. Wait budget exhaustion prints: ```json {"cli_schema_version":"1","status":"timed_out","job_id":"...","last":{"action":"get","state":"running"},"cancel_requested_by_cli":false} ``` `last` is the full last-received receipt/envelope, abbreviated above for readability. Failure of the initial HTTP operation instead uses the normal stderr error path. Timeout/abort never implies upstream cancellation or a refund. ## Output and exits Normal stdout is one JSON value plus newline; help/version are text exceptions. Diagnostics and thrown errors use stderr. No raw credential JSON, token, upstream body, stack, or network exception is included in diagnostics. | Exit | Meaning | | --- | --- | | 0 | succeeded, capabilities, offline doctor success, import success, or cancel-request acknowledgement | | 2 | invocation/configuration/transport/protocol/integrity error, failed result/job, blocked migration | | 3 | partial logical result or partial migration | | 4 | empty result | | 5 | timeout or exhausted wait budget | | 6 | cancelled result/job or caller cancellation | | 7 | pending queued/running job | A succeeded get means metadata was obtained, not that a stored partial/empty result is logically succeeded. Cancel exit 0 acknowledges the request only; inspect state and cancel_requested. Unlike the earlier CLI, empty/partial/pending are not generally exit 0. ## Explicit search-layer import Migration is an explicit operator action, not SDK legacy lookup: ```sh nb-search --import-search-layer --dry-run nb-search --import-search-layer --apply nb-search --doctor ``` Use `--source PATH` to select a source explicitly. Otherwise the first existing file wins: SEARCH_LAYER_CREDENTIALS, OS home `.openclaw/credentials/search.json`, cwd `credentials/search.json`. The selected malformed file blocks; there is no fallback to a lower-priority source. BOM is accepted. Dry-run has zero writes and no network calls; apply never modifies the source. Reports contain source path, provider/status/reason categories, and target created/merged/unchanged state, not keys, key fragments/hashes, raw endpoints/models, or source JSON. Existing identical values are no-ops; conflicting values block rather than overwrite. Migration can write to any configured `NB_SEARCH_CONFIG` path, including locations outside the home directory or targets reached through filesystem links, with the exception of reserved metadata leaf names (`.config-access.lock`, `.config-revision.json`, `..nb-search.lock`, and `..nb-search-revision.json`) which cannot be used as configuration files or redirected via metadata leaf symlinks. Temporary staging files are created adjacent to each resolved target and replace files atomically without replacing links. Canonical process environment variables continue to take precedence over saved files according to the SDK resolution order, without blocking the import from saving. Configuration schema validation and non-destructive merge rules remain strictly enforced. - Exa/Tavily preserve direct provider/proxy URL semantics and exact search suffix handling. Compatible configured results lanes form `research-evidence`; Exa then Tavily is the default preference. SDK built-in defaults are unchanged. - Grok Chat Completions does not prove Responses support. Imported Grok stays disabled with a compatibility reason. - GMA keeps its independently resolved key in NB_SEARCH_GROK_MULTI_AGENT_API_KEY via an explicit credential slot. Missing own JSON key/URL follows resolved Grok inheritance. Legacy env supports GROK_API_KEY/URL and GROK_MULTI_AGENT_MODEL/EFFORT/API_MODE; it does **not** support independent GROK_MULTI_AGENT_API_KEY/URL env names. No new legacy aliases are inferred. - The existing GMA provider now accepts explicit `options.api_mode: "chat_completions" | "messages"`, defaulting to chat when omitted. Both known modes with a valid base/matching suffix can be compatible-but-unverified. Reports expose only `normalized_protocol: chat_completions | messages | unknown` plus reasons such as API_MODE_UNKNOWN, ENDPOINT_MISSING, ENDPOINT_PROTOCOL_CONFLICT, or CREDENTIAL_MISSING. Unknown modes and cross-protocol endpoint suffixes remain disabled; no URL switching or mode fallback occurs. - Both GMA modes send `stream: true` to the provider and require an SSE-capable gateway. Messages implements the observed relay contract: POST `/messages`, top-level `system`, Bearer plus `x-api-key` and `anthropic-version`, `reasoning.effort`, and bounded JSON text blocks in the SSE response. Thinking/tool blocks are not evidence. `stream: true` describes provider-side transport; `sync`/`async` describes the local job lifecycle. The public SDK and CLI still return the completed typed result as JSON, not a user token stream. This is not a claim of generic native Anthropic compatibility. The typed research output and CLI paths are unchanged; `data.api_mode` reflects the selected protocol. GMA adapter version is now 2: finish version-1 in-flight GMA jobs before upgrading, because old execution snapshots cannot be rebuilt by the new worker. No compatibility provider or snapshot-fingerprint bypass is added. - Both local GMA modes reject every HTTP 3xx before following it or forwarding credentials/body. The redirect response stream is cancelled and the fixed error is non-retryable, even if an execution retry budget is configured. Supply the final authorized endpoint explicitly; do not rely on relay redirects. Custom host HTTP transports must honor the request's manual redirect policy. Other providers' default transport redirect behavior is unchanged. - Aggregate gateway credentials cannot become vendor keys. Per-provider timeout settings that cannot map to the shared runtime budget are reported unsupported. Migration sets local retry_count to zero without changing SDK defaults. - Imported file keys populate private provider credential slots. The runtime verifies current configuration syntax and provider-operation references without enforcing legacy endpoint or slot binding metadata. If an imported credential file contains obsolete binding entries, they are accepted for backward compatibility without acting as admission gates. A partial report means useful compatible providers may be configured while optional/incompatible/missing providers remain. It is not full search-layer parity, verified credentials, or authorization to make paid calls. No importer operation calls providers. ### Permissions, coordinated publication, and recovery The CLI creates new configuration files with 0600 mode and directories with 0700 mode, respecting the user's selected file locations and filesystem links. It does not inspect or alter existing file permissions or execute operating-system ACL management commands. If an underlying filesystem operation fails with a permission or I/O error, the CLI reports the error directly. Ordinary local readers no longer take or create `.config-access.lock`: same-home capabilities, searches, fetches and job follow-ups can run concurrently, including overlapping provider execution. Readers capture one bounded configuration/secret snapshot between two revision observations and two absent-writer-marker checks. They construct the runtime from that in-memory snapshot without a later canonical re-read. A configuration change permits only a small bounded snapshot retry, never provider replay. Missing-home capabilities, doctor, sync inline fetch and unknown-job reads leave the home missing. Only importer/config writers take persistent transaction locks. In addition to the home-level `.config-access.lock` and `.config-revision.json`, each resolved data target file `X` is protected by adjacent sidecars: `..nb-search.lock` and `..nb-search-revision.json`. This coordinates writers and prevents collision even when distinct home directories share a custom configuration file target. Writers publish complete configuration pairs, atomically advance their target revision UUIDs, and remove their transaction locks; revisions are preserved. Readers observe consistent snapshot pairs across all targets without blocking concurrent operations. First async admission initializes home and job receipt storage before JobStore writes or dispatches work. Job connection bindings use immutable per-job files under `job-connections/` with atomic no-replace publication. Different jobs do not share a configuration lock; same-ID conflicting bindings are rejected. Existing `job-connections.json` is still honored read-only and is not deleted or overwritten. A local provider write marker does not prevent authorized remote job get/read/cancel. Writer publication failures or crashes retain their transaction markers, causing subsequent operations to report the specific busy or recovery marker path. Locks record a transaction descriptor listing home, data, lock, and revision paths without sensitive credential values. Do not remove any marker based only on PID or age, and do not remove only a single lock while leaving partial state behind. To recover: stop cooperating writer processes, inspect the protected `.migration-*-backup.json` and staged files locally without printing secrets, restore the complete configuration and secrets pair, publish fresh revision UUIDs across all target revisions recorded in the transaction, and then remove all corresponding transaction locks. Never delete revision files back to an uninitialized state. If completeness cannot be established, leave storage blocked. There is no automatic recovery switch. Old reader-crash locks from earlier versions still require deliberate recovery; do not assume they are safe to erase. Routine user edits to configuration files are read directly without requiring manual revision updates. The CLI ensures atomic pairs and transaction coordination specifically for cooperating importer and writer commands; uncoordinated concurrent manual modifications to multiple files are outside this transactional guarantee. Only an interrupted transaction or crash recovery requires following the formal restoration steps. Stop older CLI/importer writer processes before upgrading: they do not publish commit revisions and must not race new writes. No serial invocation workaround is required for ordinary reader concurrency. --- Source: https://nb-corp.github.io/nb-search/en/reference/runtime # Model-facing query lane runtime The package implements configuration schema v4 with three public capabilities: `search`, `fetch`, and `capabilities`. SDK, local CLI, and MCP call the same runtime; MCP registers exactly those three tools. The explicit remote client implements the same method interface over HTTP protocol 1, with strict response validation and URL-only fetch. Its identity/job authorization and wire-content idempotency requirements are in `remote-protocol.md`; they are not implemented by the local store described here. CLI profiles, private credential loading from environment or protected files, coordinated migration, and complete artifact consumption are described in `cli.md`. ## Trusted host HTTP transport `createNbSearchRuntime({ http_transport })` accepts an in-process object implementing the root-exported `HttpTransport.send(HttpRequest): Promise>` contract. `HttpRequest`, `HttpResponse`, `HttpTransport` and `ResponseLimitError` are available from the package root. Adapters using `ProviderFactoryContext.transports.http` receive that exact object; it must enforce its own bounds, redirect policy and AbortSignal handling. This is not JSON/config/env input, a global proxy, or interception of every SDK network path: direct HTTP, wayback's direct acquisition, browser and custom provider IO may use independent paths. A cloud host must restrict enabled operations to its audited coverage rather than infer universal SSRF protection. Explicit public injection disables **all detached async run admissions on that runtime instance**, before selection/fetch preflight or job creation, returning `LANE_EXECUTION_UNSUPPORTED`. The transport object is not serialized or silently replaced in a child process. Effective `search.lanes`, presets and `fetch.pipelines` omit async; async-only browser pipelines have no effective modes and are unavailable, never relabeled sync. Provider descriptors retain static adapter support information and are not an instance execution guarantee. Sync operations, including non-egress inline/scoped-file fetch, and existing local get/read/cancel semantics remain unchanged. Omitting the option preserves the default runtime's async behavior. Internal worker/test transport injection is not subject to this public-instance policy. The SDK supplies no new cloud admission/job-store implementation through this option. ## Captured configuration and hosted workers Hosts that admit configuration themselves can call `resolveCapturedConfiguration({ home, canonicalPath, canonical }, { env, config, overrides })` instead of making the SDK reread a file. `canonical` is the admitted local patch (`{}` when no local document is used); `config` is the host patch. The resolver applies standard defaults, environment settings and patches once and returns `ResolvedConfiguration`, including effective config, revision/fingerprint, timeout provenance and in-process secret bindings. Treat that object as private host state, not a client response. After importing host-managed credential values into a captured environment, `bindCapturedConfiguration(resolved, env)` refreshes credential grants without merging configuration again. `createCapturedNbSearchRuntime(resolved, { env, launcher })` composes that exact effective configuration without disk reads, default resurrection or a second host patch. Standard `defaultConfiguration(home)` is also public; hosts should reuse it rather than copy default tables. Browser consumers can import input/routing schemas, types, built-in rule metadata and pure `selectFetchRoute` from `@nb-corp/nb-search/contracts`; that entry has no Node IO or worker startup. `launcher: WorkerLauncher` is a trusted in-process hook with `launch(jobId, jobsRoot): Promise`. Use it when a host executable must own detached worker startup. The public `runWorker(jobId, env?, jobsRoot?, onReady?)` runs the standard worker lifecycle; importing the function does not auto-run a worker. Both ordinary and captured runtime constructors preserve hosted async execution and the normal job snapshot/file admission checks. ## Offline provider operation inspection for hosts The root helper `resolveProviderOperation(providerId, operationId, instance: ProviderInstanceConfig): ResolvedProviderOperation` resolves **Exa (search/synthesis/contents), GMA (research), and script (search)** without credentials, network requests, provider construction or script imports. It returns `{ provider, operation, kind, instance, endpoints }`: registry-owned provider/operation descriptors (including adapter version and schema), `kind: 'search' | 'fetch'`, an effective instance, and the adapter's request URLs. It validates options using the registration, uses the same URL resolvers as execution, and shares GMA default resolution with its factory. Unknown operations and providers outside this narrow resolver's coverage throw rather than return guessed endpoint information. Existing `builtInProviderRegistrations()` remains the catalog source; do not duplicate descriptors in a cloud catalog. GMA defaults are materialized in `instance.options`; script resolves the module to an absolute path (relative input uses caller cwd) and preserves JSON `params`. Returned options are `Readonly>`, not string-only. No credentials are looked up, and these host-facing instance values are not automatically safe for tenant-facing display. Script has `endpoints: []` because arbitrary trusted module IO cannot be enumerated—not because scripts cannot access the network. Hosts own HTTPS/DNS/allowlist and module-deployment policy. Pass the returned instance to the SDK rather than rebuilding provider URLs or writing a second runner. Registry activation and operation descriptors are not a promise of runtime readiness or available credentials. ```ts import { resolveProviderOperation } from '@nb-corp/nb-search'; const resolved = resolveProviderOperation('grok-multi-agent', 'research', { provider_id: 'grok-multi-agent', enabled: true, base_url: 'https://relay.example/v1', options: { api_mode: 'messages' } }); // Apply host endpoint policy to resolved.endpoints. // Bind a selected credential slot and execute resolved.instance using the SDK. ``` ## Query operations A lane binds one `provider_instance_id` to one `operation_id`. The provider registration is the output-contract source of truth: - results operations declare `channel: "results"` and `schema_id: "nb-search.results@1"`; - typed operations declare `channel: "typed"` and their JSON `schema_id`. Effective execution modes are computed from registration ownership and runtime readiness. Built-in worker-installed query operations may support sync and async. Host registrations are sync-only in this version. Execution never selects another lane. `search` is a strict `run | get | read | cancel` action union. A run selects one lane, an ordered results-lane list, or a results-only preset. Typed operations require one lane. The built-in search default is the keyless general-web `duckduckgo.search` lane; a configured `defaults.search_lane` applies to both sync and async. No registration, API key or browser download is needed. Public HTML challenges, rate limits and unrecognized pages are errors, never successful empty results; explicit defaults, removals and lane selections remain authoritative without provider fallback. Tavily's credentialed typed `crawl` and `research` lanes use the official crawl and POST-then-GET research-task protocols, with default query deadlines of 180 seconds and 900 seconds respectively unless the request or configuration supplies a timeout. One query and one results lane preserve provider order. Each query×lane list canonical-deduplicates before RRF; multiple lists then use fixed RRF, global canonical deduplication, independent evidence groups, and stable plan-position/URL tie-breaking. Typed operations are treated as schema-bound JSON. The core does not branch on their business purpose. `grok.synthesis` and `grok.x-synthesis` use the Grok Responses API; `gma.research` remains the separate multi-agent relay integration, with explicit provider `options.api_mode` selecting `chat_completions` (omitted default) or `messages`. Both modes use the same typed research projection; its data.api_mode records the actual protocol. Messages accepts bounded non-streaming text blocks only, not thinking/tool data. Its reasoning.effort and dual authentication headers are the observed relay contract, not a promise of universal native Anthropic compatibility. GMA adapter version 2 changes operation fingerprints: finish version-1 in-flight jobs before upgrading; old snapshots are not silently rebuilt under a different adapter. If a sync logical output exceeds `max_inline_bytes`, the run fails with `OUTPUT_TOO_LARGE`; no truncation or implicit async conversion occurs. ## Async jobs Async run uses the same normalized request, selection, operation descriptor, provider-attempt budget, retry policy, deadline, and logical output shape as sync. It requires an idempotency key. Matching key/request pairs reuse the job; a key reused for a different normalized request conflicts. A job publishes exactly one immutable JSON result artifact as a manifest plus bounded binary chunks. The public lifecycle states are `queued`, `running`, `succeeded`, `failed`, and `cancelled`. Logical `partial` belongs to the stored query output, not the job lifecycle. Timeout is `failed` with `DEADLINE_EXCEEDED`. `search.get/read/cancel` and `fetch.get/read/cancel` use the same job store while enforcing their public namespaces. Reads return bounded base64 chunks and a cursor bound to artifact SHA and byte offset. Retention is normalized once to integer seconds. A successful artifact's TTL begins at publication; failed and cancelled jobs retain from `completed_at`. No public enumeration, selector matrix, checkpoint, or artifact revision exists. Snapshots and artifact manifests start at contract version 1. A snapshot freezes only the selected operations, fingerprints, normalized request, budget, and credential worker grant required to reconstruct that execution. Heartbeat failure during an active call or retry backoff is classified as `WORKER_LOST`, with no user cancel marker and no extra provider call. Retention pruning runs on start and worker entry. Result-directory publication retries transient Windows `EPERM`, `EACCES`, and `EBUSY` rename failures. ## Fetch `fetch` is a strict `run | get | read | cancel` action union. A run accepts a discriminated `url | inline_text | inline_bytes | file` source and requests `markdown` (default) or `text`. URL selection is **explicit pipeline → first matching enabled user rule → enabled built-in rule → input/representation fetch chain**. Only the selected ordered list is tried. Unsupported representation or execution fails preflight rather than degrading, matching another rule, or changing execution mode. Inline and file inputs do not use URL routing. `fetch.routing` defaults to enabled, with the built-in `kiki-common-direct@1` package enabled. Its four rules cover known text files on `raw.githubusercontent.com`, specific GitHub repository JSON endpoints on `api.github.com`, npm metadata, and PyPI JSON. They select only `direct.fetch`: failure does not add Jina, Tavily, or another paid fallback. GitHub webpages, unrelated GitHub APIs, binary raw files, package tarballs, and PyPI project HTML do not match these built-ins. The package and full rules are returned in `capabilities.fetch.routing`. Configure an exception with an exact HTTP(S) origin, optional `path_globs` and representation, and either a complete ordered `pipelines` list or `use: "default"`. Origins ignore case/default ports/trailing root dots. Paths are case-sensitive, fully anchored and keep percent encoding; `*` does not cross `/`, `**` does. Queries and fragments do not participate in matching, and routing does not rewrite the query sent to the source. A `use: "default"` rule suppresses the built-in match for that URL and restores the configured chain. `enabled: false` turns off routing; `builtin_enabled: false` turns off only built-ins; `disabled_builtin_rules` disables selected built-in IDs. ```json { "fetch": { "routing": { "rules": [ { "id": "raw-explicit-fallback", "match": { "origin": "https://raw.githubusercontent.com", "path_globs": ["/**/*.md"] }, "action": { "pipelines": ["direct.fetch", "jina.reader"] } }, { "id": "npm-use-my-chain", "match": { "origin": "https://registry.npmjs.org" }, "action": { "use": "default" } } ] } } } ``` Direct URL HTML uses Defuddle 0.19.3 with a linkedom DOM and `useAsync: false`; scripts, iframes and extractor secondary downloads are not executed. It extracts Markdown or readable text with links/code rather than returning the page shell. An empty extracted body fails with `QUALITY_GATE_FAILED`; authentication/challenge structures fail with `FETCH_BLOCKED`. Plain source text preserves whitespace, empty plain bodies return `status: "empty"`, and JSON is pretty-printed. GitHub contents/readme file responses decode base64 locally only when the payload is valid UTF-8 readable text within the byte budget. Directory listings stay JSON; missing content, pagination, byte truncation and LFS pointers carry warnings. No `download_url` is followed automatically. Explicit quality rules still apply after extraction; default `min_content_chars: 0` and `blocked_markers: []` do not impose an arbitrary length threshold. Direct 401/404/410 and GitHub API errors terminate the selected list instead of fetching an error page elsewhere. Each pipeline descriptor reports input kinds, media types, representations, execution modes, egress, and typed stages. File and inline sources may enter only `egress: "none"` pipelines. Scoped files retain relative-path and realpath containment, read-only byte/MIME checks, and optional host-admitted `canonical_target` plus `approved_identity`; `captureFetchFileIdentity` captures the identity for validation at read time. Capabilities expose scope IDs but not roots. The default URL chain remains `direct.fetch` then `jina.reader` for unmatched URLs. `direct.fetch` and `direct.local` support sync and async, `browser.render` is async-only, and other built-in URL pipelines are sync-only. Browser rendering connects to bounded target subresources; static Direct extraction does not. Existing SSRF, DNS pinning, redirects, abort and byte/character limits remain in the acquisition layer. URL run receipts return `routing` with origin, rule/package identity, selected pipelines, representation and execution. `lane_outcomes` retain actual provider `attempts`, failures and skipped unavailable lanes. Async admission freezes that decision and list once; workers and get/read use the captured plan even if configuration changes. Failed fetch execution can publish a diagnostic artifact with route/attempts and no successful documents. Inspect document warnings before treating projected content as complete. ## Capabilities `capabilities` is a static model-facing catalog. Its fetch section reports default representation, enabled input kinds, scope IDs, expanded chains, pipeline descriptors, availability, latency, cost, and public limits. It performs no network probe. Provider-instance `availability` describes realized runtime readiness. `ready` means activation succeeded and at least one operation declared by the provider has a realized runtime port. `PROVIDER_PORTS_PARTIAL` is a non-fatal diagnostic: the instance remains ready, but one or more declared operations are unavailable, so hosts must inspect the matching `search.lanes` and `fetch.pipelines` entries before enabling an operation. `unavailable` means activation failed or none of the provider's declared operations has a realized runtime port. `PROVIDER_PORTS_UNAVAILABLE` identifies the latter case. Operation-specific issues such as `BROWSER_NOT_INSTALLED` remain attached to their lane or pipeline and are also propagated to the provider instance. The known capability issue-code literals are `PROVIDER_NOT_REGISTERED`, `PROVIDER_DISABLED`, `CREDENTIAL_NOT_CONFIGURED`, `ENDPOINT_NOT_CONFIGURED`, `PROVIDER_PORTS_UNAVAILABLE`, `PROVIDER_PORTS_PARTIAL`, `LANE_NOT_REGISTERED`, `OPERATION_NOT_REGISTERED`, `LANE_NOT_CONFIGURED`, `BROWSER_NOT_INSTALLED`, and `RATE_LIMIT_UNAUTHENTICATED`. The public `CapabilityIssueCode` type retains an extensible string branch for future provider-defined diagnostics. Capabilities expose provider and instance identifiers, operation descriptors, activation requirements, slot identifiers, readiness, and issue codes. They do not expose raw base URLs, provider option values, credential environment-variable names, secret values, file-scope roots, retry or heartbeat internals, storage paths, or configuration provenance. --- Source: https://nb-corp.github.io/nb-search/en/reference/remote-protocol # nb-search HTTP protocol v1 Status: v1 client contract, implemented by the package's remote client and exercised by test-only loopback fixtures. This document specifies a client/service boundary, not a deployed service. The existing local runtime is not a multi-tenant server. ## Version and endpoints The API base is an absolute service URL ending in a directory path, for example `https://search.example/api/`. Append `v1/search`, `v1/fetch`, or `v1/capabilities` to that base. An origin-only base is normalized to `/`; a non-root base is normalized to one trailing slash. Never infer a different origin, protocol, or API prefix. All three endpoints accept POST, UTF-8 JSON, with these headers: ```http Authorization: Bearer Content-Type: application/json Accept: application/json X-NB-Search-Protocol: 1 X-Request-Id: ``` HTTP protocol `1` is independent of result `schema_version: "3.0"` and canonical configuration schema `4`. No protocol negotiation, downgrade, request replay, or automatic retry occurs in the client. `search` accepts the strict action union exported as `searchInputSchema`: `run`, `get`, `read`, `cancel`. `fetch` accepts `fetchActionInputSchema`, restricted in v1 to `source.kind: "url"` for `run`. `capabilities` accepts exactly `{}` (it has no action discriminator). Client-side convenience `{url: ...}` is normalized before transmission; it is not a wire request form. Execution omission means `sync`. Async run requires `idempotency_key`; sync rejects it. The bounds in `src/contracts.ts` apply on both sides. Examples (fake public data, no real keys): ```json {"action":"run","query":"Node.js release notes","lane":"exa.search"} ``` ```json {"action":"run","source":{"kind":"url","url":"https://example.com/"},"execution":"async","idempotency_key":"fetch-example-1"} ``` ```json {"action":"get","job_id":"00000000-0000-4000-8000-000000000000"} ``` ## Transport security and limits - HTTPS is required. An explicit `allow_loopback_http: true` client setting allows HTTP only for literal `127.0.0.1`, `[::1]`, or `localhost`. It is a development exception, not a general insecure-HTTP switch; no DNS lookup is used to expand that allowlist. - Reject service URLs containing userinfo, query, or fragment, and reject a missing/blank access key or a key containing whitespace/control characters. Keys are passed separately to the SDK factory, never via query strings or CLI argv. A service access key is not an upstream provider credential. - Fetch uses `redirect: 'manual'`; every 3xx is an error. No key is sent to the redirect target, including same-origin redirects. Browser-cookie authentication is not part of this protocol. - Default request limit is 1 MiB after UTF-8 encoding; default response limit is 8 MiB of decoded response-body bytes. These are client options with positive integer validation. Check Content-Length when usable, but also bound streaming reads, including chunked and decompressed responses. Cancel the response stream on overflow, invalid headers, or abort. Error bodies use the same response bound. - Default per-HTTP-call deadline is 120,000 ms, configurable from 100 through 3,600,000 ms. This is not the provider's input `timeout_ms` or the job lifetime. Compose it with `OperationContext.signal`; abort during headers or body stops the client operation. No cancellation reason or network exception text is exposed verbatim. - Requests are schema-validated before network IO. In particular, remote inline text, bytes, and file input fail locally before any request, even if a service advertises those input kinds. No upload or local fallback is inferred. - Services must separately enforce request/body/time/cost limits and SSRF-safe URL acquisition on every redirect and DNS resolution. Client endpoint checks are not service-side fetch egress enforcement. ## Response and error mapping Every conforming response includes `X-NB-Search-Protocol: 1`, `X-Request-Id` equal to the accepted request ID, and `Content-Type: application/json` (optional charset allowed). The client generates a UUID when `OperationContext.requestId` is absent. Supplied IDs must match `[A-Za-z0-9._:-]{1,128}`; invalid IDs fail before IO. IDs contain no user query, URL, secret, or personal data. A server may generate an ID for invalid/missing inbound IDs, but valid SDK requests require exact response correlation. Missing, malformed, or mismatched metadata is a protocol error, not a successful result. HTTP **200**, including queued async receipts, carries the existing method's envelope directly, without a transport wrapper. Business `failed`, `partial`, `empty`, `timed_out`, or `cancelled` results are still HTTP 200 and resolve normally. A failed async admission that the runtime represents as an envelope is also HTTP 200. SDK consumers must inspect business status/state. All other HTTP statuses reject the client promise. HTTP 201/202/204 do not stand in for a run receipt. Non-200 JSON has this service error shape: ```json {"error":{"code":"RATE_LIMITED","message":"Request rate limit exceeded.","retryable":true}} ``` Optional `error.retry_after_ms` is a nonnegative safe integer. Arbitrary `data`, stack traces, provider credentials, upstream URLs, or upstream response bodies are forbidden. The client uses its own fixed public message and status-derived code rather than trusting a server-supplied message. It does not attach raw body, headers, URL, key, or network `cause` to public errors. | HTTP status | Service error code | Meaning / client retryable metadata | | --- | --- | --- | | 400 | INVALID_REQUEST | Invalid request schema, false | | 401 | UNAUTHENTICATED | Missing/invalid service key, false | | 403 | FORBIDDEN | Authenticated identity lacks permission, false | | 404 | NOT_FOUND | Unknown endpoint, unknown/expired/unowned job, false | | 409 | CONFLICT | Idempotency key reused with different normalized request, false | | 413 | REQUEST_TOO_LARGE | Inbound body limit, false | | 415 | UNSUPPORTED_MEDIA_TYPE | Expected JSON, false | | 426 | PROTOCOL_UNSUPPORTED | Unsupported HTTP protocol version, false | | 429 | RATE_LIMITED | Service admission/quota throttling, true | | 500 | INTERNAL | Service fault, true | | 502/503/504 | UNAVAILABLE | Service unavailable, true | An unmapped non-200 status is `HTTP_ERROR`. A mapped error requires the matching wire code and retryability; a malformed JSON error or inconsistent error code is `PROTOCOL_ERROR`, retaining only the safe numeric HTTP status. Gateways which cannot provide the protocol headers likewise cause `PROTOCOL_ERROR`; they must not be misreported as business results. `Retry-After` on 429/503 accepts nonnegative integer seconds or an HTTP date; expose normalized milliseconds (past dates become zero). If both header and body delay exist, use the larger. Reject invalid or overflowing delay values as protocol errors. This is advisory metadata, **never permission to automatically retry a paid request**. Retryability describes the failure, not idempotency or user authorization. The SDK exports a distinct `NbSearchRemoteError`, not a fabricated search/fetch envelope. Its public fields are `code`, fixed `message`, `retryable`, `request_id`, optional `status`, and optional `retry_after_ms`. Local codes are `INVALID_INPUT`, `CONFIGURATION_ERROR`, `PROTOCOL_ERROR`, `RESPONSE_TOO_LARGE`, `REQUEST_TOO_LARGE`, `TRANSPORT_ERROR`, `DEADLINE_EXCEEDED`, and `CANCELLED`, plus the service/status codes above. Local validation/configuration errors are nonretryable; transport/deadline failures may be retryable but are never retried. Keep this taxonomy separate from provider-level `PublicErrorCode` so service authentication does not become `PROVIDER_AUTH`. ## Response schema validation `src/remote-protocol.ts` will validate unknown JSON before returning any exported runtime type. Reuse current input schemas; build response schemas against `src/types.ts`, not type assertions. Check: 1. Exact result schema version; action and effective execution match the request. Fetch envelopes require `mode: "fetch"`; search envelopes reject a fetch mode. Get/read/cancel `job_id` matches the request. 2. Required union fields, enum values, safe nonnegative sizes/durations, job UUIDs, timestamps, selections, hints, error objects, lane outcomes, result/provenance rows, and fetch document metadata. A queued run must have a valid receipt; a failed async admission must have an error. Failed sync runs may instead carry failed logical output/lane outcomes, as the existing runtime does; do not impose an error field that its contract makes optional. Successful search sync must have the declared logical output; whenever output is present its status agrees with the enclosing status. 3. Job receipt/get/read/cancel shapes match the current runtime's state model. A job can have state `succeeded` while its stored logical output is `partial` or `empty`; do not substitute job state for logical status. 4. Artifact metadata uses `application/json`, a 64-hex SHA-256, safe nonnegative byte length, and a valid expiry timestamp. Chunks have integer index/offset/length and canonical base64. Structural validation precedes full integrity validation during CLI result consumption. 5. Capabilities validate the full generic catalog and its declared limits. Typed search output accepts bounded JSON under a nonempty `schema_id`; clients do not assume every future typed lane is GMA. Unknown additive response object properties may pass through, but cannot replace required known fields. Unknown discriminators or result schema versions fail closed. The service must redact its successful catalogs and envelopes too: do not expose raw endpoint configuration, option values, secret values, or environment-variable names. The client can redact its own service key in error paths; it cannot discover and scrub arbitrary secrets a misconfigured service places in successful business data. ## Jobs, authorization, retries, and retention - A service authenticates every operation, including capabilities. Its catalog contains only lanes, pipelines, presets, and limits authorized for the current principal; execution reauthorizes, even after a cached catalog said ready. Remote v1 catalog input declarations must not enable file/inline input. - Every get/read/cancel requires tenant/principal ownership and capability-kind checks. Unknown, expired, wrong-kind, and unowned IDs produce indistinguishable 404 errors. An opaque UUID is not authorization. Never mount the current local job store as a shared unauthenticated job API. - Authentication/resource authorization and eligibility for **new paid execution** are separate checks. Exhausted execution quota or balance must not block get/read/cancel for an existing still-authorized job, nor matching idempotent receipt recovery which dispatches no new work. Revoked identity/resource access remains denied; ordinary abuse-protection rate limits may still apply independently of paid execution quota. These are service obligations, not features of the local store. - Async idempotency namespace is `(tenant ID, stable authenticated principal ID, capability kind, idempotency_key)`, not a bearer-token hash. Token rotation preserving the same principal preserves the namespace. Matching requests reuse the same admitted job as specified below; differing requests in that namespace yield 409. - Artifact and job retention are advertised via capabilities and artifact expiry. Expired artifacts are not regenerated by `read`. No cross-service job discovery or automatic replay is available. - SDK calls on one client instance remain on that connection. Local CLI follow-up actions default to the local connection, while remote actions explicitly select their profile, and known job bindings must match before IO. Neither local jobs nor jobs from another service are silently queried elsewhere. - `cancel` is an explicit cooperative request; `cancel_requested: true` does not guarantee all upstream work stopped, nor a refund. A get/read deadline, CLI wait timeout, or client abort never implicitly calls cancel. Sync requests also may have executed upstream after a transport failure; the client never retries them automatically. Zero client retries does not disable separately configured provider/service execution retries. ### Async idempotency normalization This is a **remote service rule**, intentionally not the local SDK's snapshot-based job fingerprint. It does not change local SDK semantics. The client sends schema-parsed wire requests and does not resolve service configuration or compute provider plans. For an async run, parse the strict v1 input schema first (including its trimming), then construct comparison JSON as follows: 1. Remove `idempotency_key`; it is the namespace key, not a content field. Retain `action: "run"` and `execution: "async"`. 2. For search, normalize `query` to an ordered array even when supplied as a string. Preserve array order and duplicates, including selector arrays. Do not equate `lane`, `lanes`, and `preset` or resolve their contents from the catalog. 3. For fetch, normalize an omitted `representation` to `"markdown"`. The URL string is the schema-parsed wire value; do not infer redirect destinations, URL canonical equivalences, or server defaults. Non-URL sources never reach admission in v1. 4. Apart from that explicit fetch representation default, keep omitted optional fields absent. In particular, do not fill in `max_results`, `timeout_ms`, `max_content_chars`, pipeline, lane, or preset using deployment defaults. An omitted field and an explicit value differ even if that value happens to equal today's default. 5. Recursively sort object keys lexicographically, preserve array order, and serialize compact JSON using JSON string/number escaping semantics. Equality is equality of these canonical UTF-8 bytes (a collision-resistant hash may index them, but a hash collision must not alias unequal content). Whitespace in the original JSON formatting is irrelevant; no Unicode normalization beyond input-schema trimming is applied. The comparison excludes configuration revisions, resolved defaults, selected lane expansions, provider options, deployment versions, and execution snapshot fingerprints. The first atomic admission binds the normalized content and key to one job and freezes its first execution plan. A matching request after defaults/deployment changes still identifies that job: reauthorize access to the originally admitted resources, then project its current state without new provider dispatch. Do not rerun selection against new defaults to decide content equality. If access to an original resource was revoked, deny recovery rather than disclose it or substitute another resource. ### Admission, replay projection, and expiry Atomically persist key/content/job/first-plan association before any worker/provider dispatch. Concurrent matching admissions converge on one job. Validation or admission rejection **before this commit** creates no association; a corrected/retried request may later be admitted. Once committed, worker startup failure, timeout, execution failure, or cancellation does not release the association. If commit/dispatch outcome is uncertain, reconcile the durable admission state; never clear the key and dispatch again merely because a response was lost. Replay is a projection of the **same admission and current job state**, not a byte-for-byte replay of an old instantaneous envelope: - queued/running/succeeded/cancelled jobs return the run-async receipt shape, `status: "queued"`, `reused: true`, the same `job_id`/`created_at`, and `job.state` set to its actual current state. The outer queued status denotes an async receipt, not a claim that terminal work is queued again. Poll hints are needed only while work is queued/running. - An admitted failed job returns the run-async failed envelope with its public error, `reused: true`, and the same receipt with `job.state: "failed"`. The optional `job` field is required for this remote admitted-failure projection so callers can distinguish it from a rejection with no admission and address get/read. No rerun occurs. - A pre-admission business rejection may use a run-async failed envelope without a job; it has no durable key association. Consumers must not infer admission from HTTP 200 alone. An association never expires while its job is active, even if the initial advertised TTL elapses. After terminal transition it remains for at least the service's advertised terminal job/result retention duration; that duration applies to failed/cancelled jobs even when they have no artifact. It must not expire earlier than any advertised retained artifact/job expiry. Replays do not shorten retention. After this documented retention has expired the same key **may admit a new job and execute again**; clients must not rely on indefinite deduplication or blindly resubmit an old key. Unknown/expired get/read/cancel returns 404, not regeneration. Services must document their retention policy; this SDK does not implement the backing admission database. ### Normative vectors Within the same tenant/principal and capability, use key `k1`. `S(q, extra)` below means `{"action":"run","execution":"async","idempotency_key":"k1","query":q,...extra}`; `F(extra)` means a fetch run with those three action/execution/key fields and `source:{"kind":"url","url":"https://example.com/"}`. All examples are schema-valid unless explicitly labeled rejection. | A then B | Normalized comparison / required outcome | | --- | --- | | `S(" q ", {})` then `S(["q"], {})` | Equal: `{"action":"run","execution":"async","query":["q"]}`; same admitted job | | `S(["a","b"], {})` then `S(["b","a"], {})` | Different ordered queries; 409 | | `S("q", {})` then `S("q", {max_results:10})` | Different: omission is not explicit 10; 409 even when default is 10 | | `S("q", {})` then `S("q", {timeout_ms:30000})` | Different: omission is not explicit timeout; 409 | | `S("q", {lane:" exa.search "})` then `S("q", {lane:"exa.search"})` | Equal after schema trim | | `S("q", {lane:"exa.search"})` then `S("q", {lanes:["exa.search"]})` | Different selector forms; 409 | | `S("q", {preset:"p"})` then `S("q", {lanes:["exa.search"]})` | Different even if today's preset p has exactly that lane; 409 | | `F({})` then `F({representation:"markdown"})` | Equal: `{"action":"run","execution":"async","representation":"markdown","source":{"kind":"url","url":"https://example.com/"}}` | | `F({})` then `F({representation:"text"})` | Different; 409 | | Same omitted-selector wire request, default lane/budget/deployment changes between A and B | Equal; return same job and frozen first plan, no new dispatch; deny if access revoked | | Matching B while A's job is queued/running/succeeded/cancelled | Same job receipt, current state, reused true; no rerun | | Matching B after admitted worker failure | Failed envelope with same failed job receipt and reused true; no rerun | | Invalid schema or rejected admission before durable commit, then valid retry | No previous association; may admit once (subject to current authorization/quota) | | Matching B after advertised TTL while job is still active | Same association; never a new job | | Matching B after terminal transition but before terminal retention expires | Same association, including failed/cancelled; no rerun | | Matching B after documented terminal association retention expires | May admit a new job; old get/read/cancel remains 404 | | Same content/key in different capability or different tenant/principal | Different namespace, independently authorized admission | These vectors constrain services and reusable conformance fixtures. A fixture test of projection/normalization is not proof of a production database's atomicity or authorization enforcement. ## Conformance boundary The repository includes an explicitly test-only loopback HTTP fixture server (`test/fixtures/remote-http.ts`), client conformance checks (`test/remote-client.test.ts`), and real CLI subprocess HTTP checks (`test/cli-remote-integration.test.ts`). These fixtures are repository test resources, not a production server or npm runtime entrypoint. Tests must cover all three methods, sync and async receipts, get/read/cancel, correlation and version headers, request validation before IO, URL-only fetch, 401/403/404/409/429, both Retry-After forms, redirects (a second listener must receive zero requests), streamed oversize bodies, malformed UTF-8/JSON/envelopes, mismatched action/job IDs, caller abort and deadlines. CLI integration must really pass through HTTP, not inject a fake SDK object alone. These tests prove a client can call the specified HTTP boundary. They do not prove production identity, job ACL, provider compatibility, quota enforcement, billing, deployment, or any live paid call. Cloud implementers own those requirements; this package does not supply a production HTTP server. --- Source: https://nb-corp.github.io/nb-search/en/reference/changelog # Changelog ## 0.5.2 - 2026-11-15 ### Added - Added the English documentation overview (`/en/`), `sitemap.xml`, `robots.txt`, OpenGraph and Twitter card metadata (`og-card.png`), JSON-LD `SoftwareSourceCode` and `SoftwareApplication` structured data, and build-generated `llms.txt` and `llms-full.txt` indexes on the VitePress documentation site. - Added npm discovery `keywords`, pointed `package.json` `homepage` to `https://nb-corp.github.io/nb-search/`, and added repository `glama.json` metadata. ### Changed - Reorganized `README.md`, `README.zh-CN.md`, and `SKILL.md` for proportional multi-source coverage, zero-key quickstart commands, MCP/Skill/SDK onboarding, and competitive trade-off comparison, while relocating provider-specific Zhihu configuration and detailed `keyUsage()` reference material to the documentation site. - Updated VitePress guides and source catalog pages for `0.5.x` behavior, including `duckduckgo.search` zero-key default search, Defuddle `direct.fetch` extraction, ordered URL routing, and `npx` MCP server configuration. ## 0.5.1 - 2026-11-15 ### Added - Added the opt-in Zhihu provider (`provider_id: zhihu`) with `zhihu.search`, `zhihu.global` and typed `zhihu.synthesis` lanes. No Zhihu provider instance, credential slot or lane is added to the defaults. The content-search operations use the official API paths; Zhida synthesis defaults to `zhida-fast-1p5` and returns `nb-search.synthesis@1`. - Added a fail-closed free-quota guard backed by `GET /api/v1/quota`, with per-API request counters from `KeyUsage.usage.meters`, a preserved legacy credits summary, Asia/Shanghai daily reset, caching only when more than 32 requests remain (up to 60 seconds), runtime-wide per-credential reservations, and retained charges across uncertain interrupted sends until upstream `TotalUsed` advances or midnight reset. Wallet billing requires explicit `allow_paid: true`. ### Changed - Zhihu `base_url` prefixes are preserved when appending configured endpoint paths; Tavily and Firecrawl usage requests also retain configured prefixes. ## 0.5.0 - 2026-11-14 ### Added - General-web search now defaults to keyless DuckDuckGo HTML without registration or lane selection. Configured defaults, explicit removals and lane choices are preserved; errors never switch providers. Challenges and unrecognized pages fail rather than returning empty results, and 429 retains Retry-After metadata. DuckDuckGo adapter version 2 changes operation fingerprints; finish version-1 jobs before upgrading. GitHub and Context7 remain explicit specialist choices. - Added default-on ordered URL routing, the versioned `kiki-common-direct@1` rule package, route/attempt receipts, and browser-safe public contracts. Explicit pipelines take precedence; direct-only rules do not silently add paid fallback. - Exported captured configuration/runtime composition, credential rebinding, hosted worker launch/run hooks, and approved file identity capture. Hosts can reuse standard defaults and resolution without substitute config files. - Added the official Tavily typed crawl and research-task lanes, preserving Bearer authentication and bounded POST/GET task polling. ### Changed - Direct URL HTML now uses Defuddle 0.19.3 with linkedom, without extractor networking. Raw text retains source whitespace; JSON and GitHub contents/readme responses produce readable bounded projections. Empty HTML fails rather than passing a page shell; empty plain bodies report `empty`. - Direct supports detached async; routing is frozen at admission and retained on get/read and execution-failure artifacts. Existing SSRF/redirect/abort/byte limits and file identity protections remain in place. Zod stays an external production dependency so public schema declaration types are nameable by downstream packages. - Configuration schema `4` and envelope schema `3.0` remain compatible through optional routing fields. Direct approved-file identity handling and Tavily API parity change adapter fingerprints; finish older Direct/Tavily jobs before upgrading. Persisted jobs are never silently rebuilt. Public DuckDuckGo HTML search may be blocked by CAPTCHA. ## 0.4.0 - 2026-09-08 ### Added - Added trusted local JavaScript and erasable TypeScript script lanes, configured through the standard SDK/CLI provider settings, with synchronous and detached asynchronous execution. - Exported script module types and included a runnable, no-network example. - Added `resolveProviderOperation` for offline host inspection of Exa, GMA and script operations, sharing validation, defaults and endpoint resolution with the actual adapters. ### Compatibility - Existing search/fetch/capabilities calls and in-process provider registrations remain supported. Script modules run with host permissions and must cooperate with cancellation; they are not sandboxed or content-pinned. - Cloud hosts can reuse SDK operation metadata instead of duplicating provider protocols. Unsupported offline endpoint resolution is reported explicitly. ## 0.3.1 - 2026-09-08 ### Fixed - GMA requests now use streaming for both chat and messages protocols, with complete SSE parsing and rejection of interrupted or failed responses. - GMA research defaults to a 10-minute deadline unless a request or configuration explicitly supplies a timeout; ordinary search keeps its 30-second default. - HTTP 524 is classified as retryable for callers with an explicit retry policy. ### Changed - Added bilingual product landing pages and themed SVG assets. - Clarified search and webpage reading in nb-search, with document conversion and resource export available through nb-extract. Existing fetch interfaces remain compatible. - Published the official CLI, MCP and Skill entry points to npm, including the changes documented under 0.3.0. ## 0.3.0 - 2026-09-08 ### Added - Added the callable remote client SDK (`createNbSearchRemoteClient`) and HTTP protocol v1 client contract with strict response validation, bounded streams, non-retry policy, and URL-only fetch boundaries. - Added formal CLI profiles, `--stdin` single UTF-8 JSON inputs, `--wait` budget execution, complete artifact reconstruction via `read --all`, default local follow-ups with explicit profile requirement for remote jobs (`get`, `read`, `cancel`), and standardized exit codes (`0`, `2`, `3`, `4`, `5`, `6`, `7`). - Added official skill launcher support for package root and nested `skills/nb-search` structures. - Added explicit offline migration tooling (`--import-search-layer --dry-run` and `--apply`) with separate protected credential storage (`secrets.json` and `remote-secrets.json`). - Added GMA `messages` relay contract and strict HTTP 3xx redirect rejection to protect credentials from unintended forwarding. - Added concurrent reader snapshot coordination, safe writer transaction markers, and lane purpose catalog guidance (`src/lane-guidance.ts` and `scripts/lane-guidance.mjs`). - Added trusted in-process `http_transport` injection option to `createNbSearchRuntime` with root exports for `HttpRequest`, `HttpResponse`, `HttpTransport`, and `ResponseLimitError`. - Added VitePress documentation site framework, continuous integration checks, and release readiness verification tooling. ### Changed - Simplified local environment admission by removing redundant ACL/owner/mode checks and initialization barriers while preserving standard file creation modes and symlink/junction support. - Updated default fetch quality rules to `min_content_chars: 0` and `blocked_markers: []`, and raised default synchronous `max_inline_bytes` to 16 MiB. - Updated GMA adapter to version 2; in-flight version-1 jobs must complete before upgrade. - Configuration schema `4` and result schema `3.0` remain unchanged; HTTP protocol `1` has its own version. - Local readers no longer require a persistent global command lock; follow-up job commands default to the local connection. ## 0.2.0 - 2026-09-04 ### Added - Added the public `runtime.fetch` action union, default fetch chains, nine built-in fetch pipelines, and sync/async fetch job envelopes. - Added schema-v4 configuration inputs and parsers for in-process hosts: `CanonicalConfigPatch`, `parseConfigPatch`, and `parseResolvedConfig`. - Added provider registrations, fifteen built-in search lanes, typed query operations, presets, and explicit default-lane selection. - Added capability descriptors for providers, configured provider instances, search lanes, fetch pipelines, defaults, readiness, redacted credential status, and the extensible `CapabilityIssueCode` type. - Added packaged `SKILL.md`, `.env.example`, and runtime contract documentation, plus the optional Playwright peer dependency for browser rendering. ### Changed - Replaced the 0.1 research-specific runtime methods with the generic `search` and `fetch` `run | get | read | cancel` action unions. - Updated public envelopes and error codes to schema version `3.0`; configuration uses schema version `4`. - A search call without an explicit selector now uses `defaults.search_lane` and returns `DEFAULT_NOT_CONFIGURED` when no default is configured. - Credential slots are provider-scoped; a provider instance cannot consume a slot declared for another provider.