iTool.KnowledgeHub.Idb 4.0.30-idbanalyze.20261113.10

iTool.KnowledgeHub

Business-neutral world knowledge primitives and AgentKernel tools.

项目定位与能力地图

本项目提供跨产品复用的知识与 RAG 基础能力。iDB 保存权威证据和任务状态，MySearch 只保存可重建的检索投影，AgentKernel/Runtime 负责宿主调度和执行循环；业务宿主负责 租户身份、授权、模型配置、业务数据源和上线运维。

这几个项目组成一个批量摄入到在线回答的闭环：批量源数据先由宿主分批调用 PrepareIngestionAsync，再由既有 Runtime 投递 ResumeIngestionAsync；读取端使用 IdbKnowledgeRagRetriever 做有界 BM25/向量混合召回，IdbKnowledgeRagAnswerService 在需要生成回答时执行引用约束和拒答；宿主维护任务调用有界 discovery、retirement、 collection、compaction 和 MaintainHistoryPageAsync 分页接口。没有独立的后台线程、 定时器或第二套 Agent 循环。

生产接入顺序

1. 宿主选定 iDB index 和知识库命名空间，注册自己的 database/search/model/chat 客户端， 并在授权后解析一个 named scoped RAG binding。
2. 每个源版本先调用 PrepareIngestionAsync(operationId, document, expectedHeadRevision, profile)。operationId 是宿主调度事实；相同意图可安全重试，旧 head 由 CAS 拒绝。
3. 既有 Runtime 按 discovery 结果投递 ResumeIngestionAsync。模型失败保留可恢复阶段， 完成阶段会写入终态回执；模型切换通过 active projection、audit、rebuild、activation 接口推进，不改写原始摄入意图。
4. 查询先取得授权后的 scope、预算和 read fence，再调用 RecallAsync。请求只提供查询 内容和允许的 limit；知识库、scope、候选预算、上下文字节数和最小 revision 由宿主授权 grant 固定。回答调用 AnswerAsync，引用必须能在返回上下文中找到原文，否则拒答。
5. 保存回答或其他长期产物前调用 RetainEvidenceAsync(ownerId, versionId)；不再需要时 用期望 revision 调用 ReleaseEvidenceAsync。终态历史默认 90 天可配置，到期会明确 返回 KnowledgeHistoryExpiredException/knowledge_history_expired，宿主应向用户显示 “已过期”，不能当作不存在或重新执行旧任务。
6. 将 discovery、retirement、collection、compaction 和 history maintenance 接入宿主 已有维护作业，保存知识库绑定的游标和 source fence。维护接口有界、可取消、可丢确认重试； 不能用单纯年龄删除仍被 current head、有效引用或 pending ingestion 使用的证据。

最小的 DI 注册形态如下（客户端生命周期仍由宿主拥有）：

services.AddIdbKnowledgeRag(
    "research",
    new KnowledgeRagBinding("knowledge-index", "research")
    {
        HistoryRetention = new KnowledgeHistoryRetentionOptions { Retention = TimeSpan.FromDays(90) },
    },
    sp => sp.GetRequiredService<IIndexedDocumentDatabaseGrain>(),
    sp => sp.GetRequiredService<IMySearchQueryGrain>(),
    (sp, profile) => ResolveEmbeddingGenerator(sp, profile), // host-owned resolver
    sp => ResolveChatClient(sp)); // host-owned chat client

IEmbeddingGenerator、IChatClient、iDB grain 和 MySearch grain 都是宿主提供的公开契约 实现；RAG 适配层不创建或释放这些客户端。宿主必须在外层完成身份和授权，不能把 KnowledgeBaseId、scope 或任意请求参数当成授权来源。iDB/MySearch 的底层实现、业务实体、 页面、告警阈值和部署发布不属于本项目 README 所描述的 RAG 适配职责。

The project intentionally fixes only the mechanics of a living world:

• WorldObservation enters the world;
• WorldIntent asks for a bounded view;
• KnowledgeContext and KnowledgeExplanation are consumer projections;
• EvolutionProposal lets an Agent submit a candidate without applying it;
• EvolutionCandidate records a data-driven proposed change;
• confidence decay retires stale links without deleting their evidence;
• conflicts can coexist, be resolved by selecting a link, and remain explainable;
• every mutation is revisioned and the reference runtime can roll back to an audited revision;
• IWorldEvolutionPolicy supplies learning behavior without adding business enums.

Predicates are opaque strings, so schema statements (for example, a relation describing a domain, range or inverse) use the same evidence-backed link model as every other world statement. No schema migration is required when a new vocabulary appears.

InMemoryWorldKnowledgeRuntime is a deterministic reference runtime for tests and simulations. Production hosts can replace it with an Orleans/DataFabric implementation without changing the agent-facing protocol.

For a repeatable learning/recall workload, WorldKnowledgeCorpusFactory.CreateReferenceCorpus() produces at least 1,000 small documents from 125 topic seeds spanning astronomy, physics, biology, earth science, mathematics, computing, language, history, geography, philosophy, art, music and other neutral world topics. DocumentKnowledgeIndex is a deliberately thin lexical projection: it stores the source text for recall, while one evidence-backed document observation is written to the same World runtime. The knowledge_learn_documents and knowledge_recall_documents AgentKernel tools use that projection without introducing a second fact model.

The synthetic corpus is only a deterministic regression fixture. For real, current information, WikimediaKnowledgeSource reads public Wikipedia summaries through the MediaWiki API and maps each page to a KnowledgeDocument with its page URL, revision id, retrieval time and reuse notice. Use FetchPagesAsync for a curated/reproducible title list, or FetchRandomAsync to sample the live encyclopedia (the default count is 1,000). Missing pages and transient HTTP/JSON failures are returned as diagnostics instead of being silently learned; only documents with non-empty extracts are admitted. A revision-aware document id keeps a later page revision as new evidence instead of overwriting the earlier snapshot.

var runtime = new InMemoryWorldKnowledgeRuntime();
var index = new DocumentKnowledgeIndex(runtime);
using var http = new HttpClient();
var source = new WikimediaKnowledgeSource(http);
var live = await source.FetchRandomAsync(new WikimediaKnowledgeSourceOptions
{
    Language = "en",
    Scope = "world",
    Count = 1_000,
    BatchSize = 20,
});
if (!live.ReachedRequestedCount)
    throw new InvalidOperationException($"Only fetched {live.ReturnedCount}/{live.RequestedCount} live pages.");

await index.LearnAsync(new KnowledgeDocumentBatch
{
    BatchId = $"wikimedia-{DateTimeOffset.UtcNow:yyyyMMdd}",
    RequestedBy = "live-source-agent",
    Documents = live.Documents,
});

Live material must retain the returned Source URL and License/SourceRevision fields and should be revalidated before publication. The source follows the public MediaWiki Action API model and does not turn web text into unverifiable synthetic facts.

To let an Agent perform this as one explicit effect, register AddWikimediaKnowledge(index, source). It exposes knowledge_fetch_wikimedia, which reports fetched/applied counts and every diagnostic; the Agent can then use the existing knowledge_recall_documents tool against the same index.

DailyWorldSimulationFactory supplies a seeded day-by-day stream for integration tests and demos: people encountered, places visited, objects noticed, sounds heard, transient signals, short information notes, and occasional contradictory reports. It emits ordinary WorldObservation records plus documents, so the same stream exercises learning, forgetting, conflict repair, emergent insight and recall. Change DailyWorldSimulationOptions.Seed to replay another trace.

var runtime = new InMemoryWorldKnowledgeRuntime();
var tools = KnowledgeAgentTools.Create(runtime);

// Register `tools` with iTool.AgentKernel. The agent can observe, understand, explain, propose and evolve
// the same world; it does not carry a second memory.

// Or use the thin builder extension:
// AgentKernel.CreateBuilder().AddWorldKnowledge(runtime);

var index = new DocumentKnowledgeIndex(runtime);
await index.LearnAsync(new KnowledgeDocumentBatch
{
    BatchId = "world-reference-v1",
    Documents = WorldKnowledgeCorpusFactory.CreateReferenceCorpus().ToList(),
});
var hits = await index.RecallAsync(new DocumentRecallRequest { Query = "photosynthesis plants" });
// hits.Hits[i].Context links each document back to its world observation/evidence.

The reference behavior is covered by tests/iTool.KnowledgeHub.Tests and can be run with:

dotnet test tests/iTool.KnowledgeHub.Tests/iTool.KnowledgeHub.Tests.csproj

Production iDB RAG foundation

The production target is a continuously maintained RAG engine with iDB as the authoritative document, evidence, knowledge and ingestion-state store. The in-memory reference runtime is not the production storage backend. AgentKernel remains the execution owner and MySearch remains the rebuildable lexical projection; business hosts supply identity, authorization and domain policies.

KnowledgeDocumentChunks now preserves exact UTF-16 source offsets with bounded UTF-8 chunk payloads, Unicode-safe overlap and content-addressed versions that include source metadata. iTool.KnowledgeHub.Idb/IdbKnowledgeDocumentStore stages immutable chunks and a manifest before conditionally publishing the current-document head through the existing iDB mutation contract. Readers must resolve a published head for current retrieval; direct version reads are evidence lookups and can include staged versions. A version manifest omits the body, which is reconstructed from its bounded chunk pages and offsets. Do not configure time-based iDB record expiry for these records: expiring evidence behind a live head would violate this contract. Use RAG reference-aware maintenance and history retention instead. Named DI registration is available; deployment and product authorization policy belong to the consuming host.

The current implementation and acceptance boundaries are listed under Current production acceptance boundary below. Earlier milestones have been superseded by durable phase execution, projection lifecycle, authorized tools, answer generation and reference-aware maintenance. World-state/ontology application semantics are separate from the user-requested RAG engine; this adapter does not supply an iDB historical-fact/graph API. Real model/corpus quality evaluation is explicitly deferred by the user; synthetic tests do not replace that later evaluation.

Storage mutation invariants are verified with a fault-injectable iDB contract double. The adapter now batches staging by provider-reported document/byte capacity, retries bounded admission changes and verifies existing immutable rows when the provider's operation history has expired. IdbKnowledgeLexicalRetriever uses existing bounded BM25 candidate queries and checks authoritative heads before returning a passage; WithdrawAsync hides a document without destroying its historical evidence. RepublishVersionAsync conditionally reuses an existing, non-retired version when the caller supplies the observed current head version and revision. It refuses a changed or withdrawn head, a foreign document, retired version or incomplete active projection; normal publication-change records are emitted. The host must independently authorize the restored version and handle any source-store race or failed repair; ambiguous acknowledgement requires rereading the head before deciding what to do next. This is not a cross-store transaction or a replacement for durable reconciliation. SynchronizeAsync returns a fence that callers can reuse for read-your-writes queries. Search scores are nullable: sorted provider responses may supply ranking without a scalar score. Budget exhaustion is explicit.

Isolated Redis/Orleans integration verifies cluster recreation, replay, retrieval, lifecycle and history expiration. Its completion trace is recorded separately because the VSTest aggregate omits the integration method after search execution. Independent-process kill/recovery and concurrent rolling probes are described below; they do not prove sustained resource bounds. The implementation uses iDB/MySearch public contracts only; changes to their implementation belong to sibling work and are outside this RAG task.

The test suite includes a 1,000-document learn/replay/recall check and a scripted AgentKernel run that calls the document recall tool and verifies the returned evidence, plus a scripted learn-then-recall tool chain.

The reference quality gates are intentionally explicit: the corpus must have at least 1,000 unique documents across 30 or more topic families and two or more languages; replay must apply zero new observations; a ten-query labeled benchmark at K=8 must reach Precision@K and Recall@K of at least 0.90, MRR of at least 0.90, and evidence coverage of at least 0.99; stale links must retire while their evidence remains available; dense observations must yield at least two evidence-backed, stable insight candidates and an Agent run must be able to explain those candidates from derived evidence; a recall-only AgentKernel run must complete in no more than two tool rounds/two calls with no retries or failed tool results. RecallQualityEvaluator and AgentToolRunMetrics provide the reusable measurements behind these assertions.

Rolling ingestion identity now excludes RetrievedAt (fingerprint protocol 2), while retaining source revision, content, title, scope and other source metadata. An unchanged published version only advances the head's LastRetrievedAt through iDB CAS; older observations do not move freshness backward. FirstRetrievedAt belongs to that head publication; the immutable version manifest retains the retrieval time of the first successfully staged manifest. Restoring a withdrawn version requires its current head revision. Legacy fingerprint-1 IDs remain explicitly readable; reingestion computes a fingerprint-2 ID (no destructive migration or historical evidence rewrite). This handles unchanged polling, not yet resumable source scheduling or reference-aware cleanup.

The RAG adapter now exposes IdbKnowledgeRagRetriever.RecallAsync(query, maximumUtf8Bytes, minimumSourceRevision, cancellationToken). It runs the lexical and optional vector channels in parallel, fuses ranks with RRF, rechecks iDB publication heads, and returns citation context plus explicit diagnostics. CandidateBudget applies per channel; combined examined counts are returned. A missing semantic configuration reports semantic_not_configured; classified embedding HTTP or timeout failures report embedding_service_unavailable while lexical retrieval remains available. Cancellation, invalid embeddings and storage failures propagate. Approximate semantic recall reports semantic_recall_approximate even when its candidate budget is not exhausted.

The host constructs an authorized store/collection and model profile, publishes documents, calls EmbedVersionAsync for the desired version/profile, and obtains a SynchronizeAsync fence before read-your-writes retrieval. For durable orchestration use Prepare/ResumeIngestionAsync, named RAG service registration, authorized AgentKernel tools and the answer service described below. An isolated real iDB integration trace verifies independent semantic recall, historical-version filtering, opposite-vector distractor exclusion, scope filtering and degraded lexical fallback with a deterministic embedding fixture. It does not establish a real model's relevance or ANN quality.

AgentKernel hosts can register the read-only production retrieval tool with builder.AddKnowledgeRag(accessResolver) from iTool.KnowledgeHub.Idb.Agents. IKnowledgeRagAccessResolver must derive the actor's identity and access from trusted host state using AgentToolContext; it returns a bound KnowledgeRagReadGrant whose Recall delegate can be IdbKnowledgeRagRetriever.RecallAsync. The grant binds scope, context bytes, per-channel candidate budget, passage limit and minimum source revision. Model input contains only Query and Limit. The handler resolves authorization on each invocation and revalidates it before returning evidence. A null grant denies access. A null grant Scope explicitly allows the entire bound base and must only be issued by a host that has authorized that access. This is an integration contract, not a built-in product identity/permission store; real product wiring remains required.

The tool serializes citations once rather than duplicating them as both JSON and typed passage content. Its description instructs consumers to treat evidence as data and abstain without support; this does not by itself implement answer validation or guarantee prompt-injection resistance.

KnowledgeAnswerGenerator makes one bounded IChatClient call using the typed citation envelope. No evidence bypasses generation and returns insufficient_evidence. The response must be structured claims with known citation labels and exact nonempty source quotes; malformed output, unknown labels or mismatched quotes are rejected as abstentions. Only verified references are returned. This checks provenance, not whether the quote entails the claim or the source is factually correct. IdbKnowledgeRagAnswerService composes hybrid retrieval with this generator and retains recall and abstention diagnostics. It requires a host-authorized binding; callers must revalidate authorization before returning the generated answer. An optional AgentKernel answer tool is available as described below.

To enable answer generation, explicitly register AddKnowledgeRag(accessResolver, includeAnswers: true) and supply the optional KnowledgeRagReadGrant.Answer delegate bound to IdbKnowledgeRagAnswerService.AnswerAsync. A read grant without that delegate cannot generate answers. The host controls MaximumOutputTokens; model arguments cannot override it. Recall and answer handlers share one grant-validation/return-time-revalidation path, including revocation during model execution. This supplies the integration boundary; real product identity wiring is still the host's responsibility.

Whitespace-only evidence chunks now persist a knowledge-embedding-skip record with reason whitespace_only. They are not passed to the embedding model and have no query buckets; subsequent runs reuse the skip record. The original evidence remains available. Embedding receipts distinguish newly Generated, Reused (including prior skips), and newly Skipped chunks.

PrepareIngestionAsync(operationId, document, expectedHeadRevision, profile) stages immutable source evidence and an immutable operation descriptor in iDB before publication or model calls. The existing host/Runtime scheduler can then deliver just the operation ID to ResumeIngestionAsync; the original document body is no longer required. Progress advances by iDB CAS through published, embedded and completed (with a search fence), or superseded. Model failure leaves published progress for retry; existing chunk vectors are reused. Completed operations do not call the model again. Old jobs do not overwrite a newer publication or resurrect a withdrawal. This is an at-least-once phase runner, not a scheduler or an exactly-once model-call guarantee. Job discovery/delivery, retry timing and single-owner execution still need host integration. Preparing source before the descriptor can leave orphan staging after a crash; RunRetentionPageAsync checks heads, references and pending work before retirement. Age alone never authorizes deletion.

DiscoverIngestionAsync queries a bounded page of RAG request records, then reads authoritative request/progress rows from iDB so stale search visibility cannot rerun completed work. Cursors bind to the knowledge base and index. RunIngestionPageAsync resolves each request's embedding profile through a host callback and executes a sequential bounded page; model availability failures return retry diagnostics without losing the persisted phase. Storage and integrity errors propagate. The existing Runtime scheduler should persist NextCursor, restart from null after end-of-scan, and schedule failed requests or another sweep with backoff. New request IDs inserted behind the current cursor are found on the next sweep. The host owns model lifetime and task delivery; this library starts no timers or autonomous processes. Requests from before discovery metadata was introduced remain resumable by explicit operation ID but need migration before indexed discovery.

Saved-answer hosts can call RetainEvidenceAsync(ownerId, versionId) before exposing durable citations, using a stable owner ID per saved artifact. References live in the same iDB knowledge base and remain active after document withdrawal; they have no implicit expiry. Repeating retain recovers a lost acknowledgement without adding another reference. ReleaseEvidenceAsync requires the reference revision and is terminal for that owner/version pair, so delayed retains cannot resurrect a released reference. A newly saved artifact uses a new owner ID. Hosts must authenticate owners and release only after the artifact no longer needs the source. Retirement must atomically fence new references/publications and account for unfinished ingestion, not merely count a stale index view.

TryRetireVersionAsync freezes an unused version without deleting its evidence. Current publication, active saved-answer references and unfinished ingestion block retirement. The bounded indexed scan uses a synchronized source revision, and retirement freezes the version guard atomically with the permanent retirement marker. Same-version writes invalidate that guard check; legacy data without a guard retains the collection-level source fence. Retired evidence IDs cannot be reused; returning content can receive a new storage identity using the current head revision, as described below. Use only after migrating prototype records that lack Kind/KnowledgeBase metadata. Contract tests cover a reference created after the scan; independent-client process probes also cover concurrent ingestion, references, retirement and collection against one shared storage host.

CollectRetiredVersionAsync performs one bounded pass (1–64 records) over an explicitly retired version. New retirement records retain an indexed fence and persist collection checkpoints; pages read authoritative rows and conditionally delete chunks and all embedding profiles without repeating synchronization. Legacy retirement records use the synchronized path. An exhausted fenced scan permits deleting the manifest. Call repeatedly until Completed; after interruption resume from the persisted checkpoint. Permanent retirement markers prevent late writers recreating old evidence IDs. Ingestion and reference receipts remain for replay, with optional request/progress consolidation; terminal ingestion no longer requires the removed manifest. This reclaims source bodies, not all lifecycle metadata. Hosts own the retention policy and pass scheduling; no automatic deletion or timer starts upon registration.

Staging now writes a knowledge-staging intent before any chunk. DiscoverStagedVersionsAsync finds intents older than a host-supplied cutoff, including interrupted staging with no manifest. Its cursor binds the index, knowledge base and cutoff; freeze the cutoff within a sweep. First staging time is not a lease or last-use time: existing heads, references and pending ingestion still block retirement. RunRetentionPageAsync combines discovery, retirement checks and one collection page per candidate, returning blockers and counts. Hosts persist NextCursor and start another sweep after reaching the end, so partially collected versions receive further passes. Collection removes the staging intent last, leaving a durable discoverable record until body cleanup is complete. Prototype orphan data written before staging intents needs explicit migration.

Semantic buckets use angular-16x16-v2: sixteen stored keys and 272 one-bit-neighbor query probes. The earlier eight-bit prototype over-selected candidates; bounded synthetic Recall@8 was 7.0% at 10k vectors/256 candidates versus 95.3% for V2. At 100k, V2 required 1024 candidates to reach 94.9% on that fixture; the default 256 still truncated. These are synthetic candidate measurements, not real-language quality or storage latency. See benchmarks/iTool.KnowledgeHub.RetrievalBenchmarks for reproducible gates and raw results. Re-run EmbedVersionAsync to build V2 projections before switching existing deployments; terminal ingestion does not automatically rebuild old protocols.

RebuildProjectionPageAsync backfills the selected model profile and the current bucket protocol over published heads. Each work unit embeds at most sixteen chunks; EmbedVersionPageAsync also exposes that bounded unit directly. The opaque cursor contains the document's chunk offset and binds to the knowledge base, index, profile and protocol. Save NextCursor only after receiving the result; replaying an earlier cursor reuses committed chunks. Model availability failures return retry without advancing that document; storage failures propagate so the host retries its last saved cursor. Source versions that are no longer current are skipped. SearchRevision fences the projections written by the page. This works independently of completed ingestion receipts.

SweepCompleted means this traversal ended, not that concurrent publications are all covered. Hosts should repeat sweeps and arrange new ingestion for the target profile during a migration; profile activation needs a separate coverage check and real-query evaluation. This API does not change the online profile or delete previous embeddings. For a protocol upgrade, run the new maintenance worker before switching query workers; the new binary queries only its own protocol. For a model upgrade within one protocol, old and new retrievers can coexist with separate profiles.

AuditProjectionPageAsync checks coverage without calling the model or writing records. Pass its opaque NextCursor unchanged until ready, incomplete, or source_changed. It checks current heads, target embedding presence, metadata bindings, whitespace skips, normalized vector payloads and recomputed bucket keys in batches of at most sixteen chunks per work unit. Counts accumulate across pages. Any change to the collection source revision invalidates the sweep, even a change in another knowledge base sharing that collection; restart at null after source_changed.

ready describes coverage at the returned SourceRevision only. It is not a lasting guarantee, semantic-quality verdict, authorization token, or atomic profile activation. Audit cursors are host-internal continuation state and must not be accepted from an untrusted client. A busy shared collection may need a quiet audit window. The staging-ledger invariant is required: migrate older published versions without staging records before relying on complete coverage. The active-profile switch uses the conditional activation API below, and ongoing ingestion must maintain the target.

ActivateProjectionAsync consumes the host's unmodified ready audit, bound to this knowledge base, index, profile and protocol. It conditions the active-record write on both the audited source revision and the expected previous active-record revision. Stale audits/configurations fail; replaying a lost acknowledgement recovers the matching active record. Audits and these management APIs are trusted-host operations, not external authorization tokens. Model quality approval remains the host's responsibility.

Construct IdbKnowledgeRagRetriever(store, search, profile => generator) to resolve the persisted active profile per request. It checks the active revision again after recall and retries once on a switch, then fails explicitly if switching continues. Without activation it provides lexical recall with the existing semantic-not-configured diagnostic. Existing explicit-profile constructors remain useful for evaluation and backfill verification; they do not follow the active record.

After activation, publication validates the active projection before the atomic head write, under the same source fence. The low-level workflow is Stage → Embed → Publish. Matching-profile ingestion requests generate embeddings before publication, so model failure leaves the previous head visible. A queued request using the single-generator overload still fails on a different active profile with knowledge_ingestion_profile_superseded. Activation and continued publication operate only through RAG APIs; direct writes to its internal rows bypass these invariants.

Use the model-resolver overload of ResumeIngestionAsync, or RunIngestionPageAsync, to migrate queued work across model switches. Each embedding phase resolves the current active profile (or the request's original profile before activation). The immutable request remains unchanged; progress records ProjectionProfile/ProjectionProtocol for the last completed embedding phase. Model failure retains pending work and the previous head. Replays reuse embeddings and terminal progress avoids resolving a model. A later source version still supersedes the old request using the original head precondition. A switch racing a publication may reject that attempt through the existing source/projection fence; retry the same operation to resolve the new active model.

Named host registration reuses the existing dependency-injection container:

services.AddIdbKnowledgeRag("articles", new KnowledgeRagBinding("knowledge", "articles"),
    sp => sp.GetRequiredService<IIndexedDocumentDatabaseGrain>(),
    sp => sp.GetRequiredService<IMySearchQueryGrain>(),
    (sp, profile) => sp.GetRequiredKeyedService<IEmbeddingGenerator<string, Embedding<float>>>(profile.Fingerprint()),
    sp => sp.GetRequiredService<IChatClient>());

The host registers the database grain for this index using the existing GetIndexedDocumentDatabase helper and the matching query grain/provider/partition, as well as each immutable model profile and optional chat client. Omit the chat callback for recall-only bindings. Resolve IdbKnowledgeDocumentStore, IdbKnowledgeRagRetriever or IdbKnowledgeRagAnswerService with GetRequiredKeyedService<T>("articles") inside the request/job's service scope. Duplicate keys fail before changing registrations; separate binding keys may target separate knowledge namespaces. Callbacks resolve host-owned clients; the RAG wrapper does not dispose them or create a timer.

This registration is not authorization. The host's IKnowledgeRagAccessResolver selects the allowed binding/scope from authenticated state, supplies that binding's RecallAsync/AnswerAsync delegates, and revalidates after the call. Do not let tool arguments choose a binding key. Long-lived agent objects must not capture scoped RAG services beyond their request scope. Existing Runtime jobs resolve the same store to run ingestion/rebuild/retention pages and persist their returned cursors.

The independent-process recovery probe in tests/iTool.KnowledgeHub.ProcessProbe forcibly kills a worker after persisted publication, then starts two new processes to verify recovery and terminal replay against real loopback Redis/Orleans. The recorded run recovered complete evidence and ready projection coverage; the first recovery made one embedding call and terminal replay made none. It does not establish Redis/machine-crash safety or sustained multi-worker performance.

Version-aware writes use the existing non-flushing published revision as an optimistic lower bound, then retry with the authoritative source revision returned on an iDB conflict. They still read retirement/active-projection state and condition the write; this avoids calling the database health API (which flushes pending indexing) before every mutation. Repeated conflicts use bounded jittered backoff. Exhausted contention is explicit and the host can retry the same operation. The rolling process probe records both contention and retained metadata growth; physical body collection alone is not a bound on total lifecycle-record count.

Ordinary chunk, embedding, request and reference writes now atomically update their version's staging row using its document revision. Retirement atomically freezes that same row and creates the permanent retirement marker. Unrelated versions therefore do not invalidate one another's retirement checks. Initial staging and head publication still use the collection source fence; publication must also serialize against an active-profile switch. Legacy versions lacking a staging row retain the conservative collection fence, with a retry if a guard appears mid-check.

The adapter reserves one mutation slot for the version guard and requires an advisory capacity of at least two records for these batches; it never splits a guarded atomic write. This changes the staging row from immutable intent to mutable technical coordination while preserving its first-staged timestamp. All writers must use the new guard protocol before per-version retirement is enabled: do not mix pre-guard binaries writing the same namespace during deployment. The code is not yet released; a live upgrade needs a quiesced writer handover.

New retirement records capture the indexed source fence and collection protocol. Collection stores its forward cursor in the frozen staging row, atomically with each page of conditional deletes. It queries at that retirement fence without flushing the index again on every page; stale indexed hits are safe because version writes are frozen and authoritative rows are read before deletion. On a lost acknowledgement, a fresh store resumes from the persisted cursor. Competing collectors cannot advance a cursor without the corresponding deletions. The manifest and staging row are removed only after the bounded scan is exhausted; permanent retirement and ingestion receipts stay. Legacy retirement records without this checkpoint protocol use the previous synchronized path.

ID-ordered maintenance and vector scans continue with an exclusive _id.keyword range rather than a live search_after anchor. The exact keyword field is required: the ordinary _id field also has analyzed tokens. This lets a saved cursor survive deletion of its anchor across worker restarts. BM25 recall fetches its bounded candidate identifiers, version IDs and ordinals in one query, using _source field selection. It then hydrates source chunks in batches of at most 64, without score pagination across changing readers. The host search window must accommodate the requested candidate budget (up to 4096; the current MySearch default is 10000); smaller host limits produce an explicit query error rather than silently reducing recall.

Lexical and vector recall share authoritative chunk materialization. Heads are checked before and after the batch read. Candidates replaced or withdrawn during that read are discarded; missing chunks for a still-current version raise knowledge_evidence_missing. Chunk identity and knowledge-base/version bindings are validated before returning source text. This is a point-in-time read, not an evidence lease: saved answers still need the existing explicit retain protocol.

CompactRetiredIngestionAsync(operationId) consolidates a terminal operation's original request into CompactedRequest on its progress record and atomically removes the standalone request. It requires a permanent retirement marker and a completed/superseded progress record; active versions keep their standalone requests, and pending operations are left unchanged. For legacy terminal receipts, the sweep may fill in the request's missing TerminalStage index hint even before retirement; it returns version_not_retired because no consolidation occurred. The host invokes it through existing maintenance work, or uses CompactRetiredIngestionPageAsync for bounded discovery and compaction (default eight operations per page). Persist its knowledge-base-bound cursor; restart a sweep to revisit blocked entries. Capacity limits or concurrent changes return an explicit blocker. Retrying a lost acknowledgement is safe, and terminal resume checks the progress receipt before requesting the now-optional standalone intent. The receipt and original intent remain available until their persisted history deadline; operation IDs remain reserved after expiration. Deploy all readers with receipt-first resume support before enabling this maintenance step in a shared namespace. This saves one record per compacted operation; history expiration removes the retained intent payload, while minimal anti-replay tombstones and retirement markers remain durable.

Returning content does not reuse retired evidence IDs. PublishAsync and revision-aware PrepareIngestionAsync derive a new storage version from the original content fingerprint and the expected current head revision when that content's original version is retired. The manifest records ContentVersionId and RestoredFromHeadRevision; neither changes source metadata or the content fingerprint. Identical retries share the same incarnation, and refreshes of its current content reuse it. A later withdrawal/return has a new head revision and therefore a new storage identity. Old retirement markers remain authoritative, so stale writes and old citations cannot accidentally bind to newly stored bodies. StageAsync(document, expectedHeadRevision) supports the same preparation path; the original staging overload does not implicitly restore retired content. This is an additive manifest format, with optional fields omitted for original versions. Readers must treat VersionId as evidence identity, not assume it always equals the content fingerprint returned by KnowledgeDocumentChunks.Describe.

Terminal ingestion commits its progress receipt and request TerminalStage hint in the same conditional iDB batch. This hint is lifecycle metadata, excluded from immutable-intent comparison; the original operation parameters remain unchanged. Pending discovery excludes indexed terminal requests. Retirement excludes released references and terminal requests at its synchronized source fence, then still verifies returned candidates against source records. Legacy requests without a hint stay on the conservative verification path until a maintenance sweep repairs them. An exact-budget scan uses one final index-only probe to distinguish end-of-results from truncation; it never reads more than the configured number of candidate source records. These changes bound scan work by potentially blocking records rather than all historical releases/completions; they do not delete those historical records or make lifetime storage bounded.

RAG operation diagnostics

KnowledgeRagDiagnostics.SourceName is iTool.KnowledgeHub.Rag for both the .NET ActivitySource and Meter. Subscribe through the host's existing telemetry configuration; the library installs no exporter, service, background loop or logging sink. Traces inherit the caller's activity and cover ingestion.prepare, ingestion.resume, embedding, recall, generation and answer. This covers the public hybrid/answer entry points; direct low-level lexical/vector calls and maintenance methods do not currently have independent operation spans.

Operation success means the method returned normally, including explicit abstention or a normal business blocker; it does not certify answer quality. Generation includes the no-evidence fast path, so its call count is not a provider billing count. Retries internal to hybrid recall may produce multiple embedding spans; candidate measurements describe only the final returned recall. No question, document text, knowledge-base ID, operation ID, provider URL, model profile or exception message is attached. The only metric labels are fixed operation/outcome values, preventing per-document or per-tenant cardinality growth. Provider exceptions propagate unchanged to the host; traces record only the fixed operation_failed description. Host-supplied telemetry callbacks and exporters remain host responsibilities.

Current production acceptance boundary

The reference Utopia README describes a complete application with hybrid retrieval, evidence-linked answers, ingestion jobs and temporal knowledge governance. This foundation reuses its relevant RAG ideas through existing iDB, model and Runtime contracts; the product UI, identity policy and graph/ontology application are outside this RAG implementation.

These are separate gates. Local synthetic-model tests and short rolling probes are not a blanket production sign-off, nor evidence of a lifetime storage bound.

The main iTool.Develop.Solution.sln includes iTool.KnowledgeHub.Idb, its adapter tests, the independent-process/rolling probe and the retrieval benchmark. Redis-backed probes remain explicitly configured test runs; ordinary adapter test runs skip the real Redis integration when no isolated endpoint is provided.

RecallQualityEvaluator.EvaluatePassagesAsync reuses the existing quality contracts for live or recorded RAG results. It computes document-level precision/recall/MRR within the first K passages, deduplicates chunk hits, checks citation identity/offset structure and retains candidate-budget diagnostics. The retrieval benchmark's --evaluate-passages mode accepts labeled recorded runs and returns a nonzero exit code for a missed gate or invalid input. See its README and fixture schema. These mechanics are tested, including a real-iDB callback; no real-model relevance gate has yet been passed, and citation structure is not semantic entailment.

Both iTool.KnowledgeHub and iTool.KnowledgeHub.Idb follow iTool.SharedVersion.props; the adapter package includes this README and its XML API documentation. The package-consumer probe restores the exact adapter package into a separate project with no source project references, verifies all locally built dependency package hashes and exercises the public registration/evidence APIs. It produces only temporary local packages; it does not publish or deploy a Foundation release.

RAG history retention

KnowledgeHistoryRetentionOptions.Retention defaults to 90 days (positive duration, maximum 36,500 days). Pass it to IdbKnowledgeDocumentStore or set KnowledgeRagBinding.HistoryRetention. The store accepts an optional TimeProvider; DI uses the host's registered provider or system UTC. Deadlines are persisted when ingestion completes/is superseded or when a reference is released. Changing configuration affects future terminal transitions, not already persisted deadlines. Current documents, pending ingestion and active evidence references have no history expiry.

At now >= ExpiresAt, getters, resume, prepare replay, retain and release return the explicit KnowledgeHistoryExpiredException (Code = "knowledge_history_expired", with ExpiresAt). Hosts should map this to their “已过期” result instead of treating it as missing or retryable work. A terminal prepare replay is read-only. Expiration atomically replaces the receipt with a minimal reserved-ID tombstone and deletes any standalone intent; compacted intent payload is removed. It does not delete published evidence or permit operation/reference ID reuse. Transport failures remain retryable: a committed tombstone survives lost acknowledgements.

Call MaintainHistoryPageAsync from existing host maintenance (up to 64 candidates/page), persist its namespace/index-bound cursor, and restart sweeps for future deadlines. Reads also enforce expiry without waiting for that job. Expired counts expired candidates observed, not unique IDs or deleted rows. Undated legacy terminal history receives a full grace period on first access; maintenance discovers legacy requests and already-compacted receipts by exact ingestion/reference storage-ID ranges, including receipts without indexed Kind/KnowledgeBase fields. Scans are bounded and do not enumerate document bodies, vectors or other knowledge bases. Upgrade all writers/readers before enabling history cleanup in a shared namespace; old binaries do not recognize expired tombstones. Tombstones are deliberately retained indefinitely to distinguish expired IDs from never-seen IDs; total metadata count is not constant.

Real model and corpus quality evaluation is deferred until host-provided fixtures are available; current tests validate engine mechanics and persistence, not real-world retrieval/answer accuracy.

Extended rolling and storage-host recovery probes

The process probe accepts ITOOL_KNOWLEDGE_ROLLING_ROUNDS (default 20, 1–10,000) and ITOOL_KNOWLEDGE_ROLLING_DAYS_PER_ROUND (default 0, 0–30). A positive day increment advances only the injected RAG test clock, runs paged history maintenance after each settled round, checks that expired replay cannot call the model, and keeps current references active. Readers continue while maintenance runs. Each round records source count/revision, elapsed time, working set, managed heap, cumulative allocations/CPU and completed reads. Cumulative allocation is allocation traffic, not retained memory; a zero private-byte reading can mean the platform does not supply that counter. Virtual calendar coverage is not equivalent to real elapsed uptime.

After building tests/iTool.KnowledgeHub.ProcessProbe, use an explicitly owned loopback Redis:

ITOOL_KNOWLEDGE_TEST_REDIS=127.0.0.1:55704 \
ITOOL_KNOWLEDGE_ROLLING_ROUNDS=60 \
ITOOL_KNOWLEDGE_ROLLING_DAYS_PER_ROUND=7 \
dotnet tests/iTool.KnowledgeHub.ProcessProbe/bin/Debug/net10.0/iTool.KnowledgeHub.ProcessProbe.dll \
  rolling-readers /tmp/unique-rolling-result.json

ITOOL_KNOWLEDGE_TEST_REDIS=127.0.0.1:55704 \
python3 tests/iTool.KnowledgeHub.ProcessProbe/run_concurrent_probe.py --restart-server

--restart-server kills the storage-host process with SIGKILL after both writers and the reader finish, restarts with the same durable identity/collection, then launches a fresh verifier. It checks all terminal receipts without model execution, released/current reference states, absence of collected bodies, current retrieval and projection coverage. This is settled-write crash recovery, not uninterrupted multi-node failover. The driver records both host PIDs and checks the killed process exit code; its default mode still tests concurrent processes without host restart. Neither probe modifies iDB implementation or publishes/deploys packages.

Verified extended probe results are in tests/iTool.KnowledgeHub.ProcessProbe/results/: 20261105-rolling-history-60.json records 636.36 seconds, 480 publications, 472 collected old bodies, 8 ready current versions, 13,907 reads (12,725 overlapping writes), 87,456 passage checks, 2,781 forbidden-scope checks and 48 rejected expired replays. The injected clock advances 420 days; real duration is 10.6 minutes. Fixture-model median/P95 are 38.12/89.77 ms. Source record count ends at 1,481, including growing anti-replay metadata; working-set samples start at 350.4 MiB and end at 467.5 MiB. These are measurements, not a lifetime memory/storage bound. 20261105-storage-host-recovery.json records SIGKILL exit -9 (PID 66242), recovery PID 66527 and verifier PID 66535, with 40 terminal receipts, 40 reference states and 38 absent historical bodies checked after recovery. Current retrieval/projection coverage is ready. All fixtures use local Redis and deterministic models, not a user's production data.

Reading evidence beyond a search result

IdbKnowledgeDocumentStore.ReadDocumentEvidenceAsync(documentId, offset, count, expectedVersionId) returns at most 64 passages with the same evidence IDs and exact source offsets as retrieval. Pass the first page's VersionId into subsequent requests; replacement or withdrawal during a read fails with knowledge_document_version_changed, so a reader cannot silently combine versions. NextOffset is null on the last page.

ReadVersionEvidenceAsync(versionId, offset, count) reads a separately authorized immutable version, including staged evidence; it must not substitute for current-publication checks. Unknown versions return null; retired versions report knowledge_evidence_retired. This does not expire live evidence or change the configurable 90-day operation/reference history retention contract.

Four focused contract tests cover Unicode offsets and citation IDs, paging, replacement, withdrawal during a read, namespace isolation, retirement and corrupt offsets. Workspace integration and the coordinated package release for these new APIs are still pending.

Pluggable evidence-backed topic candidates

Optional chat-backed topic proposals

ChatKnowledgeTopicProjector is an optional IKnowledgeTopicProjector using a host-owned IChatClient. It can propose a shared candidate from multiple passages; register it explicitly in place of DocumentKnowledgeTopicProjector and invoke it through KnowledgeTopicDiscovery:

var projector = new ChatKnowledgeTopicProjector(chatClient, "host-topic-model", immutableModelRevision);
var discovery = new KnowledgeTopicDiscovery(projector);
var topics = await discovery.DiscoverAsync(authorizedKnowledgeBaseId, question, authorizedContext,
    limit: 8, cancellationToken: cancellationToken);

The host must filter evidence before the call and recheck authorization and source versions after it. Model text is untrusted: invalid JSON or fabricated quotes fail, while valid quotes establish provenance rather than semantic truth. Model-proposed keys can vary between calls; do not use them alone as durable event identities or as an exactly-once key. Evaluate quality on the target corpus before enabling this optional provider. The provider does not own or dispose the model client.

KnowledgeTopicDiscovery accepts an authorized KnowledgeRetrievalContext, a host-selected scope, a natural-language query and a limit (1–32). It invokes IKnowledgeTopicProjector once and validates all returned candidates against that context. Providers supply stable topic keys, titles, summaries and exact quoted references using the existing KnowledgeAnswerReference contract.

var discovery = new KnowledgeTopicDiscovery(new DocumentKnowledgeTopicProjector());
var topics = await discovery.DiscoverAsync(authorizedKnowledgeBaseId, question,
    recall.Result.Context, limit: 8, cancellationToken: cancellationToken);

The built-in provider returns document candidates in retrieval order. It groups by document ID and uses a quoted passage preview; it does not perform semantic event clustering. A domain or model plugin can replace it without changing provenance checks. No vendor-specific model, scheduler, news-production rule or consumer progress is embedded in the mechanism.

Topic IDs bind scope, provider ID and provider-owned stable key. Revisions bind provider revision, candidate content and cited evidence, excluding query-local labels and scores. Stable keys are the provider's responsibility; keys derived from generated wording will not remain stable across edits. These are discovery snapshots, not persisted topic records or change cursors. Hosts must retrieve current authorized evidence and recheck permission before returning results; this API does not establish factual entailment, document publication status or business value.

Unknown citations, invented quotes, duplicate candidate keys and excess output fail explicitly. Typed evidence is reassembled before the provider call, ignoring a caller-supplied JSON envelope. No evidence returns no candidates without invoking the provider. Cancellation is checked before and after projection. Workspace/sys wiring and durable topic/change discovery remain pending.

Atomic publication change records and bounded consumption

The iDB knowledge adapter now appends KnowledgePublicationChange in the same atomic mutation batch as a meaningful head transition. Publication, version replacement, withdrawal and restoration produce records; freshness-only refetches do not. A failed head CAS cannot leave an orphan change, and a replay after a lost acknowledgement does not create another record. The existing iDB contract is consumed unchanged. Publication batches now include a head, its version guard when present, and a change record, requiring admission for up to three records.

Sequence is allocated under the collection source fence and is strictly increasing for committed transitions, with gaps allowed. It is not a wall-clock timestamp, event date or consecutive counter. ObservedAt is system observation time; the original document publication time remains separate. The record ID derives from the publication operation, and PreviousVersionId records its prior head.

GetPublicationChangeAsync(changeId) reads an individually authorized transition: unknown IDs return null; at its immutable deadline it throws KnowledgeHistoryExpiredException (knowledge_history_expired). The configurable history retention defaults to 90 days. Expiring this metadata does not withdraw live knowledge or grant access to historical evidence.

Existing heads do not gain synthetic historical events retroactively. Consumers must establish an explicit baseline when migrating pre-existing knowledge. History compaction, old-data baseline migration and sys/flow integration remain pending; these records alone do not prove complete historical coverage.

Polling changes from a flow

var page = await store.ReadPublicationChangesAsync(search, savedCursor, pageSize: 32,
    cancellationToken: cancellationToken);
foreach (var change in page.Changes)
{
    // Host/business work; deduplicate effects by ChangeId and authorize evidence separately.
    await ProcessAsync(change, cancellationToken);
}
// Save only after the whole page has succeeded, using the existing Runtime/host persistence.
await SaveCursorAsync(page.NextCursor, cancellationToken);

A null cursor starts at the beginning of recorded history. To explicitly ignore earlier changes, call CreatePublicationCheckpointAsync and persist its result. Both entry points initialize one CAS-created stream identity if needed; they do not persist a consumer position. The initial call therefore requires metadata write access. Hosts remain responsible for authorization.

Each sweep synchronizes accepted writes through the existing source-to-index fence, captures a fixed upper bound and reads at most 64 transitions per page. Sequence sorting explicitly requests numeric_type: long and a stable _id tie-breaker. Query matches are batch-read from authoritative storage and validated. Later publications wait for the next sweep. CaughtUp means the current upper bound was exhausted; continue polling with NextCursor to see future writes. An exactly-full last page may require one empty page to establish completion. Replaying a saved cursor re-delivers its transitions; it does not acknowledge them or make downstream effects exactly-once.

Cursors bind index, knowledge namespace and stream identity. They expire under the configured history policy; expired records/cursors return knowledge_history_expired. Missing matched rows, source/sort mismatches and reset streams fail explicitly, without returning an advanced cursor. Collection replacement must preserve the stream record only when preserving the entire history; otherwise old cursors are invalid. Cursor text is opaque continuation data, not an authorization credential. Filters and consumer progress belong to the caller, not a shared global subscriber.

Validation includes an isolated loopback Redis/Orleans test that resumes change paging after cluster restart and replays a page. It also exposed and fixed the lagging published-index fence and implicit string-sort assumptions. This does not establish Workspace HTTP/flow/UI or concurrency SLO readiness.

Derived artifact version descriptions (candidate)

KnowledgeDerivedArtifacts.Describe describes a derived graph, topic or other payload using its knowledge namespace, logical artifact ID, generator ID/revision and exact source document versions. Source ordering does not change the version; duplicate document IDs are rejected. Sources are copied and exposed read-only. Payloads are bounded to 1 MiB of valid UTF-8 and at most 64 source documents; source version IDs use the existing lowercase SHA-256 representation.

The version hashes exact payload text, so producers must encode deterministically. A generator revision must change when model, rules or configuration changes. This is a description contract, not a storage or publication API: it does not retain evidence, authorize reads, verify source currency, or make artifacts visible. Those operations must be integrated with the existing KnowledgeHub.Idb version/reference lifecycle before use in production. Directly constructing a record bypasses validation; storage boundaries must validate incoming descriptions again.

Validation: dotnet test tests/iTool.KnowledgeHub.Tests/iTool.KnowledgeHub.Tests.csproj covers source-order stability, input isolation, identity changes and invalid source/payload rejection.

IdbKnowledgeDocumentStore.StageArtifactAsync and ReadStagedArtifactAsync provide durable unpublished staging for these descriptions. Storage revalidates the content address and knowledge namespace. Initial staging checks source version/document bindings. Replays keep the first deadline; the configured history retention defaults to 90 days, after which reads and replays throw KnowledgeHistoryExpiredException. Parallel creation uses the first stored deadline.

Staging does not pin sources or validate that they are still current. It never changes document heads or emits publication changes. A later publication operation must recheck source validity and manage evidence references. Physical deletion of tombstones is not implemented; staging alone is not a complete production artifact lifecycle.

Expired staging reads now conditionally compact the record to a small tombstone, removing the payload and source list while keeping the version ID reserved. A later clock rollback cannot revive this tombstone. Hosts can also invoke MaintainArtifactStagingPageAsync for a bounded sweep (1–64 records). Persist its opaque cursor to resume; restart from null after the final page. Each page synchronizes the search projection and rechecks authoritative records before conditional compaction. Cursors bind collection and knowledge namespace. This does not register a scheduler; the host must arrange maintenance. Expired counts include already-expired records observed during a concurrent sweep.

PublishArtifactAsync(versionId, expectedHeadRevision) conditionally publishes a staged artifact, pins its source versions, and switches its head in one existing database mutation batch. Sources must still be current. Pass null for a new head or its observed revision for replacement. Replacing a different version also starts that previous publication's history retention in the same atomic batch. Failed replacement does not retire it. Existing retirement deadlines are preserved; idempotent publication and restoration of the same version do not start retirement. ReadCurrentArtifactAsync(artifactId) validates source currency and rejects inconsistent reads. Both use the authoritative database source revision, not the potentially lagging search revision. Concurrent collection writes can cause explicit conflicts; callers must reevaluate inputs before retrying.

Published copies survive staging expiration. Hosts own authorization and must arrange maintenance of retired publications and explicit retirement of withdrawn publications before enabling production generators.

WithdrawArtifactAsync(artifactId, expectedVersionId, expectedHeadRevision) hides exactly the observed current artifact, including when its staging has expired or its source has changed. It keeps a revisioned head tombstone; a stale creator cannot recreate an absent head. Reading current content returns null for a withdrawn head. To restore unexpired staging, explicitly publish with the tombstone revision. Withdrawal retains source references for history until that publication is explicitly retired and expires.

RetireArtifactVersionAsync(versionId) terminally retires an inactive published version. It refuses the current active head and uses a source-revision fence against concurrent restoration. Repeating retirement preserves its first deadline, based on KnowledgeHistoryRetentionOptions (90 days by default). A retired version cannot be republished; use a new generator revision for regenerated content. Withdrawal alone remains reversible while its staging is valid.

ReadArtifactVersionAsync(versionId) reads historical published content without asserting source currency. Before the retirement deadline, its evidence remains pinned. At or after the deadline, one conditional batch removes the payload, releases only this publication's references, and retains an expired marker. Reads throw KnowledgeHistoryExpiredException, including after clock rollback. Other owners' references are untouched. A conflict fails explicitly and requires a fresh read. MaintainArtifactRetirementPageAsync sweeps retired publications in pages of 1–64 records, reusing the staging sweep's ID-range pagination and authoritative expiry checks. Persist the returned cursor while a sweep is in progress; restart from null after its final page to revisit future deadlines. Cursors bind the collection, knowledge namespace and maintenance kind, so staging and retirement cursors are not interchangeable. Hosts must arrange explicit retirement for withdrawn publications and maintenance scheduling; this API registers no background service. The expired count includes concurrent expiry observations.

ReadArtifactHeadsPageAsync(search, cursor, pageSize) discovers published artifact identities in bounded pages (1–64), including withdrawn head markers. Unpublished staging is never listed. Each head is read from authoritative storage; search supplies only ordered IDs. Continuations bind the index, knowledge namespace and source revision. Any collection write during a page or between pages fails with knowledge_artifact_catalog_changed; discard the partial scan and restart from null. This deliberately favors consistent enumeration over progress under continuous writes.

A listed head does not prove that its source documents are current. Read its content with ReadCurrentArtifactAsync, check the returned version against the observed head, and apply host authorization before exposing it. Withdrawn heads have no current content. This catalog is a storage primitive for derived projections, not a graph query engine or a Workspace graph migration.