iTool.MyScriptRuntimes 4.0.0

iTool.MyScriptRuntimes

Invocation context attachments

RuntimeInvokeRequest.ContextAttachment carries a neutral, bounded immutable envelope: version 1, ASCII schema id (1–128 characters), positive schema revision, and opaque payload (at most 8192 UTF-8 bytes). The original envelope token is limited to 65536 bytes, including escapes/whitespace. The runtime does not interpret product payloads or turn them into permissions. Trusted host codecs must exclude credentials, tokens and arbitrary CLR objects.

Optional request/trigger/delayed fields use RuntimeNullableContextAttachmentJsonConverter and omit null when writing. Historical missing/explicit-null values preserve the old request digest. A non-null attachment participates in the existing digest: changing it under the same idempotency key conflicts. Admission freezes mutable inputs before awaiting storage or enrichment. F5 incarnation, generation, replay policy, retention and cancellation ownership are unchanged.

IRuntimeInvocationContextAccessor exposes the active call only. Host and Jint lifetimes revoke captured views on completion/disposal, including captured ExecutionContexts. Keywords must read it at invocation time, never at build time. Attachment is not added to JavaScript context globals. Provider bridges, library children, flows, events and delayed events propagate it explicitly. Library calls outside an active invocation fail; a waiting cancellation is not a durable child cancel.

All ordinary and debug HTTP/WebSocket creation paths and trigger HTTP paths use RuntimeInvocationIngressPolicy. Default policy rejects client non-null attachment with runtime_attachment_untrusted; explicitly configured Overwrite discards it and invokes a trusted IRuntimeInvocationAttachmentEnricher. Caller is always replaced from the validated principal, including debug's nested Invocation. No principal means null Caller, never a service identity. Protected system providers reject missing/insufficient Caller with existing system_scope_denied. This closes the former null-Caller scope-check bypass; trusted in-process callers and tests must now supply explicit least-privilege identity.

Scheduled execution requires a host IRuntimeScheduledInvocationAuthorization that resolves and revalidates durable delegation for the exact plan/occurrence. No registration means runtime_scheduled_authorization_required and no dispatch. This is a host authorization port, not an implementation of product long-lived grants, revocation or persistence. Manual trigger calls still pass through ingress authorization. Deploy compatible readers/transports/owners before enabling attachment writers; old readers must not silently drop new authority metadata.

Integration code is implemented; acceptance is gated on the coordinating build and full chain fixtures, including a real SQL Server fixture for SQL delayed authority. Product migration and scheduled delegation policy are separate requirements, not implied by these public types.

iTool.MyScriptRuntimes is a reusable development and execution foundation. It contains no Workspace concept: Workspace, a multi-project host, or another product composes this library and supplies identity, authorization, tenancy, repository placement and UI.

The stable runtime address is Application / Service / Operation. Flow, event, plan and direct calls all enter the same RuntimeInvokeRequest execution path. Repository directories, build directories, object keys and database credentials never cross that caller-facing boundary.

Authority boundaries

A native Git repository requires filesystem semantics (local disk or a suitable shared Git filesystem). IIdbFileStorage is the protocol-neutral file surface used by Git reads and runtime configuration; ObjectStoreIdbFileStorage persists its immutable projection to S3-compatible object storage. S3 is not presented as a fake Git object database and cannot advance refs. Backups or another iDB provider may copy content without creating a second live ref authority.

Protocols and extension points

Git identity with iDB file storage

MapGitSmartHttp serves upload-pack and receive-pack from the managed VirtualGitProtocolServer. It never starts a Git process, reads a local repository, or serves loose-object/pack paths from a filesystem. MapGitContentManagement routes committed-file reads and writes through the same VirtualGit object/ref authority. An API commit builds ordinary blobs, trees and commits and advances refs/heads/* through provider compare-and-swap. A standard Git push is immediately visible through the content API, and an API commit is immediately cloneable/fetchable by a standard Git client. The optional one-way iDB mirror is best-effort and cannot change any Git ref.

The content API supports branch list/create/delete, tree and file reads at any Git ref, and atomic ordered upsert/delete/move commits. ExpectedHead is a full Git object id. Competing writers get content_conflict and the current head; a create followed by delete in one commit is a net-zero change rather than a synthetic pending deletion.

Repository ids are logical identifiers normalized by VirtualGitRepositoryFactory; they are not filesystem paths. The obsolete RepositoryRoot, GitExecutable, GitExecPath and ReceivePublication option properties remain only as migration-time source-compatibility shells and are ignored by the managed routes.

Git-backed configuration and JavaScript

One resolved commit supplies the entire application snapshot:

1. app.json
2. optional app.{profile}.json deep overlay; arrays replace
3. one selected pipeline source: executable app.pipeline.js or declarative app.pipeline.json

Historical app.pipline.js and app.pipline.json spellings are read only when their canonical counterpart is absent. Canonical and legacy files with equivalent content produce an explicit configuration warning; different content fails with configuration_pipeline_ambiguous. When both JavaScript and declarative sources exist, app.json must select pipeline.source as app.pipeline.js or app.pipeline.json.

The declarative protocol is itool.runtime.pipeline revision 1 and registers flow, event and plan targets supplied by Runtime plugins:

{
  "protocol": "itool.runtime.pipeline",
  "revision": 1,
  "flows": [{ "name": "create-order", "service": "orders", "operation": "create" }],
  "events": [{ "topic": "orders.requested", "service": "orders", "operation": "create" }],
  "plans": [{ "name": "heartbeat", "intervalMilliseconds": 60000, "service": "orders", "operation": "create" }]
}

app.pipeline.js is executed by isolated, bounded Jint engines. Registrations must be deterministic across the pool. CLR namespace access is not enabled; memory, statement, promise and wall-clock limits are enforced.

runtime.register("orders", "create", async (input, context) => {
  const customer = await lib.invoke("crm", "customers", "get", { id: input.customerId });
  return { accepted: true, customer, source: context.trigger.source };
});

runtime.flow("create-order", "orders", "create");

Events and plans use file declarations. scripts/**/*.event.js and scripts/**/*.plan.js are discovered from the pinned Git tree and must export synchronous getConfig() plus invoke(input, context). Saving, moving, renaming or deleting those files is reconciled by replacing the active trigger manifest with the next successfully compiled commit.

// scripts/orders/requested.event.js
export function getConfig() { return { name: "requested orders", stream: "orders.requested" }; }
export async function invoke(input, context) { return createOrder(input, context); }

// scripts/orders/heartbeat.plan.js
export function getConfig() { return { name: "order-heartbeat", every: "1m" }; }
export async function invoke(input, context) { return checkOrders(input, context); }

config is a deep-frozen object from the Git snapshot. lib.invoke re-enters the same execution gateway and has a maximum nesting depth. A host adds real system objects through IScriptRuntimeKeyword; each object receives only application name, Git commit and a cloned configuration object.

Trigger registrations belong to the active Git commit. Event ids and flow idempotency keys are commit-scoped, and an invocation is fenced if activation changes between dispatch and execution. A failed candidate never replaces the last healthy application. In-flight calls retain a lease on their current activation while new calls move to the new one.

The trigger catalog is a projection, not an authority. Runtime trigger registrations and execution checkpoints are Orleans Grain state; the production composition registers a named Orleans Stream provider for event delivery and Orleans Reminders for plan wake-ups. The built-in local dispatcher is retained for tests and single-process hosts; a production host wires its fan-out projection to that stream provider through IRuntimeTriggerDispatcher. RuntimeTriggerChangeReconciler and the interval scheduler use owner leases/fencing supplied by Orleans. SQL, iDB, broker or another Orleans Provider can be selected by the host.

.db protocol

Committed .db documents describe intent and contain secret references, never credentials:

{
  "protocol": "itool.db",
  "revision": 1,
  "name": "orders",
  "engine": "sql",
  "secretRef": "secret://databases/orders",
  "schema": {}
}

The protocol recognizes sql, mongo, elasticsearch and the managed idb provider. The host registers concrete IDatabaseRuntimeProvider implementations and an ISecretResolver. This keeps provider SDKs and deployment-specific credentials outside the core wire contract; iDB durable pointers remain in the selected Orleans/Provider state and immutable snapshots/journals use the S3 object boundary.

Host integration

builder.Services.AddGitContentManagement();
builder.Services.AddStaticClientApplicationRuntime();
builder.Services.AddMyScriptRuntime(runtime => runtime
    .AddJavaScriptRuntime(options => options.MaximumConcurrency = 16)
    .AddScriptRuntimeKeyword<OrdersSystemObject>()
    .AddRuntimeTriggerApi(options => options.AuthorizeAsync = AuthorizeTriggerAsync)
    .AddControlPlane(options => options.AuthorizeAsync = AuthorizeRuntimeControlAsync));

var app = builder.Build();
app.UseMyScriptRuntime("/runtime");
app.MapMyScriptRuntimeTriggers("/runtime/triggers");
app.MapMyScriptRuntimeControlPlane("/runtime/control");
app.MapGitSmartHttp(new GitSmartHttpOptions
{
    EnablePush = true,
    AuthorizeAsync = AuthorizeGitAsync
});
app.MapGitContentManagement(new GitContentOptions
{
    AuthorizeAsync = AuthorizeGitContentAsync
});
app.MapClientApplications("/apps");

Control, trigger, content management and smart HTTP APIs deny access until the host supplies authorization. Smart HTTP has an independent per-repository policy and the callback can inspect the request to distinguish upload-pack from receive-pack. Product authentication remains a host concern.

For production runtime coordination, register the chosen Orleans state/stream/reminder Provider and replace local resources with Orleans Grain resources. Redis is the default composition; the provider remains replaceable:

Production hosts should make the authority split explicit at composition time:

builder.Services.AddIToolDevelopmentFoundation(runtime => runtime
    .UseGitRuntimeColdStart(new GitRuntimeColdStartOptions
    {
        RepositoryPath = configuration["Runtime:ColdStartRepository"]!,
        Reference = configuration["Runtime:ColdStartRef"] ?? "refs/heads/main"
    })
    .UseS3ObjectStorage(new S3ObjectStoreOptions
    {
        BucketName = configuration["Storage:Bucket"]!,
        ServiceUrl = new Uri(configuration["Storage:ServiceUrl"]!),
        AccessKey = configuration["Storage:AccessKey"],
        SecretKey = configuration["Storage:SecretKey"],
        ForcePathStyle = true
    })
    .UseRedisEphemeralServices(new RedisEphemeralRuntimeOptions
    {
        ConnectionString = configuration.GetConnectionString("Redis")!
    }));

builder.Host.UseOrleans(silo => silo.AddMyScriptRuntimeRedisPersistence(
    new RuntimeRedisPersistenceOptions
    {
        ConnectionString = configuration.GetConnectionString("Redis")!,
        // DataFabric reserves database 0; Runtime Orleans state is isolated in database 1.
        // Presence/cache defaults to database 2 and must never share either authority domain.
        Database = 1
    }));

UseRedisEphemeralServices registers only presence/cache and a host-owned multiplexer. Its default logical database is 2, separate from DataFabric database 0 and Runtime Orleans state database 1; a cache flush therefore cannot erase either authority. The Orleans extension above uses the same logical Redis service for Grain State, PubSubStore metadata, the named Runtime Stream provider and Reminders; Redis key prefixes/databases are configurable. A host that only uses DataFabric can call UseSharedRedisConnection directly. Runtime consumes the host-owned IConnectionMultiplexer; it never calls Connect or disposes that shared connection. The host controls endpoint and lifetime, while Runtime's database/prefix options keep its state isolated from DataFabric and ephemeral cache namespaces.

UseSqlServerAuthorities remains an explicit compatibility composition for hosts choosing SQL adapters for optional authorities; it does not replace the Orleans Runtime state provider. In a production Orleans composition, UseOrleansResources also owns neutral identity registration/credential/session state, application bindings, ClientApp publication pointers, asset upload/lifecycle metadata, neutral client properties/groups, and realtime stream partitions, subscriptions, notifications and presence. Those records are Grain State under RuntimeStorageNames.State; the asset bytes themselves remain in S3 and SQL authority adapters are compatibility-only and are not required by the Runtime. UseS3ObjectStorage registers one shared multipart object store for large immutable content and the iDB file replica. iDB addresses use the logical repository name plus Git commit SHA and relative path; a host's absolute checkout path is never part of the address, so replicas remain reusable after remounting or migration. AddIToolDevelopmentFoundation remains usable with in-memory stores for tests, while a production host should configure AddMyScriptRuntimeRedisPersistence (or another Orleans Provider) and UseGitRuntimeColdStart.

builder.Services.AddMyScriptRuntime(runtime =>
    runtime.UseOrleansResources("tenant-or-environment"));

Orleans persists application bindings, ClientApp publication pointers, neutral identity records, asset upload/lifecycle metadata, client properties/groups, realtime subscriptions/notifications/ presence, distributed script state, execution journals and idempotency records through the configured Provider. It does not persist Git files or invent a second file version. After a Redis/Provider loss, the host rehydrates declarative bindings from the fixed Git/iDB cold-start manifest; after a Silo interruption, Running work is fenced as execution_interrupted unless ReplayPolicy.RestartFromBeginning was explicitly chosen with an idempotency key. That policy is a caller declaration that restarting the operation is safe, not an exactly-once side-effect guarantee; admission deduplication alone never enables replay. Runner writes carry both a persisted lifecycle incarnation and generation, so a retired runner cannot write to a recovered or reclaimed-and-recreated execution. Unknown state writes are confirmed by a fresh provider read; an unconfirmable admission returns its execution id for reconciliation. A Git cold start is not a reconstruction of dynamic execution/audit history or user subscription cursors; those need Provider replication/backup or an explicit host rebuild policy.

ClientApp publication

IClientApplicationCompiler is the build boundary for Vite, esbuild, Razor or another compiler. The built-in static compiler copies a configured subtree from one committed Git snapshot. The publisher writes a complete content-addressed asset set through IObjectStore, then atomically switches the Orleans publication pointer. Compile/upload failure leaves the last healthy public URL online, and public responses do not expose storage keys.

.NET and JavaScript SDKs

The NuGet package contains HTTP and WebSocket runtime/control clients plus HTTP trigger and Git content clients:

IRuntimeClient runtime = new WebSocketRuntimeClient(
    new Uri("wss://runtime.example.com/runtime/connect"));
var triggers = new HttpRuntimeTriggerClient(
    httpClient,
    new Uri("https://runtime.example.com/runtime/triggers/"));
var content = new HttpGitContentClient(gitContentHttpClient);

The packaged ES module exports the same integration surface:

import {
  RuntimeClient,
  RuntimeControlClient,
  RuntimeTriggerClient,
  GitContentClient
} from "@itool/runtime-sdk";

Custom transports can implement the protocol interfaces without changing the runtime execution model. HTTP and WebSocket are built in; SignalR, webhooks and external brokers are intentionally extensions.

Verification

The persistent test suite covers every mapped HTTP/WebSocket/Git/ClientApp endpoint, native Git push/clone/fetch interoperability, atomic content mutations, exact conflict/error behavior, two language SDK execution, real JavaScript/system-object calls, lib nesting, event/flow idempotency, plan scheduling, activation rollover, Orleans restart/elasticity and S3-compatible operations.

The two real loopback 200-concurrency gates require every individual content or JavaScript runtime HTTP call to complete in less than 500 ms. The S3 gate requires an isolated loopback endpoint:

ITOOL_S3_TEST_ENDPOINT=http://127.0.0.1:19090 \
ITOOL_S3_TEST_ACCESS_KEY=itool-test \
ITOOL_S3_TEST_SECRET_KEY=itool-test-secret \
dotnet test --filter 'Category=storage-integration' \
  tests/iTool.MyScriptRuntimes.Tests/iTool.MyScriptRuntimes.Tests.csproj

The fixture creates unique repositories, branches, buckets and objects and removes them after each scenario.

Literal dynamic imports

The JavaScript adapter executes literal import('./module.js') expressions from the immutable catalog, including JSON modules with literal { with: { type: 'json' } } options. Dependency discovery captures these targets before activation. Execution does not read mutable Git refs or the filesystem. Repeated imports share the module namespace within one invocation; a later invocation receives a fresh engine. Computed specifiers such as import(path) remain rejected by this adapter, even when a catalog declares finite targets. String compilation also remains disabled. The current Jint public loader API cannot distinguish every computed dynamic cache lookup from a static lookup, so finite catalog declarations alone are not execution authorization.