On-device inference
A hash-verified ONNX model runs inference on-device with zero egress by default, the hosted lane switches on only when you name its host in the privacy allowlist.
What it is
local-inference implements one InferenceBackend port two ways: OnnxEmbeddingBackend runs a MiniLM-class ONNX model on-device via transformers.js, every fetched model file SHA-256-verified before use; RentedInferenceBackend calls a hosted provider (OpenRouter, Azure OpenAI, Bedrock) only once its host is allowlisted in the privacy gate, metering every call. A deterministic stub backs CI, the live paths are proven, never exercised in tests.
What ships in the module
Hash-verified model load, fail closed on mismatch
OnnxEmbeddingBackend's guarded fetch checks every pinned file's SHA-256 digest against the integrity map in OnnxBackendConfig with the constant-time safeEqualFixed before the bytes reach transformers.js; resolveConfig refuses to construct the backend at all with zero hash-pins.
One egress chokepoint, two purpose-bound sink kinds
Both backends route through the shared EgressGuard's assertAllowedFor (the ONNX backend allowlists only modelHost under the model-fetch sink kind, the rented backend only its endpoint under rented-backend) so a host sanctioned for one purpose can never receive traffic meant for the other.
Rented inference is off by default
RentedInferenceBackend's constructor calls guard.assertAllowedFor(endpoint, "rented-backend") before it will even build, and re-asserts the same gate on every embed/complete call, a zero-egress privacy policy makes construction itself throw, with no silent hosted fallback.
Every rented call meters exactly once
#emitMeter builds one UsageMetering record (integer quantity, a fresh idempotencyKey per call) and hands it to the buyer-wired MeterSink before the result returns; if the sink throws, the call fails, because a paid call that can't be recorded must not silently succeed.
Four wire dialects, one RentedTransport port
createLiveRentedTransport speaks a first-party /embed + /complete wire; createOpenRouterRentedTransport, createAzureOpenAIRentedTransport, and createBedrockRentedTransport map the same port onto OpenRouter, Azure OpenAI, and Bedrock, every response re-validated against the strict RentedEmbedResponse/RentedCompleteResponse shape regardless of which one answered.
Deterministic stub, byte-identical in CI
StubInferenceBackend seeds a mulberry32 PRNG from the SHA-256 of the input text, so identical text always embeds to the byte-identical vector and CI never touches a model or a socket, the same InferenceBackend port the live backends implement, so swapping to production changes zero call sites.
constructor(config: RentedBackendConfig) {
const dim = config.dim ?? EMBEDDING_DIM;
if (!Number.isInteger(dim) || dim <= 0) {
throw new ValidationError(
"rented backend dim must be a positive integer",
{ received: dim },
);
}
assertNonEmpty(config.tenantId, "tenantId");
assertNonEmpty(config.feature, "feature");
assertNonEmpty(config.model, "model");
// OFF BY DEFAULT. `assertAllowedFor` throws unless the endpoint is HTTPS, the host
// is on the privacy allowlist (a zero-egress default policy fails closed), AND the sanctioned
// sink KIND is `rented-backend` specifically — a host allowlisted only for the model fetch
// can never double as a hosted-inference egress.
const url = config.guard.assertAllowedFor(
config.endpoint,
"rented-backend",
);
this.dim = dim;
this.model = config.model;
this.#endpoint = url;
this.#guard = config.guard;
this.#transport = config.transport;
this.#meter = config.meter;
this.#tenantId = config.tenantId;
this.#feature = config.feature;
}- assertAllowedFor throws right here, at construction, unless the endpoint is HTTPS and allowlisted under the rented-backend sink kind specifically, a host sanctioned only for the model fetch can't double as a hosted-inference egress.
- The guard check runs before any field is assigned, throw here and `this.#endpoint`, `this.#guard`, and the rest of the private state are never set, so there's no partially-built instance and no silent hosted fallback to fall into.
- The dim/EMBEDDING_DIM guard runs before assertAllowedFor is ever called, so a misconfigured embedding width fails closed before the privacy gate is even consulted.