can you self-host jev?

Self-Hosting Means Controlling the Inference Runtime

A self-hosted model runs on infrastructure your organization controls, with weights and serving software available under suitable terms. Installing an SDK locally does not meet that definition when the SDK sends the request to a remote API. A proxy, gateway or private network connection can improve control without becoming self-hosted inference.

Use precise deployment language in architecture and procurement documents: direct hosted API, managed provider, customer-controlled gateway, private connectivity, or customer-run model. Each creates different security and operational responsibilities.

Distinguish the Available Deployment Shapes

Check the Model Behind a Local Serving Route

Ollama’s Nimble and Tev1 listings show local /v1/systemone examples and require Ollama 0.35 or later. SGLang documents native /v1/decisions and /v1/systemone routes for supported chat models; its docs say to use a nightly build until a release includes them. Ollaya is a separate runtime. None is an official Jev deployment, and a listed context window does not validate answer quality across that length.

Pin the exact model tag, downloaded artifact, runtime version and hardware. Verify that required weights are present and inference works without an outbound dependency. Use the open-model guide to test protocol conformance and the benchmark protocol to test behavior.

Translate “Self-Hosted” into the Real Requirement

Teams often mean data residency, no public internet, fixed cost, offline operation, custom throughput, model inspection or vendor independence. A managed provider may satisfy some requirements through regional processing or private networking; only downloadable weights satisfy offline independent inference.

Write each requirement as a testable control. “Private” is too vague; “request bodies never leave region X and are deleted within Y days under contract” can be verified. See the privacy and retention checklist.

Choose the Deployment Shape from the Constraint

This prevents a costly category mistake. A private gateway can centralize credentials and logs but still sends state to remote inference. A managed regional endpoint can meet some residency needs but is still not customer-run. Only an available model artifact and runtime under acceptable licenses enable independent offline inference.

A Local Alternative Must Be Revalidated as a New Evaluator

A model that implements Choice, Score and Noul-shaped responses is not automatically Jev. It can differ in weights, training objective, probability meaning, language coverage, adversarial behavior and latency. Preserve the interface adapter, but treat the implementation identifier as part of the evaluator version.

Run the same blind labels, calibration metrics, stress cases and operating thresholds used for hosted Jev. Do not transfer a 0.8 threshold across models. The evaluator comparison protocol and open-source status explain the evidence boundary.

Local Control Adds Operating Responsibility

Model Local Capacity Before Choosing on Token Price

A self-hosted service needs a peak-arrival model, not only an average requests-per-second estimate. Measure input-length and question-count distributions, service-time percentiles, batching behavior, accelerator memory, queue limits and timeout budget. Include redundancy and deployment headroom. An idle accelerator can cost more than hosted inference; an overloaded one can turn a fast model into a slow application.

Benchmark the complete request path with representative concurrency and record rejected, timed-out and retried calls. Compare hosted and local options at the same quality and automation coverage, including engineering, monitoring and incident-response effort. The pricing guide covers complete workflow cost and the retry guide defines failure-budget accounting.

Self-Hosting Moves—Not Removes—the Security Boundary

Local inference can reduce exposure to an external processor, but the operator now owns every control in this table. The semantic model also remains vulnerable to bad or adversarial state; deployment location does not convert its output into authorization. The privacy review maps data flow, and known limitations covers adversarial inputs.

Verify a Self-Hosting Claim Before Publishing It

• Locate the official weight files and serving instructions.
• Read the model and code licenses for the intended use.
• Confirm architecture, tokenizer, context and supported primitives.
• Run inference without an outbound dependency.
• Benchmark calibration and limitations independently.
• Name community implementations separately from TypeSafe Jev.