Mohdel 1.0: a self-hosted LLM gateway and SDK for Node

I released mohdel 1.0.0 this week. It is a self-hosted LLM gateway and SDK for Node, MIT licensed, and it runs the inference layer of a document analysis product with hundreds of thousands of users. The closest thing to it is LiteLLM, which lives in Python. If your stack is JavaScript, your options have been a Python sidecar to deploy and monitor, a SaaS router sitting in your request path, or N provider SDKs with N shapes and no accounting. Mohdel is the Node-native version of that layer. Here is what it does, section by section. One interface, 13 providers Anthropic, OpenAI, Gemini, Mistral, Groq, xAI, Cerebras, Fireworks, DeepSeek, Qwen Cloud, Xiaomi, OpenRouter, Novita, plus local inference. One call: import mohdel from 'mohdel' const mo = await mohdel() const result = await mo.use('anthropic/claude-sonnet-4-6').answer('Hello') console.log(result.output, result.cost) Switching provider is one string. The result shape does not move: { status: 'completed', // 'completed' | 'tool_use' | 'incomplete' output: 'Generated text', inputTokens: 42, outputTokens: 128, // visible output, excludes thinking thinkingTokens: 0, cost: 0.002046, // USD, from your catalog timestamps: { start, first, end } } Tool calls, streaming, vision and speech to text go through the same call. Adapter differences stay inside mohdel: max_tokens versus max_output_tokens, top-level system versus instructions, five different usage shapes. Local models are the local/ provider, pointed at any OpenAI-compatible server: Ollama, vLLM, llama.cpp server, LM Studio. There is no default endpoint and no per-call override, so a call to a local model can never fall through to a cloud provider by accident. Two integration paths, same API In-process. import mohdel from 'mohdel' and call it. No subprocess, nothing to run. Right for scripts, CLI tools, tests and single-process services, which is most projects. Cross-process. A Rust supervisor, thin-gate, owns a pool of session subprocesses. Your app talks to it over a unix socket; the provider SDKs run somewhere else entirely. An adapter that hangs, leaks or panics takes down one session, and the supervisor respawns it while the caller gets a recoverable error. The wire between them is documented NDJSON, so a caller does not have to be JavaScript. There are reference clients in Lua, Gleam, Rust and OCaml in the repo, all validated against the same conformance fixtures. Switching between the two paths is configuration, not code. Real per-call cost result.cost is a USD number, computed from a catalog that belongs to you: { "anthropic/claude-sonnet-4-6": { "model": "claude-sonnet-4-6", "creator": "anthropic", "provider": "anthropic", "inputPrice": 3, "outputPrice": 15, "cacheWritePrice": 3.75, "cacheReadPrice": 0.3, "contextTokenLimit": 1000000, "outputTokenLimit": 128000, "tags": ["chat", "tool-loop", "vision"] } } Yours, so it carries the rate you negotiated and the tags your routing code selects on, rather than a central price map you inherit and hope is current. The catch is that no provider publishes prices in machine-readable form: no endpoint, just a marketing page with a table and a footnote about cached reads. So mohdel writes a brief and hands the transcription to the coding agent already running in your terminal: mo model instructions anthropic > mohdel-brief.md claude "read mohdel-brief.md, then add claude-haiku-5 to my mohdel catalog" mo model check --entry mohdel-candidate.json # schema, types, required fields mo model apply mohdel-candidate.json # prints the diff, waits for you The agent never writes the catalog. It writes a candidate file, check validates it, and apply prints a full diff before anything lands. Entries record the URL the numbers came from and the date they were read, and the brief tells the agent to leave a field out rather than guess it. OpenRouter needs none of this: it publishes per-token prices in its own model list, so setup offers to add every free model in one keystroke. Observability without instrumentation Set OTEL_EXPORTER_OTLP_ENDPOINT and you get spans, trace-linked logs and OTLP metrics. The call span follows the GenAI semantic conventions (gen_ai.request.model, gen_ai.usage.input_tokens) and adds mohdel's own mohdel.cost and mohdel.time_to_first_token_ms. Pass a traceparent on the call and it parents under your span. Running the gate, you also get sessions alive and respawned, calls by provider and status, a call-duration histogram, and cooldown, quota and policy rejections. Per-model and per-provider rate limits live in the catalog, and a provider that keeps failing goes into cooldown so calls fast-fail instead of hitting the wire. Types, and a wire that does not move The source is JavaScript with JSDoc. Declarations are generated from it and ship with the package, so a TypeScript consumer typechecks against the real contract: CallEnvelope, Event, AnswerResult and a single MohdelError. No @types package, nothing to keep in sync. The NDJSON protocol between client, gate and session has been frozen since 0.90 and is enforced by fixtures that both the JS and the Rust side round-trip. 1.0.0 puts that under SemVer along with the library API. Your keys, and not much to steal Self-hosted means the keys stay in your infrastructure and calls go straight to the provider. Nothing routes through a third party and nothing marks up your tokens. The gateway itself is built to hold nothing worth taking: no network listener (the supervisor binds unix sockets at mode 0600), no credential store (the key rides on each call and goes to the SDK client), and nothing that executes, meaning no eval, no child_process, no automatic tool loop. The session subprocess starts from an empty environment and gets back only the variables the runtime reads, so a key for a provider it will never call is not in its address space. Tool execution stays in your process, where you can see it. What it is not Not an orchestrator: no chains, no agents, no memory, no retrieval. Wrap it with whatever you like. Not a retry or fallback engine; errors are classified with retryable, severity and type, and the caller decides. Not a response cache. Not a token budgeter. It does not expose an OpenAI-compatible endpoint, so a cross-process caller uses the JS client or implements the documented NDJSON protocol. The library and CLI run anywhere Node 22 does; the prebuilt supervisor binary is Linux x64 only so far. Try it npm install -g mohdel mo # pick a provider, paste a key mo ask openai/gpt-5.6-luna "why is the sky blue" npm install mohdel for the library. MIT licensed, and the issues are open: https://github.com/clbrge/mohdel