Documentation / Overview
DownloadsGitHub ↗

Native inference · 4D integration

Fast by design.
Made for Apple.

A local inference server for Apple Silicon, with a 4D component to start it, connect to it, and shut it down with your application.

Make your first request Find a native build
The local stackOn your machine
Your application4D component or your own HTTP client
HTTP / 127.0.0.1:8080
UzuNative inference server for Apple Silicon
loads
A model you chooseCatalog ID, Hugging Face repo or local directory
Local inference. A familiar HTTP interface.
Open source · MITLoopback-first examplesNo hosted inference service required

Two ways in

One engine. Your workflow.

01 / Native server

Start with the command line

Run the cli executable and send HTTP requests from your own tools or application.

Download & verify → 02 / 4D component

Make it part of your app

Let the component manage the server process, model resolution, and application lifecycle.

Integration guide →

Before you begin

Built for Apple Silicon.

This native release target is arm64 macOS 26.4 or later, using Metal 4 on an Apple GPU. It is not an Intel, Windows, Linux, or CPU-only server distribution.

Runtime requirements
You needWhat to know
An Apple Silicon MacmacOS 26.4+ and a Metal 4-capable Apple GPU.
A compatible modelModel downloads are separate from the server. Check the model's license and memory requirements.
4D 20+ (optional)Required only for the component integration, not for direct HTTP use.
Native means self-contained shaders.

Metal shaders are embedded; no external .metallib is needed at runtime. End users need the system Apple frameworks, not Homebrew or Rust.

A short path to inference

From terminal to conversation.

After obtaining and verifying a native build, open its extracted directory. The first launch may download model assets; leave the server running in this terminal.

01 — Start the serverShell
./cli server \
  --model 'alibaba:qwen3.5:0.8b:mirai:mirai-m:4' \
  --host 127.0.0.1 \
  --port 8080

In another terminal, poll this endpoint until it succeeds, then read the loaded model identifier. Process launch alone does not mean the HTTP server is ready; model downloads and loading may take time.

02 — Discover the modelShell
curl http://127.0.0.1:8080/v1/models

Replace MODEL_ID_FROM_RESPONSE with the returned model ID, then send a chat request:

03 — Send a messageShell
curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "MODEL_ID_FROM_RESPONSE",
    "messages": [{"role": "user", "content": "Hello from my Mac."}],
    "max_tokens": 128,
    "stream": false
  }'

For streaming, set "stream": true and use curl -N. Tool use depends on the loaded model's support. Keep the listener on 127.0.0.1 unless you have deliberately secured network access.

Know the boundary

A focused server API.

Use GET /v1/models and POST /v1/chat/completions for model discovery and chat. Decision-model specializations use /v1/decisions or /v1/systemone; they are not general chat endpoints.

Do not assume embeddings, router, image generation, or files endpoints, or GGUF model compatibility. Other upstream APIs and executables are not interchangeable with this cli server.

Explore the supported server options →