Documentation Tiny weights. Great intelligence.
Get the componentGitHub

Local inference / 4D integration

Tiny weights. Great intelligence.

Bring OxiBonsai into your 4D application. Run a local model server on macOS or Windows, and connect through a familiar HTTP API.

Get the component Read the guide
The local stackOn your machine
Your application4D component or any HTTP client
HTTP / 127.0.0.1:8080
OxiBonsai serverMetal on Apple Silicon / CPUoxibonsai serve
loads
A model you chooseGGUF weights & compatible tokenizer
Local by defaultChat & completions
4D 20+GGUF modelsOpen source

Choose your ground

One integration. Mac and Windows.

Native executables for the platforms your 4D app calls home.

01 / macOS

Apple Silicon

Metal acceleration

Universal Mac executable
02 / macOS

Intel

CPU inference

Universal Mac executable
03 / Windows

x64 & ARM64

CPU inference

Native executables

Bring your own compatible GGUF model and tokenizer, with enough disk space and RAM. Model downloads are separate. No Linux component or CUDA-enabled executable is packaged here.

Make it yours

From a process
to part of your app.

Use the 4D component to manage the server, or work directly with the bundled command-line executable.

THE 4D ROUTE

A local worker.
A familiar workflow.

Install the .4dbase in your application's Components folder and restart 4D. The wrapper selects the platform executable and starts the server with your model and options.

Connect 4D AI Kit's OpenAI-compatible provider to http://127.0.0.1:8080/v1. Use the model ID returned by the server.

See the 4D example
THE HTTP ROUTE

Your tools.
The same local server.

Run oxibonsai serve from a terminal, discover the loaded model, and send chat or completion requests from your own client.

Keep the loopback binding by default. Deliberate network access needs deliberate protection; bearer authentication is available.

Start with HTTP

A first conversation

Start. Check. Connect.

After installing the component, open a terminal in its platform-specific Resources/bin/ directory. The examples below use macOS; on Windows, use oxibonsai.exe and adapt the shell syntax.

01

Load your model

Replace both paths with your GGUF and its compatible tokenizer. Keep this terminal running.

./oxibonsai serve \
  --model /path/to/model.gguf \
  --tokenizer /path/to/tokenizer.json \
  --host 127.0.0.1 \
  --port 8080
02

Wait for readiness

In another terminal, repeat the health request until it succeeds. Then discover the model ID. A started process is not a ready server.

curl --fail http://127.0.0.1:8080/health
curl --fail http://127.0.0.1:8080/v1/models
03

Send a message

Replace MODEL_ID_FROM_RESPONSE with the returned ID. For streaming, set "stream": true and use curl -N.

curl --fail http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "MODEL_ID_FROM_RESPONSE",
    "messages": [{"role": "user", "content": "Hello from my app."}],
    "max_tokens": 128,
    "stream": false
  }'
Using a bearer token? Add Authorization: Bearer YOUR_TOKEN to model and inference requests. /health and /metrics are exempt from authentication. Read the server options →

Know what is inside

A focused API.
Clear boundaries.

The integration runs upstream's oxibonsai serve, not the separate oxibonsai-serve binary. Capabilities belong to this entry point, not every upstream tool.

Explore the API & options
Models, chat & completions
/v1/models, /v1/chat/completions and /v1/completions are available.
Lexical vectors, not model embeddings
/v1/embeddings uses TF-IDF/byte-hash vectors, not learned semantic embeddings from your GGUF.
No image, file or moderation API
These endpoints are not exposed by this server.

Open at the roots

An engine you can trace.

The integration is MIT-licensed; the upstream Rust engine is Apache-2.0. Source builds pin OxiBonsai v0.2.3, its dependencies, build features and Rust 1.99.0. An explicit native refresh commits verified resources to Git; component-only Publish reuses them without rebuilding Rust. Native refresh never dispatches a release. No performance or inference-quality guarantees.

Build provenance