Documentation Overview
ReleasesGitHub ↗

Local inference / 4D integration

Local RWKV.
Connected to 4D.

Bring RWKV language models into your application. Run locally on Mac or Windows, with a 4D component that handles the server lifecycle.

Make your first request Get the component
The local stackOn your machine
Your application4D 20+ component
HTTP · 127.0.0.1:8080
/api/oai/v1
Ai00RWKV inference / WebGPU
loads
RWKV model weightsSafetensors · downloaded separately
Models · Chat · Text completionsGPU backend required; no CUDA or Python for inference.
Apple SiliconIntel MacWindows x644D 20+

Two ways in

Your model. Your workflow.

01 / 4D component

Make it part of your app

Start the server, resolve a model, and manage its process from 4D. Talk to it over a local HTTP API.

Integrate with 4D → 02 / Native server

Take the direct route

Build the pinned upstream source for your machine, configure a compatible model, and use your own HTTP client.

Build & run →

Before you begin

Local doesn't mean one platform.

Ai00 supports Apple Silicon, Intel macOS, and Windows x64. Inference needs a WebGPU-compatible GPU/backend and enough memory for your model. It is not a CPU-only RWKV server.

You need What to know
A supported host macOS 11+ (also subject to your 4D version), or Windows x64 with current graphics drivers and the Visual C++ x64 runtime.
A compatible GPU Metal on macOS; Vulkan/DX12 where available on Windows. No CUDA or Python required for inference.
A compatible model Safetensors (.st) RWKV weights. The default 4D model downloads separately on first launch. Check its license and memory requirements.
4D 20+ For component integration. Make event, workers, and tcp dependencies available to the host project.

Recurrent by design. RWKV keeps a fixed-size recurrent inference state instead of a growing transformer KV cache. That is not a promise of unlimited effective model context.

A short path to inference

From 4D to a first conversation.

Install the released component and its dependencies, then create the shared instance in On Startup. The first launch downloads the default model into ~/.Ai00 if it is missing.

01 / Start the component4D
var $Ai00 : cs.Ai00.Ai00
$Ai00:=cs.Ai00.Ai00.new()

The default listener is 127.0.0.1:8080. Model downloads and GPU loading take time: the wrapper’s onSuccess callback signals worker startup, not inference readiness. Retry model discovery until it succeeds before sending a request.

02 / Discover the modelShell
curl --fail http://127.0.0.1:8080/api/oai/v1/models
03 / Send a messageShell
curl --fail http://127.0.0.1:8080/api/oai/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "messages": [{"role": "user", "content": "Hello from 4D!"}],
    "max_tokens": 64,
    "stream": false
  }'

Keep it local. The generated 4D configuration does not require authentication. Keep the loopback binding unless you deliberately secure remote access with authentication and network controls.

A familiar interface, with clear boundaries

HTTP, without the round trip to a hosted service.

Use the OpenAI-compatible base URL /api/oai/v1, not just /v1. Model discovery, chat completions, and text completions are supported. Embeddings require a separate upstream embedding model; they are not RWKV embeddings. Don’t assume images, moderation, or file APIs.

Explore configuration, models, and the full integration guide →