WASM Plugins¶

WebAssembly (WASM) plugins enable custom middleware logic in SMG's request pipeline without recompiling or restarting the gateway.

Overview¶

How It Works¶

WASM plugins execute in SMG's middleware layer, intercepting requests before they reach workers and responses before they return to clients.

The middleware runs on the inference routes: /v1/chat/completions, /v1/completions, /generate, /v1/responses and /v1/conversations (with their sub-paths), /v1/messages and /v1/messages/count_tokens, /v1/embeddings, /v1/rerank and /rerank, /v1/classify, /v1/interactions, /v1/tokenize, /v1/detokenize, and the Realtime REST endpoints. It does not run on /v1/audio/transcriptions, the Realtime WebSocket and WebRTC routes, or the public, admin, and worker-management routes.

Plugin Chain Execution¶

SMG runs every plugin attached at a point, in no defined order: don't rely on deployment order. Each plugin receives the request/response and returns an action:

1. Continue: Pass through to the next plugin (or worker)
2. Reject(status): Stop processing and return an error response immediately
3. Modify(changes): Apply transformations and continue

If any plugin returns Reject, subsequent plugins are skipped and the error response is returned to the client: on a request, the status with an empty body; on a response, the status with the response body so far. A plugin that fails or runs past its time limit is skipped, and processing continues with the next one.

Attach Points¶

Plugins register at specific points in the request lifecycle:

Example Plugins¶

SMG includes ready-to-use example plugins demonstrating common middleware patterns:

Find complete source code and build instructions in examples/wasm/.

Plugin Development¶

Interface¶

Plugins implement the SMG middleware interface using the WebAssembly Component Model:

// OnRequest: Called before forwarding to worker
fn on_request(req: Request) -> Action {
    // Validate, modify, or reject the request
    Action::Continue
}

// OnResponse: Called after receiving worker response
fn on_response(resp: Response) -> Action {
    // Transform or log the response
    Action::Continue
}

Actions¶

Request Context¶

Plugins receive rich context for decision-making:

Configuration¶

Enabling WASM Support¶

smg --enable-wasm --worker-urls http://worker:8000

Runtime Settings¶

These values are fixed in v1.11.0; no flag changes them. Plugins run on a pool of runtime threads, one per CPU core up to 4.

With an OnRequest plugin attached, SMG reads the whole request body before running it, and answers 400 when the body exceeds max_body_size. With an OnResponse plugin attached, it reads each non-streaming response body the same way, and answers 500 when the body is too large.

Module Management¶

Plugins are managed via the Admin API at runtime. These routes need --enable-wasm (without it they answer 500) and are protected like the other admin routes.

Deploy Plugins¶

curl -X POST http://localhost:30000/wasm \
  -H "Content-Type: application/json" \
  -d '{
    "modules": [
      {
        "name": "auth-middleware",
        "file_path": "/plugins/auth.component.wasm",
        "module_type": "Middleware",
        "attach_points": [{"Middleware": "OnRequest"}]
      },
      {
        "name": "logging-middleware",
        "file_path": "/plugins/logging.component.wasm",
        "module_type": "Middleware",
        "attach_points": [
          {"Middleware": "OnRequest"},
          {"Middleware": "OnResponse"}
        ]
      }
    ]
  }'

SMG loads each module before it answers. The response lists every module with an add_result: {"Success": "<module uuid>"}, or {"Error": "<message>"} for a module that failed, in which case the status is 400.

List Plugins¶

curl http://localhost:30000/wasm

Remove Plugin¶

curl -X DELETE http://localhost:30000/wasm/{uuid}

Security Model¶

WASM plugins execute in a sandboxed environment with multiple protection layers:

Blocked directories: /etc/, /proc/, /sys/, /dev/, /boot/, /root/, /var/log/, /var/run/

Performance¶

The first run of a component on each runtime thread compiles it, and later runs on that thread reuse the compiled component. Every invocation still instantiates the component fresh. GET /wasm reports execution counts (total, successful, failed) and the total, average, and maximum execution time, so you can measure your plugins' overhead.

What's Next?¶