Authentication¶

SMG supports multiple authentication methods for securing access to inference APIs and the control plane, including JWT/OIDC integration, API keys, and role-based access control.

Overview¶

Authentication Methods¶

When to Use Each Method¶

• Control plane JWT/OIDC: Use for enterprise deployments with existing identity providers (Keycloak, Auth0, Azure AD, Okta). Provides centralized user management and SSO for control plane operations.
• Control plane API keys: Use for service-to-service automation against admin endpoints (CI/CD pipelines, tooling). Simpler to set up but requires manual key management.
• Data plane API key: Use when you want a single shared secret that clients present on data plane routes (chat, completions, responses, and so on), and that workers started with the same key accept.
• Data plane tenant keys: Use when each team or application needs its own data plane key, so that tenant rate limits and priority scheduling can tell callers apart. Each --tenant-api-key tenant_id:key resolves to the tenant auth:<tenant_id>. Tenant keys never unlock control plane routes, and the flag belongs to the Rust smg binary (the Python launcher does not accept it).

JWTs and control plane API keys are checked only on control plane routes. Data plane routes accept only --api-key and --tenant-api-key credentials, and are open when neither is set.

JWT/OIDC Authentication¶

JWT (JSON Web Token) authentication allows integration with OIDC-compliant identity providers for enterprise single sign-on on the control plane.

Configuration Options¶

The environment variables and --jwt-role-claim belong to the Rust smg binary. The Python launcher (smg launch from pip, and the container image) takes only the flags and always uses the default roles claim (with the fallbacks in Supported Claims). See Python Launcher Differences.

Basic Setup¶

Enable JWT authentication by providing the issuer and audience:

smg \
  --worker-urls http://worker:8000 \
  --jwt-issuer "https://auth.example.com/realms/myrealm" \
  --jwt-audience "smg-gateway"

JWKS Discovery¶

By default, SMG discovers the JWKS (JSON Web Key Set) endpoint automatically via OIDC discovery (/.well-known/openid-configuration). You can override this with an explicit JWKS URI:

smg \
  --worker-urls http://worker:8000 \
  --jwt-issuer "https://auth.example.com" \
  --jwt-audience "smg-gateway" \
  --jwt-jwks-uri "https://auth.example.com/.well-known/jwks.json"

Without --jwt-jwks-uri, SMG fetches the discovery document at startup. It fetches the key set on first use, caches it for an hour, and fetches it again when a token names a key it doesn't have. The discovery URL and the JWKS URI must use HTTPS (plain HTTP is allowed only for localhost, 127.0.0.1, and ::1), must not point at a private, loopback, link-local, or other internal IP address literal, and must not use a host name ending in .internal or .local. SMG does not follow redirects on these requests. Tokens must carry a kid header that names a key in the set.

Role Mapping¶

Map identity provider roles to SMG gateway roles:

smg \
  --worker-urls http://worker:8000 \
  --jwt-issuer "https://auth.example.com" \
  --jwt-audience "smg-gateway" \
  --jwt-role-mapping "Gateway.Admin=admin" \
  --jwt-role-mapping "Gateway.User=user"

Role Mapping Format: idp_role=gateway_role

Supported Claims¶

SMG reads role values from the configured --jwt-role-claim (default: roles). Only when the token has no such claim does it fall back to the role, roles, groups, and group claims, collecting the values of all of them in that order. Each claim may be a string or an array of strings.

• Without --jwt-role-mapping, the first value equal to admin or user (ignoring case) sets the role.
• With --jwt-role-mapping, the first value that has a mapping sets the role; unmapped values are ignored.

If no role is found, the user defaults to the user role.

Supported Algorithms¶

SMG supports the following JWT signing algorithms:

• RSA: RS256, RS384, RS512
• ECDSA: ES256, ES384

Identity Provider Setup¶

Keycloak¶

smg \
  --worker-urls http://worker:8000 \
  --jwt-issuer "https://keycloak.example.com/realms/myrealm" \
  --jwt-audience "smg-gateway" \
  --jwt-role-mapping "admin=admin" \
  --jwt-role-mapping "user=user"

Auth0¶

exports.onExecutePostLogin = async (event, api) => {
  const namespace = 'https://smg.example.com';
  if (event.authorization) {
api.accessToken.setCustomClaim(
  `${namespace}/roles`,
  event.authorization.roles
);
  }
};

smg \
  --worker-urls http://worker:8000 \
  --jwt-issuer "https://your-tenant.auth0.com/" \
  --jwt-audience "https://smg.example.com/api" \
  --jwt-role-claim "https://smg.example.com/roles" \
  --jwt-role-mapping "smg-admin=admin" \
  --jwt-role-mapping "smg-user=user"

Azure AD / Entra ID¶

smg \
  --worker-urls http://worker:8000 \
  --jwt-issuer "https://login.microsoftonline.com/{tenant-id}/v2.0" \
  --jwt-audience "api://smg-gateway" \
  --jwt-role-mapping "Gateway.Admin=admin" \
  --jwt-role-mapping "Gateway.User=user"

Okta¶

smg \
  --worker-urls http://worker:8000 \
  --jwt-issuer "https://your-org.okta.com/oauth2/default" \
  --jwt-audience "api://smg" \
  --jwt-role-mapping "smg_admins=admin" \
  --jwt-role-mapping "smg_users=user"

API Key Authentication¶

API keys provide a simpler authentication method for service accounts and programmatic access.

Control Plane API Keys¶

Configure API keys for control plane access:

smg \
  --worker-urls http://worker:8000 \
  --control-plane-api-keys "key1:Service Account:admin:sk-your-secret-key-here"

Format: id:name:role:key

Multiple API Keys¶

smg \
  --worker-urls http://worker:8000 \
  --control-plane-api-keys "admin1:Admin Service:admin:sk-admin-key-12345" \
  --control-plane-api-keys "user1:Read Only Service:user:sk-readonly-key-67890"

A user key authenticates, but every control plane route answers it with 403 (see Role-Based Access Control).

Environment Variable Configuration¶

For security, pass API keys via environment variable:

export CONTROL_PLANE_API_KEYS="admin1:Admin Service:admin:sk-admin-key-12345"
smg --worker-urls http://worker:8000

The variable holds one key, and only the Rust smg binary reads it; the Python launcher (smg launch from pip, and the container image) takes keys only from --control-plane-api-keys.

Using API Keys¶

Clients authenticate by including the API key in the Authorization header:

curl -H "Authorization: Bearer sk-admin-key-12345" \
  https://smg.example.com/workers

Security Features¶

• Hashed Storage: Keys are SHA-256 hashed immediately; plaintext keys are never stored in memory
• Constant-Time Comparison: Key verification uses constant-time comparison to prevent timing attacks
• Role-Based Access: Each key is assigned a specific role limiting its permissions

Data Plane API Key (--api-key)¶

The --api-key option configures a single bearer token that does two things:

smg \
  --worker-urls http://worker:8000 \
  --api-key "shared-secret-key"

1. Gates incoming data plane requests. The gateway requires every client request to data plane routes (/v1/chat/completions, /v1/completions, /v1/responses, /v1/embeddings, /v1/rerank, /v1/messages, /v1/realtime/*, etc.) to present Authorization: Bearer <api-key> (or a --tenant-api-key key). Requests without a valid token receive 401 Unauthorized. Public routes such as /health and /v1/models stay open.
2. Becomes the API key of startup and discovered workers. Workers from --worker-urls (and from --prefill and --decode) and workers that Kubernetes service discovery registers get the --api-key value as their worker API key. A worker added through POST /workers uses only the api_key in its own spec; SMG logs a warning when one arrives without a key while --api-key is set.

What reaches a worker depends on the path:

Service discovery registers each pod as a bare host:port, so SMG reaches it over plain HTTP (or plaintext gRPC), and a worker API key crosses the network unencrypted.

This is useful when:

• Clients and workers share a common token
• Workers require authentication (e.g., deployed with API key protection)
• Using DP-aware scheduling that requires authenticated worker queries
• Workers are behind an authentication proxy

Control plane routes (/workers, /wasm, /v1/tokenizers, etc.) use their own middleware stack. When --control-plane-api-keys or --jwt-* are configured they take over as the admin auth backend (with role-based access control and audit logging); when neither is set, admin routes fall back to the same --api-key bearer check that guards the data plane (tenant keys are never accepted there, and with only --tenant-api-key set, admin routes reject every request with 401). If you run with only --api-key, the same shared secret therefore gates both the data plane and the control plane — which is rarely what you want in production. See Control Plane Auth for configuring JWT/OIDC or dedicated control-plane keys.

Role-Based Access Control¶

SMG implements role-based access control (RBAC) with two primary roles:

Admin Role¶

Full access to all control plane APIs:

• Worker management (/workers, /workers/{id})
• WASM module management (/wasm/*)
• Tokenizer configuration
• System administration

User Role¶

No control plane access. Every control plane route answers a user credential with 403 Admin role required for control plane access.

Roles don't apply to data plane routes (chat completions, completions, embeddings, and the rest): those accept only --api-key and --tenant-api-key credentials, never JWTs or control plane API keys. /v1/models is public.

Role Assignment¶

Roles are assigned through:

1. JWT Claims: Via --jwt-role-mapping configuration
2. API Key Configuration: Via the role component in --control-plane-api-keys

If no role can be determined, the user defaults to user role for safety.

Audit Logging¶

SMG provides audit logging for control plane operations to support security monitoring and compliance.

Configuration¶

With control plane authentication configured (--jwt-issuer with --jwt-audience, or --control-plane-api-keys), the Rust smg binary logs audit events by default. To disable:

smg \
  --worker-urls http://worker:8000 \
  --jwt-issuer "https://auth.example.com" \
  --jwt-audience "smg-gateway" \
  --disable-audit-logging

The Python launcher (smg launch from pip, and the container image) works the other way around: audit logging is off unless you pass --control-plane-audit-enabled, and it has no --disable-audit-logging flag. Neither launcher audits admin requests checked by the --api-key fallback.

Audit Log Format¶

Each audit event is an INFO log record with target smg::audit and message control_plane_audit. SMG records every request that reaches the control plane authentication check: successes, 403 role denials, and 401 authentication failures. The layout of the record follows your log format (--log-json for JSON).

Audit Event Fields¶

Viewing Audit Logs¶

# Filter for audit logs
RUST_LOG=smg::audit=info smg ...

# Or view in combined logs
kubectl logs -n inference -l app=smg | grep "control_plane_audit"

Production Configuration¶

A production setup combining JWT and API key authentication:

smg \
  --worker-urls http://worker1:8000 http://worker2:8000 \
  --host 0.0.0.0 \
  --port 443 \
  --tls-cert-path /etc/certs/server.crt \
  --tls-key-path /etc/certs/server.key \
  --jwt-issuer "https://auth.example.com/realms/production" \
  --jwt-audience "smg-gateway" \
  --jwt-role-mapping "Gateway.Admin=admin" \
  --jwt-role-mapping "Gateway.User=user" \
  --control-plane-api-keys "ci-cd:CI/CD Pipeline:admin:${CI_CD_API_KEY}" \
  --api-key "${DATA_PLANE_API_KEY}"

JWT and the control plane key protect the control plane routes. --api-key protects the data plane routes, which are open without it.

Kubernetes Deployment¶

Secret for API Keys¶

apiVersion: v1
kind: Secret
metadata:
  name: smg-auth
  namespace: inference
type: Opaque
stringData:
  CONTROL_PLANE_API_KEYS: "admin1:Admin:admin:sk-secret-key"

ConfigMap for JWT Configuration¶

apiVersion: v1
kind: ConfigMap
metadata:
  name: smg-config
  namespace: inference
data:
  JWT_ISSUER: "https://auth.example.com/realms/production"
  JWT_AUDIENCE: "smg-gateway"

Deployment¶

apiVersion: apps/v1
kind: Deployment
metadata:
  name: smg
  namespace: inference
spec:
  template:
    spec:
      containers:
        - name: smg
          image: lightseekorg/smg:latest
          envFrom:
            - configMapRef:
                name: smg-config
            - secretRef:
                name: smg-auth
          args:
            - --service-discovery
            - --selector
            - app=sglang-worker
            - --jwt-issuer
            - $(JWT_ISSUER)
            - --jwt-audience
            - $(JWT_AUDIENCE)
            - --control-plane-api-keys
            - $(CONTROL_PLANE_API_KEYS)
            - --jwt-role-mapping
            - "Gateway.Admin=admin"
            - --jwt-role-mapping
            - "Gateway.User=user"
            - --control-plane-audit-enabled

The container image runs the Python launcher, which does not read JWT_ISSUER, JWT_AUDIENCE, or CONTROL_PLANE_API_KEYS itself, so the args pass them as flags; Kubernetes expands each $(VAR) reference from the container's environment. --control-plane-audit-enabled turns on audit logging, which the Python launcher leaves off by default.

Troubleshooting¶

JWT Validation Failures¶

Symptom: Invalid JWT or Token validation failed errors

Solutions:

1. Verify issuer URL matches exactly (including trailing slash):

   Copy

   curl https://auth.example.com/.well-known/openid-configuration

2. Verify audience claim matches your configuration:

   Copy

   echo "YOUR_JWT" | cut -d. -f2 | base64 -d | jq .

3. Check clock synchronization (JWT validation uses time-based claims and allows 30 seconds of clock skew)

4. Verify JWKS endpoint is accessible from the SMG pod

API Key Not Working¶

Symptom: Invalid authentication token errors

Solutions:

1. Verify key format is correct: id:name:role:key
2. Check for special characters that may need escaping
3. Ensure the Authorization header format is correct: Bearer <key>

Role Mapping Issues¶

Symptom: Users getting wrong permissions

Solutions:

1. Check which claim contains roles in your JWT
2. Verify role mapping syntax: idp_role=gateway_role
3. Check if role claim name needs to be specified with --jwt-role-claim

Security Best Practices¶

What's Next?¶