Model Context Protocol (MCP)¶

The Model Context Protocol (MCP) gives models access to external tools such as search, databases, and internal APIs through a standard interface. SMG acts as the MCP client: it connects to MCP servers, offers their tools to the model as function tools, runs the calls the model makes, and feeds the results back until the model answers. The model never connects to an MCP server and never sees its credentials.

Overview¶

Where MCP Runs¶

SMG runs the MCP tool loop on these paths:

No other path runs MCP tools. Chat Completions has no MCP loop, SMG forwards /v1/responses requests to HTTP workers with their mcp tools left in place, and on gRPC workers the Messages API ignores mcp_toolset tools.

How It Works¶

1. Resolve servers. An mcp tool with only a server_label refers to a static server from the config file. An mcp tool with a server_url opens (or reuses) a pooled connection to a dynamic server. A built-in tool such as web_search_preview adds the static server configured for it.
2. Expose tools. SMG adds each server's tools to the model request as function tools, after allowed_tools filtering, aliases, and name de-duplication.
3. Generate. The model runs, and SMG parses its tool calls.
4. Execute. Calls to MCP tools pass the approval check and run on their server under the per-call timeout. On gRPC workers, calls to the client's own function tools go back to the client.
5. Loop. SMG appends the calls and their results to the conversation and calls the model again, until the model answers without tool calls or a limit stops the loop.
6. Respond. In the Responses API, mcp_list_tools and mcp_call items (or hosted-tool items such as web_search_call) come before the model's answer in output. In the Messages API, executed calls appear as mcp_tool_use and mcp_tool_result blocks.

Servers and Transports¶

Transports¶

Static Servers¶

Static servers are declared under servers in the configuration file.

• Registration. SMG registers each server in the background after the gateway starts, with up to 100 connection attempts. The log shows Successfully connected and registered MCP server: <name> once a server is ready. The gateway keeps serving while registration runs, and also when it fails; see required in the server fields.
• Referencing. A Responses request uses a static server by setting server_label to the server's name and leaving out server_url. Built-in tool routing only uses static servers.
• Tool discovery. SMG lists the server's tools when it connects and lists them again when the server sends a tools/list_changed (or resource or prompt list-changed) notification. There is no periodic refresh.
• Restarts. When a Streamable HTTP server restarts and reports the old session as expired (HTTP 404), the client opens a new session on the next call. SMG does not currently restart a stdio server whose process exits; restart SMG instead.
• Server logs. Log messages that a static server sends are written to SMG's log at the matching level.

Dynamic Servers¶

A dynamic server comes from the request: server_url on a Responses mcp tool, or url in a Messages API mcp_servers entry.

• Transport. Only Streamable HTTP. Stdio is not available for dynamic servers.
• Pooling. SMG keeps connections in a pool keyed by the URL plus a hash of the credentials (authorization and headers). A later request with the same URL and credentials reuses the connection and the tool list fetched when it was opened.
• Capacity. The pool holds up to 100 connections. When it is full, opening a new connection drops the oldest one and removes its tools. Reusing a connection does not refresh its place in the pool. Idle connections are not closed.
• Credential isolation. Two different credentials for the same URL create two pooled connections, and calls to that URL then fail closed. See Ambiguous Pooled Connections.
• Failures. SMG skips a server it cannot connect to and logs a warning. If no MCP server is left for the request, gRPC workers return 424 with code connect_mcp_server_failed, and the Messages API returns 502 with code mcp_connection_failed.

Tool Names¶

The model sees each tool under the server's tool name, or under the alias you configure for it. When two servers in the same request expose the same tool name, SMG renames both to mcp_<server_label>_<tool>. Letters are lowercased, any character other than a letter, digit, or _ becomes _, and a numeric suffix is added if the name is still taken. For example, two servers labeled server-a and server-b that both expose run_query become mcp_server_a_run_query and mcp_server_b_run_query. The name on each mcp_call item is the name the model used.

Tool Loop Limits¶

On gRPC workers, the Responses tool loop for regular (non-Harmony) models behaves the same in streaming and non-streaming mode:

• Budget. A request can execute at most max_tool_calls MCP calls, and never more than 10, the internal safety cap. Without max_tool_calls, the budget is 10.
• Partial batches. When one model turn asks for more MCP calls than the budget has left, SMG executes the calls that fit in order, keeps their results, and ends the response. Calls beyond the budget are dropped.
• User cap or safety cap. Reaching your own max_tool_calls (10 or lower) is a normal finish, and the response completes with the results so far. Reaching the internal cap (no max_tool_calls, or a value above 10) fails the response with error.code max_tool_calls_exceeded (a response.failed event when streaming).
• Final answer. When a turn's calls fit the remaining budget exactly, SMG still calls the model again, so the model can answer after its last permitted call.
• Interrupted generation. If a model turn ends with finish reason length, failed, or error, SMG executes none of that turn's calls, even complete ones. Results from earlier turns stay in the response. length gives status: "incomplete", and failed or error give status: "failed" with the upstream error, or a server_error fallback.
• Mixed batches. When a turn calls both MCP tools and the client's own function tools, SMG executes the permitted MCP calls first, then returns the response with the client's function_call items for the client to run.
• No leaked calls. Calls that SMG executes or drops never appear as function_call items, and in streaming mode their argument deltas are not sent as function-call events. They only show up as mcp_call or hosted-tool items.
• Usage. usage adds up input, output, total, cached, and reasoning tokens across every model turn in the loop.

Other Paths¶

The other paths also budget MCP calls, but enforce the limit differently:

On the OpenAI-compatible provider and in the Messages API, SMG runs every tool call in the loop through MCP, so a call to one of the client's own function tools comes back to the model as a failed tool call. Mix client function tools with MCP tools only on gRPC workers.

Call Timeouts and Failures¶

Per-Call Timeout¶

Every MCP tool call has a 120-second deadline. A call that runs longer fails with an unknown outcome and is never retried, because the server may already have acted on it:

Tool call timed out after 120s on server 'docs' while executing 'search_docs'; the outcome is unknown and the call was not retried

The deadline covers the whole call, including any re-issue after a reconnect. The value comes from pool.call_timeout, but the gateway does not apply the pool section of the configuration file, so the timeout is always 120 seconds (see Accepted but Not Applied).

Interrupted Calls¶

If the connection to an MCP server drops while a call is in flight, SMG cannot tell whether the server ran it, so it does not send the call again. The call fails with:

Tool call outcome unknown: server 'orders' disconnected while executing 'place_order'; the call was not retried because the tool is not marked idempotent or read-only

Only a call to a tool that declares the MCP idempotentHint or readOnlyHint annotation is eligible to be re-issued on a new connection. SMG does not currently use these annotations from a server's tool list for this check, so every interrupted call fails this way.

Ambiguous Pooled Connections¶

Dynamic connections are pooled by URL and credentials. When the same server_url is pooled under two different credentials (different authorization or headers values), SMG cannot tell which connection belongs to the current caller. It refuses to run the call rather than risk using another caller's authenticated session:

Server access denied: ambiguous MCP connection for 'https://mcp.example.com/mcp': multiple credentials or tenants are pooled for this URL

Calls to that URL keep failing while more than one of those connections stays in the pool. Send a single credential to each server_url.

What the Model and Client See¶

A failed call does not fail the request. SMG sends the error back to the model as the tool result (for example {"error": "Tool call failed: ..."}), and the model usually explains the failure in its answer. On gRPC workers, the client sees the call as an mcp_call item with status: "failed", an empty output, and a structured error:

{
  "type": "mcp_call",
  "id": "mcp_abc123",
  "status": "failed",
  "name": "search_docs",
  "server_label": "docs",
  "arguments": "{\"query\":\"rate limits\"}",
  "output": "",
  "error": {
    "type": "mcp_tool_execution_error",
    "content": "Tool call failed: Tool call timed out after 120s on server 'docs' while executing 'search_docs'; the outcome is unknown and the call was not retried"
  }
}

• error follows OpenAI's mcp_call.error schema: mcp_protocol_error and http_error (each with code and message), or mcp_tool_execution_error (with content). SMG emits mcp_tool_execution_error.
• A legacy item whose error is a plain string, such as one stored by an older SMG, still parses, with its error converted to mcp_tool_execution_error.
• For regular (non-Harmony) models in streaming mode, SMG also sends a response.mcp_call.failed event. Its error field is the plain message, an SMG extension.
• Hosted-tool items such as web_search_call always end as completed. Failure details are in their content.
• On the OpenAI-compatible provider, non-streaming responses currently report a failed call as a completed mcp_call with the error text in output.

Approval¶

Every MCP call passes through SMG's approval check before it runs, in one of two modes:

• require_approval: "never", the object form ({"always": ..., "never": ...}), streaming requests, gRPC workers, and the Messages API all use policy-only mode.
• SMG does not currently act on an mcp_approval_response input item, so a paused call cannot be approved through SMG.
• The loop stops at the first call that needs approval, so a response carries at most one mcp_approval_request item.

Built-in Tools and Response Formats¶

A static server can stand in for an OpenAI hosted tool. Set builtin_type and builtin_tool_name together on the server. When a Responses request includes the matching hosted tool, for example {"type": "web_search_preview"}, SMG adds that server to the request and returns the configured tool's results in the hosted format. If no server is configured for a requested hosted tool, SMG passes the tool through without MCP.

Configure each builtin_type on one server only. For tools with a hosted format, SMG also:

• Merges the options declared on the request's hosted tool (for example size and quality on image_generation) into the call arguments. The request's values win over the model's.
• Adds the request's user field as a user argument, unless the model already supplied a non-null value.

response_format on a tool sets its output item explicitly:

An explicit response_format overrides the default that builtin_type gives the builtin_tool_name tool. The mcp_list_tools item for a server with builtin_type set is never shown to clients.

MCP in the Messages API¶

On an Anthropic provider, the Messages API runs its own MCP tool loop when a request has the X-SMG-MCP: enabled header and at least one mcp_toolset tool. Without the header, SMG forwards mcp_servers and mcp_toolset to the provider unchanged.

The loop differs from the Responses loop in a few ways:

• Servers. They come from the request's mcp_servers list and connect as dynamic servers over Streamable HTTP. mcp_toolset entries choose which of their tools the model sees.
• Approval. Every call runs in policy-only approval mode.
• Execution. SMG runs the calls of each round one after another, for at most 10 rounds.
• Output. Each executed tool_use block is returned as an mcp_tool_use block followed by an mcp_tool_result block.

See the Messages API reference for the request fields, tool filtering, a full example, and the response format.

Configuration Reference¶

CLI Option¶

smg launch --worker-urls grpc://localhost:50051 --mcp-config-path /etc/smg/mcp.yaml

--mcp-config-path points at a YAML file. SMG reads and parses it at startup and refuses to start if the file can't be read or parsed. Unknown keys are ignored, but an invalid value, such as an unknown protocol, is a parse error.

Configuration File¶

servers:                             # required; use [] for no static servers
  # Remote server over Streamable HTTP
  - name: brave
    protocol: streamable
    url: "https://mcp.example.com/mcp"
    token: "<token>"                 # sent as "Authorization: Bearer <token>"
    headers:
      X-Tenant-ID: "tenant-a"
    required: true
    proxy:                           # optional, this server only
      https: "http://proxy.internal:8080"
      no_proxy: "localhost,127.0.0.1"
    builtin_type: web_search_preview # optional hosted-tool routing
    builtin_tool_name: brave_web_search
    tools:
      brave_web_search:
        alias: web_search            # the model sees "web_search"
        response_format: web_search_call
        arg_mapping:
          renames:
            q: query                 # the model's "q" is sent as "query"
          defaults:
            count: 10                # added when the model leaves it out
          overrides:
            safesearch: strict       # always replaces the model's value

  # Local server over stdio (static servers only)
  - name: filesystem
    protocol: stdio
    command: npx
    args: ["-y", "@modelcontextprotocol/server-filesystem", "/workspace"]
    envs:
      NODE_ENV: production

  # Tools stay callable, but hidden from client-visible output
  - name: memory
    protocol: streamable
    url: "http://127.0.0.1:28080/mcp"
    internal: true

servers[] Fields¶

tools Fields¶

SMG applies renames first, then defaults, then overrides. It then converts string values to numbers where the tool's input schema declares the property as number or integer. A tools entry for a tool the server doesn't list at connect time is skipped with a warning.

Accepted but Not Applied¶

The gateway accepts these keys but does not apply them. The defaults shown are the values in effect.

A pool section that asks for a 60-second call timeout looks like this. The gateway accepts it but still uses 120 seconds:

servers: []
pool:
  call_timeout: 60   # seconds per tool call; 0 disables the bound

Keeping Credentials Out of Config Files¶

SMG parses mcp.yaml as plain YAML and does not expand environment variables in it, including in server token values. To keep credentials out of the checked-in config, substitute placeholders before the gateway loads the file: render the YAML through envsubst from a shell wrapper, use a templating tool (Helm, Kustomize, Jinja), or mount the final config from a secret.

HTTP Proxies¶

Set proxy on each server that needs one. http and https are proxy URLs, no_proxy is a comma-separated list of hosts to reach directly, and username with password adds basic authentication. A server without its own proxy block uses the standard HTTP_PROXY, HTTPS_PROXY, ALL_PROXY, and NO_PROXY environment variables (upper- or lowercase). SMG currently ignores the top-level proxy section and the MCP_HTTP_PROXY, MCP_HTTPS_PROXY, and MCP_NO_PROXY variables. Proxy settings in mcp.yaml apply to MCP traffic only.

Internal Servers¶

Set internal: true on a static server when its tools support the model behind the scenes and should not show up in responses. The model can still call them, but SMG removes the server's mcp_list_tools item and its mcp_call items from non-streaming Responses output, and on an OpenAI-compatible provider also from a stream's final response.completed event. Other streaming events are not filtered. See Internal MCP Servers for the exact rules.

Security¶

• The model never sees credentials. It receives tool names, descriptions, input schemas, and results. Server URLs, tokens, and headers stay in SMG.
• Clients see tool metadata. mcp_list_tools output items list each server's tool names, descriptions, and input schemas. Mark a static server internal to keep them out of non-streaming responses.
• Static credentials live in the config file. See Keeping Credentials Out of Config Files.
• Dynamic credentials come from the request (authorization and headers, or authorization_token in the Messages API). They are used only for connections pooled under those exact credentials.
• No exported audit trail. SMG keeps its approval decisions in an in-memory buffer (the latest 10,000), but does not currently expose that buffer through an API or the logs. Use the metrics and your MCP servers' own logs to audit tool activity.

Metrics¶

tool_name is the name the model used. The model and tool_name labels each keep up to 1,024 distinct values, after which new values are reported as other. See the Metrics Reference for all gateway metrics.

Troubleshooting¶

What's Next?¶