# Design 0 — Agent Smith Framework
> **Scope.** This is the v1 design for **Agent Smith**, a C++20
> library for building LLM-powered agents. The full functional
> requirement is captured in [`prd.md`](../prd.md); this document
> turns those requirements into a buildable architecture: directory
> layout, class APIs, threading model, error contract, sequence
> diagrams, file-by-file responsibilities, and a step-by-step
> implementation order.
---
## Table of Contents
1. [Background and Glossary](#1-background-and-glossary)
2. [External References](#2-external-references)
3. [Top-Level Architecture](#3-top-level-architecture)
4. [Directory Layout and Build System](#4-directory-layout-and-build-system)
5. [Runtime and Concurrency](#5-runtime-and-concurrency)
6. [Error Model](#6-error-model)
7. [Message Model](#7-message-model)
8. [LLM Client](#8-llm-client)
9. [Streaming Pipeline](#9-streaming-pipeline)
10. [Tool Subsystem](#10-tool-subsystem)
11. [MCP Client Subsystem](#11-mcp-client-subsystem)
12. [Skill Subsystem](#12-skill-subsystem)
13. [Memory Subsystem](#13-memory-subsystem)
14. [Sub-Agents](#14-sub-agents)
15. [Agent Core (the loop)](#15-agent-core-the-loop)
16. [Observability and Logging](#16-observability-and-logging)
17. [Cancellation and Timeouts](#17-cancellation-and-timeouts)
18. [Testing Strategy](#18-testing-strategy)
19. [Demo CLI](#19-demo-cli)
20. [Implementation Order (Milestones)](#20-implementation-order-milestones)
21. [Open Questions / Deferred to v2](#21-open-questions--deferred-to-v2)
---
## 1. Background and Glossary
Before diving into the design, fix the meaning of every term that
will appear later. Read this section once carefully; the rest of the
document assumes these definitions.
### 1.1 What is an LLM agent?
An **LLM agent** is a program that wraps a large language model (LLM)
in a control loop. The loop sends conversation messages to the LLM,
the LLM may answer with text *or* with a request to call one or more
**tools** (function-call requests), the program executes those tools,
appends the results to the conversation, and asks the LLM again. The
loop terminates when the LLM produces a plain-text answer with no
tool calls, when a stop condition is hit (max iterations, deadline,
explicit "done" sentinel), or when an error occurs.
### 1.2 What is the OpenAI Chat Completions API?
The de-facto industry HTTP/JSON contract for chatting with an LLM. A
client `POST`s a JSON body containing `model`, a `messages` array
(each `{role, content}`), and optionally a `tools` array describing
callable functions. The server responds with one or more **choices**,
each containing an assistant message that may have a `content` string
and/or a `tool_calls` array. See the
[OpenAI Chat Completions reference](https://platform.openai.com/docs/api-reference/chat/create).
**llama.cpp server** implements a compatible subset; see
[llama.cpp server README](https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md).
### 1.3 What is Server-Sent Events (SSE) streaming?
When `"stream": true` is set on the request, the server keeps the
HTTP connection open and emits a sequence of frames of the form:
```
data: {"choices":[{"delta":{"content":"Hel"}}]}\n
\n
data: {"choices":[{"delta":{"content":"lo"}}]}\n
\n
data: [DONE]\n
\n
```
Each frame is a single JSON object containing an incremental
**delta**. The terminal frame is the literal string `[DONE]`. See
[MDN: Server-sent events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events).
### 1.4 What is MCP?
The **Model Context Protocol** is an open JSON-RPC 2.0 protocol that
lets an LLM client talk to "MCP servers" that expose tools,
resources, and prompts. We target the
[2025-06-18 spec revision](https://modelcontextprotocol.io/specification/2025-06-18).
Two transports matter for v1:
* **stdio** — the MCP server is a subprocess; JSON-RPC frames are
exchanged on its stdin/stdout, one frame per line.
* **Streamable HTTP** — JSON-RPC messages are sent as HTTP `POST`
requests to the MCP endpoint. Responses may be plain JSON or
`text/event-stream`.
### 1.5 What is `mw::E<T>`?
`mw::E<T>` is `std::expected<T, mw::Error>` (see
[cppreference: std::expected](https://en.cppreference.com/w/cpp/utility/expected)).
`mw::Error` is the error carrier defined in `mw/error.hpp`. It can
carry libmw's built-in transport errors and caller-defined concrete
error structs. Agent Smith defines its own parse, tool, MCP, LLM,
timeout, cancellation, and skill errors and returns them through
`mw::E<T>`. Two helper macros from `mw/utils.hpp` make this ergonomic:
* `ASSIGN_OR_RETURN(var, expr)` — assigns the value or `return`s the
error.
* `DO_OR_RETURN(expr)` — same but discards the value.
Agent Smith uses `mw::E<T>` as the **only** failure channel.
Exceptions are reserved for true programmer bugs (precondition
violations) and never used for control flow.
### 1.6 What is a C++20 coroutine?
A function declared with `co_await`, `co_yield`, or `co_return` in
its body becomes a **coroutine**: it can suspend mid-execution and
resume later. The framework returns `mw::Task<T>` from any public
function that does asynchronous work. Read
[cppreference: coroutines](https://en.cppreference.com/w/cpp/language/coroutines)
for a primer and the libmw task implementation before changing the
public async contract.
The agent's API will look like:
```cpp
mw::Task<mw::E<AssistantMessage>> reply =
co_await agent.send("Hello!");
```
### 1.7 What does `mw::HTTPSessionAsync` give us?
`mw::HTTPSessionAsync` (in `mw/http_client_async.hpp`) wraps libcurl's
multi interface behind coroutine-friendly methods. It owns its own
curl driver thread and supports concurrent HTTP requests. Agent
Smith uses one runtime-owned `mw::HTTPSessionAsync` for all LLM
HTTP and MCP Streamable HTTP traffic.
Streaming uses the async client's chunk-callback API. The callback
runs on the HTTP driver thread, so it must stay small: parse incoming
bytes, push typed events to a queue, and return whether the transfer
should continue.
### 1.8 What blocking work remains?
HTTP is not blocking at the framework layer. Agent Smith still owns a
small internal blocking runner for operations that are inherently
blocking in v1:
* synchronous native tools;
* MCP stdio pipe reads and writes;
* subprocess management around `mw::exec`;
* compatibility shims if an implementation detail requires them.
### 1.9 What does `mw::exec` give us?
`mw::Process` (in `mw/exec.hpp`) spawns a child process via
`fork`/`execvp`. It accepts `Input` and `Output` of type
`std::variant<None, std::string_view, Pipe*>` and
`std::variant<None, std::string*, Pipe*>` respectively. For MCP
stdio servers we will use `Pipe*` for **both** input and output so
that we can keep writing requests and reading responses for the
lifetime of the server. `Process::wait()` reaps the child and
returns its exit code.
---
## 2. External References
This section centralizes every external link the implementer should
keep open in a browser tab.
* [PRD](../prd.md) — source of truth for requirements.
* [OpenAI Chat Completions API](https://platform.openai.com/docs/api-reference/chat/create)
* [llama.cpp server README](https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md)
* [MCP spec, revision 2025-06-18](https://modelcontextprotocol.io/specification/2025-06-18)
* [JSON-RPC 2.0 spec](https://www.jsonrpc.org/specification)
* [SSE: HTML Living Standard](https://html.spec.whatwg.org/multipage/server-sent-events.html)
* [cppreference: std::expected](https://en.cppreference.com/w/cpp/utility/expected)
* [cppreference: coroutines](https://en.cppreference.com/w/cpp/language/coroutines)
* [nlohmann/json](https://github.com/nlohmann/json)
* [spdlog](https://github.com/gabime/spdlog)
* [RFC 7807 — Problem Details for HTTP APIs](https://www.rfc-editor.org/rfc/rfc7807)
(consult when designing error JSON payloads)
* [RFC 6750 — Bearer Token Usage](https://www.rfc-editor.org/rfc/rfc6750)
(HTTP `Authorization` header format)
---
## 3. Top-Level Architecture
Agent Smith is a single static (or shared) library `libagent_smith`
plus a small CLI demo binary `agent_smith_demo`. The library exposes
one public include directory (`include/agent_smith/`) and a single
namespace `agent_smith`. Internal helpers live in
`agent_smith::internal` and are not part of the public API.
The library is composed of nine subsystems, each owning one
sub-namespace and one or two header files:
```
┌──────────────────────┐
│ Agent │ agent.hpp
│ (loop orchestrator) │
└──────────┬───────────┘
┌──────────┬──────────┼──────────┬───────────┐
│ │ │ │ │
┌────▼────┐ ┌───▼───┐ ┌────▼────┐ ┌───▼───┐ ┌─────▼─────┐
│ LLM │ │ Tools │ │ Skills │ │Memory │ │ MCP Client│
│ Client │ │ │ │ │ │ │ │ │
└────┬────┘ └───┬───┘ └─────────┘ └───────┘ └─────┬─────┘
│ │ │
│ └──────────┐ ┌────────┘
│ │ │
┌────▼──────────────────── ▼──────────────▼──────┐
│ Runtime │
│ mw::HTTPSessionAsync + small blocking runner │
│ + queues for streaming / stdio MCP │
└─────────────────────────────────────────────────┘
│
┌──────────────────────▼──────────────────────────┐
│ libmw (http_client, exec, │
│ url, error, utils) │
└─────────────────────────────────────────────────┘
```
Subsystem responsibilities:
| Subsystem | Owns | Talks to |
| -------------- | ---------------------------------------- | ----------------------------- |
| Agent | the loop, stop conditions, hooks | every other subsystem |
| LLM Client | OpenAI-compat HTTP, SSE parsing | Runtime |
| Tools | `Tool` interface, schemas, dispatch | Agent, MCP Client (adapter) |
| Skills | bundles of (prompt, tools, MCP, data) | Agent |
| Memory | message history, truncation/compaction | Agent (read/write) |
| Sub-Agents | a `Tool` that spawns child `Agent`s | Agent (recursive) |
| MCP Client | JSON-RPC, transports, tool adapters | Tools |
| Observability | hook dispatch, spdlog wiring | every subsystem (passive) |
| Runtime | async HTTP, blocking runner, queues | LLM, MCP, Tools |
---
## 4. Directory Layout and Build System
### 4.1 Directory layout
```
agent-smith/
├── CMakeLists.txt # top-level
├── prd.md
├── designs/
│ └── design-0-framework.md # (this file)
├── include/
│ └── agent_smith/
│ ├── agent.hpp
│ ├── cancellation.hpp
│ ├── error.hpp # re-exports mw::E and adds AS-specific tags
│ ├── llm_client.hpp
│ ├── memory.hpp
│ ├── message.hpp
│ ├── mcp_client.hpp
│ ├── observability.hpp
│ ├── runtime.hpp # async HTTP + blocking runner
│ ├── skill.hpp
│ ├── stream_event.hpp
│ ├── sub_agent.hpp
│ ├── tool.hpp
│ └── version.hpp
├── src/
│ ├── agent.cpp
│ ├── llm_client.cpp
│ ├── memory.cpp
│ ├── mcp/
│ │ ├── client.cpp
│ │ ├── transport_stdio.cpp
│ │ ├── transport_http.cpp
│ │ ├── jsonrpc.cpp
│ │ └── tool_adapter.cpp
│ ├── observability.cpp
│ ├── runtime.cpp
│ ├── skill.cpp
│ ├── sse_parser.cpp
│ ├── sub_agent.cpp
│ └── tool.cpp
├── src/detail/
│ └── started_task.hpp # private eager task helper
├── tests/
│ ├── CMakeLists.txt
│ ├── test_sse_parser.cpp
│ ├── test_llm_client.cpp # uses async HTTP mock/seam
│ ├── test_tool.cpp
│ ├── test_memory.cpp
│ ├── test_mcp_jsonrpc.cpp
│ ├── test_mcp_transport_stdio.cpp
│ ├── test_skill.cpp
│ ├── test_agent_loop.cpp
│ └── test_sub_agent.cpp
└── demo/
├── CMakeLists.txt
├── main.cpp # REPL
├── tools_calculator.cpp
├── tools_filesystem.cpp
└── mcp_server_example.py # bundled stdio MCP server
```
### 4.2 CMake structure
* The top-level `CMakeLists.txt` calls `project(agent_smith CXX)`,
requires `cmake_minimum_required(VERSION 3.25)`, and sets
`CMAKE_CXX_STANDARD 20` plus `CMAKE_CXX_STANDARD_REQUIRED ON`.
* It looks for required dependencies via `find_package`:
* `nlohmann_json` (3.11+),
* `spdlog` (1.12+),
* `libmw` is consumed as a subdirectory or installed package; the
`add_subdirectory(${LIBMW_DIR})` path is configurable via
`-DLIBMW_DIR=`.
* Two targets are defined:
* `agent_smith` (library, default `STATIC`; toggle with
`-DAGENT_SMITH_BUILD_SHARED=ON`).
* `agent_smith_demo` (executable, optional, behind
`-DAGENT_SMITH_BUILD_DEMO=ON`).
* Tests live under `tests/` and are built only when
`-DAGENT_SMITH_BUILD_TESTS=ON`. They link against
[GoogleTest](https://github.com/google/googletest) (matching
libmw's choice; verify by inspecting libmw's CMakeLists when
implementing).
* Per the user's coding style: indentation is 4 spaces, opening
braces on their own line, no space before `(` in calls. Apply this
to **all** C++ source.
### 4.3 Public include structure
The top-level header `agent_smith/agent_smith.hpp` is an umbrella
header that includes the public subsystem headers. Each subsystem
header is also independently includable.
---
## 5. Runtime and Concurrency
This is the most subtle part of the design. Read it twice.
### 5.1 Why the runtime is small
`mw::HTTPSessionAsync` already provides coroutine-friendly HTTP and
owns its curl driver thread. Agent Smith does not wrap HTTP in a
worker pool, does not expose asio types, and does not own per-worker
HTTP sessions. The runtime owns:
* one `mw::HTTPSessionAsync` used by LLM and MCP HTTP code;
* one small blocking runner used only for non-HTTP blocking work;
* configuration shared by subsystems.
The public async paradigm is still unified: framework APIs return
`mw::Task<mw::E<T>>` and callers use `co_await`. The implementation is
not flattened into a single "run this closure on a worker" mechanism.
HTTP uses `mw::HTTPSessionAsync` directly because it has streaming,
connection, timeout, and cancellation semantics that are better handled
by libmw's curl driver. Only operations that are inherently blocking at
the framework layer are lifted into coroutines with `Runtime::runOnWorker`.
### 5.2 The `Runtime` type
```cpp
// runtime.hpp
#include <mw/http_client_async.hpp>
namespace agent_smith
{
struct RuntimeConfig
{
std::size_t blocking_pool_size = 4;
std::chrono::seconds connection_timeout{60};
std::chrono::seconds transfer_timeout{60};
};
class Runtime
{
public:
explicit Runtime(RuntimeConfig cfg = {});
~Runtime();
Runtime(const Runtime&) = delete;
Runtime& operator=(const Runtime&) = delete;
// Shared async HTTP session for all LLM and MCP HTTP calls.
mw::HTTPSessionAsync& http();
// Run non-HTTP blocking work on the internal worker pool and return
// an awaitable result. Do not use for HTTP.
template <typename F>
mw::Task<internal::ExpectedResult<std::invoke_result_t<F>>>
runOnWorker(F&& f);
// Coroutine-friendly timer used for retry backoff and internal
// timeout races. Does not expose asio or scheduler-specific types.
mw::Task<void> sleepFor(std::chrono::milliseconds duration);
private:
RuntimeConfig cfg_;
mw::HTTPSessionAsync http_;
internal::BlockingPool blocking_pool_;
};
} // namespace agent_smith
```
### 5.3 Blocking runner
`internal::BlockingPool` is a private utility implemented with
`std::jthread`, `std::mutex`, `std::condition_variable`, and a FIFO work
queue. Subsystems do not access it directly; they call
`Runtime::runOnWorker`:
```cpp
template <typename F>
mw::Task<internal::ExpectedResult<std::invoke_result_t<F>>>
Runtime::runOnWorker(F&& f);
```
The submitted callable must not throw as part of normal failure. It
returns an `mw::E<T>` directly or a plain value wrapped by the
adapter. The first implementation may support only the exact shapes
needed by synchronous tools and stdio transport.
`internal::ExpectedResult<T>` normalizes the return type:
```cpp
T -> mw::E<T>
mw::E<T> -> mw::E<T>
```
This keeps callers simple. A synchronous tool callable returning
`mw::E<ToolResult>` still produces `mw::Task<mw::E<ToolResult>>`,
not a nested `mw::E<mw::E<ToolResult>>`.
### 5.4 Timers and timeout helpers
First inspect libmw for an existing coroutine sleep/timer primitive. If
one exists, `Runtime::sleepFor` wraps it. If not, implement a small
runtime-owned timer facility without exposing asio or scheduler-specific
types in the Agent Smith public API.
Retry backoff and timeout races use `Runtime::sleepFor`; they must not
block a coroutine thread with `std::this_thread::sleep_for`.
An internal helper in `include/agent_smith/detail/timeout.hpp` provides the common race
logic:
```cpp
template <typename T>
mw::Task<mw::E<T>> withTimeout(Runtime& rt,
mw::Task<mw::E<T>> task,
std::chrono::milliseconds timeout,
CancellationSource* cancel);
```
On timeout, `withTimeout` cancels the supplied source when non-null and
returns `TimeoutError`. Tool dispatch converts that timeout error into a
tool-result message as described in §10.4. HTTP retry backoff uses
`Runtime::sleepFor` directly.
### 5.5 HTTP usage
LLM and MCP HTTP code call the runtime HTTP session directly:
```cpp
mw::HTTPRequest http_req = buildRequest(...);
auto http_res = co_await rt.http().post(std::move(http_req));
```
For streaming:
```cpp
auto http_res = co_await rt.http().postStream(
std::move(http_req),
[&](std::span<const std::byte> chunk)
{
return parser.feed(chunk, event_sink);
});
```
`Runtime` does not define an `onHttp` helper and does not expose a
session factory for the old blocking `mw::HTTPSession` API.
### 5.6 Test seam
Prefer libmw's async HTTP mock if one exists. If not, add a tiny
Agent Smith interface only for tests:
```cpp
class HttpClient
{
public:
virtual ~HttpClient() = default;
virtual mw::Task<mw::E<mw::HTTPResponse>>
post(mw::HTTPRequest req) = 0;
virtual mw::Task<mw::E<mw::HTTPResponse>>
postStream(mw::HTTPRequest req, mw::ChunkCallback cb) = 0;
};
```
Only add this wrapper when tests need it. Production code should use
`mw::HTTPSessionAsync` directly through `Runtime::http()`.
### 5.7 Concurrency invariants
1. `Agent` is not thread-safe. One coroutine drives one agent at a
time.
2. HTTP concurrency is governed by `mw::HTTPSessionAsync`. Agent
Smith adds no HTTP worker pool, semaphore, or token bucket in v1.
3. The blocking runner is used only for synchronous native tools, MCP
stdio pipe I/O, and subprocess management.
4. Sub-agents are separate `Agent` instances. They may run
concurrently when invoked by parallel tool calls.
5. Hooks are invoked synchronously by the agent coroutine, not from
the HTTP driver thread.
---
## 6. Error Model
### 6.1 Re-using `mw::E<T>`
Every fallible function in the public API returns
`mw::E<T>`. When the function is a coroutine, the return type is
`mw::Task<mw::E<T>>`.
Agent Smith defines concrete, copyable error structs in
`agent_smith/error.hpp` and stores them in `mw::Error`. Callers can
inspect the error with `err.as<agent_smith::ParseError>()` instead
of parsing message prefixes.
```cpp
namespace agent_smith
{
struct ParseError
{
std::string msg;
std::size_t offset = 0;
};
struct ToolError
{
std::string msg;
std::string tool_name;
bool retryable = false;
};
struct McpError
{
std::string msg;
std::string server;
std::optional<int> rpc_code;
};
struct TransportError
{
std::string msg;
std::string operation;
std::string endpoint_or_server;
};
struct McpProtocolError
{
std::string msg;
std::string server;
std::optional<int> rpc_code;
};
struct LlmApiError
{
std::string msg;
int http_status = 0;
std::string body_excerpt;
};
struct TimeoutError
{
std::string msg;
std::string operation;
std::chrono::milliseconds budget;
};
struct CancelledError
{
std::string msg;
std::string operation;
};
struct SkillError
{
std::string msg;
std::string skill_name;
};
} // namespace agent_smith
```
Error helpers may live in `agent_smith::error`, but they construct
typed errors:
```cpp
mw::Error parse(std::string_view msg, std::size_t offset = 0);
mw::Error tool(std::string_view tool_name, std::string_view msg);
mw::Error mcp(std::string_view server, std::optional<int> rpc_code,
std::string_view msg);
mw::Error transport(std::string_view operation,
std::string_view endpoint_or_server,
std::string_view msg);
mw::Error mcpProtocol(std::string_view server,
std::optional<int> rpc_code,
std::string_view msg);
mw::Error cancel(std::string_view operation, std::string_view msg);
mw::Error timeout(std::string_view operation,
std::chrono::milliseconds budget);
mw::Error llmApi(int http_status, std::string_view body_excerpt);
```
Use `TransportError` only when the raw libmw transport error needs
Agent Smith operation context, such as the MCP server alias or LLM
endpoint. Otherwise preserve the original libmw error in `mw::Error`.
Use `McpProtocolError` for JSON-RPC errors and invalid MCP message flow.
Do not conflate those with subprocess, pipe, HTTP, or TLS failures.
Return typed errors as:
```cpp
co_return std::unexpected(mw::Error(ParseError{
.msg = "expected object",
.offset = offset
}));
```
### 6.2 No exceptions across the API boundary
All public functions are `noexcept` whenever possible. The only
exceptions allowed:
* `std::bad_alloc` (truly fatal)
* nlohmann::json may throw on malformed JSON when callers use the
`[]` operator; the library uses `.contains` + `.value` or
`.find()` to avoid that.
### 6.3 HTTP retry policy
Retry is applied at the HTTP layer only. A small `HTTPRetryPolicy`
wrapper inside `llm_client.cpp` and `mcp/transport_http.cpp`:
| Trigger | Action |
| ----------------------------- | --------------------------------------- |
| Connect failure or request timeout | Retry up to N times (default 3) with backoff. |
| HTTP 5xx | Retry, same. |
| HTTP 429 with `Retry-After` | Sleep that long (capped at policy max), then retry. |
| HTTP 429 without `Retry-After`| Backoff schedule. |
| Streaming response that already delivered ≥1 byte | **No retry.** Surface error. |
| Tool error | **Never retry.** Pass to LLM as result. |
Backoff: `delay = base * 2^(attempt-1) + jitter` where `base =
500ms`, jitter is `uniform(0, 250ms)`, capped at 8s. See
[AWS architecture blog: exponential backoff and jitter](https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/)
for rationale.
---
## 7. Message Model
### 7.1 Roles
```cpp
// message.hpp
namespace agent_smith
{
enum class Role
{
SYSTEM,
USER,
ASSISTANT,
TOOL
};
} // namespace agent_smith
```
Mapping to the OpenAI wire role string is done in `llm_client.cpp`
via a small lookup table.
### 7.2 The `Message` variant
Each turn in the conversation is one of four concrete shapes. We
model this as a `std::variant`:
```cpp
struct SystemMessage
{
std::string content;
};
struct UserMessage
{
std::string content;
// future: attachments (omitted in v1; PRD: no multimodal)
};
struct ToolCall
{
std::string id; // server-generated id, opaque
std::string name; // function name
nlohmann::json arguments; // already-parsed JSON object
};
struct AssistantMessage
{
std::string content; // may be empty when there are tool calls
std::vector<ToolCall> tool_calls;
// Echoed usage info if available (only present after a finished call).
std::optional<TokenUsage> usage;
};
struct ToolMessage
{
std::string tool_call_id; // matches ToolCall::id
std::string name;
std::string content; // may be JSON-stringified structured result
// Internal marker only. The OpenAI-compatible wire message does
// not get a non-standard `is_error` field.
bool is_error = false;
};
using Message = std::variant<SystemMessage,
UserMessage,
AssistantMessage,
ToolMessage>;
struct TokenUsage
{
std::uint64_t prompt = 0;
std::uint64_t completion = 0;
std::uint64_t total = 0;
};
```
**Why a variant and not inheritance?** `std::variant` gives us
value semantics, no heap allocations, exhaustive `std::visit`
matching at compile time, and trivial serialization. The downside
(adding a new message kind requires touching every visitor) is
acceptable — the OpenAI message taxonomy has been stable for
years.
### 7.3 JSON helpers
Define free functions in `message.cpp`:
```cpp
nlohmann::json toOpenAIJson(const Message& m);
mw::E<Message> assistantMessageFromOpenAIJson(const nlohmann::json& j);
```
`toOpenAIJson` is total (no errors). The "from" direction can fail
because the server payload is untrusted — return
`agent_smith::error::parse(...)` on shape mismatches.
---
## 8. LLM Client
### 8.1 Public API
```cpp
// llm_client.hpp
namespace agent_smith
{
struct SamplingParams
{
std::optional<double> temperature;
std::optional<double> top_p;
std::optional<int> max_tokens;
std::vector<std::string> stop;
// Opaque pass-through for structured output / JSON mode.
nlohmann::json response_format = nullptr;
// Provider-specific request fields. See merge rules below.
nlohmann::json extra = nullptr;
};
struct LlmClientConfig
{
mw::URL endpoint; // e.g. http://localhost:8080
std::string model; // e.g. "qwen2.5-7b-instruct"
std::optional<std::string> api_key;
std::chrono::seconds request_timeout{60};
int max_retries = 3;
SamplingParams default_sampling;
};
struct LlmRequest
{
std::vector<Message> messages;
std::vector<ToolSchema> tools; // see Tool subsystem
SamplingParams sampling; // overrides defaults
};
struct LlmResponse
{
AssistantMessage message; // includes usage
};
class LlmClient
{
public:
LlmClient(Runtime& rt, LlmClientConfig cfg);
// Non-streaming. Posts to /v1/chat/completions.
mw::Task<mw::E<LlmResponse>> complete(LlmRequest req);
// Streaming. The HTTP callback only pushes queue events; this
// coroutine drains them and invokes `on_event`. See §9.
mw::Task<mw::E<LlmResponse>> completeStream(
LlmRequest req,
std::function<void(const StreamEvent&)> on_event);
private:
Runtime* rt_;
LlmClientConfig cfg_;
};
} // namespace agent_smith
```
### 8.2 Request body construction
`buildRequestBody(req, cfg)` builds:
```json
{
"model": "qwen2.5-7b-instruct",
"messages": [...],
"tools": [{"type":"function","function":{...}}, ...],
"tool_choice": "auto",
"temperature": 0.2,
"top_p": 0.95,
"max_tokens": 1024,
"stop": ["</s>"],
"response_format": {...}, // only if non-null
"stream": false // or true for streaming
}
```
The `tools` field is omitted when `req.tools` is empty (some
servers reject empty arrays). The `Authorization: Bearer <key>`
header is added when `cfg.api_key` is set.
Sampling is merged deterministically:
1. Start with `cfg.default_sampling`.
2. Apply request-level `req.sampling` fields when set.
3. Build first-class request fields (`model`, `messages`, `tools`,
`tool_choice`, `stream`, and named sampling fields).
4. Merge `sampling.extra` last, except framework-owned structural keys
are reserved and cannot be overwritten.
Reserved keys are `model`, `messages`, `tools`, `tool_choice`, and
`stream`. If `extra` contains one of those keys, request construction
fails with a typed configuration/parse error. Provider-specific knobs
such as llama.cpp extensions may be passed through `extra`.
### 8.3 The non-streaming path (sequence)
1. Caller awaits `complete(req)` from an `mw::Task` coroutine.
2. `complete` builds the JSON body and an `mw::HTTPRequest` with
path `/v1/chat/completions` (resolved via `mw::URL::resolve`).
3. `complete` calls `co_await rt.http().post(std::move(http_req))`.
4. With retry-policy in effect the coroutine may loop: build, post,
classify, wait according to the backoff policy, retry.
5. On 2xx, parse the body and produce an `AssistantMessage`.
6. The coroutine returns `mw::E<LlmResponse>`.
### 8.4 The streaming path
Defined in [§9](#9-streaming-pipeline) because it is intricate
enough to warrant its own section.
---
## 9. Streaming Pipeline
### 9.1 Goal
Translate the byte stream produced by
`mw::HTTPSessionAsync::postStream` into a typed `StreamEvent` stream.
The HTTP chunk callback runs on the libmw HTTP driver thread. It
parses bytes and pushes events to a queue; the Agent Smith coroutine
drains that queue and invokes user hooks.
### 9.2 Event types
```cpp
// stream_event.hpp
namespace agent_smith
{
struct StreamTokenDelta { std::string text; };
struct StreamToolCallStarted { std::string id; std::string name; int index; };
struct StreamToolCallCompleted { int index; nlohmann::json arguments; };
struct StreamFinished { std::string finish_reason; std::optional<TokenUsage> usage; };
using StreamEvent = std::variant<StreamTokenDelta,
StreamToolCallStarted,
StreamToolCallCompleted,
StreamFinished>;
} // namespace agent_smith
```
### 9.3 Two SSE sub-parsers
* **Frame parser.** Statefully splits incoming raw bytes on the
boundary `"\n\n"`. Inside a frame, lines starting with `data:`
contribute to the payload; other lines are ignored. The literal
payload `[DONE]` terminates the stream.
* **Delta parser.** Reads each JSON object from the frame parser and
emits `StreamEvent`s. State it must keep:
* `current_text` accumulator (rarely needed, we emit deltas as
they come).
* `tool_calls_by_index` — `std::vector<PartialToolCall>` indexed
by the integer `index` field on each tool-call delta. Each
`PartialToolCall` accumulates `id`, `name`, and an
`arguments_buffer` string. When the stream finishes, each
completed entry is parsed once into a `nlohmann::json` object
and emitted as `StreamToolCallCompleted`.
Argument fragments are internal parser state. They are never
exposed through public stream events because they are not valid
JSON until accumulation completes.
* `usage` — captured from the terminal frame, which llama.cpp
server emits as `{"usage": {...}}` in the *final* `data:` frame
before `[DONE]` (per
[llama.cpp PR #6614](https://github.com/ggerganov/llama.cpp/pull/6614)).
The frame parser is tested in isolation (`tests/test_sse_parser.cpp`)
with hand-crafted byte inputs that include multi-byte chunk boundary
splits ("data: {\"choi" then "ces\":[...]}\n\n").
### 9.4 The cross-thread queue
```cpp
template <typename T>
class AsyncQueue
{
public:
void push(T v);
mw::Task<std::optional<T>> pop();
void close();
private:
std::mutex m_;
std::deque<T> q_;
bool closed_ = false;
// Implementation detail: use the wake primitive supplied by
// mw::Task, or a small condition-variable bridge hidden in
// detail/. Do not expose asio types.
};
```
`pop()` returns `std::nullopt` only after `close()` and after all
queued events are drained.
### 9.5 End-to-end streaming sequence
`mw::Task` is lazy and does not expose a public spawn/select primitive.
Therefore `completeStream` must not simply `co_await
rt.http().postStream(...)` and then drain the queue: that would suspend
the LLM coroutine until the HTTP transfer completed, causing all stream
events to be delivered to hooks only after the response ended.
First inspect `libmw` for an existing task-spawn, scheduling, or async
queue primitive. If none exists, Agent Smith provides a tiny internal
`StartedTask<T>` helper in `detail/started_task.hpp`. It owns a
coroutine handle, starts it immediately, records its result, and lets
the caller `co_await` completion after the foreground work is done. It
is not part of the public API and is used only where the framework must
run two internal coroutines concurrently.
Keep `StartedTask` intentionally narrow in v1:
* Support only `mw::Task<mw::E<T>>`.
* Store exactly one result.
* Close the associated queue when the task completes.
* Make observation coroutine-shaped: callers `co_await` the started
task to retrieve the stored result.
* Require the caller to await the result before destruction.
* Test success, HTTP error, parse error, cancellation, and early queue
close.
```
agent coroutine HTTP task / driver thread
──────────────── ─────────────────────────
co_await llm.completeStream(req, on_event)
startTask(http_task):
──▶ co_await rt.http().postStream(
req,
[&](chunk) {
feed frame parser;
for each StreamEvent ev:
queue.push(ev);
return !cancelled;
});
store http_result on http_task;
queue.close();
while ev = co_await queue.pop():
on_event(ev);
accumulate final assistant message
http_result = co_await http_task
if http_result failed:
co_return error
co_return accumulated response
```
The chunk callback never calls user code. `completeStream` returns
only after the HTTP stream has closed, all queued events have been
drained, and the started HTTP task's result has been observed. Hooks see
token and completed tool-call events while the HTTP stream is active.
Tool execution begins after the stream finishes, not while tool-call
argument fragments are still arriving.
### 9.6 Tool-call accumulation
Per OpenAI semantics, when the model emits tool calls, each
`tool_calls[i].function.arguments` arrives as a stream of partial
JSON strings. The library **must not** expose those partial strings
as parsed JSON or as public string fragments. It buffers them per
`index` and parses once at the end of the stream (or when a new tool
call at a higher index arrives, signaling that index `k`'s arguments
are complete — though the conservative implementation is to parse
only on `finish_reason == "tool_calls"` to handle out-of-order edge
cases). If trace-level diagnostics need raw fragments, log them
through the redaction path instead of adding them to `StreamEvent`.
If parsing fails, emit `agent_smith::error::parse(...)` from the
overall `completeStream` result. Already-delivered stream events remain
observable through `on_stream_event`, but v1 does not attach partial
assistant output to the failed result. If a caller later needs partial
state on errors, add a typed `LlmStreamError` that carries it rather
than complicating the success response shape.
---
## 10. Tool Subsystem
### 10.1 The `Tool` interface
```cpp
// tool.hpp
namespace agent_smith
{
struct ToolSchema
{
std::string name;
std::string description;
nlohmann::json parameters; // JSON-Schema object
};
struct ToolContext
{
CancellationToken cancel; // honored on a best-effort basis
std::chrono::milliseconds budget; // remaining time
std::string call_id; // matches ToolCall::id
spdlog::logger* logger; // sub-logger "agent_smith.tool"
Runtime* runtime; // for tools that need HTTP
};
struct ToolResult
{
std::string content; // freeform text or JSON-stringified
bool is_error = false;
nlohmann::json structured = nullptr; // optional richer return
};
struct ToolOptions
{
std::optional<std::chrono::milliseconds> timeout;
};
class Tool
{
public:
virtual ~Tool() = default;
virtual const ToolSchema& schema() const = 0;
virtual mw::Task<mw::E<ToolResult>>
invoke(nlohmann::json args, ToolContext ctx) = 0;
};
} // namespace agent_smith
```
### 10.2 Adapting plain C++ callables
To avoid forcing every user to derive from `Tool`, provide concept-
constrained adapter factories. The core `Tool::invoke` interface stays
normalized to `mw::Task<mw::E<ToolResult>>`, but user callables may
return simpler shapes that can be normalized:
```cpp
mw::E<ToolResult> normalizeToolResult(std::string value);
mw::E<ToolResult> normalizeToolResult(const char* value);
mw::E<ToolResult> normalizeToolResult(nlohmann::json value);
mw::E<ToolResult> normalizeToolResult(ToolResult value);
mw::E<ToolResult> normalizeToolResult(mw::E<ToolResult> value);
```
`std::string` and `const char*` become plain text results.
`nlohmann::json` is serialized into `ToolResult::content` and also
stored in `ToolResult::structured`. `mw::E<ToolResult>` is forwarded
unchanged.
```cpp
template <typename F>
concept SyncToolCallable =
requires(F f, nlohmann::json a, ToolContext c)
{
{ normalizeToolResult(f(a, c)) } -> std::same_as<mw::E<ToolResult>>;
};
template <typename F>
concept AsyncToolCallable =
requires(F f, nlohmann::json a, ToolContext c)
{
{ f(a, c) } -> std::same_as<mw::Task<mw::E<ToolResult>>>;
};
std::unique_ptr<Tool>
makeTool(ToolSchema schema, SyncToolCallable auto fn);
std::unique_ptr<Tool>
makeTool(ToolSchema schema, AsyncToolCallable auto fn);
```
Implementation: each overload returns a small derived class that
holds `schema`/`fn`. The async overload awaits the callable directly.
The sync overload returns a `SyncToolAdapter` whose `invoke` submits
the callable to `ctx.runtime->runOnWorker()` and awaits that task, so
blocking user code never runs on the agent coroutine:
```cpp
mw::Task<mw::E<ToolResult>> invoke(nlohmann::json args,
ToolContext ctx) override
{
co_return co_await ctx.runtime->runOnWorker(
[fn = fn_, args = std::move(args), ctx]() mutable
{
return normalizeToolResult(fn(std::move(args), ctx));
});
}
```
`ToolContext::runtime` is non-null for all framework-created tool
invocations. Hand-written `Tool` implementations may do their own
asynchronous work, but adapters created by `makeTool` preserve the
PRD's execution rule: coroutine callables run as coroutines;
synchronous callables run on the internal blocking runner.
### 10.3 Registry
```cpp
class ToolRegistry
{
public:
// Returns false if name collision (caller-driven; Skill subsystem
// tolerates collisions per PRD).
bool add(std::unique_ptr<Tool> t, ToolOptions opts = {});
void replace(std::unique_ptr<Tool> t, ToolOptions opts = {});
Tool* find(std::string_view name);
ToolOptions options(std::string_view name) const;
std::vector<ToolSchema> schemas() const;
std::size_t size() const;
private:
struct Entry
{
std::unique_ptr<Tool> tool;
ToolOptions options;
};
std::unordered_map<std::string, Entry> by_name_;
std::vector<std::string> insertion_order_; // for deterministic schemas()
};
```
### 10.4 Dispatching tool calls
In each loop iteration, after the LLM response contains
`tool_calls`, the agent:
1. Looks each call up in its `ToolRegistry`. Missing tool → produce
a `ToolMessage{is_error=true, content="unknown tool: X"}` and
continue (no abort; let the model retry).
2. Wraps each `Tool::invoke` in `withTimeout`, using the tool's
registered `ToolOptions::timeout` when present, else
`AgentConfig::default_tool_timeout`.
3. Starts every wrapped task immediately with the same internal
`StartedTask` helper used by the streaming pipeline. This is required
because `mw::Task` is lazy and cannot be assumed to run concurrently
just because several task objects exist.
4. Waits for all started tool tasks with a small internal
`gatherAllStarted` helper that preserves result order.
5. On timeout, marks the call's `CancellationToken` cancelled and
produces an error `ToolMessage` for that call.
6. Truncates `content` to `cfg.tool_result_max_bytes` (default 64
KiB), appending the marker
`\n…[truncated, original size N bytes]`.
7. Wraps each result into a `ToolMessage` and appends to the
conversation in the **same order as the tool_calls array** (this
is required by some servers' template rendering).
Async tool callables run as coroutines inside their started task. Sync
tools created through `makeTool` submit their blocking callable to
`Runtime::runOnWorker()` from inside their adapter. `StartedTask`
remains an internal helper; the public API still exposes only
`mw::Task<mw::E<T>>`.
Per-tool failures are part of the conversation, not agent-loop aborts.
This includes an unknown tool, `ToolResult{.is_error = true}`, an
`mw::E<ToolResult>` error returned by a tool implementation, and a
per-call timeout. Each is converted into an ordered `ToolMessage` with
compact JSON error content. Framework-level failures outside a specific
tool call still abort the loop: LLM transport/API errors, malformed LLM
responses, memory snapshot errors, whole-agent cancellation/deadline,
and internal invariant violations.
Milestone 8 exposes this batch operation internally as `dispatchTools`
for the later agent loop. A timed-out invocation retains its tool and
coroutine state until it finishes. The runtime drains these operations
before destroying HTTP, worker, or timer services; non-cooperative tools
can therefore delay runtime shutdown. Timers are cancelled when the tool
finishes first. Result truncation keeps a UTF-8 boundary and appends the
marker after the configured content-byte allowance.
`ToolMessage::is_error` is internal. When serializing to the
OpenAI-compatible wire format, emit only standard fields:
```json
{
"role": "tool",
"tool_call_id": "...",
"content": "..."
}
```
For tool errors, encode a compact JSON object in `content`, for
example `{"error":true,"message":"unknown tool: X"}`. Do not add
non-standard top-level fields to chat messages.
### 10.5 Schemas: author-supplied
v1 does not generate schemas from C++ types. Authors pass either a
JSON-string literal or a `nlohmann::json` value. A small helper
validates structurally that the parameters object is itself a valid
JSON-Schema object root (`{"type":"object","properties":{...}}`)
and warns via spdlog otherwise.
### 10.6 Sub-agent as a tool
`SubAgentTool` (see [§14](#14-sub-agents)) implements `Tool` and is
the only "framework-internal" tool the agent ever auto-registers.
---
## 11. MCP Client Subsystem
### 11.1 Wire protocol layer
JSON-RPC 2.0 framing is identical regardless of transport. Define a
small frame type and a transport interface:
```cpp
// mcp_client.hpp (public)
namespace agent_smith::mcp
{
struct Notification
{
std::string method;
nlohmann::json params;
};
enum class LifecycleEventKind
{
CONNECTED,
DISCONNECTED,
RECONNECTING,
RECONNECT_FAILED
};
struct LifecycleEvent
{
LifecycleEventKind kind;
std::string server_alias;
std::string message;
};
using Event = std::variant<Notification, LifecycleEvent>;
using EventHook = std::function<void(const Event&)>;
struct Request
{
std::int64_t id;
std::string method;
nlohmann::json params;
};
struct Response
{
std::int64_t id;
std::variant<nlohmann::json /*result*/, RpcError> body;
};
struct RpcError
{
int code;
std::string message;
nlohmann::json data;
};
struct ServerInfo
{
std::string name;
std::string version;
nlohmann::json raw = nullptr;
};
struct ResourceInfo
{
std::string uri;
std::string name;
std::optional<std::string> description;
std::optional<std::string> mime_type;
nlohmann::json raw = nullptr;
};
struct PromptInfo
{
std::string name;
std::optional<std::string> description;
nlohmann::json arguments = nlohmann::json::array();
nlohmann::json raw = nullptr;
};
struct PromptRendering
{
std::vector<Message> messages;
nlohmann::json raw = nullptr;
};
class Transport
{
public:
virtual ~Transport() = default;
virtual mw::Task<mw::E<void>> send(nlohmann::json frame) = 0;
// Receives the next inbound frame (either Response or Notification).
virtual mw::Task<mw::E<nlohmann::json>> recv() = 0;
virtual mw::Task<mw::E<void>> close() = 0;
};
} // namespace agent_smith::mcp
```
The `raw` fields retain the original server payload so callers can
inspect MCP fields that v1 does not model directly. `PromptRendering`
converts MCP prompt messages into Agent Smith `Message` values. Because
the PRD excludes multimodal inputs in v1, prompt content types that
cannot be represented as text messages fail with `McpError`.
The stdio transport implements `send`/`recv` directly because it is a
bidirectional stream. The HTTP transport may implement those methods
over an internal request map, but its primary primitive is
`sendRequest(frame)`: one `POST`, one response matched by JSON-RPC id.
### 11.2 Server configuration
`ServerConfig` carries transport, authentication, filtering, timeout,
and reconnect policy for one MCP server:
```cpp
enum class McpTransportKind
{
STDIO,
STREAMABLE_HTTP
};
struct StdioServerConfig
{
std::vector<std::string> argv;
};
struct HttpServerConfig
{
mw::URL endpoint;
};
struct ServerConfig
{
std::string alias;
McpTransportKind kind;
std::variant<StdioServerConfig, HttpServerConfig> transport;
std::optional<std::string> bearer_token;
std::vector<std::pair<std::string, std::string>> extra_headers;
std::chrono::milliseconds request_timeout{30'000};
std::vector<std::string> allow_tools;
std::vector<std::string> deny_tools;
bool tools_required = true;
bool reconnect_on_transport_error = true;
};
```
Filtering is applied after `tools/list` and before tools are adapted
into local `Tool` instances. `deny_tools` wins over `allow_tools`. If
`allow_tools` is empty, all discovered tools are allowed unless denied.
Reconnect is deliberately simple in v1: on a transport error, the
client may reconnect once for the failed high-level operation. It must
not automatically replay a tool call after a request was partially
written to a stdio server. Reconnect attempts, failures, and final
disconnects are surfaced through MCP observability events and returned
as `mw::E<>` failures when the operation cannot complete.
### 11.3 `Client`
```cpp
class Client
{
public:
static mw::Task<mw::E<std::unique_ptr<Client>>>
connect(Runtime& rt, ServerConfig cfg, EventHook hook = {});
// Capability discovery (called automatically by connect()).
const ServerInfo& info() const;
const std::vector<ToolSchema>& tools() const;
const std::vector<ResourceInfo>& resources() const;
const std::vector<PromptInfo>& prompts() const;
mw::Task<mw::E<ToolResult>>
callTool(std::string_view name, nlohmann::json args,
std::chrono::milliseconds timeout);
mw::Task<mw::E<std::string>>
readResource(std::string_view uri);
mw::Task<mw::E<PromptRendering>>
getPrompt(std::string_view name, nlohmann::json args);
void setEventHook(EventHook hook);
mw::Task<mw::E<void>> shutdown();
};
```
`connect` accepts an optional event hook for lifecycle events that occur
during initial connection. `setEventHook` lets an `Agent` forward later
MCP notifications and lifecycle events into `AgentHooks::on_mcp_event`
when a client is bound through a skill, direct registration, or a
sub-agent spec. Event hooks are best-effort observers like the agent
hooks in §16.3.
### 11.4 stdio transport (`transport_stdio.cpp`)
Implementation outline:
1. `connect()` calls `mw::Process::exec(input_pipe, argv,
output_pipe)` where both pipes are owned by the transport.
2. A reader task submits blocking reads to `Runtime::runOnWorker()`.
It uses `read(pipe.outputFD(), ...)` in a loop, splits on `\n`
(newline-delimited JSON per the MCP stdio profile), and pushes
parsed `nlohmann::json` frames into an `AsyncQueue<json>`.
3. `send(frame)` serializes the JSON, appends `\n`, and writes via
`Runtime::runOnWorker()`.
4. `close()` closes the pipes and `wait()`s the child.
Caveat: the libmw `Pipe::read()` reads until EOF, which is
unsuitable for line-delimited streaming. The transport reads raw
bytes via `::read(fd, buf, n)` and reuses a small accumulator
buffer. This is the only place where the framework reaches past
libmw's API — document it inline.
### 11.5 Streamable HTTP transport (`transport_http.cpp`)
MCP 2025-06-18 Streamable HTTP has a request/response path over `POST`
and optional long-lived server-to-client listening streams over `GET`.
v1 implements the request/response-compatible subset only. This is
enough for tool calls, resource reads, prompt rendering, initialization,
and request-scoped SSE responses. It is not a full implementation of
resumability, redelivery, or out-of-band server notifications.
This is the v1 compatibility level: Agent Smith can talk to Streamable
HTTP servers that complete agent-facing operations through `POST`
responses, including request-scoped SSE responses. Servers that require
a persistent `GET` listening stream for normal tool/resource/prompt
operation are outside v1 support.
1. `sendRequest(frame)` performs one async HTTP `POST` to the
configured MCP endpoint using `rt.http()`.
2. Include `Accept: application/json, text/event-stream` on every
request.
3. After initialization, include `MCP-Protocol-Version:
2025-06-18`.
4. If the server returns `Mcp-Session-Id`, store it and include it
on subsequent requests.
5. If the response content type is `application/json`, parse one
JSON-RPC response.
6. If the response content type is `text/event-stream`, consume SSE
events until the response for the request id is received.
7. After a successful `initialize` response, send
`notifications/initialized`.
Server notifications are surfaced only when they arrive on an active
request response stream. Optional `GET` listening streams, resumability,
and redelivery are deferred until a demo or test server requires them.
### 11.6 Capability discovery
After the transport is connected, `Client::connect` issues:
1. `initialize` with `protocolVersion: "2025-06-18"`, the library
name (`"agent_smith"`), and the version. Stores the server's
reported `serverInfo`.
2. `tools/list`, `resources/list`, `prompts/list`.
* Unsupported optional `resources/list` or `prompts/list`: warn and
continue with an empty list.
* Transport, authentication, parse, and protocol failures: return
`mw::E<>` failure.
* `tools/list` unsupported or failed: return failure when
`ServerConfig::tools_required` is true. Otherwise warn and continue
with no MCP tools.
3. Apply `allow_tools` / `deny_tools` filtering to discovered tools.
4. The filtered list of tools is exposed via `tools()`.
### 11.7 Adapting MCP tools to local tools
`mcp::tool_adapter::adapt(Client& c, ToolSchema s) ->
std::unique_ptr<Tool>` returns a tool whose `invoke` calls
`c.callTool(name, args, ctx.budget)`. The adapter:
* Prefixes the tool name with `mcp_<server_alias>__` to avoid
collisions across servers (configurable; default is the alias
given in the `ServerConfig`).
* Forwards MCP error results into `ToolResult::is_error` and a compact
textual/JSON error payload in `ToolResult::content`.
* Honors `ctx.cancel` by sending a JSON-RPC `notifications/cancelled`
to the server (optional in MCP 2025-06-18; if unsupported, just
stop awaiting).
### 11.8 Resources and prompts
Per PRD:
* Resources are exposed via a synthetic tool registered alongside the
MCP-adapted tools. The concrete tool name uses the same server-alias
prefixing rule as MCP tools:
`mcp_<server_alias>__read_resource(uri: string)`.
* Prompts are not auto-applied; the public API offers
`Client::getPrompt(name, args)` returning the rendered messages.
The resource tool adapter closes over the `mcp::Client&` for that
server, so routing is unambiguous even when an agent has multiple MCP
connections. Always using the prefixed name avoids changing tool names
when a second MCP server is added later.
Agents, skills, and sub-agents all bind MCP clients through the same
mechanism: discovered MCP tools and the synthetic resource reader are
adapted into local `Tool` instances and registered in the agent's
`ToolRegistry`. This keeps the execution path identical for native
tools, MCP tools, resource readers, and sub-agents.
### 11.9 Authentication
No OAuth, no token storage. Tokens never leave process memory and are
redacted from logs (see [§16.4](#164-redaction)).
---
## 12. Skill Subsystem
### 12.1 Definition
```cpp
// skill.hpp
namespace agent_smith
{
struct Skill
{
std::string name;
std::string system_prompt;
std::vector<std::unique_ptr<Tool>> tools;
std::vector<std::shared_ptr<mcp::Client>> mcp_clients;
nlohmann::json context_data = nullptr; // freeform extra material
};
} // namespace agent_smith
```
`mcp::Client` is `shared_ptr` because two skills may bind the same
MCP server connection.
### 12.2 Manifest loading
```cpp
mw::Task<mw::E<Skill>>
loadFromManifest(Runtime& rt, const std::filesystem::path& dir);
```
1. Read `dir/skill.json`. Required fields: `name`,
`system_prompt_file` (path relative to `dir`), `context_file`
(optional), and `mcp_servers` (optional list of `ServerConfig`).
2. Read the system prompt file as UTF-8 text.
3. If `context_file` is present, parse it as `nlohmann::json`.
4. For each entry in `mcp_servers`, asynchronously connect through
the provided `Runtime`.
5. Manifest-defined native tools are out of scope for v1. Native code
tools are registered from C++ code. This keeps manifest loading
simple and avoids loading executable code from data files.
### 12.3 Activation/deactivation
`Agent::activate(Skill)` and `Agent::deactivate(name)` mutate the
agent's effective tool set and concatenated system prompt. Activation
returns `mw::E<void>` because context-size validation and MCP tool
registration can fail.
Skill-owned native tools are exposed to the LLM with a stable prefix:
```
<normalized_skill_name>__<tool_name>
```
The normalized skill name contains only `[A-Za-z0-9_]`; manifest loading
and `Agent::activate` reject names that cannot be normalized
unambiguously. MCP tools bound through a skill keep the MCP prefixing
rule from §11.7, for example `mcp_<server_alias>__search`, because the
MCP server alias is already the relevant namespace.
The unprefixed original tool name remains inside the `Skill` object.
Prefixing happens when the skill is activated and the effective tool
registry is rebuilt. This keeps skill manifests readable while making
the LLM-visible tool names deterministic.
Agent-owned base tools are stored separately from active skills. After
every activation or deactivation, the agent rebuilds an effective
`ToolRegistry`:
1. Insert base tools in registration order.
2. Insert active skill tools in activation order, with the skill prefix.
3. Insert MCP-bound tools from active skills using their MCP prefixes.
After prefixing, a collision is still possible, for example two skills
whose names normalize to the same prefix. Per PRD, last-registered wins
and spdlog warns at `warn` level. Rebuilding the effective registry
keeps deactivation simple: when a skill is removed, any still-active
tool that was shadowed by that skill becomes visible again during the
next rebuild.
### 12.4 System prompt composition
When constructing the LLM request, the agent builds the system
message as:
```
<base_system_prompt>
# Skill: <skill1.name>
<skill1.system_prompt>
# Skill Context: <skill1.name>
<skill1.context_data.dump()>
# Skill: <skill2.name>
<skill2.system_prompt>
```
In activation order. Empty sections are omitted.
`Skill::context_data` is deterministic prompt context. It is not hidden
tool state, is not persisted, and is not treated as MCP resource
content. MCP resources remain available only through the synthetic
resource-reader tools described in §11.8.
When a skill is activated, non-null `context_data` is serialized with
`nlohmann::json::dump()` and compared against
`AgentConfig::skill_context_max_bytes` (default 16 KiB). If it exceeds
the cap, activation fails with `SkillError`; the library does not
truncate because that could produce invalid or misleading JSON context.
---
## 13. Memory Subsystem
### 13.1 Interface
```cpp
// memory.hpp
namespace agent_smith
{
class Memory
{
public:
virtual ~Memory() = default;
virtual void append(Message m) = 0;
// Build the messages slice to send to the LLM for this turn.
// base_system is provided by the agent; the implementation
// may prepend it, drop older entries, summarize, etc.
virtual mw::Task<mw::E<std::vector<Message>>>
snapshot(const SystemMessage& base_system) = 0;
virtual std::size_t size() const = 0;
virtual void clear() = 0;
};
} // namespace agent_smith
```
### 13.2 Built-in implementations
* `FullMemory` — never truncates. `snapshot` simply prepends the
system message.
* `RollingWindowMemory(max_messages | max_chars)` — keeps the most
recent N messages or trims oldest until char-count is below a
budget. The character estimate uses `~4 chars/token` for budgets
expressed in tokens, per PRD.
* `SummaryMemory(LlmClient&, threshold)` — when `size() >
threshold`, calls the LLM with a special "summarize this" system
prompt and replaces the older half with a single
`SystemMessage` containing the summary. The summarizing call
uses `complete` (non-streaming) to keep the implementation
simple. Failure to summarize falls back to rolling-window
truncation so the loop never stalls.
The rolling-window constructor accepts `RollingWindowConfig` with optional
`max_messages`, `max_chars`, and `max_tokens` limits; all configured limits
apply. A message-count convenience constructor is also available. Budgets
apply to stored history, excluding the separately supplied base system
prompt. Character accounting uses UTF-8 bytes of each serialized OpenAI
message, including tool arguments and metadata; token estimates allow four
such bytes per token. Trimming occurs on append, drops whole messages, and
discards a message that exceeds the entire budget. `size()` reports retained
messages, and `snapshot()` returns an independent copy without changing them.
### 13.3 Tool-message pairing invariant
A `ToolMessage` MUST always be preceded (eventually, with no
intervening assistant turn) by an `AssistantMessage` whose
`tool_calls` includes the matching `tool_call_id`. Truncation
strategies must respect this invariant — when dropping an
assistant message, drop the entire group of corresponding tool
results too. The base class provides a static helper
`Memory::stripOrphans(std::vector<Message>&)` that enforces this
post-truncation.
---
## 14. Sub-Agents
### 14.1 SubAgentTool
```cpp
// sub_agent.hpp
namespace agent_smith
{
struct SubAgentSpec
{
std::string name;
std::string description;
std::string system_prompt;
std::vector<std::function<std::unique_ptr<Tool>()>> tool_factories;
std::vector<std::shared_ptr<mcp::Client>> mcp_clients;
std::function<std::shared_ptr<Memory>()> memory_factory;
AgentConfig config;
};
struct SubAgentRuntimeState
{
int parent_depth = 0;
int max_depth = 3;
int tree_max_iterations = 50;
std::shared_ptr<AgentTreeState> tree_state;
};
std::unique_ptr<Tool> makeSubAgentTool(SubAgentSpec spec, Runtime& rt,
LlmClient& llm,
SubAgentRuntimeState state);
} // namespace agent_smith
```
The exposed schema is:
```json
{
"name": "<spec.name>",
"description": "<spec.description>",
"parameters": {
"type": "object",
"properties": {
"task": {"type": "string", "description": "Instruction for the sub-agent"}
},
"required": ["task"]
}
}
```
`SubAgentSpec` is a reusable factory specification, not a bag of
already-owned runtime objects. A `SubAgentTool` may be invoked multiple
times and may be invoked concurrently, so each invocation constructs
fresh per-agent state:
1. Call `memory_factory` to create a fresh `Memory`; if it is empty,
use `std::make_shared<FullMemory>()`.
2. Call every `tool_factories` entry to create fresh `Tool` instances
for this child agent.
3. Construct an `Agent` instance using the spec, shared `Runtime`, and
shared `LlmClient`.
4. Register the fresh tools and MCP-bound tools.
5. Send the `task` as a user message, run the loop, and return the
final assistant `content` as `ToolResult.content`.
MCP clients are intentionally shared with `std::shared_ptr<mcp::Client>`
because a single server connection may be bound into several skills or
sub-agents. `mcp::Client` must therefore make concurrent `callTool`,
`readResource`, and `getPrompt` calls safe by correlating JSON-RPC ids
and protecting transport writes with an internal mutex. If a transport
cannot support concurrent in-flight calls, the client serializes them
internally.
### 14.2 Depth and iteration counter
The PRD requires two independent guards: recursive depth and total loop
iterations across the sub-agent tree. Root agents start at depth `0`.
With the default `max_subagent_depth = 3`, child depths `1`, `2`, and
`3` are allowed. Setting `max_subagent_depth = 0` disables sub-agent
creation.
`AgentConfig` carries:
```cpp
int max_subagent_depth = 3;
int current_subagent_depth = 0;
int tree_max_iterations = 50;
std::shared_ptr<AgentTreeState> tree_state;
```
The root `Agent` constructs `tree_state` if the caller did not provide
one. Child agents receive the same `tree_state` and
`current_subagent_depth = parent.current_subagent_depth + 1`. The shared
state is intentionally small: one atomic iteration counter and one
cancellation source for the whole sub-agent tree.
Before constructing a child agent, `SubAgentTool::invoke` checks:
```cpp
if(state_.parent_depth >= state_.max_depth)
{
co_return ToolResult{
.content = "sub-agent depth limit reached",
.is_error = true,
};
}
```
Depth-limit failures are returned as tool errors and fed back to the
parent agent, because the failed operation is a tool invocation. The
shared tree iteration cap is a global runaway-loop guard: each loop
iteration of any agent in the tree increments
`tree_state->iterations`; at `tree_max_iterations` (default 50), any
further iteration returns `agent_smith::error::cancel("subagent tree
iteration cap")`.
`SubAgentRuntimeState` is passed into `makeSubAgentTool` when the parent
agent registers the sub-agent tool. `ToolContext::cancel` still carries
the per-call parent cancellation token at invocation time, so
agent-specific depth and tree-counter fields do not need to be added to
every `ToolContext`.
### 14.3 Cancellation propagation
Each `Agent` owns a `CancellationSource`. The root agent also owns, or
creates, the shared `AgentTreeState`. Sub-agent tools receive the parent
token through `ctx.cancel`, forward cancellation to their child agent's
source, and cancel `tree_state->cancel` so live descendants in the same
tree can observe the same shutdown signal.
---
## 15. Agent Core (the loop)
### 15.1 Public API
```cpp
// agent.hpp
namespace agent_smith
{
struct AgentTreeState
{
std::atomic<int> iterations{0};
CancellationSource cancel;
};
struct AgentConfig
{
int max_iterations = 16;
std::optional<std::chrono::seconds> deadline;
std::optional<std::uint64_t> max_total_tokens;
std::chrono::milliseconds default_tool_timeout{30'000};
std::size_t tool_result_max_bytes = 64 * 1024;
std::size_t skill_context_max_bytes = 16 * 1024;
bool stream_default = true;
int max_subagent_depth = 3;
int current_subagent_depth = 0;
int tree_max_iterations = 50;
std::shared_ptr<AgentTreeState> tree_state;
};
struct AgentHooks
{
std::function<void(const Message&)> on_message;
std::function<void(const ToolCall&)> on_tool_call;
std::function<void(const ToolMessage&)> on_tool_result;
std::function<void(const StreamEvent&)> on_stream_event;
std::function<void(const mcp::Event&)> on_mcp_event;
std::function<void(const mw::Error&)> on_error;
};
struct AgentRequest
{
UserMessage message;
SamplingParams sampling;
std::optional<bool> stream;
};
class Agent
{
public:
Agent(Runtime& rt, LlmClient& llm,
std::shared_ptr<Memory> mem,
AgentConfig cfg = {});
void setSystemPrompt(std::string s);
mw::E<void> activate(Skill skill);
void deactivate(std::string_view name);
mw::E<void> addTool(std::unique_ptr<Tool> t,
ToolOptions opts = {});
mw::E<void> addMcpClient(std::shared_ptr<mcp::Client> client);
mw::Task<mw::E<void>> connectMcp(mcp::ServerConfig cfg);
AgentHooks& hooks();
// Send one user message and run the loop until a final
// assistant message (no tool calls) or a stop condition.
mw::Task<mw::E<AssistantMessage>> send(std::string user);
// Send one user message with per-turn sampling and stream options.
mw::Task<mw::E<AssistantMessage>> send(AgentRequest req);
};
} // namespace agent_smith
```
### 15.2 The loop, step by step
Milestone 10 implements native tools and the memory/LLM loop; skill and MCP
registration arrive in their respective milestones. Null constructor memory
selects `FullMemory`. `Agent::cancel()` permanently cancels the agent and its
shared tree. Tool tokens observe tree cancellation, and streaming requests
abort at the next received chunk. Loop deadlines are checked between awaited
operations, without preempting an in-flight request. Iteration and token
counts reset for each send; the shared tree counter persists with its state.
When several tools return a successful structured `_done` result, the first
in call order supplies the final message, after all tool results are recorded.
```
1. Normalize the input into an `AgentRequest` named `agent_req`. The
`send(std::string)` overload creates `{UserMessage{user}, {}, {}}`.
2. Append `agent_req.message` to memory (via mem->append).
3. iter = 0; total_tokens = 0;
If cfg.tree_state is null, create one.
deadline_tp = now + cfg.deadline (if set).
4. LOOP:
a. iter += 1; tree_state->iterations += 1.
b. If iter > cfg.max_iterations OR
tree_state->iterations > cfg.tree_max_iterations OR
(cfg.max_total_tokens is set AND
total_tokens > *cfg.max_total_tokens) OR
now > deadline_tp: return error::cancel(...).
c. snapshot = co_await memory.snapshot(base_system_with_skills).
d. tool_schemas = registry.schemas().
e. llm_req = LlmRequest{snapshot, tool_schemas,
agent_req.sampling}.
f. If agent_req.stream.value_or(cfg.stream_default):
resp = co_await llm.completeStream(
llm_req, hooks.on_stream_event).
else:
resp = co_await llm.complete(llm_req).
On error, fire hooks.on_error and return it.
g. If resp.message.usage is present, add usage.total to
total_tokens.
h. Append resp.message to memory; fire hooks.on_message.
i. If resp.message.tool_calls is empty: return resp.message.
j. For each tool_call: fire hooks.on_tool_call.
k. Run all tool invocations concurrently (see §10.4).
l. For each ToolMessage produced, append to memory and fire
hooks.on_tool_result.
m. continue LOOP.
```
### 15.3 Stop conditions
* **Iteration cap.** Per `cfg.max_iterations`.
* **Tree iteration cap.** Per `cfg.tree_max_iterations`.
* **Token cap.** Per `cfg.max_total_tokens`, using server-reported
`usage.total`. If the server does not report usage, this stop
condition is best-effort and does not estimate missing tokens.
* **Deadline.** Per `cfg.deadline`.
* **Explicit "done" signal.** A tool may return a `ToolResult`
whose `structured` field contains `{"_done": true,
"final_message": "..."}`. The agent treats this as a final
assistant message and exits.
### 15.4 Concurrency restated
* One coroutine drives one `Agent` instance at a time.
* `complete*` and MCP Streamable HTTP use `Runtime::http()` directly.
* Synchronous native tools and MCP stdio operations use
`Runtime::runOnWorker()`.
* Hooks fire synchronously from the agent coroutine. They are
best-effort observers and are expected not to throw. All hook
invocations go through a single internal helper that catches
`std::exception` and unknown exceptions, logs a warning, and continues.
Hook failures are never converted into agent or tool failures. Hooks
must be fast and non-blocking; the PRD makes this explicit.
---
## 16. Observability and Logging
### 16.1 Logger naming
The library acquires a logger named `agent_smith` via
`spdlog::get("agent_smith")`. If `nullptr`, the log call is skipped.
The library does not fall back to `spdlog::info(...)`,
`spdlog::default_logger()`, or `spdlog::null_logger_mt(...)`, because
those would route logs through caller-global state or register logger
state from inside the library. **It never registers a logger, registers
a sink, or configures a level.** Per PRD, sink/level configuration is
the caller's responsibility.
Sub-loggers used inside subsystems:
* `agent_smith.llm`
* `agent_smith.tool`
* `agent_smith.mcp`
* `agent_smith.skill`
* `agent_smith.agent`
Subsystem code follows the same rule: call `spdlog::get(...)` for the
specific logger name, and skip the log call when it is not registered.
### 16.2 Levels
| Level | Used for |
| ------- | --------------------------------------------------- |
| `trace` | Wire-level JSON: full request/response bodies (redacted), every SSE frame, every JSON-RPC frame. |
| `debug` | Loop step labels, tool dispatch decisions, retry decisions. |
| `info` | Lifecycle: client created, MCP connected, skill activated. |
| `warn` | Tool-name collision, optional MCP method missing, retry triggered. |
| `error` | Final, surfaced-to-caller errors. |
### 16.3 Hooks
Already covered in [§15.1](#151-public-api). Hooks are **not** an
async I/O channel; they are callbacks. They are useful for UIs,
metric counters, and for the demo CLI to render the streaming token
stream.
Hooks are best-effort observers. They must be fast and should not
throw. Every hook invocation goes through a single internal helper:
```cpp
template <typename F, typename Arg>
void invokeHook(F& hook, const Arg& arg);
```
The helper returns immediately when the hook is empty, catches
`std::exception` and unknown exceptions, logs a warning, and continues.
Hook failures are not converted into agent, tool, MCP, or LLM failures.
### 16.4 Redaction
Before any header dictionary or body string is logged, run it
through `agent_smith::internal::redact`:
* Replace any value of header keys equal (case-insensitive) to
`Authorization`, `X-Api-Key`, `Api-Key`, `Cookie`, or
`Set-Cookie` with `"<redacted>"`.
* In JSON bodies, replace values whose keys match
`/api[-_]?key/i`, `/access[-_]?token/i`, `/bearer/i`, or
`/password/i` with `"<redacted>"`.
A `tests/test_redaction.cpp` file pins these rules.
---
## 17. Cancellation and Timeouts
### 17.1 Cancellation primitive
Use a small framework-owned cancellation pair:
```cpp
// cancellation.hpp
class CancellationToken
{
public:
bool cancelled() const;
};
class CancellationSource
{
public:
CancellationToken token() const;
void cancel();
};
```
The implementation can be an atomic shared state. It should remain
independent from any particular coroutine scheduler.
### 17.2 Where cancellation is checked
* Before each loop iteration (cheap, deterministic).
* Before and after each awaited framework operation.
* Before submitting synchronous native tools or stdio work through
`Runtime::runOnWorker()`.
* Streaming: the HTTP chunk callback returns `false` if cancellation
is observed. The LLM client maps the resulting HTTP abort to
`CancelledError`.
* Synchronous tools and stdio operations may only observe
cancellation at explicit checks or framework-defined suspension
points.
* Parent cancellation propagates to live sub-agents by cancelling
their `CancellationSource`.
### 17.3 Timeouts
Three nested budgets, all configurable:
* HTTP request timeout — set on the `mw::HTTPRequest` or
`mw::HTTPSessionAsync` request options, according to libmw's API.
Default 60 s.
* Tool-call timeout — enforced by racing the tool task against the
framework timer primitive. The default is
`AgentConfig::default_tool_timeout` (30 s). A per-tool override is
stored in `ToolOptions::timeout` at registration time.
* Whole-agent-loop deadline — checked at the top of each loop
iteration; a separate timer is **not** used in v1 to keep
cancellation paths simple.
---
## 18. Testing Strategy
### 18.1 Unit tests
* **Pure logic**, no I/O: SSE parser, redaction, schema
composition, JSON conversion, memory truncation, tool-result
truncation.
* **HTTP**: prefer libmw's async HTTP mock if one exists. Otherwise
use the narrow Agent Smith `HttpClient` test seam from §5.6.
* **MCP stdio**: spawn `python3 demo/mcp_server_example.py` as a
child via `mw::Process::exec` in tests. Tests should be opt-in
(CMake option `-DAGENT_SMITH_TESTS_NEED_PYTHON=ON`) so CI without
Python still builds.
### 18.2 Integration tests
A small test that runs against a real `llama.cpp` server is
gated behind `-DAGENT_SMITH_TESTS_NEED_LLAMA_SERVER=ON` and
expects an env var `LLAMA_SERVER_URL`. CI does not run it by
default.
### 18.3 Property-style tests for the SSE parser
Use a hand-written byte fuzzer that splits a known SSE stream at
every possible byte boundary and asserts the parser emits the same
events regardless of split. This catches the "data:" prefix split
across chunks bug, which is the most common SSE parser failure
mode.
### 18.4 Coverage targets
Library code (`src/`) ≥ 80% line coverage in CI, measured via
`gcovr`. Demo and MCP transport HTTP code paths need special
coverage exemptions until an integration env exists.
---
## 19. Demo CLI
`demo/main.cpp` builds an `agent_smith_demo` binary that:
1. Reads `LLAMA_SERVER_URL` (default `http://127.0.0.1:8080`) and
optional `LLAMA_API_KEY`.
2. Constructs a `Runtime`, an `LlmClient`, a `FullMemory`, and an
`Agent`.
3. Registers two C++ tools:
* `calculator(expression: string) -> number` —
evaluates a tiny arithmetic expression (no `eval`; a
hand-written shunting-yard parser).
* `read_local_file(path: string) -> string` — reads a
file under a hard-coded sandbox directory (e.g. `./demo_fs`)
and refuses any path that escapes via `..`.
4. Connects to the bundled `mcp_server_example.py` via stdio.
5. Enters a REPL: each user line is forwarded to `agent.send`; the
`on_stream_event` hook prints token deltas to stdout; the
`on_tool_call` and `on_tool_result` hooks print short status
lines to stderr.
The Python MCP server uses the
[official mcp-server-python](https://github.com/modelcontextprotocol/python-sdk)
quickstart to expose two trivial tools (`echo` and `now`).
---
## 20. Implementation Order (Milestones)
The intern should implement in this order; each milestone produces a
passing test suite before the next begins. The internal concurrency
foundation comes early because LLM streaming, parallel tool dispatch,
timeouts, and MCP stdio all depend on the same primitives. Do not build
separate one-off runners for each subsystem.
| # | Milestone | Deliverable |
| - | ------------------------------------ | -------------------------------------------- |
| 1 | Skeleton + CMake + libmw integration | `Runtime` builds, empty `Agent` constructs. |
| 2 | Internal concurrency core | `Runtime::runOnWorker`, awaitable `StartedTask`, `AsyncQueue`, timer/sleep, and `gatherAll` tested. |
| 3 | Message model + JSON serialization | `tests/test_message.cpp` green. |
| 4 | SSE parser (offline) | `tests/test_sse_parser.cpp` green. |
| 5 | LLM client non-streaming | `tests/test_llm_client.cpp` with `mw::HTTPSessionAsyncMock` green. |
| 6 | LLM client streaming | `StartedTask` + `AsyncQueue` stream events while HTTP is active. |
| 7 | Tool subsystem | Sync adapters use `Runtime::runOnWorker`; async adapters are awaited directly. |
| 8 | Parallel tool dispatch + timeouts | Ordered `gatherAll` results and per-tool timeout tests green. |
| 9 | Memory: Full + RollingWindow | `tests/test_memory.cpp` green. |
| 10| Agent loop | Non-streaming and streaming loop tests green. |
| 11| MCP JSON-RPC + stdio transport | `tests/test_mcp_jsonrpc.cpp` green; demo MCP works. |
| 12| Demo CLI | End-to-end manual verification. |
The following capabilities are useful but should not block the first
stable slice. Implement them after the core loop and demo are working,
or keep them experimental until they have integration coverage:
| # | Milestone | Deliverable |
| - | ------------------------------------ | -------------------------------------------- |
| 13| Skill subsystem | `tests/test_skill.cpp` green. |
| 14| Sub-agents | `tests/test_sub_agent.cpp` green. |
| 15| MCP HTTP transport | Optional integration test only. |
| 16| SummaryMemory | Manual demo with long history. |
---
## 21. Open Questions / Deferred to v2
* **Stronger cancellation semantics for in-flight HTTP**. v1 is
limited to what `mw::HTTPSessionAsync` exposes.
* **JSON Schema generation from C++ types** — explicitly deferred.
Look at [Boost.Describe](https://www.boost.org/doc/libs/release/libs/describe/)
or [glaze](https://github.com/stephenberry/glaze) when revisiting.
* **Token-accurate budgeting**. Would require shipping a tokenizer
(e.g. [tiktoken-cpp](https://github.com/openai/tiktoken)) or
invoking the server's `/tokenize` endpoint where exposed.
* **MCP sampling and roots** — explicit non-goals for v1.
* **Persistence** — explicit non-goal; document how a caller would
serialize `std::vector<Message>` with `nlohmann::json` if they
want it.
* **Windows port** — non-goal. The major blocker is `mw::Process`
using POSIX `fork`/`execvp`.
---
*End of design-0-framework.md*