the stack · a field guide
The same ideas behind TokenOps MCP — exact cost, graduated budget guard, cheapest model — plug into any agentic framework with four OSS packages. TypeScript or Python.
the 4 layers
ledger · budgets · local agents
The product you already saw: exact cost ledger (SQLite), 4-state budget guard and route_llm for your local agents — OpenCode, Claude Code, Cursor, Windsurf.
steering · which model
Model routing brain. Picks the cheapest / most reliable model for a task, falls back when one is down or rate-limited. The engine behind route_llm.
guard · enforcement
The graduated circuit breaker: allow → warn (80%) → degrade (95%) → block (100%). Runs in-process, before and after every LLM call. No proxy, zero extra hops.
observability · audit
Trace wrapper that captures latency, tokens and exact cost per call, user and session. Feeds Grafana / OTel / MLflow. The dashboard side of run_audit.
| tokenops mcp tool | cloud component |
|---|---|
| route_llm | styrr — which model is cheapest for this task |
| set_budget + guard (warn→degrade→block) | sayay — enforcement at the call site |
| track_usage · get_budget · session_report | qhaway spans + sayay storage |
| run_audit | qhaway dashboards · OTel / MLflow / Grafana exporters |
tokenops-mcp = local, single-user (your machine, ~/.tokenops/usage.db). sayay + styrr + qhaway = in the process of your deployed, multi-tenant agents.
the pattern
Every framework reduces to the same shape: a routing layer that picks the model, a guard that checks before and records after, and a trace wrapper. Here it is, generic:
// 1 · decide — styrr (steering)
const router = new StyrRouter({ models, apiKey })
// 2 · enforce — sayay (guard)
const guard = new SayayGuard({ storage, budget: { dailyUsd: 5 } })
// 3 · wire it together
async function call(userId, prompt) {
const d = await guard.check(userId, estimateCost(prompt))
// → allow | warn | degrade | block
if (d.action === 'block') throw new Error(d.reason)
const res = await router.prompt(
prompt,
d.action === 'degrade' ? { models: [d.suggestedModel] } : undefined,
)
await guard.record(userId, costFromUsage(res.usage))
// → exact tokens from the response, never an estimate
return res
}
// 4 · observe — qhaway
const traced = trace.wrap(call) // latency, tokens, cost, user, sessionframework recipes
the one that ties it — strands agents, ts, in one block:
import { Agent } from '@strands-agents/sdk'
import { StyrModelProvider } from '@carloscortezcloud/styrr-strands'
import { SayayPlugin } from '@carloscortezcloud/sayay-guard'
const agent = new Agent({
model: new StyrModelProvider({ // steering
models: [
'anthropic.claude-opus-4-7',
'meta-llama/llama-3.3-70b-instruct:free',
],
apiKey: process.env.OPENROUTER_API_KEY,
}),
plugins: [new SayayPlugin({ // guard
budget: { dailyUsd: 5.0 },
userId: 'user-123', onExceeded: 'degrade',
degradeModel: 'meta-llama/llama-3.3-70b-instruct:free',
})],
systemPrompt: 'You are a helpful assistant.',
})ts vs python
| concern | typescript | python |
|---|---|---|
| steering (routing) | @carloscortezcloud/styrr-llm | styrr (PyPI) |
| strands adapter | @carloscortezcloud/styrr-strands | StyrRouter como model |
| langchain adapter | @carloscortezcloud/styrr-langchain | styrr.langchain.StyrLLM |
| budget guard | @carloscortezcloud/sayay-guard | sayay (PyPI) |
| observabilidad | @carloscortezcloud/qhaway | qhaway (PyPI) |
¿styrr, sayay o qhaway? No es "o" — es "y", según el caso:
routing / elegir modelo → styrr · enforcement de budget → sayay · tracing / audit / dashboards → qhaway
en python: pip install styrr + sayay, y qhawaycuando el caso es observabilidad. El guard y el router son ortogonales — se combinan, no compiten.