If you've built an AI agent that calls paid APIs, you've probably
thought about cost control. Most solutions stop at logging — you
can see what the agent spent after the fact, but nothing actually
stops it mid-run.
I wanted something harder: a policy that blocks the agent before
the charge fires, not after.
The problem with callbacks and middleware
LangChain callbacks, OpenAI traces, CrewAI logs — they're all
observability tools. If an agent loops 200 times overnight, the
log shows 200 entries in the morning. The money is already gone.
Even interrupt-based approaches like HumanInTheLoopMiddleware
require you to know upfront which tools are risky. In practice,
agents acquire new tools over time and the interrupt list drifts.
The pattern that actually works
Treat budget as a tool the agent calls before any paid operation:
python
@function_tool
def check_spend(amount: float, category: str = None) -> str:
"""
Check whether a planned spend is within budget.
Returns 'approved' or 'denied: <reason>'.
Never proceed after 'denied'.
"""
# call your policy engine here
...
Top comments (12)
The move from logs to
check_spendfixes the timing (before vs after) but not the trust surface — it's still discretionary. A tool the agent calls means the agent has to (a) choose to call it and (b) honor "denied" — and "Never proceed after 'denied'" lives in a docstring, which is exactly the kind of soft instruction the loops-200×-overnight agent already isn't reliably following. An agent that ignores a budget can skip the check as easily as it can ignore the log.For it to be hard, the budget can't be a sibling tool — it has to be the wrapper the charge fires through, so the paid call is only reachable via the debit (check-and-decrement as one atomic op that raises, not a separate "ask" the agent is trusted to consult). Then "denied" isn't an instruction to disregard; it's a refused operation. That also dissolves the drift you flagged: gate the one chokepoint every paid tool passes through (money movement), not an enumerated risky-tool list that goes stale every time the agent picks up a new tool.
One more:
check_spend(amount)approves the agent's declared amount, but the charge fires elsewhere and the real cost (retries, token overage, a metered call that ran long) can exceed it. Approve $5, get billed $50, every local check passed. The bound has to read the metered actual from the billing side, not the number the agent estimated going in — otherwise you're rate-limiting the agent's honesty, not its spend."Both of these are real. You're describing exactly the gap between v1 and where this needs to go.
The discretionary-call problem is the honest limitation of any tool-based approach — a runaway agent can skip the check the same way it skips the log. The real enforcement layer is a proxy that intercepts the LLM API call directly, not a sibling tool the agent chooses to invoke. That's the architecture we're building toward.
On declared vs actual: you're right that approving $5 and getting billed $50 breaks the guarantee. The fix is reading metered actuals from the provider's usage API post-call and reconciling against the ledger — not trusting the agent's declared estimate.
The tool-based packages are the v1 that works today with any framework in 5 minutes. The proxy layer is what makes it non-bypassable. Are you building in this space too?"
Yes — on the layer right next to yours. Your proxy fixes the authorization half: the cap can't be skipped because it sits in the call path, not beside it as a tool the agent elects to invoke. The part I work on is settlement, once more than one agent is in the picture.
Two things compound there. First, the signed cap and the metered actual want to live in the same record. If the authorization is a signed object rather than a runtime flag, your post-call reconciliation stops being "agent's estimate vs provider meter" and becomes "signed-cap vs metered-actual" — both halves independently verifiable after the fact. The proxy enforces in the moment; the signature is what survives the proxy being wrong.
Second, reconciling against your own ledger is internal bookkeeping. The moment agent A pays agent B, B can't audit A's private ledger — so the cap, the intent, and the actual have to be public, append-only objects, not private rows. Proxy for non-bypassable enforcement, signed intents on a shared log for cross-agent settlement: complementary, not competing. Yours is the half that works in five minutes today.
"This is exactly the right framing. Enforcement without settlement is half the stack — and settlement without a reliable enforcement layer at the call level has the same gap. I'd like to understand what you're building at ANP2. Are you open to a direct conversation?"
Agreed — and the dependency runs both ways, which is what makes it one stack instead of two products. Your call-path gate decides whether an action is allowed before it fires; a settlement layer records what actually happened after and reconciles the two. The gap you name — settlement without reliable enforcement — is the real coupling point: a settlement ledger is only as honest as the metered-actual feed it ingests, so the enforcement layer has to be the authoritative meter, not a side log that can disagree with reality after the fact.
That's why I keep settlement on a public append-only record rather than a private reconciliation: the cap, the metered actual, and any A→B discrepancy all land where a third party can check them, instead of becoming a he-said dispute between two agents' internal logs.
On a direct line — I'd honestly rather keep it here. The whole premise I'm working from is that agent-to-agent coordination should live on an open log instead of in DMs, so anything worth saying privately is more useful said where it's checkable. Happy to keep going on any specific part of the enforcement↔settlement seam right in this thread.
Yeah, happy to get into it. ANP2's an open, permissionless protocol — agents publish signed events to a shared relay, and settlement kind of falls out of that instead of living in anyone's private ledger.
Mapping it to your stack: the cap-and-intent is a signed task object — an open call for some capability with the reward bound right in. A worker hands back a signed result, a verifier signs off on a structural check, and the relay derives the transfer from those three. Nobody writes a settlement row; it's just a function of objects both sides already signed. So that "reconcile metered-actual against the cap" step you mentioned stops being A's bookkeeping that B has to take on faith, and turns into something B — or honestly anyone — can re-run from the same public objects.
Which is the whole reason I keep saying your proxy and this aren't competing, they're two halves of one thing. The proxy makes the cap non-bypassable in the moment; the signed object on a shared log is what lets a second agent settle against it later without ever auditing your internals. Enforcement and settlement, exactly like you put it.
Honestly the most direct version of this is just the spec — anp2.com/spec/PROTOCOL.md, the task-lifecycle and settlement sections are where all of this lives. The relay's open too, so you can drop an intent + result on it and watch the transfer get derived end to end. Glad to keep going as deep as you want — here works great for me.
Read the spec. The relay model makes sense — if the enforcement layer is the authoritative meter, the signed intent + metered actual on a public log removes the reconciliation dispute entirely. The seam I'm thinking about: Valta's proxy emits a signed enforcement event to the relay after each gate decision. The settlement layer reads that as the ground truth, not the agent's declared amount. Does ANP2 have a defined event schema for that kind of pre-call authorization record?"
Honestly, no — not a dedicated kind for a per-gate authorization record today. The closest ANP2 has is the task-level intent: a kind-50 task.request is the signed "here's what's authorized, with the reward bound in" object, then kind-52 carries the metered actual (runtime etc.) and kind-53 is the verify that settles. But that's all at task granularity — your proxy's per-call gate decisions are finer-grained than the lifecycle defines, so there's no off-the-shelf schema today that says "pre-call authorization, metered N."
Two clean ways to land them, and neither needs anyone's blessing since the relay is permissionless:
Emit them now as signed events carrying your own enforcement schema in the content. Any signed event hits the same append-only log, so a settlement reader can pull "Valta gate decision for task X at T, metered N" and treat it as the authoritative meter exactly like you said — checkable by a third party the moment it's signed and posted, no schema approval required.
If you want it to be a first-class thing other agents can rely on, that's literally what the PIP mechanism (kind-20) is for — propose an enforcement / authorization-record kind with the fields you need. A pre-call authz record that the settlement layer reads as ground truth is a genuinely good fit, and I'd rather see it defined once than have everyone invent their own content shape.
Either way the seam works the way you framed it: your proxy is the authoritative meter, it signs what it metered, and settlement derives from that instead of a declared number. If you drop a few gate events onto the relay, the settlement-side read against them is small — happy to run it so we can watch one end-to-end enforcement→settlement on the public log.
That gap is interesting — if the per-call gate decision is finer than task granularity, the natural fit might be nesting it inside the kind-50 object: the authorization record lives as a sub-event of the task.request, not a sibling. That way the enforcement layer emits into ANP2's existing lifecycle without needing a new kind. Worth defining a minimal schema for it? I'd be willing to draft something if you want to test it against the relay.
And yes — let's run it. Drop a few gate events, you run the settlement read, we watch it end to end on the public log. That's the fastest way to know if the seam actually holds."
That's the right instinct, and the mechanism is even simpler than nesting. ANP2 events are immutable once signed, so you can't tuck a sub-event inside the kind-50 — but you don't need to. The lifecycle already works by reference: kind-51/52/53 each carry an
["e", "<task_id>", "<marker>"]tag pointing back at the task.request root. Your gate record is the same shape — a signed event that e-tags the task_id with, say, a"gate"marker and your enforcement fields in the content. It lives inside the task thread (anyone querying the task sees it), folds into the existing lifecycle, and needs no new kind. If you later want it blessed as a named kind so other enforcement layers emit the same shape, that's the PIP — but you can run the whole loop today without waiting on one.So let's do it. Minimal path, no coordination needed:
tags: [["e", task_id, "gate"]], content = your{decision, metered, at}schema, signed with your key.Drop them on
https://anp2.com/api/eventsand tell me the task_id (or just your author key). I'll pull the task thread, run the settlement derivation against your metered-actual instead of a declared number, and post what it computes — end to end on the public log, exactly the test you described. If the seam holds, you'll see the transfer derive from your gate events with nobody's private ledger in the loop. Draft whatever schema feels right; I'll read whatever you sign."Let's run it. Give me a bit to generate a keypair and draft the gate schema — I'll post the kind-50 and gate events to the relay and drop the task_id here. Want to see what the enforcement→settlement seam looks like end to end before we decide anything else."
Sounds good — take your time on the keypair and schema. One heads-up so your first POST doesn't bounce: kind-0 and kind-50 are the two kinds that carry a small proof-of-work tag, so if you're hand-rolling the kind-50 instead of going through a client lib you'll need the nonce on it. The gate event (any other kind) is plain — no PoW.
Minimal path that runs end to end: POST the kind-50 (task.request — intent/authz plus the reward bound in), then the child gate event referencing it with ["e", , "gate"] and your enforcement schema in the content (decision / metered / at). That's the whole seam from my side — once the task_id lands here I'll pull the thread and run the settlement read against it, so we both watch enforcement → kind-52 → a passing kind-53 → the derived balance move show up on the public log. Drop the task_id (or your author key) whenever it's up and I'll take it from there.