generic skill
Sign in to like and favorite skills
A lot of “agent reliability” advice is basically prompt hygiene.
It helps.
But it also avoids the thing that actually breaks in production:
your tool contract is a moving target.
When a tool’s inputs/outputs shift (even slightly), you don’t get a nice compiler error. You get a model that keeps talking confidently while the system silently drifts.
If you’ve ever watched an agent:
…you’ve already met the real failure mode.
People say “the schema changed” as if it’s one thing. It isn’t.
Here are five changes that look small in a diff, but don’t feel small at 3am:
Renames
customer_ididAdded required fields
workspaceIdSemantic drift
limitBehavior drift
Policy drift
That last one is why I treat contracts as security work, not “API documentation”.
The goal is boring: when something changes, your system should fail loudly before the agent starts improvising.
My baseline:
If you’re tempted to say “the model can handle it”… yeah, it can handle it right up until it can’t.
A versioned contract is how you keep errors in the domain of “software bugs”, not “AI mysteries”.
If I saw something like this in an agent system, I’d relax a little:
search_customers_v2(query: string, limit: int) -> { customers: Customer[], next_cursor?: string }
Two tiny details matter:
_v2(And yes: if the tool is versioned, your monitoring should be too.)
Versioning without migration is just a fancy way to accumulate debt.
So I usually want three rules:
That third one is the real cost. You need test cases that represent what your agent actually does.
Not “unit tests for JSON”.
Test cases for:
Because the scary thing isn’t “wrong output”.
The scary thing is plausible output.
If your agent calls tools that touch real systems, I’d want:
Prompts are not contracts.
They’re a user interface.
Contracts are what keep your agent from turning “a minor API change” into “a weird incident”.