Blog

Limiting the number of turns in agent loops through manual caching

I’m creating my own reusable skills. This week I learned something more about the ergonomics: it can be worth it to design what is loaded into context not just around limiting the size, but also around how many turns and tool calls are needed, reducing latency while using the skill.

For example, in the verification step of my /setup skill, I wanted the AI to double-check it had properly filled in all <placeholders> when processing a template. Initially I had this:

Syntactic: did I write valid files? The characteristic failure is an unfilled placeholder. Check it precisely: extract the placeholder tokens the template itself declares, then assert none of them survive in the file that was written.

For example, for the README, look for placeholder tokens the template declares:

SKILL_DIR=[skill base directory]
grep -ohE '<[^>]{1,80}>' ${SKILL_DIR}/assets/README-template.md | sort -u
# -> <how to use the project> <medium description> <Name> <other sections>
#    <pointers at docs/> <project> <short description>
And then assert each of those is absent from the actual README.md.

A bare grep ‘<’ is the wrong check — it false-positives on ordinary prose and on the comments inside the mise templates.

These instructions started out as codifying how the AI had been instructing itself to do the verification, taking into account that the template can change and that there’s multiple templates. But having the AI invoke bash to run grep to then process a list of placeholder tokens further is slowing things down for no good reason.

While the skill its templates do change, they do not change often, and when they do, they go into GitHub and through CI/CD tooling. So the grep commands can be run once, without AI involved, and the results can be inlined into the skill file. The result becomes:

Syntactic: did I write valid files? The characteristic failure is an unfilled placeholder. Check it precisely: for every file written from a template, assert that none of the tokens that template declares survive in it.

The tokens, by template:

<!– placeholders: assets/ –>

assets/CODE_OF_CONDUCT-template.md
  ...
assets/README-template.md
  <how to use the project>
  <medium description>
  <Name>
  <other sections>
...

<!– /placeholders –>

That list is generated from the templates; a template not listed declares no placeholders. If the list is empty, this skill is broken: stop and report it rather than reading no tokens as no placeholders.

A bare grep '<' is the wrong check — it false-positives on ordinary prose and on the comments inside the mise templates.

A lot of the words for the initial skill and also for this update rewrite was done by AI. It also wrote the script that keeps the list updated, and suggested this specific improvement in the first place.

You can tell this is AI-written, I think. Besides the em-dash, another clear tell is the sentence “if the list is empty, this skill is broken: stop and report it rather than reading no tokens as no placeholders”. Opus 5 is often too careful when it is trying to instruct other AI; I think the CI setup is good and so I don’t think it is likely I will mess this up.

In software development we learned “premature optimization is the root of all evil” and then “There are only two hard things in Computer Science: cache invalidation and naming things“, with caching being one kind of optimization. So we developed a sense of taste to avoid caching until really needed.

But now with agentic coding, I find myself reaching for such caches and other optimizations sooner, to limit AI cost but especially to limit AI latency. I feel that, when I keep my agent skills fast, I reach for them more, my AI stays on the rails better, I experience less in-session frustration, and then I can spend my cognitive energy on judging the things that matter, like whether it is properly accomplishing the thing I want it to do.

(You can probably also tell that like this post, last sentence is not written by AI. Too long, too many commas, ambiguous grammar. I’m leaving it in.)