`/ak-test`: fewer box-ticking tests, tests in the right places, run by blast radius

Article 15 in the AgentKit v3 onboarding series. About a 6 minute read.

AI is now trained to produce more stable output, so it writes tests in huge quantities. Great, until you have a few thousand tests and the CI bill is jumping around, yet the app still breaks when it breaks. In that pile there are "obvious tests" (tautological tests, like true == true): tests that vouch for what they just wrote, rather than proving the app won't break. Worse are tests that exist only to turn CI green: rewriting the test to be trivially true, or commenting out the failing test with "unrelated, will deal with later". Later is when exactly? 😂

The ak-test skill was built to deal with exactly these tricks.

1/ What does it do?

• Writes tests with a reason. Before creating a test, the agent must answer 5 questions: what behavior actually changed, what could break, which existing tests already prove this, what is the lowest trustworthy layer to verify at, and do we really need another test.

• Investigates failing tests first. It does not weaken assertions to get past them, does not comment tests out before the cause is explained, and does not skip unless you allow it.

• Cleans up box-ticking tests. Meaningless, obvious, duplicated ones get removed.

• Runs by blast radius. Tests are split into lanes, and each push only runs the lanes the change affects. The full suite is saved for stable release builds.

The result I measured on AK itself: CI time went from about 35 minutes to about 9 minutes after auditing and optimizing. That is one project's number (AK), not a promise for your repo.

2/ When to use it

• You just finished implementing and need verification.

• The suite has ballooned and CI is slow and expensive.

• You do not trust the existing test suite (suspect skipped tests, box-ticking tests).

• Bootstrapping tests for a repo that has none.

• Testing the UI of a web page (ui [url]).

Don't use it when you just want "CI green". This skill exists to block exactly that.

3/ The modes

Per the SKILL.md of ak-test (v1.3.0) in the kit, the syntax is [context] OR ui [url] OR create|optimize|audit [scope] [--advice] [--ultra] [--interview]:

Note: my original post wrote /ak:test; in v3 the skill name is ak-test, so you type /ak-test. Called with no arguments, the skill asks you to pick an operation.

Extra flags:

• --advice: runs under the supervision of kongming (the advisor agent).

• --ultra: the analysis/design step runs as best-of-5, then a verifier picks the strongest direction (the mechanism is described in the ak-brainstorm docs).

• --interview: lists every change it plans to make and waits for your approval before applying. Without this flag, create and optimize apply directly and then report the list of changes, while audit only reports.

For repo-managed manifests, SKILL.md also mentions ak test optimize to pick tests in a deterministic ranked order. This command really exists in ak 3.0.0; the example in the help text is ak test optimize --base origin/dev --json. I have not tried the other flags, see ak test optimize --help.

4/ Using it, step by step

1. Run audit first to see where the suite is sick:

/ak-test audit packages/api

1. Read the report. Audit does not fix anything, you decide which ones to fix.

2. To cut CI time, run optimize, ideally with --interview the first time so you can approve each change:

/ak-test optimize .github/workflows --interview

1. If coverage is missing in an important place, create:

/ak-test create modules/billing

1. If you want more directions weighed, add --ultra.

The principle I keep: we are the ones paying the bill, not the agent. Be as strict as you like.

5/ Status

Wrap-up

1/ Tests in the right places matter more than lots of tests.

2/ A failing test means finding the cause: don't weaken it, don't comment it out, don't skip without permission.

3/ audit to see the disease, optimize to cut the bill, create to patch coverage holes.

4/ Measure before and after. CI time and money are the real evidence.

Sources

• Introducing the /ak:test skill: fewer box-ticking tests, tests in the right places (09/10/2026)

• SKILL.md of the ak-test v1.3.0 skill in the ak 3.0.0 kit