Try it
Copy the code ↓- Data source
- Decision schema
- Model inference
- Structured result
2 Decision schema 3 typed questions
prioritychoicerun_firstrun_laterskiprelevancescorelowmediumhighcovers_diffnoulprobability 0–1
3 Model inference typesafe/jev-1.13
Waiting for a run.
4 Structured result
Raw JSON
Run the demo to see the JSON answers, the highlighted pick and the final action.
FLATKEY_API_KEY and run.curl -sS https://router.flatkey.ai/api/alpha/decisions \
-H "Authorization: Bearer $FLATKEY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "typesafe/jev-1.13",
"state": "PR #4821 — auth session refresh\nChanged files:\n src/auth/session.ts (refresh token rotation, revoke on reuse)\n src/auth/middleware.ts (attach refreshed cookie)\n tests/auth/session_refresh.test.ts (updated)\nNot touched: checkout, billing, search.\n\nCandidate test:\n name: tests/e2e/checkout_flow.spec.ts\n description: Full browser checkout from cart → payment → receipt\n avg_duration: 4m12s\n tags: e2e, checkout, stripe\n\nTask: priority (run_first / run_later / skip), relevance, covers_diff.",
"questions": {
"priority": {
"type": "choice",
"instructions": "Given this diff and this candidate test, how should CI schedule it? Prefer run_first when the test clearly covers changed behavior; skip when it is unrelated; use run_later when it is weakly related or only a smoke check.",
"criteria": {
"run_first": "The test exercises the changed modules, APIs, or failure modes—run it in the first wave before merge-critical gates",
"run_later": "Weak or indirect overlap, or a broad smoke/e2e that can wait until the targeted wave passes",
"skip": "No meaningful overlap with the diff; spending minutes here wastes the budget"
}
},
"relevance": {
"type": "score",
"instructions": "How strongly does this test protect the changed behavior, based only on the paths and test name/description provided.",
"criteria": [
"low",
"medium",
"high"
]
},
"covers_diff": {
"type": "noul",
"instructions": "This test would exercise at least one changed file, symbol, or user-visible path described in the diff—true only when there is concrete overlap, not a vague suite membership."
}
}
}'// Node.js 18+ (built-in fetch). Run: FLATKEY_API_KEY=... node jev.mjs
const body = {
"model": "typesafe/jev-1.13",
"state": "PR #4821 — auth session refresh\nChanged files:\n src/auth/session.ts (refresh token rotation, revoke on reuse)\n src/auth/middleware.ts (attach refreshed cookie)\n tests/auth/session_refresh.test.ts (updated)\nNot touched: checkout, billing, search.\n\nCandidate test:\n name: tests/e2e/checkout_flow.spec.ts\n description: Full browser checkout from cart → payment → receipt\n avg_duration: 4m12s\n tags: e2e, checkout, stripe\n\nTask: priority (run_first / run_later / skip), relevance, covers_diff.",
"questions": {
"priority": {
"type": "choice",
"instructions": "Given this diff and this candidate test, how should CI schedule it? Prefer run_first when the test clearly covers changed behavior; skip when it is unrelated; use run_later when it is weakly related or only a smoke check.",
"criteria": {
"run_first": "The test exercises the changed modules, APIs, or failure modes—run it in the first wave before merge-critical gates",
"run_later": "Weak or indirect overlap, or a broad smoke/e2e that can wait until the targeted wave passes",
"skip": "No meaningful overlap with the diff; spending minutes here wastes the budget"
}
},
"relevance": {
"type": "score",
"instructions": "How strongly does this test protect the changed behavior, based only on the paths and test name/description provided.",
"criteria": [
"low",
"medium",
"high"
]
},
"covers_diff": {
"type": "noul",
"instructions": "This test would exercise at least one changed file, symbol, or user-visible path described in the diff—true only when there is concrete overlap, not a vague suite membership."
}
}
};
const res = await fetch("https://router.flatkey.ai/api/alpha/decisions", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.FLATKEY_API_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify(body),
});
if (!res.ok) throw new Error(`Jev request failed: ${res.status} ${await res.text()}`);
const { answers, usage } = await res.json();
console.log(JSON.stringify(answers, null, 2), usage);# Python 3.8+ (pip install requests). Run: FLATKEY_API_KEY=... python jev.py
import json, os, requests
body = json.loads(r"""{
"model": "typesafe/jev-1.13",
"state": "PR #4821 — auth session refresh\nChanged files:\n src/auth/session.ts (refresh token rotation, revoke on reuse)\n src/auth/middleware.ts (attach refreshed cookie)\n tests/auth/session_refresh.test.ts (updated)\nNot touched: checkout, billing, search.\n\nCandidate test:\n name: tests/e2e/checkout_flow.spec.ts\n description: Full browser checkout from cart → payment → receipt\n avg_duration: 4m12s\n tags: e2e, checkout, stripe\n\nTask: priority (run_first / run_later / skip), relevance, covers_diff.",
"questions": {
"priority": {
"type": "choice",
"instructions": "Given this diff and this candidate test, how should CI schedule it? Prefer run_first when the test clearly covers changed behavior; skip when it is unrelated; use run_later when it is weakly related or only a smoke check.",
"criteria": {
"run_first": "The test exercises the changed modules, APIs, or failure modes—run it in the first wave before merge-critical gates",
"run_later": "Weak or indirect overlap, or a broad smoke/e2e that can wait until the targeted wave passes",
"skip": "No meaningful overlap with the diff; spending minutes here wastes the budget"
}
},
"relevance": {
"type": "score",
"instructions": "How strongly does this test protect the changed behavior, based only on the paths and test name/description provided.",
"criteria": [
"low",
"medium",
"high"
]
},
"covers_diff": {
"type": "noul",
"instructions": "This test would exercise at least one changed file, symbol, or user-visible path described in the diff—true only when there is concrete overlap, not a vague suite membership."
}
}
}""")
res = requests.post(
"https://router.flatkey.ai/api/alpha/decisions",
headers={"Authorization": f"Bearer {os.environ['FLATKEY_API_KEY']}"},
json=body,
timeout=60,
)
res.raise_for_status()
data = res.json()
print(json.dumps(data["answers"], indent=2), data.get("usage"))
No Flatkey key yet? Get one for Jev 1.13 →
About this use case

Who / when
Audience
- CI owners who cannot afford full suites on every push
- harness builders handing the model an option list, not a blank plan
- teams that want cheap relevance scoring before expensive test minutes
Triggers
- a PR changes a few files and you need a ranked subset of tests
- full suite cost or wall time is the bottleneck, not inventing new tests
- the harness already has the test catalog, thresholds, and budget
- you want calibrated run_first / defer / skip instead of an LLM rewriting the plan
The workflow
- 01Diff + candidateChanged files and one testInput
- 02Priorityrun_first / run_later / skipJev
- 03RelevanceScore + covers_diff flagJev
- 04Code gateQueue, defer, or dropYour code
Question shape
{
"priority": {
"type": "choice",
"instructions": "Given this diff and this candidate test, how should CI schedule it? Prefer run_first when the test clearly covers changed behavior; skip when it is unrelated; use run_later when it is weakly related or only a smoke check.",
"criteria": {
"run_first": "The test exercises the changed modules, APIs, or failure modes—run it in the first wave before merge-critical gates",
"run_later": "Weak or indirect overlap, or a broad smoke/e2e that can wait until the targeted wave passes",
"skip": "No meaningful overlap with the diff; spending minutes here wastes the budget"
}
},
"relevance": {
"type": "score",
"instructions": "How strongly does this test protect the changed behavior, based only on the paths and test name/description provided.",
"criteria": [
"low",
"medium",
"high"
]
},
"covers_diff": {
"type": "noul",
"instructions": "This test would exercise at least one changed file, symbol, or user-visible path described in the diff—true only when there is concrete overlap, not a vague suite membership."
}
}Gate policy
≥ 0.90
Honor priority: enqueue run_first immediately; schedule run_later after the first wave; never spend budget on skip when covers_diff is false.
0.65–0.90
Prefer run_later over run_first; log probabilities and widen the first wave only if the budget allows.
< 0.65
Do not auto-promote to run_first; treat as skip or human review of the selection policy.
Failure modes
Pre-launch checks before you trust auto-routing:
- Promoting unrelated e2e suites to run_first because the name sounds important;
- Skipping a unit test that imports a changed module because the path string did not match literally;
- Asking a chat model to rewrite the whole verification plan instead of scoring a fixed catalog;
- Letting Jev invent new tests or authorize merge—selection only, enforcement stays in CI;
- Ignoring duration budgets and scheduling every high-name test first;
FAQ
How is this different from coverage-based test selection?
Coverage and import graphs give hard overlaps; they miss renamed symbols, shared fixtures, and "this e2e would catch the auth cookie bug" judgment. Jev answers run_first / run_later / skip with calibrated scores you can threshold on top of those exact indexes.
Should Jev invent which tests exist?
No. The harness owns the option list, permissions, and budget. Jev only chooses among candidates it was handed—same pattern as routing notes or tools.
What does the Flatkey call look like?
POST https://router.flatkey.ai/api/alpha/decisions with Authorization: Bearer …, Content-Type: application/json, and a body of state, model (typesafe/jev-1.13), and questions (choice / score / noul). This is not an OpenAI-compatible /v1 chat endpoint.
Can I bring my own API key?
Yes. Try the free demo above without signing in. To run it live on your own text, sign in with Google and connect a Flatkey or OpenRouter key (a quick test call runs when you save it). Manage or delete it anytime under Dashboard → BYOK. Keys are stored encrypted on the server, used only for runs you trigger, and never sent back to the browser. Need a Flatkey key? Get one for Jev 1.13 on Flatkey.
Is this the official TypeSafe Jev product page?
No. jev-llm.com is an independent research and practice site. Verify pricing, models and terms at the original TypeSafe sources.
Sources
which tests to run first… harness → option list, thresholds, permissions, budget; verifier → reads the file, the diff and the test output again.
— @DarkFactorr View the post on X →
Wire this gate with your own key
Bring your own key through Flatkey / OpenRouter. Keep exact checks in code.