Classification & Triage

Ticket classification and triage is the work of reading something that just arrived — a support ticket, a GitHub issue, a crash report, a paper submission, a job application — and deciding what it is, who owns it and how urgent it is. It is high volume, repetitive and consequential: a mis-routed outage ticket costs hours, a buried urgent issue costs customers. This page compares the usual approaches to automated triage and shows how to build it with typed Jev decisions.

Why triage is worth automating

Manual triage does not scale linearly. Queues grow, rotations burn people out, and the first hour after something arrives — when routing matters most — is when it is least likely to be read. Automation helps when it does three things reliably:

• Puts each item in one known category or team.
• Assigns a priority on a scale everyone understands.
• Knows when to ask a human instead of guessing.

The same pattern covers customer support routing, GitHub issue classification, crash report screening, research paper triage and resume screening.

Common approaches and trade-offs

Keyword rules and forms. Dropdowns, required fields, regexes on subjects. Cheap and predictable, but users pick the wrong dropdown and describe problems in their own words.

Traditional text classifiers. Logistic regression, gradient boosting or a fine-tuned small transformer on historical labels. Fast and cheap at inference, often accurate on a stable taxonomy. The costs are upfront: labelled data, training pipelines, and retraining every time a team is renamed or a product launches.

Prompted LLMs. Describe the categories, ask for JSON. No training, flexible taxonomy, good with messy language. But each ticket costs a full generative call, outputs occasionally include invented categories or broken JSON, and the "confidence" is whatever number the model chose to write.

Zero-shot embeddings. Compare the ticket with category descriptions in vector space. Cheap, but topic similarity is not ownership: "refund for the outage" belongs to billing, not to reliability.

How Jev's typed decisions fit

Jev, from TypeSafe AI, takes the ticket as state and answers typed questions:

• Choice for category, team or queue — up to 255 options, each with a description. You get the chosen option and a probability for every option.
• Score for urgency or severity on an ordered scale of 2 to 10 levels described in words, such as low, normal, high, outage.
• Noul for yes/no flags as a probability between 0 and 1: needs_human, mentions_legal_threat, is_duplicate_candidate.

Because categories live in the request, a taxonomy change ships with your next deploy. Outputs are always inside the schema. TypeSafe reports 70–500 ms per request, and Jev 1.13 costs $0.042 per million input tokens on Flatkey with free output, so classifying every incoming item is a small cost next to the human time it saves.

Triage architecture walkthrough

1. Normalise the input. Subject, body, a few structured fields (plan, product area, reporter role). Strip signatures and quoted history.
2. Run deterministic routes first. VIP accounts, security keywords that must page someone, items from monitoring systems.
3. Ask Jev once. Category Choice, urgency Score, needs_human Noul.
4. Apply thresholds per category. Auto-assign when confidence clears the bar; send the rest to a general triage queue. High urgency with low confidence should page a person rather than wait.
5. Write back typed fields. Set labels, assignee and priority in your helpdesk or issue tracker so dashboards and SLAs work.
6. Close the loop. When a human re-routes an item, log it. Those corrections are your evaluation set.

Choosing question types for triage

Decision	Question type	Example options or levels
Which team owns it	Choice	billing, auth, api, mobile, other
How urgent	Score	low → normal → high → outage
Needs a person first	Noul	probability from 0 to 1
Possible duplicate	Noul	probability from 0 to 1
Issue type	Choice	bug, feature_request, question, docs

Two practical notes. First, write each option description from the receiving team's point of view ("billing: charges, refunds, invoices, plan changes; not login problems"). Descriptions are your policy, and vague ones produce split probabilities. Second, keep urgency as a Score rather than a Choice: the ordered scale lets your code reason about "at least high," and the level probabilities show whether the model hesitated between two adjacent levels.

For GitHub issues, the same request can label type, area and whether reproduction steps are present, which lets a bot ask the reporter for missing details before a maintainer ever looks. For crash reports and paper submissions, add a Noul for "contradicts the structured fields" to catch records whose narrative disagrees with the coded data.

Pitfalls and evaluation tips

• Taxonomy debt. Categories that overlap ("Account" vs "Login") split probability and lower confidence. Fix the taxonomy before tuning thresholds.
• Class imbalance. Rare but critical categories (security, legal) need their own review of misses; overall accuracy hides them.
• Measure re-route rate. In production the best signal is how often humans move an auto-routed item.
• Bias in screening. For resumes and other people-related decisions, keep protected attributes out of the state, audit outcomes by group, and keep a human decision for anything consequential.
• Confident errors exist. Spot-check high-confidence items too, not only the low-confidence queue.

When not to use Jev

If a structured field already determines the route, use it. If you have a mature, well-performing classifier on a taxonomy that never changes, keep it and maybe use Jev only for the long tail. And Jev should not write the response; pair it with a generator or a person.

Glossary

• Triage: deciding category, owner and priority for incoming work.
• Taxonomy: the set of categories items are sorted into.
• Re-route rate: share of auto-assigned items moved by a human.
• SLA: the response-time promise tied to priority.
• Catch-all: an other option that keeps unusual items out of real queues.

Open a use case below to run the demo and copy the code.

curl -sS https://router.flatkey.ai/api/alpha/decisions \
  -H "Authorization: Bearer $FLATKEY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "typesafe/jev-1.13",
  "state": "Customer was charged twice for the same invoice and wants a refund today.",
  "questions": {
    "department": {
      "type": "choice",
      "criteria": {
        "billing": "Payments, invoices, refunds or subscriptions",
        "technical": "Bugs, APIs or integrations",
        "sales": "Plans, pricing or sales questions"
      },
      "instructions": "Pick the department that best fits the state."
    },
    "urgency": {
      "type": "score",
      "criteria": [
        "normal",
        "elevated",
        "urgent",
        "critical"
      ],
      "instructions": "Rate the urgency of the state."
    },
    "needs_human": {
      "type": "noul",
      "instructions": "Is this true for the state: needs human?"
    }
  }
}'

// Node.js 18+ (built-in fetch). Run: FLATKEY_API_KEY=... node jev.mjs
const body = {
  "model": "typesafe/jev-1.13",
  "state": "Customer was charged twice for the same invoice and wants a refund today.",
  "questions": {
    "department": {
      "type": "choice",
      "criteria": {
        "billing": "Payments, invoices, refunds or subscriptions",
        "technical": "Bugs, APIs or integrations",
        "sales": "Plans, pricing or sales questions"
      },
      "instructions": "Pick the department that best fits the state."
    },
    "urgency": {
      "type": "score",
      "criteria": [
        "normal",
        "elevated",
        "urgent",
        "critical"
      ],
      "instructions": "Rate the urgency of the state."
    },
    "needs_human": {
      "type": "noul",
      "instructions": "Is this true for the state: needs human?"
    }
  }
};

const res = await fetch("https://router.flatkey.ai/api/alpha/decisions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.FLATKEY_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify(body),
});
if (!res.ok) throw new Error(`Jev request failed: ${res.status} ${await res.text()}`);
const { answers, usage } = await res.json();
console.log(JSON.stringify(answers, null, 2), usage);

# Python 3.8+ (pip install requests). Run: FLATKEY_API_KEY=... python jev.py
import json, os, requests

body = json.loads(r"""{
  "model": "typesafe/jev-1.13",
  "state": "Customer was charged twice for the same invoice and wants a refund today.",
  "questions": {
    "department": {
      "type": "choice",
      "criteria": {
        "billing": "Payments, invoices, refunds or subscriptions",
        "technical": "Bugs, APIs or integrations",
        "sales": "Plans, pricing or sales questions"
      },
      "instructions": "Pick the department that best fits the state."
    },
    "urgency": {
      "type": "score",
      "criteria": [
        "normal",
        "elevated",
        "urgent",
        "critical"
      ],
      "instructions": "Rate the urgency of the state."
    },
    "needs_human": {
      "type": "noul",
      "instructions": "Is this true for the state: needs human?"
    }
  }
}""")

res = requests.post(
    "https://router.flatkey.ai/api/alpha/decisions",
    headers={"Authorization": f"Bearer {os.environ['FLATKEY_API_KEY']}"},
    json=body,
    timeout=60,
)
res.raise_for_status()
data = res.json()
print(json.dumps(data["answers"], indent=2), data.get("usage"))